EmbodiedAI
/
RoboWaiter
mirror of https://github.com/HPCL-EI/RoboWaiter.git
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' of github.com:HPCL-EI/RoboWaiter

This commit is contained in:
ChenXL97 2023-11-28 17:22:12 +08:00
parent 0e8976e846 8bb061e7ee
commit 2e4d563aad
8 changed files with 541 additions and 280 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

57
robowaiter/behavior_tree/dataset/expansion_out/output2.txt Normal file
View File

@ -0,0 +1,57 @@
At(Robot,Bar) 请问您需要什么帮助吗？我就在吧台附近，能否请您自己过来呢？
At(Robot,Bar) 请问你能去一下吧台吗？
At(Robot,Bar) 请问你能帮我到吧台那个位置吗？
At(Robot,WaterTable) 请问你能过来一下吗？我正在茶水桌旁。
At(Robot,WaterTable) 请问你可以去茶水桌一下吗？
At(Robot,WaterTable) 你能带路去茶水桌吗？
At(Robot,CoffeeTable) 请问你可以过来一下吗？我在这张咖啡桌旁边。
At(Robot,CoffeeTable) 请问你可以帮我一下，去一下咖啡桌吗？
At(Robot,CoffeeTable) 请问你能帮我前往咖啡桌那个位置吗？
At(Robot,Bar2) 您好，请问您需要什么帮助吗？我正在另一个吧台处理一些事情。
At(Robot,Bar2) 请问你可以去一下另一个吧台吗？
At(Robot,Bar2) 你能去另一个吧台的位置吗？
At(Robot,Table1) 请问你能过来一下吗？我目前在第一桌，需要你的帮助。
At(Robot,Table1) 请问你能去一下第一桌吗？
At(Robot,Table1) 请问你能帮我到第一张桌子那个位置吗？
At(Robot,Table2) 请问您能过来一下吗？我正在第二张桌子这里。
At(Robot,Table2) 请问你可以去第二张桌子一下吗？
At(Robot,Table2) 请问你能帮我前往第二桌吗？
At(Robot,Table3) 请问你能过来一下吗？我正在第三张桌子旁。
At(Robot,Table3) 请问你能去第三张桌子一下吗？
At(Robot,Table3) 你能告诉我第三张桌子的位置在哪里吗？
On(Softdrink,Bar) 您好，请问您需要我帮您把盒装冰红茶放到哪个位置呢？
On(Softdrink,Bar) 服务员，能否帮我拿来一盒冰红茶放到吧台呢？
On(Softdrink,WaterTable) 您好，请问您需要我帮忙将盒装冰红茶放到哪个位置吗？
On(Softdrink,WaterTable) 服务员，能否帮我把盒装冰红茶拿到茶水桌呢？
On(Softdrink,CoffeeTable) 请问你能把盒装冰红茶放到咖啡桌那个位置吗？
On(Softdrink,CoffeeTable) 服务员，把盒装冰红茶拿到咖啡桌 position 好吗？
On(Softdrink,Bar2) 请问你能把盒装冰红茶放到另一个吧台的位置吗？
On(Softdrink,Bar2) 请问你能把盒装冰红茶拿到另一个吧台位置吗？
On(Softdrink,Table1) 请问您能否把盒装冰红茶放在第一桌的指定位置呢？
On(Softdrink,Table1) 请您把盒装冰红茶拿到第一桌的位置。
On(Softdrink,Table2) 服务员，您好，请问能否帮我將這盒裝冰红茶放 到第二張桌子那個位置呢？
On(Softdrink,Table2) 服务员，请把盒装冰红茶拿到第二张桌子的位置。
On(Softdrink,Table3) 请问你可以把盒装冰红茶放到第三张桌子的那个位置吗？
On(Softdrink,Table3) 请问你能把盒装冰红茶拿到第三张桌子的位置吗？
On(BottledDrink,Bar) 您好，请问您需要我将瓶装饮料放到哪个位置呢？
On(BottledDrink,Bar) 请把瓶装饮料拿到吧台的位置。
On(BottledDrink,WaterTable) 请问你可以把瓶装饮料放到茶水桌那个位置吗？
On(BottledDrink,WaterTable) 请问你能把瓶装饮料拿到茶水桌位置吗？
On(BottledDrink,CoffeeTable) 服务员，能否把瓶装饮料放在咖啡桌那个位置呢？
On(BottledDrink,CoffeeTable) 服务员，能否把瓶装饮料拿到咖啡桌附近呢？
On(BottledDrink,Bar2) 请问你能把瓶装饮料放到另一个吧台的那个位置吗？
On(BottledDrink,Bar2) 请把瓶装饮料拿到另一个吧台位置。
On(BottledDrink,Table1) 请问您能否帮我將瓶裝飲料放至第一張桌子的那個位置呢？
On(BottledDrink,Table1) 请问你能把瓶装饮料拿到第一桌的位置吗？
On(BottledDrink,Table2) 请问，你可以把瓶装饮料放到第二张桌子的那个位置吗？
On(BottledDrink,Table2) 请问你能把瓶装饮料拿到第二张桌子的位置吗？
On(BottledDrink,Table3) 请问，你能把瓶装饮料放到第三桌的哪个位置吗？
On(BottledDrink,Table3) 请问你能把瓶装饮料拿到第三张桌子的位置吗？
On(Yogurt,Bar) 请问你能把酸奶放到吧台那个位置吗？
On(Yogurt,Bar) 请问你能把酸奶拿到吧台位置吗？
On(Yogurt,WaterTable) 请问你能把酸奶放到茶水桌那个位置吗？
On(Yogurt,WaterTable) 服务员，请把酸奶拿到茶水桌的位置。
On(Yogurt,CoffeeTable) 请问，你能把酸奶放在咖啡桌那个位置吗？
On(Yogurt,CoffeeTable) 服务员，能否把酸奶拿到咖啡桌的位置呢？
On(Yogurt,Bar2) 请问你能把酸奶放到另一个吧台的那个位置吗？
On(Yogurt,Bar2) 请问你能把酸奶拿到另一个吧台位置吗？

21
robowaiter/behavior_tree/dataset/sentence_expansion.py
View File

@ -1,6 +1,8 @@
import os
import requests
import urllib3
from tqdm import tqdm
########################################
# 该文件实现了与大模型的简单通信
########################################
@ -18,7 +20,12 @@ def single_round(question,prefix=""):
{
"role": "system",
# "content": "你是一个机器人服务员：RoboWaiter. 你的职责是为顾客提供对话及具身服务。"
"content": "请将以下你对咖啡厅服务员说的话改写成更清晰更合理的顾客表述。"
"content": """
假设现在你是咖啡厅的一个顾客，请将以下你对咖啡厅服务员说的话改写成更清晰更合理的顾客表述。注意：句中的你指的是咖啡厅服务员，也不要说能帮我。
例如：麻烦你去一下吧台。可以转述成：服务员，你能去下吧台吗？
另一个例子：请你拿一下酸奶到吧台位置。可以转述成：服务员，拿一杯酸奶来吧台。
"""
},
{
"role": "user",
@ -37,14 +44,18 @@ def single_round(question,prefix=""):
if __name__ == '__main__':
with open('goal_states_with_description.txt', 'r', encoding='utf-8') as file:
with open('./goal_states_with_description.txt', 'r', encoding='utf-8') as file:
lines = file.readlines()
for line in lines:
output_file = './expansion_out/output2.txt'
with open(output_file, 'a', encoding='utf-8') as file:
file.truncate(0)
for line in tqdm(lines):
tmp = line[:-1].split('\t')
# file.write("""{"title":"%s","text":"%s"}\n""" % (tmp[1], tmp[0]))
question = tmp[1]
# print(single_round(question))
# print(tmp[1])
with open('output1.txt', 'a',encoding='utf-8') as file:
file.write(tmp[0]+"\t"+single_round(question)+'\n')
with open(output_file, 'a', encoding='utf-8') as file:
file.write(tmp[0] + "\t" + single_round(question, prefix="现在改写一下句子：") + '\n')
print("输出完成")

16
robowaiter/llm_client/single_round.py
View File

@ -1,6 +1,8 @@
import time
import requests
import urllib3
########################################
# 该文件实现了与大模型的简单通信
########################################
@ -18,6 +20,13 @@ def single_round(question,prefix=""):
{
"role": "system",
"content": "你是一个机器人服务员：RoboWaiter. 你的职责是为顾客提供对话及具身服务。"
# "content":
# """
# 你是一个熟悉行为树的工程师，你的职责是根据用户需求，为一个人型的机器人设计完成用户需求的行为树序列。
# 行为树是一种有向根树，他的内部节点称为控制节点，控制节点包括选择、顺序，顺序节点只有所有子节点返回成功之后他才成功。
# 选择节点只要有一个子节点返回成功最后就成功。叶节点称为执行节点，执行节点包括动作节点和条件节点。动作节点执行具体的动作，条件节点检查环境中的条件是否满足。
# """
# "content": "你是一个优秀的目标状态规划师，能够根据用户输入，规划出所需要的目标状态。例如：打开窗帘的目标状态可以是Is(Curtain, On)"
},
{
"role": "user",
@ -37,7 +46,10 @@ def single_round(question,prefix=""):
if __name__ == '__main__':
question = '''
给我一杯拿铁
嘿！能做一杯咖啡吗？咖啡就行！
'''
import timeit
print(single_round(question))
cur_time = time.time()
print(single_round(question, prefix='现在给出符合这句话要求的目标状态: '))
print(f"单次生成耗时：{time.time() - cur_time} s \n")

6
robowaiter/scene/scene.py
View File

@ -15,6 +15,8 @@ from robowaiter.proto import GrabSim_pb2
from robowaiter.proto import GrabSim_pb2_grpc
import copy
import os
from robowaiter.scene.ui import scene_ui
from robowaiter.utils import get_root_path
from sklearn.cluster import DBSCAN
from matplotlib import pyplot as plt
@ -1063,7 +1065,7 @@ class Scene:
# cur_objs, objs_name_set = camera.get_semantic_map(GrabSim_pb2.CameraName.Head_Segment, cur_objs,
# objs_name_set)
cur_obstacle_world_points, cur_objs_id = camera.get_obstacle_point(plt, db, scene,
cur_obstacle_world_points, cur_objs_id = camera.get_obstacle_point(self, db, scene,
cur_obstacle_world_points, map_ratio)
# if scene.info == "Unreachable":
@ -1452,7 +1454,9 @@ class Scene:
# plt.text(0, 0.5, f'新增语义信息：{walker_detect_count}', fontsize=10)
# plt.text(0, 0.3, f'更新语义信息：{update_info_count}', fontsize=10)
# plt.text(0, 0.1, f'已存语义信息：{self.infoCount}', fontsize=10)
# 元组传参 pyqt5中的函数名， 四个参数
self.ui_func(("get_info", walker_detect_count, obj_detect_count, update_info_count, self.infoCount))
# draw figures

31
robowaiter/scene/ui/pyqt5.py
View File

@ -80,13 +80,22 @@ class UI(QMainWindow, Ui_MainWindow):
self.btn_say.clicked.connect(self.btn_say_on_click)
self.img_view_bt.setDragMode(QGraphicsView.ScrollHandDrag)
# 下拉菜单绑定函数
# 下拉菜单.seleted.connect(self.你写的函数)
# # BT树的显示
# self.setScaledContents(True) # 图片将根据label的大小自动缩放
# self.label_moved = False # 用于判断是否移动了标签
# self.drag_start_position = QPoint() # 记录鼠标按下时的位置
self.cb_task.setCurrentIndex(-1)
self.cb_task.setPlaceholderText("请选择城市")
# 多个添加条目
self.cb_task.addItems(['Java', 'C#', 'PHP'])
# 当下拉索引发生改变时发射信号触发绑定的事件
self.cb_task.currentIndexChanged.connect(self.selectionchange)
# 绑定任务演示按钮
for example in example_list:
@ -107,6 +116,25 @@ class UI(QMainWindow, Ui_MainWindow):
sys.exit(app.exec_())
def selectionchange(self, i):
print(i)
# self.create_example_click(dic[i])
# 标签用来显示选中的文本
# currentText()：返回选中选项的文本
# self.btn1.setText(self.cb.currentText())
# print('Items in the list are:')
# # 输出选项集合中每个选项的索引与对应的内容
# # count()：返回选项集合中的数目
# for count in range(self.cb.count()):
# print('Item' + str(count) + '=' + self.cb.itemText(count))
# print('current index', i, 'selection changed', self.cb.currentText())
def get_info(self, walker_detect_count, obj_detect_count, update_info_count, infoCount):
print(obj_detect_count)
self.textEdit_5.clear()
self.textEdit_5.append(str(obj_detect_count))
# self.textEdit_5.setText(walker_detect_count + "_" + obj_detect_count + "_" + update_info_count + "_" + infoCount)
def new_history(self,customer_name,chat):
role = chat["role"]
content = chat["content"].strip()
@ -239,6 +267,7 @@ class UI(QMainWindow, Ui_MainWindow):
control.setPixmap(self.scale_pixmap_to_label(pixmap, control))
# control.setPixmap(pixmap)
def draw_canvas(self,control_name,canvas):
control = getattr(self,control_name,None)
# 加载并显示图片

7
robowaiter/scene/ui/scene_ui.py
View File

@ -101,8 +101,14 @@ class SceneUI(Scene):
# plt.title("地图构建过程")
# self.send_img("img_label_map")
# plt.subplot(2, 7, 14) # 这里的2,1表示总共2行，1列，2表示这个位置是第2个子图
# walker_count 新增行人信息
# 新增语义信息
new_add_info = len(cur_objs) - added_info + walker_count
# plt.text(0, 0.5, f'新增语义信息：{new_add_info}', fontsize=10) # 在图中添加文字，x和y坐标是在这个图片大小内的相对位置，fontsize是字体大小
# 已存语义信息
added_info += new_add_info
# plt.text(0, 0.3, f'已存语义信息：{added_info}', fontsize=10) # 在图中添加文字，x和y坐标是在这个图片大小内的相对位置，fontsize是字体大小
self.infoCount = added_info
@ -126,6 +132,7 @@ class SceneUI(Scene):
with open(file_json_name, 'w') as file:
json.dump(obj_json_data, file)
print("已绘制完成地图！！！")
print("------------检测到的所有物品信息--------------")
print(obj_json_data)

251
robowaiter/scene/ui/window.py
View File

@ -14,92 +14,23 @@ from PyQt5 import QtCore, QtGui, QtWidgets
class Ui_MainWindow(object):
def setupUi(self, MainWindow):
MainWindow.setObjectName("MainWindow")
MainWindow.resize(896, 967)
MainWindow.resize(895, 967)
MainWindow.setAutoFillBackground(False)
self.centralwidget = QtWidgets.QWidget(MainWindow)
self.centralwidget.setObjectName("centralwidget")
self.gridLayout_4 = QtWidgets.QGridLayout(self.centralwidget)
self.gridLayout_4.setObjectName("gridLayout_4")
self.gridLayout = QtWidgets.QGridLayout()
self.gridLayout = QtWidgets.QGridLayout(self.centralwidget)
self.gridLayout.setObjectName("gridLayout")
self.label_9 = QtWidgets.QLabel(self.centralwidget)
font = QtGui.QFont()
font.setFamily("等线")
font.setPointSize(10)
font.setBold(True)
font.setWeight(75)
self.label_9.setFont(font)
self.label_9.setObjectName("label_9")
self.gridLayout.addWidget(self.label_9, 0, 0, 1, 2)
self.edit_say = QtWidgets.QLineEdit(self.centralwidget)
font = QtGui.QFont()
font.setFamily("Times New Roman")
font.setPointSize(14)
self.edit_say.setFont(font)
self.edit_say.setObjectName("edit_say")
self.gridLayout.addWidget(self.edit_say, 0, 2, 1, 1)
self.btn_say = QtWidgets.QPushButton(self.centralwidget)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.btn_say.setFont(font)
self.btn_say.setObjectName("btn_say")
self.gridLayout.addWidget(self.btn_say, 0, 3, 1, 1)
self.verticalLayout = QtWidgets.QVBoxLayout()
self.verticalLayout.setObjectName("verticalLayout")
self.btn_AEM = QtWidgets.QPushButton(self.centralwidget)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.btn_AEM.setFont(font)
self.btn_AEM.setObjectName("btn_AEM")
self.verticalLayout.addWidget(self.btn_AEM)
self.btn_VLN = QtWidgets.QPushButton(self.centralwidget)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.btn_VLN.setFont(font)
self.btn_VLN.setObjectName("btn_VLN")
self.verticalLayout.addWidget(self.btn_VLN)
self.btn_VLM = QtWidgets.QPushButton(self.centralwidget)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.btn_VLM.setFont(font)
self.btn_VLM.setObjectName("btn_VLM")
self.verticalLayout.addWidget(self.btn_VLM)
self.btn_GQA = QtWidgets.QPushButton(self.centralwidget)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.btn_GQA.setFont(font)
self.btn_GQA.setObjectName("btn_GQA")
self.verticalLayout.addWidget(self.btn_GQA)
self.btn_OT = QtWidgets.QPushButton(self.centralwidget)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.btn_OT.setFont(font)
self.btn_OT.setObjectName("btn_OT")
self.verticalLayout.addWidget(self.btn_OT)
self.btn_AT = QtWidgets.QPushButton(self.centralwidget)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.btn_AT.setFont(font)
self.btn_AT.setObjectName("btn_AT")
self.verticalLayout.addWidget(self.btn_AT)
self.btn_reset = QtWidgets.QPushButton(self.centralwidget)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.btn_reset.setFont(font)
self.btn_reset.setObjectName("btn_reset")
self.verticalLayout.addWidget(self.btn_reset)
self.gridLayout.addLayout(self.verticalLayout, 1, 0, 2, 1)
self.verticalLayout_4 = QtWidgets.QVBoxLayout()
self.verticalLayout_4.setObjectName("verticalLayout_4")
self.horizontalLayout_2 = QtWidgets.QHBoxLayout()
self.horizontalLayout_2.setSpacing(100)
self.horizontalLayout_2.setObjectName("horizontalLayout_2")
self.label_10 = QtWidgets.QLabel(self.centralwidget)
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Maximum, QtWidgets.QSizePolicy.Preferred)
sizePolicy.setHorizontalStretch(0)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.label_10.sizePolicy().hasHeightForWidth())
self.label_10.setSizePolicy(sizePolicy)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
@ -107,16 +38,27 @@ class Ui_MainWindow(object):
self.label_10.setObjectName("label_10")
self.horizontalLayout_2.addWidget(self.label_10)
self.label_11 = QtWidgets.QLabel(self.centralwidget)
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Fixed, QtWidgets.QSizePolicy.Preferred)
sizePolicy.setHorizontalStretch(0)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.label_11.sizePolicy().hasHeightForWidth())
self.label_11.setSizePolicy(sizePolicy)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.label_11.setFont(font)
self.label_11.setObjectName("label_11")
self.horizontalLayout_2.addWidget(self.label_11)
self.gridLayout.addLayout(self.horizontalLayout_2, 1, 1, 1, 3)
self.horizontalLayout_3 = QtWidgets.QHBoxLayout()
self.horizontalLayout_3.setObjectName("horizontalLayout_3")
self.verticalLayout_4.addLayout(self.horizontalLayout_2)
self.horizontalLayout = QtWidgets.QHBoxLayout()
self.horizontalLayout.setObjectName("horizontalLayout")
self.list_customer = QtWidgets.QListWidget(self.centralwidget)
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Expanding)
sizePolicy.setHorizontalStretch(0)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.list_customer.sizePolicy().hasHeightForWidth())
self.list_customer.setSizePolicy(sizePolicy)
self.list_customer.setMinimumSize(QtCore.QSize(50, 250))
font = QtGui.QFont()
font.setPointSize(11)
self.list_customer.setFont(font)
@ -125,17 +67,23 @@ class Ui_MainWindow(object):
self.list_customer.addItem(item)
item = QtWidgets.QListWidgetItem()
self.list_customer.addItem(item)
self.horizontalLayout_3.addWidget(self.list_customer)
self.horizontalLayout.addWidget(self.list_customer)
self.edit_local_history = QtWidgets.QTextEdit(self.centralwidget)
self.edit_local_history.setEnabled(True)
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Expanding)
sizePolicy.setHorizontalStretch(0)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.edit_local_history.sizePolicy().hasHeightForWidth())
self.edit_local_history.setSizePolicy(sizePolicy)
self.edit_local_history.setMinimumSize(QtCore.QSize(340, 0))
font = QtGui.QFont()
font.setPointSize(11)
self.edit_local_history.setFont(font)
self.edit_local_history.setReadOnly(True)
self.edit_local_history.setObjectName("edit_local_history")
self.horizontalLayout_3.addWidget(self.edit_local_history)
self.gridLayout.addLayout(self.horizontalLayout_3, 2, 1, 1, 3)
self.gridLayout_4.addLayout(self.gridLayout, 0, 0, 1, 1)
self.horizontalLayout.addWidget(self.edit_local_history)
self.verticalLayout_4.addLayout(self.horizontalLayout)
self.gridLayout.addLayout(self.verticalLayout_4, 1, 1, 5, 2)
self.gridLayout_3 = QtWidgets.QGridLayout()
self.gridLayout_3.setObjectName("gridLayout_3")
self.verticalLayout_2 = QtWidgets.QVBoxLayout()
@ -162,7 +110,7 @@ class Ui_MainWindow(object):
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.img_label_map.sizePolicy().hasHeightForWidth())
self.img_label_map.setSizePolicy(sizePolicy)
self.img_label_map.setMinimumSize(QtCore.QSize(300, 200))
self.img_label_map.setMinimumSize(QtCore.QSize(345, 200))
self.img_label_map.setStyleSheet("border: 2px solid black;")
self.img_label_map.setText("")
self.img_label_map.setObjectName("img_label_map")
@ -174,7 +122,7 @@ class Ui_MainWindow(object):
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.img_label_seg.sizePolicy().hasHeightForWidth())
self.img_label_seg.setSizePolicy(sizePolicy)
self.img_label_seg.setMinimumSize(QtCore.QSize(280, 200))
self.img_label_seg.setMinimumSize(QtCore.QSize(250, 200))
self.img_label_seg.setStyleSheet("border: 2px solid black;")
self.img_label_seg.setText("")
self.img_label_seg.setObjectName("img_label_seg")
@ -186,7 +134,7 @@ class Ui_MainWindow(object):
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.img_label_obj.sizePolicy().hasHeightForWidth())
self.img_label_obj.setSizePolicy(sizePolicy)
self.img_label_obj.setMinimumSize(QtCore.QSize(280, 200))
self.img_label_obj.setMinimumSize(QtCore.QSize(250, 200))
self.img_label_obj.setStyleSheet("border: 2px solid black;")
self.img_label_obj.setText("")
self.img_label_obj.setObjectName("img_label_obj")
@ -237,28 +185,119 @@ class Ui_MainWindow(object):
self.horizontalLayout_4.addWidget(self.label_8)
self.gridLayout_2.addLayout(self.horizontalLayout_4, 0, 0, 1, 1)
self.gridLayout_3.addLayout(self.gridLayout_2, 0, 0, 1, 1)
self.gridLayout_4.addLayout(self.gridLayout_3, 1, 0, 1, 1)
self.gridLayout.addLayout(self.gridLayout_3, 6, 0, 1, 3)
self.verticalLayout_3 = QtWidgets.QVBoxLayout()
self.verticalLayout_3.setObjectName("verticalLayout_3")
self.textEdit_5 = QtWidgets.QTextEdit(self.centralwidget)
self.textEdit_5.setObjectName("textEdit_5")
self.verticalLayout_3.addWidget(self.textEdit_5)
self.gridLayout.addLayout(self.verticalLayout_3, 5, 0, 1, 1)
self.verticalLayout = QtWidgets.QVBoxLayout()
self.verticalLayout.setObjectName("verticalLayout")
self.btn_reset = QtWidgets.QPushButton(self.centralwidget)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.btn_reset.setFont(font)
self.btn_reset.setObjectName("btn_reset")
self.verticalLayout.addWidget(self.btn_reset)
self.btn_AEM = QtWidgets.QPushButton(self.centralwidget)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.btn_AEM.setFont(font)
self.btn_AEM.setObjectName("btn_AEM")
self.verticalLayout.addWidget(self.btn_AEM)
self.btn_VLN = QtWidgets.QPushButton(self.centralwidget)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.btn_VLN.setFont(font)
self.btn_VLN.setObjectName("btn_VLN")
self.verticalLayout.addWidget(self.btn_VLN)
self.btn_VLM = QtWidgets.QPushButton(self.centralwidget)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.btn_VLM.setFont(font)
self.btn_VLM.setObjectName("btn_VLM")
self.verticalLayout.addWidget(self.btn_VLM)
self.btn_GQA = QtWidgets.QPushButton(self.centralwidget)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.btn_GQA.setFont(font)
self.btn_GQA.setObjectName("btn_GQA")
self.verticalLayout.addWidget(self.btn_GQA)
self.btn_OT = QtWidgets.QPushButton(self.centralwidget)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.btn_OT.setFont(font)
self.btn_OT.setObjectName("btn_OT")
self.verticalLayout.addWidget(self.btn_OT)
self.btn_AT = QtWidgets.QPushButton(self.centralwidget)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.btn_AT.setFont(font)
self.btn_AT.setObjectName("btn_AT")
self.verticalLayout.addWidget(self.btn_AT)
self.cb_task = QtWidgets.QComboBox(self.centralwidget)
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Preferred, QtWidgets.QSizePolicy.Minimum)
sizePolicy.setHorizontalStretch(0)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.cb_task.sizePolicy().hasHeightForWidth())
self.cb_task.setSizePolicy(sizePolicy)
self.cb_task.setMinimumSize(QtCore.QSize(0, 25))
self.cb_task.setEditable(True)
self.cb_task.setCurrentText("")
self.cb_task.setInsertPolicy(QtWidgets.QComboBox.InsertAtBottom)
self.cb_task.setObjectName("cb_task")
self.verticalLayout.addWidget(self.cb_task)
self.gridLayout.addLayout(self.verticalLayout, 1, 0, 1, 1)
self.label_9 = QtWidgets.QLabel(self.centralwidget)
font = QtGui.QFont()
font.setFamily("等线")
font.setPointSize(10)
font.setBold(True)
font.setWeight(75)
self.label_9.setFont(font)
self.label_9.setObjectName("label_9")
self.gridLayout.addWidget(self.label_9, 0, 0, 1, 1)
self.btn_say = QtWidgets.QPushButton(self.centralwidget)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.btn_say.setFont(font)
self.btn_say.setObjectName("btn_say")
self.gridLayout.addWidget(self.btn_say, 0, 2, 1, 1)
self.label = QtWidgets.QLabel(self.centralwidget)
font = QtGui.QFont()
font.setFamily("黑体")
font.setPointSize(14)
self.label.setFont(font)
self.label.setObjectName("label")
self.gridLayout.addWidget(self.label, 4, 0, 1, 1)
self.edit_say = QtWidgets.QLineEdit(self.centralwidget)
font = QtGui.QFont()
font.setFamily("Times New Roman")
font.setPointSize(14)
self.edit_say.setFont(font)
self.edit_say.setObjectName("edit_say")
self.gridLayout.addWidget(self.edit_say, 0, 1, 1, 1)
MainWindow.setCentralWidget(self.centralwidget)
self.statusbar = QtWidgets.QStatusBar(MainWindow)
self.statusbar.setObjectName("statusbar")
MainWindow.setStatusBar(self.statusbar)
self.retranslateUi(MainWindow)
self.cb_task.setCurrentIndex(-1)
QtCore.QMetaObject.connectSlotsByName(MainWindow)
def retranslateUi(self, MainWindow):
_translate = QtCore.QCoreApplication.translate
MainWindow.setWindowTitle(_translate("MainWindow", "MainWindow"))
self.label_9.setText(_translate("MainWindow", "任务演示：（播放动画时需等待动画播放完毕才会重置场景）"))
self.edit_say.setText(_translate("MainWindow", "Is(AC,On)"))
self.btn_say.setText(_translate("MainWindow", "说话"))
self.btn_AEM.setText(_translate("MainWindow", "环境主动探索"))
self.btn_VLN.setText(_translate("MainWindow", "视觉语言导航"))
self.btn_VLM.setText(_translate("MainWindow", "视觉语言操作"))
self.btn_GQA.setText(_translate("MainWindow", "具身多轮对话"))
self.btn_OT.setText(_translate("MainWindow", "开放具身任务"))
self.btn_AT.setText(_translate("MainWindow", "自主具身任务"))
self.btn_reset.setText(_translate("MainWindow", "重置"))
self.label_10.setText(_translate("MainWindow", "顾客列表："))
self.label_11.setText(_translate("MainWindow", "顾客历史对话："))
__sortingEnabled = self.list_customer.isSortingEnabled()
@ -272,6 +311,18 @@ class Ui_MainWindow(object):
self.label_6.setText(_translate("MainWindow", "可达性地图"))
self.label_7.setText(_translate("MainWindow", "实例分割"))
self.label_8.setText(_translate("MainWindow", "目标检测"))
self.btn_reset.setText(_translate("MainWindow", "重置场景"))
self.btn_AEM.setText(_translate("MainWindow", "环境主动探索"))
self.btn_VLN.setText(_translate("MainWindow", "视觉语言导航"))
self.btn_VLM.setText(_translate("MainWindow", "视觉语言操作"))
self.btn_GQA.setText(_translate("MainWindow", "具身多轮对话"))
self.btn_OT.setText(_translate("MainWindow", "开放具身任务"))
self.btn_AT.setText(_translate("MainWindow", "自主具身任务"))
self.cb_task.setPlaceholderText(_translate("MainWindow", "其他任务"))
self.label_9.setText(_translate("MainWindow", "任务演示：（播放动画时需等待动画播放完毕才会重置场景）"))
self.btn_say.setText(_translate("MainWindow", "说话"))
self.label.setText(_translate("MainWindow", "语义信息"))
self.edit_say.setText(_translate("MainWindow", "Is(AC,On)"))
if __name__ == "__main__":

380
robowaiter/scene/ui/window.ui
View File

@ -6,7 +6,7 @@
<rect>
<x>0</x>
<y>0</y>
<width>896</width>
<width>895</width>
<height>967</height>
</rect>
</property>
@ -17,149 +17,22 @@
<bool>false</bool>
</property>
<widget class="QWidget" name="centralwidget">
<layout class="QGridLayout" name="gridLayout_4">
<item row="0" column="0">
<layout class="QGridLayout" name="gridLayout">
<item row="0" column="0" colspan="2">
<widget class="QLabel" name="label_9">
<property name="font">
<font>
<family>等线</family>
<pointsize>10</pointsize>
<weight>75</weight>
<bold>true</bold>
</font>
</property>
<property name="text">
<string>任务演示：（播放动画时需等待动画播放完毕才会重置场景）</string>
</property>
</widget>
</item>
<item row="0" column="2">
<widget class="QLineEdit" name="edit_say">
<property name="font">
<font>
<family>Times New Roman</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>Is(AC,On)</string>
</property>
</widget>
</item>
<item row="0" column="3">
<widget class="QPushButton" name="btn_say">
<property name="font">
<font>
<family>黑体</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>说话</string>
</property>
</widget>
</item>
<item row="1" column="0" rowspan="2">
<layout class="QVBoxLayout" name="verticalLayout">
<item row="1" column="1" rowspan="5" colspan="2">
<layout class="QVBoxLayout" name="verticalLayout_4">
<item>
<widget class="QPushButton" name="btn_AEM">
<property name="font">
<font>
<family>黑体</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>环境主动探索</string>
</property>
</widget>
</item>
<item>
<widget class="QPushButton" name="btn_VLN">
<property name="font">
<font>
<family>黑体</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>视觉语言导航</string>
</property>
</widget>
</item>
<item>
<widget class="QPushButton" name="btn_VLM">
<property name="font">
<font>
<family>黑体</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>视觉语言操作</string>
</property>
</widget>
</item>
<item>
<widget class="QPushButton" name="btn_GQA">
<property name="font">
<font>
<family>黑体</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>具身多轮对话</string>
</property>
</widget>
</item>
<item>
<widget class="QPushButton" name="btn_OT">
<property name="font">
<font>
<family>黑体</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>开放具身任务</string>
</property>
</widget>
</item>
<item>
<widget class="QPushButton" name="btn_AT">
<property name="font">
<font>
<family>黑体</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>自主具身任务</string>
</property>
</widget>
</item>
<item>
<widget class="QPushButton" name="btn_reset">
<property name="font">
<font>
<family>黑体</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>重置</string>
</property>
</widget>
</item>
</layout>
</item>
<item row="1" column="1" colspan="3">
<layout class="QHBoxLayout" name="horizontalLayout_2">
<property name="spacing">
<number>100</number>
</property>
<item>
<widget class="QLabel" name="label_10">
<property name="sizePolicy">
<sizepolicy hsizetype="Maximum" vsizetype="Preferred">
<horstretch>0</horstretch>
<verstretch>0</verstretch>
</sizepolicy>
</property>
<property name="font">
<font>
<family>黑体</family>
@ -173,6 +46,12 @@
</item>
<item>
<widget class="QLabel" name="label_11">
<property name="sizePolicy">
<sizepolicy hsizetype="Fixed" vsizetype="Preferred">
<horstretch>0</horstretch>
<verstretch>0</verstretch>
</sizepolicy>
</property>
<property name="font">
<font>
<family>黑体</family>
@ -186,10 +65,22 @@
</item>
</layout>
</item>
<item row="2" column="1" colspan="3">
<layout class="QHBoxLayout" name="horizontalLayout_3">
<item>
<layout class="QHBoxLayout" name="horizontalLayout">
<item>
<widget class="QListWidget" name="list_customer">
<property name="sizePolicy">
<sizepolicy hsizetype="Minimum" vsizetype="Expanding">
<horstretch>0</horstretch>
<verstretch>0</verstretch>
</sizepolicy>
</property>
<property name="minimumSize">
<size>
<width>50</width>
<height>250</height>
</size>
</property>
<property name="font">
<font>
<pointsize>11</pointsize>
@ -212,6 +103,18 @@
<property name="enabled">
<bool>true</bool>
</property>
<property name="sizePolicy">
<sizepolicy hsizetype="Minimum" vsizetype="Expanding">
<horstretch>0</horstretch>
<verstretch>0</verstretch>
</sizepolicy>
</property>
<property name="minimumSize">
<size>
<width>340</width>
<height>0</height>
</size>
</property>
<property name="font">
<font>
<pointsize>11</pointsize>
@ -226,7 +129,7 @@
</item>
</layout>
</item>
<item row="1" column="0">
<item row="6" column="0" colspan="3">
<layout class="QGridLayout" name="gridLayout_3">
<item row="1" column="0">
<layout class="QVBoxLayout" name="verticalLayout_2">
@ -265,7 +168,7 @@
</property>
<property name="minimumSize">
<size>
<width>300</width>
<width>345</width>
<height>200</height>
</size>
</property>
@ -290,7 +193,7 @@
</property>
<property name="minimumSize">
<size>
<width>280</width>
<width>250</width>
<height>200</height>
</size>
</property>
@ -315,7 +218,7 @@
</property>
<property name="minimumSize">
<size>
<width>280</width>
<width>250</width>
<height>200</height>
</size>
</property>
@ -418,6 +321,193 @@
</item>
</layout>
</item>
<item row="5" column="0">
<layout class="QVBoxLayout" name="verticalLayout_3">
<item>
<widget class="QTextEdit" name="textEdit_5"/>
</item>
</layout>
</item>
<item row="1" column="0">
<layout class="QVBoxLayout" name="verticalLayout">
<item>
<widget class="QPushButton" name="btn_reset">
<property name="font">
<font>
<family>黑体</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>重置场景</string>
</property>
</widget>
</item>
<item>
<widget class="QPushButton" name="btn_AEM">
<property name="font">
<font>
<family>黑体</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>环境主动探索</string>
</property>
</widget>
</item>
<item>
<widget class="QPushButton" name="btn_VLN">
<property name="font">
<font>
<family>黑体</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>视觉语言导航</string>
</property>
</widget>
</item>
<item>
<widget class="QPushButton" name="btn_VLM">
<property name="font">
<font>
<family>黑体</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>视觉语言操作</string>
</property>
</widget>
</item>
<item>
<widget class="QPushButton" name="btn_GQA">
<property name="font">
<font>
<family>黑体</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>具身多轮对话</string>
</property>
</widget>
</item>
<item>
<widget class="QPushButton" name="btn_OT">
<property name="font">
<font>
<family>黑体</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>开放具身任务</string>
</property>
</widget>
</item>
<item>
<widget class="QPushButton" name="btn_AT">
<property name="font">
<font>
<family>黑体</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>自主具身任务</string>
</property>
</widget>
</item>
<item>
<widget class="QComboBox" name="cb_task">
<property name="sizePolicy">
<sizepolicy hsizetype="Preferred" vsizetype="Minimum">
<horstretch>0</horstretch>
<verstretch>0</verstretch>
</sizepolicy>
</property>
<property name="minimumSize">
<size>
<width>0</width>
<height>25</height>
</size>
</property>
<property name="editable">
<bool>true</bool>
</property>
<property name="currentText">
<string/>
</property>
<property name="currentIndex">
<number>-1</number>
</property>
<property name="insertPolicy">
<enum>QComboBox::InsertAtBottom</enum>
</property>
<property name="placeholderText">
<string extracomment="其他任务">其他任务</string>
</property>
</widget>
</item>
</layout>
</item>
<item row="0" column="0">
<widget class="QLabel" name="label_9">
<property name="font">
<font>
<family>等线</family>
<pointsize>10</pointsize>
<weight>75</weight>
<bold>true</bold>
</font>
</property>
<property name="text">
<string>任务演示：（播放动画时需等待动画播放完毕才会重置场景）</string>
</property>
</widget>
</item>
<item row="0" column="2">
<widget class="QPushButton" name="btn_say">
<property name="font">
<font>
<family>黑体</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>说话</string>
</property>
</widget>
</item>
<item row="4" column="0">
<widget class="QLabel" name="label">
<property name="font">
<font>
<family>黑体</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>语义信息</string>
</property>
</widget>
</item>
<item row="0" column="1">
<widget class="QLineEdit" name="edit_say">
<property name="font">
<font>
<family>Times New Roman</family>
<pointsize>14</pointsize>
</font>
</property>
<property name="text">
<string>Is(AC,On)</string>
</property>
</widget>
</item>
</layout>
</widget>
<widget class="QStatusBar" name="statusbar"/>