摘要
机器人抓取滑动检测对于抓取任务而言具有重要的意义。在抓取过程中,视觉和触觉作为判断抓取状态的关键模态信息,实现其高效融合仍具有挑战性。基于视触觉模态信息融合理念,提出一种新型视触觉融合模型,用于高效解决机器人抓取滑动检测问题。首先,该模型通过卷积神经网络、多尺度时序卷积网络提取视触觉数据的空间和时序特征信息;然后,使用注意力机制为视触觉特征分配权重,并通过多尺度时序卷积网络进行多模态信息融合;最后,通过全连接层输出机器人抓取状态的检测结果。使用7自由度XArm机械臂、D455 RGB摄像头和XELA触觉传感器进行数据采集。实验结果表明,基于该模型的机器人抓取滑动检测的准确率高达98.98%,该模型在可靠、顺利执行机器人抓取任务方面具有较好的研究与应用价值。
Slip detection for robot grasping is of great significance for grip tasks.In the grasping process,vision and touch are the key modal information for judging the grasping state.To achieve efficient visuotactile fusion remains challenging.A novel visual and tactile fusion model is proposed to address the issues of efficient slip detection for robot grasping based on the idea of visual and tactile fusion.First,the model extracts the spatial and temporal feature information of visual and haptic data using a convolutional neural network and a multiscale temporal convolutional network.Second,an attention mechanism is used to assign weights to visuotactile features,and multi-modal information fusion is performed through a multiscale temporal convolutional network.Finally,the detection results of the grasping state are acquired through one fully connected layer.Data acquisition is implemented using an XArm 7DOF robotic arm,D455 RGB camera,and XELA tactile sensor.The experimental results show that the accuracy rate of slip detection for robot grasping using the proposed model is as high as 98.98%.The proposed model has good research and application value in the reliable and smooth execution of robot grip tasks.
作者
黄兆基
高军礼
唐兆年
宋海涛
郭靖
HUANG Zhaoji;GAO Junli;TANG Zhaonian;SONG Haitao;GUO Jing(School of Automation,Guangdong University of Technology,Guangzhou 510006,China;School of Business Administration,South China University of Technology,Guangzhou 510640,China)
出处
《信息与控制》
CSCD
北大核心
2024年第2期191-198,共8页
Information and Control
基金
国家自然科学基金项目(61803103)
国家留学基金项目(201908440537)
关键词
机器人抓取
视触融合
多尺度时序卷积网络
注意力机制
robot grasp
visuo-tactile fusion
multi-scale temporal convolution network
attention mechanism
