The fusion of large language models and robotic systems has introduced a transformative paradigm in human–robot interaction,offering unparalleled capabilities in natural language understanding and task execution.This...The fusion of large language models and robotic systems has introduced a transformative paradigm in human–robot interaction,offering unparalleled capabilities in natural language understanding and task execution.This review paper offers a comprehensive analysis of this nascent but rapidly evolving domain,spotlighting the recent advances of Large Language Models(LLMs)in enhancing their structures and performances,particularly in terms of multimodal input handling,high-level reasoning,and plan generation.Moreover,it probes the current methodologies that integrate LLMs into robotic systems for complex task completion,from traditional probabilistic models to the utilization of value functions and metrics for optimal decision-making.Despite these advancements,the paper also reveals the formidable challenges that confront the field,such as contextual understanding,data privacy and ethical considerations.To our best knowledge,this is the first study to comprehensively analyze the advances and considerations of LLMs in Human–Robot Interaction(HRI)based on recent progress,which provides potential avenues for further research.展开更多
This paper presents an innovative investigation on prototyping a digital twin(DT)as the platform for human-robot interactive welding and welder behavior analysis.This humanrobot interaction(HRI)working style helps to ...This paper presents an innovative investigation on prototyping a digital twin(DT)as the platform for human-robot interactive welding and welder behavior analysis.This humanrobot interaction(HRI)working style helps to enhance human users'operational productivity and comfort;while data-driven welder behavior analysis benefits to further novice welder training.This HRI system includes three modules:1)a human user who demonstrates the welding operations offsite with her/his operations recorded by the motion-tracked handles;2)a robot that executes the demonstrated welding operations to complete the physical welding tasks onsite;3)a DT system that is developed based on virtual reality(VR)as a digital replica of the physical human-robot interactive welding environment.The DT system bridges a human user and robot through a bi-directional information flow:a)transmitting demonstrated welding operations in VR to the robot in the physical environment;b)displaying the physical welding scenes to human users in VR.Compared to existing DT systems reported in the literatures,the developed one provides better capability in engaging human users in interacting with welding scenes,through an augmented VR.To verify the effectiveness,six welders,skilled with certain manual welding training and unskilled without any training,tested the system by completing the same welding job;three skilled welders produce satisfied welded workpieces,while the other three unskilled do not.A data-driven approach as a combination of fast Fourier transform(FFT),principal component analysis(PCA),and support vector machine(SVM)is developed to analyze their behaviors.Given an operation sequence,i.e.,motion speed sequence of the welding torch,frequency features are firstly extracted by FFT and then reduced in dimension through PCA,which are finally routed into SVM for classification.The trained model demonstrates a 94.44%classification accuracy in the testing dataset.The successful pattern recognition in skilled welder operations should benefit to accelerate n展开更多
Study results in the last decades show that amount and quality of physical exercises,then the active participation,and now the cognitive involvement of patient in rehabilitation training are crucial to enhance recover...Study results in the last decades show that amount and quality of physical exercises,then the active participation,and now the cognitive involvement of patient in rehabilitation training are crucial to enhance recovery outcome of motor dysfunction patients after stroke.Rehabilitation robots mainly have been developed along this direction to satisfy requirements of recovery therapy,or focused on one or more of the above three points.Therefore,rehabilitation robot based on neuro-machine interaction has been proposed for the paralyzed limb training of post-stroke patient,which utilizes motor related EEG,UCSDI(Ultrasound Current Source Density Imaging),EMG for the robot control and feeds back the multi-sensory interaction information such as visual,auditory,force,haptic sensation to the patient simultaneously.This neuro-controlled and perceptual rehabilitation robot will bring great benefits to post-stroke patients.In order to develop such a kind of rehabilitation robot,some key technologies,such as non-invasive precise measurement and decoding of neural signals,realistic sensation feedback,coordinated control for both the rehabilitation robot and the patient,need to be solved.In this paper,some fundamental problems in developing and optimizing such a kind of rehabilitation robot based on neuro-machine interaction are proposed and discussed.展开更多
文摘The fusion of large language models and robotic systems has introduced a transformative paradigm in human–robot interaction,offering unparalleled capabilities in natural language understanding and task execution.This review paper offers a comprehensive analysis of this nascent but rapidly evolving domain,spotlighting the recent advances of Large Language Models(LLMs)in enhancing their structures and performances,particularly in terms of multimodal input handling,high-level reasoning,and plan generation.Moreover,it probes the current methodologies that integrate LLMs into robotic systems for complex task completion,from traditional probabilistic models to the utilization of value functions and metrics for optimal decision-making.Despite these advancements,the paper also reveals the formidable challenges that confront the field,such as contextual understanding,data privacy and ethical considerations.To our best knowledge,this is the first study to comprehensively analyze the advances and considerations of LLMs in Human–Robot Interaction(HRI)based on recent progress,which provides potential avenues for further research.
文摘This paper presents an innovative investigation on prototyping a digital twin(DT)as the platform for human-robot interactive welding and welder behavior analysis.This humanrobot interaction(HRI)working style helps to enhance human users'operational productivity and comfort;while data-driven welder behavior analysis benefits to further novice welder training.This HRI system includes three modules:1)a human user who demonstrates the welding operations offsite with her/his operations recorded by the motion-tracked handles;2)a robot that executes the demonstrated welding operations to complete the physical welding tasks onsite;3)a DT system that is developed based on virtual reality(VR)as a digital replica of the physical human-robot interactive welding environment.The DT system bridges a human user and robot through a bi-directional information flow:a)transmitting demonstrated welding operations in VR to the robot in the physical environment;b)displaying the physical welding scenes to human users in VR.Compared to existing DT systems reported in the literatures,the developed one provides better capability in engaging human users in interacting with welding scenes,through an augmented VR.To verify the effectiveness,six welders,skilled with certain manual welding training and unskilled without any training,tested the system by completing the same welding job;three skilled welders produce satisfied welded workpieces,while the other three unskilled do not.A data-driven approach as a combination of fast Fourier transform(FFT),principal component analysis(PCA),and support vector machine(SVM)is developed to analyze their behaviors.Given an operation sequence,i.e.,motion speed sequence of the welding torch,frequency features are firstly extracted by FFT and then reduced in dimension through PCA,which are finally routed into SVM for classification.The trained model demonstrates a 94.44%classification accuracy in the testing dataset.The successful pattern recognition in skilled welder operations should benefit to accelerate n
基金supported by the National Natural Science Foundation of China ( Grant no. 61272379, 61325018)the Technique Support Project of Jiangsu Province ( Grantno. BE2014132)
文摘Study results in the last decades show that amount and quality of physical exercises,then the active participation,and now the cognitive involvement of patient in rehabilitation training are crucial to enhance recovery outcome of motor dysfunction patients after stroke.Rehabilitation robots mainly have been developed along this direction to satisfy requirements of recovery therapy,or focused on one or more of the above three points.Therefore,rehabilitation robot based on neuro-machine interaction has been proposed for the paralyzed limb training of post-stroke patient,which utilizes motor related EEG,UCSDI(Ultrasound Current Source Density Imaging),EMG for the robot control and feeds back the multi-sensory interaction information such as visual,auditory,force,haptic sensation to the patient simultaneously.This neuro-controlled and perceptual rehabilitation robot will bring great benefits to post-stroke patients.In order to develop such a kind of rehabilitation robot,some key technologies,such as non-invasive precise measurement and decoding of neural signals,realistic sensation feedback,coordinated control for both the rehabilitation robot and the patient,need to be solved.In this paper,some fundamental problems in developing and optimizing such a kind of rehabilitation robot based on neuro-machine interaction are proposed and discussed.
