摘要
通信感知一体化技术局限于雷达感知与通信在频谱和硬件层面上的共享,不足以提升新兴应用场景中通信与感知的性能.在涵盖海量多模态感知和通信数据的场景中,通信感知一体化技术应向考虑多模态感知的方向进行范式演进,即通信与多模态感知的智能融合.受人类联觉现象启发,文中系统化建立并论述通信和多模态感知智能融合的范式——机器联觉.首先,总结机器联觉3种典型工作模式:唤起模式、增强模式、合作模式,系统全面给出通信和多模态感知之间相互辅助增强的目的与方式.然后,介绍机器联觉研究的数据基础(通信与多模态感知智能融合仿真数据集)和理论基础(通信与多模态感知联觉机理).最后,综述当前机器联觉的研究现状,并展望未来的研究方向.
Integrated sensing and communications(ISAC)technique is limited to the sharing of radar sensing and communications at the spectrum and hardware levels,and it fails to enhance the performance of communication and sensing in future emerging application scenarios.In scenarios involving massive multi-modal sensing and communication data,ISAC should evolve towards the incorporation of multi-modal sensing,specifically intelligent multi-modal sensing-communication integration.Inspired by human synesthesia,a paradigm for intelligent multi-modal sensing-communication integration,synesthesia of machines(SoM),is systematically established and discussed in this paper.Firstly,three typical operational modes of SoM,SoM-evoke,SoM-enhance and SoM-concert,are systematically summarized,and thus the purposes and methods of the mutual assistance and enhancement between communications and multi-modal sensing are given comprehensively.Then,the data foundation of SoM research,mixed multi-modal sensing and communication(M 3SC)simulation dataset,and the theoretical foundation of SoM research,SoM mechanism,are also discussed.Finally,the current research status of SoM is reviewed and future research directions are prospected.
作者
程翔
张浩天
李思江
黄子蔚
杨宗辉
高诗简
白露
张嘉楠
郑心湖
杨柳青
CHENG Xiang;ZHANG Haotian;LI Sijiang;HUANG Ziwei;YANG Zonghui;GAO Shijian;BAI Lu;ZHANG Jia′nan;ZHENG Xinhu;YANG Liuqing(School of Electronics,Peking University,Beijing 100871;Samsung Semiconductor,Samsung SoC Research and Deve-lopment Lab,San Diego,CA 92121,USA;Shandong Research Institute of Industrial Technology,Jinan 250100;Joint SDU-NTU Centre for Artificial Intelligence Research,Shangdong University,Jinan 250101;Intelligent Transportation Thrust,The Hong Kong University of Science and Technology(Guangzhou),Guangzhou 511455;Internet of Things Thrust,The Hong Kong University of Science and Technology(Guangzhou),Guangzhou 511455;Department of Electronic and Computer Engineering,The Hong Kong University of Science and Technology,Hong Kong 999077,China)
出处
《模式识别与人工智能》
EI
CSCD
北大核心
2023年第11期967-986,共20页
Pattern Recognition and Artificial Intelligence
基金
国家重点研发计划项目(No.2020AAA0108101)
国家自然科学基金项目(No.62125101,62341101,62001018,62301011,62373315,U23A20339,62371273)
新基石科学基金会科学探索奖
山东省自然科学基金项目(No.ZR2023YQ058)
第九届青年人才托举工程项目(No.2023QNRC001)
泰山学者工程
广州市科技计划项目(No.2023A03J0011,2023A03J06831)
广东省普通高校重点科研项目(No.2023ZDZX1037)资助。
关键词
机器联觉(SoM)
通信感知一体化
多模态感知
人工神经网络
机器联觉机理
Synesthesia of Machines(SoM)
Integrated Sensing and Communications
Multi-modal Sensing
Artificial Neural Networks
Synesthesia of Machine Mechanism
