摘要
针对手语手势姿态待规范、识别率低的问题,提出一种AR辅助手语字母识别算法MSMobileNet。设计多尺度卷积模块提取底层特征,增强网络的特征提取能力;利用ELU激活函数来保留更全面的负值特征信息;结合适用于Web的轻量级MobileNet模型,提高面向移动AR应用的手语字母识别准确率和实时性。实验结果表明:MS-MobileNet在数据集ASL-M、NUS-II和Creative Senz3D上识别准确率较原模型分别提高了2.58%、5.32%和3.04%。基于MS-MobileNet网络设计一套WebAR辅助的手语字母协同交互系统,经评估测试,用户平均参与度达到8.2分,单次识别耗时低于0.115 s,能较好地满足用户沉浸式的实时手语字母交互需求。
An AR-assisted sign language letter recognition algorithm MS-MobileNet is proposed for the problems of sign language gestures needing to be standardized and low recognition rate.A multi-scale convolution module is designed to extract the low-level features and enhance the feature extraction ability.ELU activation function is used to retain the negative feature information,which combined with a lightweight MobileNet model for the web to improve the recognition accuracy and real-time performance for mobile AR applications.Test results show that compared with the original model,the recognition accuracy of MS-MobileNet on the datasets ASL-M,NUS-II and Creative Senz3D is improved by 2.58%,5.32% and 3.04%,respectively.Based on improved network,a WebAR-assisted sign language collaborative interaction system is designed.After the evaluation test,the average user participation rate reached 8.2 points,and the single recognition time is less than 0.115 s.User's needs for immersive realtime sign language letter interaction is better met.
作者
刘春宏
王松
王赋攀
唐文生
裴云强
田东生
吴亚东
Liu Chunhong;Wang Song;Wang Fupan;Tang Wensheng;Pei Yunqiang;Tian Dongsheng;Wu Yadong(School of Computer Science&Technology,Southwest University of Science and Technology,Mianyang 621010,China;School of Computer Science&Engineering,Sichuan University of Science and Engineering,Zigong 643002,China)
出处
《系统仿真学报》
CAS
CSCD
北大核心
2023年第6期1308-1321,共14页
Journal of System Simulation
基金
国家自然科学基金面上项目(61872304,61802320)
四川省自然科学基金(2022NSFSC0961)
西南科技大学博士基金(19zx7144)
西南科技大学素质类教改(青年发展研究)专项(20szjg17)。
