摘要
基于深度图的3D手部姿态估计通常需要大量人工标注数据以达到高精确度和鲁棒性,然而关节点标注过程冗杂且存在一定误差.现有研究工作使用自监督方法解决对标注数据的依赖,通过在虚拟数据集上预训练网络,并在无标注的真实数据集上进行模型拟合,实现3D姿态估计.自监督方法的关键在于设计模型拟合的能量函数以减小模型在真实数据集上的精度下降程度.为了减小模型拟合难度,本文提出局部深度一致性损失,依据初始姿态估计结果,提取输入与输出深度图的局部表征,将深度图显式地解耦为以关节点为中心的不同区域.通过有针对性地对不同关节点进行局部优化,减少虚拟与真实深度图之间的固有领域误差对网络学习的影响,增加训练的稳定性.本文方法在NYU数据集上相比基础方法平均关节点误差提升了21.9%.
Depth-based 3D hand pose estimation requires manually labelled data to achieve high accuracy and robust⁃ness.However,the labeling process is laborsome and bares inevitable biases.Researchers solve this problem by using selfsupervised methods.They pretrain model on synthetic dataset then finetune on unlabelled real dataset through model fit⁃ting.The biggest challenge is the design of model fitting term in fintuning stage to prevent severe accuracy drop.We pro⁃posed the regional depth correspondence loss which utilized initial pose estimation results to extract regional representation of input and output depth maps and transparently divided them into different regions.This allows network to finetune re⁃gions around joints without being affected by overall domain gaps between synthetic and real depth images.The proposed method outperforms baseline method by 21.9%on NYU hand pose dataset.
作者
王敬宇
黄伟亭
刘聪
戚琦
孙海峰
廖建新
WANG Jing-yu;HUANG Wei-ting;LIU Cong;QI Qi;SUN Hai-feng;LIAO Jian-xin(State key laboratory of Networking and Switching Techonology,Beijing University of Posts and Telecommunications,Beijing 100876,China;China Mobile Group Design Institute Co.,Ltd.,Beijing 100053,China)
出处
《电子学报》
EI
CAS
CSCD
北大核心
2023年第6期1644-1653,共10页
Acta Electronica Sinica
基金
国家重点研发计划(No.2020YFB1807800)
国家自然科学基金(No.62071067,No.62001054,No.61771068)
教育部-中国移动科研基金(No.MCM20200202,No.MCM20180101)
博士后创新人才支持计划(No.BX20200067)
中国博士后科学基金资助(No.2021M690469)。
关键词
自监督
手部姿态估计
局部一致性
深度图
深度学习
self-supervised
hand pose estimation
regional consistency
depth images
deep learning
