摘要
针对难以获取银行业智能客服系统的核心算法以及真实用户语料等关键信息,导致难以对智能客服系统性能进行统一、客观评价的现状,本文深入调研和分析了银行业智能客服支持场景及业务逻辑,建立了一套基于典型范式的智能客服性能评测方法。并基于实际产品进行验证,选取3个国内领先银行的智能对话客服进行性能评测,在两组不同的评测语料上取得了一致的排名结果,降低人为主观因素的影响,实现了对客服系统性能的客观对标。该评测方法可广泛应用于各领域智能客服系统,不仅可以提升评测效率,同时降低人工评估过程中的主观性影响。
Addressing the issue of difficulty in obtaining crucial information such as the core algorithms of banking intelligent customer service systems and realistic user dialogue,which results in challenges in a uniform and objective evaluation of the performance of these systems,this article conducts in-depth research and analysis on the support scenarios and business logic of intelligent customer service in the banking industry,and establishes a performance evaluation method for intelligent customer service based on typical paradigms.This method was verified using actual products,selecting three leading domestic banks'intelligent dialogue customer service systems for performance evaluation,achieving consistent ranking results on two different sets of evaluation dialogues,reducing the influence of human subjectivity and achieving objective benchmarking of customer service system performance.This evaluation method can be widely applied to intelligent customer service systems in various sectors,not only improving evaluation efficiency but also reducing the influence of subjectivity in manual evaluation processes.
作者
段宣翡
刘伟东
高东辉
黄艳华
刘硕
焦若丹
DUAN Xuanfei;LIU Weidong;GAO Donghui;HUANG Yanhua;LIU Shuo;JIAO Ruodan(China Mobile Communications Corporation Research Institute,Beijing 100053,China)
出处
《电子测试》
2023年第1期53-59,共7页
Electronic Test
关键词
智能客服
人机交互
性能评测
intelligent customer service
human-machine interaction
performance evaluation
