摘要
文本按体裁自动分类属于按文本的形式分类的范畴,所以它与按内容自动分类问题有许多的不同之处,本文提出了一种关于中文文本体裁自动分类的新机制。在体裁分类过程中首要的问题是分类特征的选取,体裁分类特征项分为两种方式加以描述,一是集合形式,如基于分类词典和语料统计的政论性词汇和情感词汇等,二是规则形式,如公文标识信息和条文句等。基于根据特征之间的关联性和差异性,采用样本分布决策的方法抽取相应的特征项。最后利用支撑向量机算法进行自动分类。该机制已经在五类体裁的语料上得到实现,并获得了较好的效果。
Genre is defined as a category on the basis of external criteria, so its classification is different firm the. classification based on content. A new mechanism for automatic classification of Chinese text genre is presented, and its main idea is as follows. Features for genre classification, as an essential factor in the mechanism, are described in two ways： one is in word-set, such as affective wards and political words derived from some related dictionaries and corpus statistics; another one is in rule format, such as document identifiers and items. In terms of the correlativeness and variance of features, an approach of parametric distribution is applied to evaluate various features of the genres and extract the features for genre classification. Support Voctor Machine is then used as the learning algorithm to build the classifier. The experiment on automatic classification of Chinese text genres, running on a text sorpus consisting of five genres, shows that it can improve the precision of classification.
出处
《中文信息学报》
CSCD
北大核心
2006年第2期24-32,共9页
Journal of Chinese Information Processing
基金
国家自然科学基金资助项目(60373095)
关键词
计算机应用
中文信息处理
体裁分类
特征项选取
样本分布决策
支撑向量机
computer application
Chinese information processing
text genre classification, feature selection
parametric distribution
support vector machine
