摘要
在深入研究开源网页爬虫——Heritrix系统框架和源代码的基础上,对其搜索内容和范围增加过滤处理功能。这些功能扩展的实现是通过面向对象分析与设计技术,定义类对Heritrix中相关的类进行继承。在校园网范围内运行Heritrix进行网页收集实验。实验结果表明,新增的过滤功能运行有效。
On the basis of in-depth study of the system framework of Web-Crawler, Heritrix and its source code, adds the filtering functions on its target contents as well as the search scope. Realizes the function expansion through object-oriented analysis and design techniques, and constructs some classes that inherit the related classes in Heritrix. Makes Website collection experiment by runnning Hertrix in the campus network, and the result shows that the newly additional functions run well.
出处
《现代计算机》
2012年第14期18-22,共5页
Modern Computer
