摘要
针对Web论坛站点结构复杂、内容更新快等特点,提出一种针对论坛的增量信息采集算法,使用站点地图重建技术及网页更新频繁度估计方法,根据站点地图选择有效的链接,按照网页更新频度确定网页的采集频度。实验结果表明,该方法是有效的。
According to the characters of Web forum site such as the complex structure and quickly updating contents,an algorithm of forum incremental information sampling is presented.The technologies of site map rebuilding and estimating the frequency of page update are used.According to the site map,the crawler selects effective links.According to the frequency of Web page update,the crawler determines the crawling frequency of the Web page.Experimental results indicate this method is effective.
出处
《计算机工程》
CAS
CSCD
北大核心
2010年第9期285-287,共3页
Computer Engineering
基金
国家自然科学基金资助项目(60673092)
2008年江苏省重大科技支撑与自主创新基金资助项目(BE2008044)
