摘要
随着网络技术的迅猛发展,网络已经成为信息的载体,网络随处可见,打开网络,大量的信息充斥而来。如何有效提取并利用网络中有价值的信息将会成为未来一个很大的挑战。网络爬虫是一个可以自动提取网页的程序,从万维网上下载网页,提取信息。通常爬取的页面比较多,如何快速有效地爬取页面是关键,该文通过当当网新书畅销排行榜实例来分析对比现有的四种方式,得出结果异步爬虫和scrapy框架爬虫速度最快。
With the rapid development of network technology,the network has become the carrier of information.The network can be seen everywhere.Opening the network,a large number of information flooded.How to effectively extract and utilize valuable information in the network will become a great challenge in the future.Web crawler is a program that can automatically extract web pages,Download Web pages from the World Wide Web and extract information.Usually there are many pages crawled.How to crawl pages quickly and effectively is the key.This paper compares the four existing ways through examples,and finds that asynchronous crawler and scrapy framework crawler are the fastest.
作者
翟普
ZHAI Pu(Henan Light Industry Vocational College,Zhengzhou 450008,China)
出处
《电脑知识与技术》
2020年第1期29-30,34,共3页
Computer Knowledge and Technology
