The refined algorithm is put forward mainly based on the Wget. Combining the advantages of HTTrackand Wget, the download algorithm is rewritten on the following aspects: the download process is changed fromdepth-first...The refined algorithm is put forward mainly based on the Wget. Combining the advantages of HTTrackand Wget, the download algorithm is rewritten on the following aspects: the download process is changed fromdepth-first retrieval to width-first retrieval; from single-socket to multi-socket; and from recursive retrieval to loopretrieval. The original process,which includes the system calls that maybe cause the network blocking,is cutted intodifferent sectors,and runs in the way of multitask. The results of experiments and feedbacks from actual applicationshow that the refined algorithm gets great improvement on the download efficiency and memory occupancy.展开更多
文摘The refined algorithm is put forward mainly based on the Wget. Combining the advantages of HTTrackand Wget, the download algorithm is rewritten on the following aspects: the download process is changed fromdepth-first retrieval to width-first retrieval; from single-socket to multi-socket; and from recursive retrieval to loopretrieval. The original process,which includes the system calls that maybe cause the network blocking,is cutted intodifferent sectors,and runs in the way of multitask. The results of experiments and feedbacks from actual applicationshow that the refined algorithm gets great improvement on the download efficiency and memory occupancy.
