为了解决传统分布式搜索引擎存在的搜索性能问题,从索引结构、查询算法方面改进了传统模型.提出了一种非集中的高并行化搜索模型,该模型按照文档主题对索引分类,对较长的倒排记录表采用位图结构,利用多线程技术对索引节点实现并行搜索算...为了解决传统分布式搜索引擎存在的搜索性能问题,从索引结构、查询算法方面改进了传统模型.提出了一种非集中的高并行化搜索模型,该模型按照文档主题对索引分类,对较长的倒排记录表采用位图结构,利用多线程技术对索引节点实现并行搜索算法(multi max score heap,MMSH).实验结果表明:改进模型中的索引分类方法与倒排表结构的位图策略,能够增强Merge层查询的针对性,降低Merge层节点的CPU和内存开销;在倒排表不能完全存入内存情况下,MMSH算法能够实现高度并行化查询,其查询效率高于经典的term-at-a-time算法,缩短了平均查找时间,提高了系统吞吐量.索引分类、位图结构以及并行查询算法能够避免查询的盲目性,改善了分布式搜索引擎的性能.展开更多
To improve efficiency of search engines, the query result cache has drawn much attention re- cently. According to the query processing and user＇s query logs locality, a new hybrid result cache strategy which associat...To improve efficiency of search engines, the query result cache has drawn much attention re- cently. According to the query processing and user＇s query logs locality, a new hybrid result cache strategy which associates with caching heat and worth is proposed to compute cache score in accord- ance with cost-aware strategies. Exactly, query repeated distance and query length factor are utilized to improve the static result policy, and the dynamic policy is adjusted by the caching worth. The hy- brid result cache is implemented in term of the document content and document ids （docIds） se- quence. Based on a score format and the new hybrid structure, an initial algorithm and a new rou- ting algorithm are designed for result cache. Experiments＇ results show that the improved caching policies decrease the average response time effectively, and increase the system throughput signifi- cantly. By choosing comfortable combination of page cache and docIds cache, the new hybrid cac- hing strategy almost reduces more than 20% of the only cache and docId-only cache. average query time compared with the basic page-展开更多
文摘为了解决传统分布式搜索引擎存在的搜索性能问题,从索引结构、查询算法方面改进了传统模型.提出了一种非集中的高并行化搜索模型,该模型按照文档主题对索引分类,对较长的倒排记录表采用位图结构,利用多线程技术对索引节点实现并行搜索算法(multi max score heap,MMSH).实验结果表明:改进模型中的索引分类方法与倒排表结构的位图策略,能够增强Merge层查询的针对性,降低Merge层节点的CPU和内存开销;在倒排表不能完全存入内存情况下,MMSH算法能够实现高度并行化查询,其查询效率高于经典的term-at-a-time算法,缩短了平均查找时间,提高了系统吞吐量.索引分类、位图结构以及并行查询算法能够避免查询的盲目性,改善了分布式搜索引擎的性能.
基金Supported by the National Natural Science Foundation of China(No.61173024)
文摘To improve efficiency of search engines, the query result cache has drawn much attention re- cently. According to the query processing and user＇s query logs locality, a new hybrid result cache strategy which associates with caching heat and worth is proposed to compute cache score in accord- ance with cost-aware strategies. Exactly, query repeated distance and query length factor are utilized to improve the static result policy, and the dynamic policy is adjusted by the caching worth. The hy- brid result cache is implemented in term of the document content and document ids （docIds） se- quence. Based on a score format and the new hybrid structure, an initial algorithm and a new rou- ting algorithm are designed for result cache. Experiments＇ results show that the improved caching policies decrease the average response time effectively, and increase the system throughput signifi- cantly. By choosing comfortable combination of page cache and docIds cache, the new hybrid cac- hing strategy almost reduces more than 20% of the only cache and docId-only cache. average query time compared with the basic page-
