Parallel web crawler architecture for clickstream analysis
The tremendous growth of the Web causes many challenges for single-process crawlers including the presence of some irrelevant answers among search results and the coverage and scaling issues. As a result, more robust algorithms needed to produce more precise and relevant search results in an appropr...
| Main Authors: | , |
|---|---|
| Format: | Book Section |
| Published: |
Springer
2012
|
| Subjects: |
| _version_ | 1796857444463280128 |
|---|---|
| author | Ahmadi-Abkenari, Fatemeh Selamat, Ali |
| author_facet | Ahmadi-Abkenari, Fatemeh Selamat, Ali |
| author_sort | Ahmadi-Abkenari, Fatemeh |
| collection | ePrints |
| description | The tremendous growth of the Web causes many challenges for single-process crawlers including the presence of some irrelevant answers among search results and the coverage and scaling issues. As a result, more robust algorithms needed to produce more precise and relevant search results in an appropriate timely manner. The existed Web crawlers mostly implement link dependent Web page importance metrics. One of the barriers of applying this metrics is that these metrics produce considerable communication overhead on the multi agent crawlers. Moreover, they suffer from the shortcoming of high dependency to their own index size that ends in their failure to rank Web pages with complete accuracy. Hence more enhanced metrics need to be addressed in this area. Proposing new Web page importance metric needs define a new architecture as a framework to implement the metric. The aim of this paper is to propose architecture for a focused parallel crawler. In this framework, the decision-making on Web page importance is based on a combined metric of clickstream analysis and context similarity analysis to the issued queries. |
| first_indexed | 2024-03-05T18:57:50Z |
| format | Book Section |
| id | utm.eprints-35741 |
| institution | Universiti Teknologi Malaysia - ePrints |
| last_indexed | 2024-03-05T18:57:50Z |
| publishDate | 2012 |
| publisher | Springer |
| record_format | dspace |
| spelling | utm.eprints-357412017-02-02T04:57:11Z http://eprints.utm.my/35741/ Parallel web crawler architecture for clickstream analysis Ahmadi-Abkenari, Fatemeh Selamat, Ali QA75 Electronic computers. Computer science The tremendous growth of the Web causes many challenges for single-process crawlers including the presence of some irrelevant answers among search results and the coverage and scaling issues. As a result, more robust algorithms needed to produce more precise and relevant search results in an appropriate timely manner. The existed Web crawlers mostly implement link dependent Web page importance metrics. One of the barriers of applying this metrics is that these metrics produce considerable communication overhead on the multi agent crawlers. Moreover, they suffer from the shortcoming of high dependency to their own index size that ends in their failure to rank Web pages with complete accuracy. Hence more enhanced metrics need to be addressed in this area. Proposing new Web page importance metric needs define a new architecture as a framework to implement the metric. The aim of this paper is to propose architecture for a focused parallel crawler. In this framework, the decision-making on Web page importance is based on a combined metric of clickstream analysis and context similarity analysis to the issued queries. Springer 2012 Book Section PeerReviewed Ahmadi-Abkenari, Fatemeh and Selamat, Ali (2012) Parallel web crawler architecture for clickstream analysis. In: Communications In Computer And Information Science. Springer, Berlin, pp. 123-132. ISBN 978-3-642-32825-1 (Print); 978-3-642-32826-8 (Electronic) http://dx.doi.org/10.1007/978-3-642-32826-8_13 DOI:10.1007/978-3-642-32826-8_13 |
| spellingShingle | QA75 Electronic computers. Computer science Ahmadi-Abkenari, Fatemeh Selamat, Ali Parallel web crawler architecture for clickstream analysis |
| title | Parallel web crawler architecture for clickstream analysis |
| title_full | Parallel web crawler architecture for clickstream analysis |
| title_fullStr | Parallel web crawler architecture for clickstream analysis |
| title_full_unstemmed | Parallel web crawler architecture for clickstream analysis |
| title_short | Parallel web crawler architecture for clickstream analysis |
| title_sort | parallel web crawler architecture for clickstream analysis |
| topic | QA75 Electronic computers. Computer science |
| work_keys_str_mv | AT ahmadiabkenarifatemeh parallelwebcrawlerarchitectureforclickstreamanalysis AT selamatali parallelwebcrawlerarchitectureforclickstreamanalysis |