Maintaining the search engine freshness using mobile agent
Search engines must keep an up-to-date image to all Web pages and other web resources hosted in web servers in their index and data repositories, to provide better and accurate results to its clients. The crawlers of these search engines have to retrieve the pages continuously to keep the index up-t...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2013-03-01
|
| Series: | Egyptian Informatics Journal |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1110866512000503 |
| _version_ | 1819149765928026112 |
|---|---|
| author | Marwa Badawi Ammar Mohamed Ahmed Hussein Mervat Gheith |
| author_facet | Marwa Badawi Ammar Mohamed Ahmed Hussein Mervat Gheith |
| author_sort | Marwa Badawi |
| collection | DOAJ |
| description | Search engines must keep an up-to-date image to all Web pages and other web resources hosted in web servers in their index and data repositories, to provide better and accurate results to its clients. The crawlers of these search engines have to retrieve the pages continuously to keep the index up-to-date. It is reported in the literature that 40% of the current Internet traffic and bandwidth consumption is due to these crawlers. So we are interested in detecting the significant changes in web pages which reflect effectively in search engine’s index and minimize the network load. In this paper, we suggest a document index based change detection technique and distributed indexing using mobile agents. The experimental results have shown that the proposed system can considerably reduce the network traffic and the computational load on the search engine side and keep its index up-to-date with significant changes. |
| first_indexed | 2024-12-22T14:06:49Z |
| format | Article |
| id | doaj.art-ba7423152cbf43bd8297d7acf50ba24b |
| institution | Directory Open Access Journal |
| issn | 1110-8665 |
| language | English |
| last_indexed | 2024-12-22T14:06:49Z |
| publishDate | 2013-03-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Egyptian Informatics Journal |
| spelling | doaj.art-ba7423152cbf43bd8297d7acf50ba24b2022-12-21T18:23:17ZengElsevierEgyptian Informatics Journal1110-86652013-03-01141273610.1016/j.eij.2012.11.001Maintaining the search engine freshness using mobile agentMarwa Badawi0Ammar Mohamed1Ahmed Hussein2Mervat Gheith3Institute of Statistical Studies and Research, Cairo University, Cairo, EgyptInstitute of Statistical Studies and Research, Cairo University, Cairo, EgyptFaculty of Computers and Information, Cairo University, Cairo, EgyptInstitute of Statistical Studies and Research, Cairo University, Cairo, EgyptSearch engines must keep an up-to-date image to all Web pages and other web resources hosted in web servers in their index and data repositories, to provide better and accurate results to its clients. The crawlers of these search engines have to retrieve the pages continuously to keep the index up-to-date. It is reported in the literature that 40% of the current Internet traffic and bandwidth consumption is due to these crawlers. So we are interested in detecting the significant changes in web pages which reflect effectively in search engine’s index and minimize the network load. In this paper, we suggest a document index based change detection technique and distributed indexing using mobile agents. The experimental results have shown that the proposed system can considerably reduce the network traffic and the computational load on the search engine side and keep its index up-to-date with significant changes.http://www.sciencedirect.com/science/article/pii/S1110866512000503Search engineMobile crawlingDistributed indexingWeb page change detection |
| spellingShingle | Marwa Badawi Ammar Mohamed Ahmed Hussein Mervat Gheith Maintaining the search engine freshness using mobile agent Egyptian Informatics Journal Search engine Mobile crawling Distributed indexing Web page change detection |
| title | Maintaining the search engine freshness using mobile agent |
| title_full | Maintaining the search engine freshness using mobile agent |
| title_fullStr | Maintaining the search engine freshness using mobile agent |
| title_full_unstemmed | Maintaining the search engine freshness using mobile agent |
| title_short | Maintaining the search engine freshness using mobile agent |
| title_sort | maintaining the search engine freshness using mobile agent |
| topic | Search engine Mobile crawling Distributed indexing Web page change detection |
| url | http://www.sciencedirect.com/science/article/pii/S1110866512000503 |
| work_keys_str_mv | AT marwabadawi maintainingthesearchenginefreshnessusingmobileagent AT ammarmohamed maintainingthesearchenginefreshnessusingmobileagent AT ahmedhussein maintainingthesearchenginefreshnessusingmobileagent AT mervatgheith maintainingthesearchenginefreshnessusingmobileagent |