The Data Extraction Using Distributed Crawler Inside Multi-Agent System
The paper discusses the use of web crawler technology. We created an application based on standard web crawler. Our application is determined for data extraction. Primarily, the application was designed to extract data using keywords from a social network Twitter. First, we created a standard crawle...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
VSB-Technical University of Ostrava
2013-01-01
|
| Series: | Advances in Electrical and Electronic Engineering |
| Subjects: | |
| Online Access: | http://advances.utc.sk/index.php/AEEE/article/view/867 |
| _version_ | 1797827030980493312 |
|---|---|
| author | Karel Tomala Jan Plucar Patrik Dubec Lukas Rapant Miroslav Voznak |
| author_facet | Karel Tomala Jan Plucar Patrik Dubec Lukas Rapant Miroslav Voznak |
| author_sort | Karel Tomala |
| collection | DOAJ |
| description | The paper discusses the use of web crawler technology. We created an application based on standard web crawler. Our application is determined for data extraction. Primarily, the application was designed to extract data using keywords from a social network Twitter. First, we created a standard crawler, which went through a predefined list of URLs and gradually download page content of each of the URLs. Page content was then parsed and important text and metadata were stored in a database. Recently, the application was modified in to the form of the multi-agent system. The system was developed in the C# language, which is used to create web applications and sites etc. Obtained data was evaluated graphically. The system was created within Indect project at the VSB-Technical University of Ostrava. |
| first_indexed | 2024-04-09T12:43:05Z |
| format | Article |
| id | doaj.art-5c3df63e7841404db067cc076345f7f9 |
| institution | Directory Open Access Journal |
| issn | 1336-1376 1804-3119 |
| language | English |
| last_indexed | 2024-04-09T12:43:05Z |
| publishDate | 2013-01-01 |
| publisher | VSB-Technical University of Ostrava |
| record_format | Article |
| series | Advances in Electrical and Electronic Engineering |
| spelling | doaj.art-5c3df63e7841404db067cc076345f7f92023-05-14T20:50:08ZengVSB-Technical University of OstravaAdvances in Electrical and Electronic Engineering1336-13761804-31192013-01-0111645546010.15598/aeee.v11i6.867626The Data Extraction Using Distributed Crawler Inside Multi-Agent SystemKarel Tomala0Jan PlucarPatrik DubecLukas RapantMiroslav VoznakProofreader, VSB - Technical University of Ostrava Faculty of Electrical Engineering and Computer Science, Department of TelecommunicationsThe paper discusses the use of web crawler technology. We created an application based on standard web crawler. Our application is determined for data extraction. Primarily, the application was designed to extract data using keywords from a social network Twitter. First, we created a standard crawler, which went through a predefined list of URLs and gradually download page content of each of the URLs. Page content was then parsed and important text and metadata were stored in a database. Recently, the application was modified in to the form of the multi-agent system. The system was developed in the C# language, which is used to create web applications and sites etc. Obtained data was evaluated graphically. The system was created within Indect project at the VSB-Technical University of Ostrava.http://advances.utc.sk/index.php/AEEE/article/view/867class diagrammulti-agent systemtwitterweb crawler. |
| spellingShingle | Karel Tomala Jan Plucar Patrik Dubec Lukas Rapant Miroslav Voznak The Data Extraction Using Distributed Crawler Inside Multi-Agent System Advances in Electrical and Electronic Engineering class diagram multi-agent system web crawler. |
| title | The Data Extraction Using Distributed Crawler Inside Multi-Agent System |
| title_full | The Data Extraction Using Distributed Crawler Inside Multi-Agent System |
| title_fullStr | The Data Extraction Using Distributed Crawler Inside Multi-Agent System |
| title_full_unstemmed | The Data Extraction Using Distributed Crawler Inside Multi-Agent System |
| title_short | The Data Extraction Using Distributed Crawler Inside Multi-Agent System |
| title_sort | data extraction using distributed crawler inside multi agent system |
| topic | class diagram multi-agent system web crawler. |
| url | http://advances.utc.sk/index.php/AEEE/article/view/867 |
| work_keys_str_mv | AT kareltomala thedataextractionusingdistributedcrawlerinsidemultiagentsystem AT janplucar thedataextractionusingdistributedcrawlerinsidemultiagentsystem AT patrikdubec thedataextractionusingdistributedcrawlerinsidemultiagentsystem AT lukasrapant thedataextractionusingdistributedcrawlerinsidemultiagentsystem AT miroslavvoznak thedataextractionusingdistributedcrawlerinsidemultiagentsystem AT kareltomala dataextractionusingdistributedcrawlerinsidemultiagentsystem AT janplucar dataextractionusingdistributedcrawlerinsidemultiagentsystem AT patrikdubec dataextractionusingdistributedcrawlerinsidemultiagentsystem AT lukasrapant dataextractionusingdistributedcrawlerinsidemultiagentsystem AT miroslavvoznak dataextractionusingdistributedcrawlerinsidemultiagentsystem |