Building Advanced Web Applications Using Data Ingestion and Data Processing Tools
Today, advanced websites serve as robust data repositories that constantly collect various user-centered information and prepare it for subsequent processing. The data collected can include a wide range of important information from email addresses, usernames, and passwords to demographic informatio...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-02-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/13/4/709 |
| _version_ | 1797298395603271680 |
|---|---|
| author | Šimun Šprem Nikola Tomažin Jelena Matečić Marko Horvat |
| author_facet | Šimun Šprem Nikola Tomažin Jelena Matečić Marko Horvat |
| author_sort | Šimun Šprem |
| collection | DOAJ |
| description | Today, advanced websites serve as robust data repositories that constantly collect various user-centered information and prepare it for subsequent processing. The data collected can include a wide range of important information from email addresses, usernames, and passwords to demographic information such as age, gender, and geographic location. User behavior metrics are also collected, including browsing history, click patterns, and time spent on pages, as well as different preferences like product selection, language preferences, and individual settings. Interactions, device information, transaction history, authentication data, communication logs, and various analytics and metrics contribute to the comprehensive range of user-centric information collected by websites. A method to systematically ingest and transfer such differently structured information to a central message broker is thoroughly described. In this context, a novel tool—Dataphos Publisher—for the creation of ready-to-digest data packages is presented. Data acquired from the message broker are employed for data quality analysis, storage, conversion, and downstream processing. A brief overview of the commonly used and freely available tools for data ingestion and processing is also provided. |
| first_indexed | 2024-03-07T22:34:15Z |
| format | Article |
| id | doaj.art-1b753b9e3d014f0491113857bfa65317 |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-07T22:34:15Z |
| publishDate | 2024-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-1b753b9e3d014f0491113857bfa653172024-02-23T15:14:44ZengMDPI AGElectronics2079-92922024-02-0113470910.3390/electronics13040709Building Advanced Web Applications Using Data Ingestion and Data Processing ToolsŠimun Šprem0Nikola Tomažin1Jelena Matečić2Marko Horvat3Syntio, Trg Dražena Petrovića 3, HR-10000 Zagreb, CroatiaSyntio, Trg Dražena Petrovića 3, HR-10000 Zagreb, CroatiaSyntio, Trg Dražena Petrovića 3, HR-10000 Zagreb, CroatiaDepartment of Applied Computing, Faculty of Electrical Engineering and Computing, University of Zagreb, Unska 3, HR-10000 Zagreb, CroatiaToday, advanced websites serve as robust data repositories that constantly collect various user-centered information and prepare it for subsequent processing. The data collected can include a wide range of important information from email addresses, usernames, and passwords to demographic information such as age, gender, and geographic location. User behavior metrics are also collected, including browsing history, click patterns, and time spent on pages, as well as different preferences like product selection, language preferences, and individual settings. Interactions, device information, transaction history, authentication data, communication logs, and various analytics and metrics contribute to the comprehensive range of user-centric information collected by websites. A method to systematically ingest and transfer such differently structured information to a central message broker is thoroughly described. In this context, a novel tool—Dataphos Publisher—for the creation of ready-to-digest data packages is presented. Data acquired from the message broker are employed for data quality analysis, storage, conversion, and downstream processing. A brief overview of the commonly used and freely available tools for data ingestion and processing is also provided.https://www.mdpi.com/2079-9292/13/4/709data engineeringbig data analyticsbig data managementdata acquisitiondata ingestionchange data capture (CDC) |
| spellingShingle | Šimun Šprem Nikola Tomažin Jelena Matečić Marko Horvat Building Advanced Web Applications Using Data Ingestion and Data Processing Tools Electronics data engineering big data analytics big data management data acquisition data ingestion change data capture (CDC) |
| title | Building Advanced Web Applications Using Data Ingestion and Data Processing Tools |
| title_full | Building Advanced Web Applications Using Data Ingestion and Data Processing Tools |
| title_fullStr | Building Advanced Web Applications Using Data Ingestion and Data Processing Tools |
| title_full_unstemmed | Building Advanced Web Applications Using Data Ingestion and Data Processing Tools |
| title_short | Building Advanced Web Applications Using Data Ingestion and Data Processing Tools |
| title_sort | building advanced web applications using data ingestion and data processing tools |
| topic | data engineering big data analytics big data management data acquisition data ingestion change data capture (CDC) |
| url | https://www.mdpi.com/2079-9292/13/4/709 |
| work_keys_str_mv | AT simunsprem buildingadvancedwebapplicationsusingdataingestionanddataprocessingtools AT nikolatomazin buildingadvancedwebapplicationsusingdataingestionanddataprocessingtools AT jelenamatecic buildingadvancedwebapplicationsusingdataingestionanddataprocessingtools AT markohorvat buildingadvancedwebapplicationsusingdataingestionanddataprocessingtools |