The impact of pre-processing and feature selection on text classification
Nowadays text classification is dealing with unstructured and high-dimensionality text document. These textual data can be easily retrieved from social media platforms. However, this textual data is hard to manage and process for classification purposes. Pre-processing activities and feature selecti...
| Main Authors: | , |
|---|---|
| Format: | Conference or Workshop Item |
| Language: | English English |
| Published: |
Springer
2020
|
| Subjects: | |
| Online Access: | http://umpir.ump.edu.my/id/eprint/30799/2/17.1%20The%20impact%20of%20pre-processing%20and%20feature%20selection.pdf http://umpir.ump.edu.my/id/eprint/30799/3/17.%20The%20impact%20of%20pre-processing%20and%20feature%20selection.pdf |
| _version_ | 1825813748875526144 |
|---|---|
| author | Suryanti, Awang Nur Syafiqah, Mohd Nafis |
| author_facet | Suryanti, Awang Nur Syafiqah, Mohd Nafis |
| author_sort | Suryanti, Awang |
| collection | UMP |
| description | Nowadays text classification is dealing with unstructured and high-dimensionality text document. These textual data can be easily retrieved from social media platforms. However, this textual data is hard to manage and process for classification purposes. Pre-processing activities and feature selection are two methods to process the text documents. Therefore, this paper is presented to evaluate the effect of pre-processing and feature selection on the text classification performance. A tweet dataset is utilized and pre-processed using several combinations of pre-processing activities (tokenization, removing stop-words and stemming). Later, two feature selection techniques (Bag-of-Words and Term Frequency-Inverse Document Frequency) are applied on the pre-processed text. Finally, Support Vector Machine classifier is used to test the classification performances. The experimental results reveal that the combination of pre-processing technique and TF-IDF approach achieved greater classification performances compared to BoW approach. Better classification performances hit when the number of features is decreased. However, it is depending on the number of features obtained from the pre-processing activities and feature selection technique chosen. |
| first_indexed | 2024-03-06T12:48:38Z |
| format | Conference or Workshop Item |
| id | UMPir30799 |
| institution | Universiti Malaysia Pahang |
| language | English English |
| last_indexed | 2024-03-06T12:48:38Z |
| publishDate | 2020 |
| publisher | Springer |
| record_format | dspace |
| spelling | UMPir307992024-01-08T04:38:46Z http://umpir.ump.edu.my/id/eprint/30799/ The impact of pre-processing and feature selection on text classification Suryanti, Awang Nur Syafiqah, Mohd Nafis T Technology (General) Nowadays text classification is dealing with unstructured and high-dimensionality text document. These textual data can be easily retrieved from social media platforms. However, this textual data is hard to manage and process for classification purposes. Pre-processing activities and feature selection are two methods to process the text documents. Therefore, this paper is presented to evaluate the effect of pre-processing and feature selection on the text classification performance. A tweet dataset is utilized and pre-processed using several combinations of pre-processing activities (tokenization, removing stop-words and stemming). Later, two feature selection techniques (Bag-of-Words and Term Frequency-Inverse Document Frequency) are applied on the pre-processed text. Finally, Support Vector Machine classifier is used to test the classification performances. The experimental results reveal that the combination of pre-processing technique and TF-IDF approach achieved greater classification performances compared to BoW approach. Better classification performances hit when the number of features is decreased. However, it is depending on the number of features obtained from the pre-processing activities and feature selection technique chosen. Springer 2020 Conference or Workshop Item PeerReviewed pdf en http://umpir.ump.edu.my/id/eprint/30799/2/17.1%20The%20impact%20of%20pre-processing%20and%20feature%20selection.pdf pdf en http://umpir.ump.edu.my/id/eprint/30799/3/17.%20The%20impact%20of%20pre-processing%20and%20feature%20selection.pdf Suryanti, Awang and Nur Syafiqah, Mohd Nafis (2020) The impact of pre-processing and feature selection on text classification. In: Advances in Electronics Engineering: Proceedings of the ICCEE 2019 , 29-30 April 2019 , Kuala Lumpur, Malaysia. pp. 269-280., 619. ISBN 978-981-15-1289-6 (Published) https://doi.org/10.1007/978-981-15-1289-6_25 |
| spellingShingle | T Technology (General) Suryanti, Awang Nur Syafiqah, Mohd Nafis The impact of pre-processing and feature selection on text classification |
| title | The impact of pre-processing and feature selection on text classification |
| title_full | The impact of pre-processing and feature selection on text classification |
| title_fullStr | The impact of pre-processing and feature selection on text classification |
| title_full_unstemmed | The impact of pre-processing and feature selection on text classification |
| title_short | The impact of pre-processing and feature selection on text classification |
| title_sort | impact of pre processing and feature selection on text classification |
| topic | T Technology (General) |
| url | http://umpir.ump.edu.my/id/eprint/30799/2/17.1%20The%20impact%20of%20pre-processing%20and%20feature%20selection.pdf http://umpir.ump.edu.my/id/eprint/30799/3/17.%20The%20impact%20of%20pre-processing%20and%20feature%20selection.pdf |
| work_keys_str_mv | AT suryantiawang theimpactofpreprocessingandfeatureselectionontextclassification AT nursyafiqahmohdnafis theimpactofpreprocessingandfeatureselectionontextclassification AT suryantiawang impactofpreprocessingandfeatureselectionontextclassification AT nursyafiqahmohdnafis impactofpreprocessingandfeatureselectionontextclassification |