A set theory based similarity measure for text clustering and classification
Abstract Similarity measures have long been utilized in information retrieval and machine learning domains for multi-purposes including text retrieval, text clustering, text summarization, plagiarism detection, and several other text-processing applications. However, the problem with these measures...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2020-09-01
|
| Series: | Journal of Big Data |
| Subjects: | |
| Online Access: | http://link.springer.com/article/10.1186/s40537-020-00344-3 |
| _version_ | 1819044551655948288 |
|---|---|
| author | Ali A. Amer Hassan I. Abdalla |
| author_facet | Ali A. Amer Hassan I. Abdalla |
| author_sort | Ali A. Amer |
| collection | DOAJ |
| description | Abstract Similarity measures have long been utilized in information retrieval and machine learning domains for multi-purposes including text retrieval, text clustering, text summarization, plagiarism detection, and several other text-processing applications. However, the problem with these measures is that, until recently, there has never been one single measure recorded to be highly effective and efficient at the same time. Thus, the quest for an efficient and effective similarity measure is still an open-ended challenge. This study, in consequence, introduces a new highly-effective and time-efficient similarity measure for text clustering and classification. Furthermore, the study aims to provide a comprehensive scrutinization for seven of the most widely used similarity measures, mainly concerning their effectiveness and efficiency. Using the K-nearest neighbor algorithm (KNN) for classification, the K-means algorithm for clustering, and the bag of word (BoW) model for feature selection, all similarity measures are carefully examined in detail. The experimental evaluation has been made on two of the most popular datasets, namely, Reuters-21 and Web-KB. The obtained results confirm that the proposed set theory-based similarity measure (STB-SM), as a pre-eminent measure, outweighs all state-of-art measures significantly with regards to both effectiveness and efficiency. |
| first_indexed | 2024-12-21T10:14:29Z |
| format | Article |
| id | doaj.art-dad798d9e45f466a8c87a1f6f97a8751 |
| institution | Directory Open Access Journal |
| issn | 2196-1115 |
| language | English |
| last_indexed | 2024-12-21T10:14:29Z |
| publishDate | 2020-09-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Journal of Big Data |
| spelling | doaj.art-dad798d9e45f466a8c87a1f6f97a87512022-12-21T19:07:38ZengSpringerOpenJournal of Big Data2196-11152020-09-017114310.1186/s40537-020-00344-3A set theory based similarity measure for text clustering and classificationAli A. Amer0Hassan I. Abdalla1Computer Science Department, Taiz UniversityCollege of Technological Innovation, Zayed UniversityAbstract Similarity measures have long been utilized in information retrieval and machine learning domains for multi-purposes including text retrieval, text clustering, text summarization, plagiarism detection, and several other text-processing applications. However, the problem with these measures is that, until recently, there has never been one single measure recorded to be highly effective and efficient at the same time. Thus, the quest for an efficient and effective similarity measure is still an open-ended challenge. This study, in consequence, introduces a new highly-effective and time-efficient similarity measure for text clustering and classification. Furthermore, the study aims to provide a comprehensive scrutinization for seven of the most widely used similarity measures, mainly concerning their effectiveness and efficiency. Using the K-nearest neighbor algorithm (KNN) for classification, the K-means algorithm for clustering, and the bag of word (BoW) model for feature selection, all similarity measures are carefully examined in detail. The experimental evaluation has been made on two of the most popular datasets, namely, Reuters-21 and Web-KB. The obtained results confirm that the proposed set theory-based similarity measure (STB-SM), as a pre-eminent measure, outweighs all state-of-art measures significantly with regards to both effectiveness and efficiency.http://link.springer.com/article/10.1186/s40537-020-00344-3Information retrievalText retrievalText classificationSimilarity measuresEmpirical study |
| spellingShingle | Ali A. Amer Hassan I. Abdalla A set theory based similarity measure for text clustering and classification Journal of Big Data Information retrieval Text retrieval Text classification Similarity measures Empirical study |
| title | A set theory based similarity measure for text clustering and classification |
| title_full | A set theory based similarity measure for text clustering and classification |
| title_fullStr | A set theory based similarity measure for text clustering and classification |
| title_full_unstemmed | A set theory based similarity measure for text clustering and classification |
| title_short | A set theory based similarity measure for text clustering and classification |
| title_sort | set theory based similarity measure for text clustering and classification |
| topic | Information retrieval Text retrieval Text classification Similarity measures Empirical study |
| url | http://link.springer.com/article/10.1186/s40537-020-00344-3 |
| work_keys_str_mv | AT aliaamer asettheorybasedsimilaritymeasurefortextclusteringandclassification AT hassaniabdalla asettheorybasedsimilaritymeasurefortextclusteringandclassification AT aliaamer settheorybasedsimilaritymeasurefortextclusteringandclassification AT hassaniabdalla settheorybasedsimilaritymeasurefortextclusteringandclassification |