Development of machine learning sentiment analyzer and quality classifier (MLSAQC) and its application in analysing hospital patient satisfaction from Facebook reviews in Malaysia
Background: Patient online reviews (POR) on social media platforms have been proposed as novel strategies for assessing patient satisfaction and monitoring healthcare quality. Social media data, on the other hand, is unstructured and huge in volume. Furthermore, no empirical study has been undertake...
| Main Author: | |
|---|---|
| Format: | Thesis |
| Language: | English |
| Published: |
2022
|
| Subjects: | |
| Online Access: | http://eprints.usm.my/53447/1/Afiq%20Izzudin%20A%20Rahim-OCR..pdf |
| _version_ | 1825906883912794112 |
|---|---|
| author | A Rahim, Afiq Izzudin |
| author_facet | A Rahim, Afiq Izzudin |
| author_sort | A Rahim, Afiq Izzudin |
| collection | USM |
| description | Background: Patient online reviews (POR) on social media platforms have been proposed as novel strategies for assessing patient satisfaction and monitoring healthcare quality. Social media data, on the other hand, is unstructured and huge in volume. Furthermore, no empirical study has been undertaken in Malaysia on the use of social media data and the perceived quality of care in hospitals based on POR, as well as the relationship between these variables and hospital accreditation. The objectives of this study were to (1) develop a machine learning system for automatically classifying Facebook (FB) reviews of public hospitals in Malaysia using service quality (SERVQUAL) dimensions and sentiment analysis, (2) determine the validity of FB Reviews as a supplement to a standard patient satisfaction survey, (3) investigate associations between SERVQUAL dimensions and sentiment and patient satisfaction and (4) determine the associations between hospital accreditation status and patient satisfaction and sentiment.
Method: Between 2017 and 2019, we collected comments from 48 official public hospital FB pages. By manually annotating many batches of randomly chosen reviews, we constructed a machine learning quality classifier (MLQC) based on the SERVQUAL model and a machine learning sentiment analyzer (MLSA). The classifiers were trained using logistic regression (LR), naïve Bayes (NB), support vector machine (SVM), and other approaches. Each classifier's performance was evaluated using 5-fold cross validation. We used logistic regression analysis to determine the associations.
Results: The average F1-score for topic classification was between 0.687 and 0.757 for all models. In addition, SVM consistently outperformed other approaches in a 5-fold cross validation of each SERVQUAL dimension and in sentiment analysis. We analysed 1852 reviews in total and discovered that 72.1% of positive reviews and 27.9% of negative reviews were accurately recognised by MLSA. Also, 73.5% of respondents reported being satisfied with public hospital services, while 26.5% reported being dissatisfied. 240 reviews were classified as tangible, 1257 as reliability, 125 as responsive, 356 as assurance, and 1174 as empathetic using the MLQC. After adjusting for hospital covariates, all SERVQUAL indicators except tangible were associated with positive sentiment. Furthermore, after correcting for hospital variables, it was shown that all SERVQUAL dimensions except tangible and assurance were significantly linked with patient dissatisfaction. However, no statistically significant association between hospital accreditation and internet sentiment and patient satisfaction has been identified.
Conclusion: Using data acquired from FB reviews and machine learning algorithms, a pragmatic and practical strategy for eliciting patient perceptions of service quality and supplementing standard patient satisfaction surveys has been created. Additionally, online patient reviews provide a hitherto untapped measure of quality, which may benefit all healthcare stakeholders. Our findings complement earlier studies and the use of FB reviews, in addition to other approaches for assessing the quality of hospital care in Malaysia. Additionally, the findings give critical data that will assist hospital administrators in capitalising on POR through real-time monitoring and evaluation of service quality. |
| first_indexed | 2024-03-06T15:56:10Z |
| format | Thesis |
| id | usm.eprints-53447 |
| institution | Universiti Sains Malaysia |
| language | English |
| last_indexed | 2024-03-06T15:56:10Z |
| publishDate | 2022 |
| record_format | dspace |
| spelling | usm.eprints-534472022-07-19T01:35:46Z http://eprints.usm.my/53447/ Development of machine learning sentiment analyzer and quality classifier (MLSAQC) and its application in analysing hospital patient satisfaction from Facebook reviews in Malaysia A Rahim, Afiq Izzudin R Medicine Background: Patient online reviews (POR) on social media platforms have been proposed as novel strategies for assessing patient satisfaction and monitoring healthcare quality. Social media data, on the other hand, is unstructured and huge in volume. Furthermore, no empirical study has been undertaken in Malaysia on the use of social media data and the perceived quality of care in hospitals based on POR, as well as the relationship between these variables and hospital accreditation. The objectives of this study were to (1) develop a machine learning system for automatically classifying Facebook (FB) reviews of public hospitals in Malaysia using service quality (SERVQUAL) dimensions and sentiment analysis, (2) determine the validity of FB Reviews as a supplement to a standard patient satisfaction survey, (3) investigate associations between SERVQUAL dimensions and sentiment and patient satisfaction and (4) determine the associations between hospital accreditation status and patient satisfaction and sentiment. Method: Between 2017 and 2019, we collected comments from 48 official public hospital FB pages. By manually annotating many batches of randomly chosen reviews, we constructed a machine learning quality classifier (MLQC) based on the SERVQUAL model and a machine learning sentiment analyzer (MLSA). The classifiers were trained using logistic regression (LR), naïve Bayes (NB), support vector machine (SVM), and other approaches. Each classifier's performance was evaluated using 5-fold cross validation. We used logistic regression analysis to determine the associations. Results: The average F1-score for topic classification was between 0.687 and 0.757 for all models. In addition, SVM consistently outperformed other approaches in a 5-fold cross validation of each SERVQUAL dimension and in sentiment analysis. We analysed 1852 reviews in total and discovered that 72.1% of positive reviews and 27.9% of negative reviews were accurately recognised by MLSA. Also, 73.5% of respondents reported being satisfied with public hospital services, while 26.5% reported being dissatisfied. 240 reviews were classified as tangible, 1257 as reliability, 125 as responsive, 356 as assurance, and 1174 as empathetic using the MLQC. After adjusting for hospital covariates, all SERVQUAL indicators except tangible were associated with positive sentiment. Furthermore, after correcting for hospital variables, it was shown that all SERVQUAL dimensions except tangible and assurance were significantly linked with patient dissatisfaction. However, no statistically significant association between hospital accreditation and internet sentiment and patient satisfaction has been identified. Conclusion: Using data acquired from FB reviews and machine learning algorithms, a pragmatic and practical strategy for eliciting patient perceptions of service quality and supplementing standard patient satisfaction surveys has been created. Additionally, online patient reviews provide a hitherto untapped measure of quality, which may benefit all healthcare stakeholders. Our findings complement earlier studies and the use of FB reviews, in addition to other approaches for assessing the quality of hospital care in Malaysia. Additionally, the findings give critical data that will assist hospital administrators in capitalising on POR through real-time monitoring and evaluation of service quality. 2022-03 Thesis NonPeerReviewed application/pdf en http://eprints.usm.my/53447/1/Afiq%20Izzudin%20A%20Rahim-OCR..pdf A Rahim, Afiq Izzudin (2022) Development of machine learning sentiment analyzer and quality classifier (MLSAQC) and its application in analysing hospital patient satisfaction from Facebook reviews in Malaysia. Masters thesis, Universiti Sains Malaysia. |
| spellingShingle | R Medicine A Rahim, Afiq Izzudin Development of machine learning sentiment analyzer and quality classifier (MLSAQC) and its application in analysing hospital patient satisfaction from Facebook reviews in Malaysia |
| title | Development of machine learning sentiment analyzer and quality classifier (MLSAQC) and its application in analysing hospital patient satisfaction from Facebook reviews in Malaysia |
| title_full | Development of machine learning sentiment analyzer and quality classifier (MLSAQC) and its application in analysing hospital patient satisfaction from Facebook reviews in Malaysia |
| title_fullStr | Development of machine learning sentiment analyzer and quality classifier (MLSAQC) and its application in analysing hospital patient satisfaction from Facebook reviews in Malaysia |
| title_full_unstemmed | Development of machine learning sentiment analyzer and quality classifier (MLSAQC) and its application in analysing hospital patient satisfaction from Facebook reviews in Malaysia |
| title_short | Development of machine learning sentiment analyzer and quality classifier (MLSAQC) and its application in analysing hospital patient satisfaction from Facebook reviews in Malaysia |
| title_sort | development of machine learning sentiment analyzer and quality classifier mlsaqc and its application in analysing hospital patient satisfaction from facebook reviews in malaysia |
| topic | R Medicine |
| url | http://eprints.usm.my/53447/1/Afiq%20Izzudin%20A%20Rahim-OCR..pdf |
| work_keys_str_mv | AT arahimafiqizzudin developmentofmachinelearningsentimentanalyzerandqualityclassifiermlsaqcanditsapplicationinanalysinghospitalpatientsatisfactionfromfacebookreviewsinmalaysia |