Detecting premature departure in online text-based counseling using logic-based pattern matching
Background: More so than face-to-face counseling, users of online text-based services might drop out from a session before establishing a clear closure or expressing the intention to leave. Such premature departure may be indicative of heightened risk or dissatisfaction with the service or counselor...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2021-12-01
|
| Series: | Internet Interventions |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2214782921001263 |
| _version_ | 1818723362158936064 |
|---|---|
| author | Yucan Xu Christian S. Chan Christy Tsang Florence Cheung Evangeline Chan Jerry Fung James Chow Lihong He Zhongzhi Xu Paul S.F. Yip |
| author_facet | Yucan Xu Christian S. Chan Christy Tsang Florence Cheung Evangeline Chan Jerry Fung James Chow Lihong He Zhongzhi Xu Paul S.F. Yip |
| author_sort | Yucan Xu |
| collection | DOAJ |
| description | Background: More so than face-to-face counseling, users of online text-based services might drop out from a session before establishing a clear closure or expressing the intention to leave. Such premature departure may be indicative of heightened risk or dissatisfaction with the service or counselor. However, there is no systematic way to identify this understudied phenomenon. Purpose: This study has two objectives. First, we developed a set of rules and used logic-based pattern matching techniques to systematically identify premature departures in an online text-based counseling service. Second, we validated the importance of premature departure by examining its association with user satisfaction. We hypothesized that the users who rated the session as less helpful were more likely to have departed prematurely. Method: We developed and tested a classification model using a sample of 575 human-annotated sessions from an online text-based counseling platform. We used 80% of the dataset to train and develop the model and 20% of the dataset to evaluate the model performance. We further applied the model to the full dataset (34,821 sessions). We compared user satisfaction between premature departure and completed sessions based on data from a post-session survey. Results: The resulting model achieved 97% and 92% F1 score in detecting premature departure cases in the training and test sets, respectively, suggesting it is highly consistent with the judgment of human coders. When applied to the full dataset, the model classified 15,150 (43.5%) sessions as premature departure and the remaining 19,671 (56.5%) as completed sessions. Completed cases (15.2%) were more likely to fill the post-chat survey than premature departure cases (4.0%). Premature departure was significantly associated with lower perceived helpfulness and effectiveness in distress reduction. Conclusions: The model is the first that systematically and accurately identifies premature departure in online text-based counseling. It can be readily modified and transferred to other contexts for the purpose of risk mitigation and service evaluation and improvement. |
| first_indexed | 2024-12-17T21:09:19Z |
| format | Article |
| id | doaj.art-0fe884e474984a2a978ddfbed61ae2bb |
| institution | Directory Open Access Journal |
| issn | 2214-7829 |
| language | English |
| last_indexed | 2024-12-17T21:09:19Z |
| publishDate | 2021-12-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Internet Interventions |
| spelling | doaj.art-0fe884e474984a2a978ddfbed61ae2bb2022-12-21T21:32:29ZengElsevierInternet Interventions2214-78292021-12-0126100486Detecting premature departure in online text-based counseling using logic-based pattern matchingYucan Xu0Christian S. Chan1Christy Tsang2Florence Cheung3Evangeline Chan4Jerry Fung5James Chow6Lihong He7Zhongzhi Xu8Paul S.F. Yip9Centre for Suicide Research and Prevention, The University of Hong Kong, Pokfulam, Hong KongDepartment of Psychology, The University of Hong Kong, Pokfulam, Hong Kong; Corresponding author.Centre for Suicide Research and Prevention, The University of Hong Kong, Pokfulam, Hong KongCentre for Suicide Research and Prevention, The University of Hong Kong, Pokfulam, Hong KongCentre for Suicide Research and Prevention, The University of Hong Kong, Pokfulam, Hong KongCentre for Suicide Research and Prevention, The University of Hong Kong, Pokfulam, Hong KongCentre for Suicide Research and Prevention, The University of Hong Kong, Pokfulam, Hong KongCentre for Suicide Research and Prevention, The University of Hong Kong, Pokfulam, Hong KongCentre for Suicide Research and Prevention, The University of Hong Kong, Pokfulam, Hong KongCentre for Suicide Research and Prevention, The University of Hong Kong, Pokfulam, Hong Kong; Department of Social Work and Social Administration, The University of Hong Kong, Pokfulam, Hong Kong; Correspondence to: P.S.F. Yip, Centre for Suicide Research and Prevention, The University of Hong Kong, Pokfulam, Hong Kong.Background: More so than face-to-face counseling, users of online text-based services might drop out from a session before establishing a clear closure or expressing the intention to leave. Such premature departure may be indicative of heightened risk or dissatisfaction with the service or counselor. However, there is no systematic way to identify this understudied phenomenon. Purpose: This study has two objectives. First, we developed a set of rules and used logic-based pattern matching techniques to systematically identify premature departures in an online text-based counseling service. Second, we validated the importance of premature departure by examining its association with user satisfaction. We hypothesized that the users who rated the session as less helpful were more likely to have departed prematurely. Method: We developed and tested a classification model using a sample of 575 human-annotated sessions from an online text-based counseling platform. We used 80% of the dataset to train and develop the model and 20% of the dataset to evaluate the model performance. We further applied the model to the full dataset (34,821 sessions). We compared user satisfaction between premature departure and completed sessions based on data from a post-session survey. Results: The resulting model achieved 97% and 92% F1 score in detecting premature departure cases in the training and test sets, respectively, suggesting it is highly consistent with the judgment of human coders. When applied to the full dataset, the model classified 15,150 (43.5%) sessions as premature departure and the remaining 19,671 (56.5%) as completed sessions. Completed cases (15.2%) were more likely to fill the post-chat survey than premature departure cases (4.0%). Premature departure was significantly associated with lower perceived helpfulness and effectiveness in distress reduction. Conclusions: The model is the first that systematically and accurately identifies premature departure in online text-based counseling. It can be readily modified and transferred to other contexts for the purpose of risk mitigation and service evaluation and improvement.http://www.sciencedirect.com/science/article/pii/S2214782921001263E-counselingText-based counselingDropoutsPattern matchingText matching |
| spellingShingle | Yucan Xu Christian S. Chan Christy Tsang Florence Cheung Evangeline Chan Jerry Fung James Chow Lihong He Zhongzhi Xu Paul S.F. Yip Detecting premature departure in online text-based counseling using logic-based pattern matching Internet Interventions E-counseling Text-based counseling Dropouts Pattern matching Text matching |
| title | Detecting premature departure in online text-based counseling using logic-based pattern matching |
| title_full | Detecting premature departure in online text-based counseling using logic-based pattern matching |
| title_fullStr | Detecting premature departure in online text-based counseling using logic-based pattern matching |
| title_full_unstemmed | Detecting premature departure in online text-based counseling using logic-based pattern matching |
| title_short | Detecting premature departure in online text-based counseling using logic-based pattern matching |
| title_sort | detecting premature departure in online text based counseling using logic based pattern matching |
| topic | E-counseling Text-based counseling Dropouts Pattern matching Text matching |
| url | http://www.sciencedirect.com/science/article/pii/S2214782921001263 |
| work_keys_str_mv | AT yucanxu detectingprematuredepartureinonlinetextbasedcounselingusinglogicbasedpatternmatching AT christianschan detectingprematuredepartureinonlinetextbasedcounselingusinglogicbasedpatternmatching AT christytsang detectingprematuredepartureinonlinetextbasedcounselingusinglogicbasedpatternmatching AT florencecheung detectingprematuredepartureinonlinetextbasedcounselingusinglogicbasedpatternmatching AT evangelinechan detectingprematuredepartureinonlinetextbasedcounselingusinglogicbasedpatternmatching AT jerryfung detectingprematuredepartureinonlinetextbasedcounselingusinglogicbasedpatternmatching AT jameschow detectingprematuredepartureinonlinetextbasedcounselingusinglogicbasedpatternmatching AT lihonghe detectingprematuredepartureinonlinetextbasedcounselingusinglogicbasedpatternmatching AT zhongzhixu detectingprematuredepartureinonlinetextbasedcounselingusinglogicbasedpatternmatching AT paulsfyip detectingprematuredepartureinonlinetextbasedcounselingusinglogicbasedpatternmatching |