Exploiting contextual target attributes for target sentiment classification
In the past few years, pre-trained language models (PTLMs) have brought significant improvements to target sentiment classification (TSC). Existing PTLM-based models can be categorized into two groups: 1) fine-tuning-based models that adopt PTLM as the context encoder; 2) prompting-based models that...
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/179897 |
| _version_ | 1826112318993334272 |
|---|---|
| author | Xing, Bowen Tsang, Ivor Wai-Hung |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Xing, Bowen Tsang, Ivor Wai-Hung |
| author_sort | Xing, Bowen |
| collection | NTU |
| description | In the past few years, pre-trained language models (PTLMs) have brought significant improvements to target sentiment classification (TSC). Existing PTLM-based models can be categorized into two groups: 1) fine-tuning-based models that adopt PTLM as the context encoder; 2) prompting-based models that transfer the classification task to the text/word generation task. Despite the improvements achieved by these models, we argue that they have their respective limitations. For fine-tuning-based models, they cannot make the best use of the PTLMs’ strong language modeling ability because the pre-train task and downstream fine-tuning task are not consistent. For prompting-based models, although they can sufficiently leverage the language modeling ability, it is hard to explicitly model the target-context interactions, which are widely realized as a crucial point of this task. In this paper, we present a new perspective of leveraging PTLM for TSC: simultaneously leveraging the merits of both language modeling and explicit target-context interactions via contextual target attributes. Specifically, we design the domain- and target-constrained cloze test, which can leverage the PTLMs’ strong language modeling ability to generate the given target’s attributes pertaining to the review context. The attributes contain the background and property information of the target, which can help to enrich the semantics of the review context and the target. To exploit the attributes for tackling TSC, we first construct a heterogeneous information graph by treating the attributes as nodes and combining them with (1) the syntax graph automatically produced by the off-the-shelf dependency parser and (2) the semantics graph of the review context, which is derived from the self-attention mechanism. Then we propose a heterogeneous information gated graph convolutional network to model the interactions among the attribute information, the syntactic information, and the contextual information. The experimental results on three benchmark datasets demonstrate the superiority of our model, which achieves new state-of-the-art performance. |
| first_indexed | 2024-10-01T03:05:03Z |
| format | Journal Article |
| id | ntu-10356/179897 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T03:05:03Z |
| publishDate | 2024 |
| record_format | dspace |
| spelling | ntu-10356/1798972024-09-06T15:36:11Z Exploiting contextual target attributes for target sentiment classification Xing, Bowen Tsang, Ivor Wai-Hung School of Computer Science and Engineering CFAR, Agency for Science, Technology and Research IHPC, Agency for Science, Technology and Research Computer and Information Science Classification tasks Sentiment classification In the past few years, pre-trained language models (PTLMs) have brought significant improvements to target sentiment classification (TSC). Existing PTLM-based models can be categorized into two groups: 1) fine-tuning-based models that adopt PTLM as the context encoder; 2) prompting-based models that transfer the classification task to the text/word generation task. Despite the improvements achieved by these models, we argue that they have their respective limitations. For fine-tuning-based models, they cannot make the best use of the PTLMs’ strong language modeling ability because the pre-train task and downstream fine-tuning task are not consistent. For prompting-based models, although they can sufficiently leverage the language modeling ability, it is hard to explicitly model the target-context interactions, which are widely realized as a crucial point of this task. In this paper, we present a new perspective of leveraging PTLM for TSC: simultaneously leveraging the merits of both language modeling and explicit target-context interactions via contextual target attributes. Specifically, we design the domain- and target-constrained cloze test, which can leverage the PTLMs’ strong language modeling ability to generate the given target’s attributes pertaining to the review context. The attributes contain the background and property information of the target, which can help to enrich the semantics of the review context and the target. To exploit the attributes for tackling TSC, we first construct a heterogeneous information graph by treating the attributes as nodes and combining them with (1) the syntax graph automatically produced by the off-the-shelf dependency parser and (2) the semantics graph of the review context, which is derived from the self-attention mechanism. Then we propose a heterogeneous information gated graph convolutional network to model the interactions among the attribute information, the syntactic information, and the contextual information. The experimental results on three benchmark datasets demonstrate the superiority of our model, which achieves new state-of-the-art performance. Agency for Science, Technology and Research (A*STAR) Published version This work was supported by National Key Research and Development Project of China (No. 2022YFC3502303) and Australian Research Council Grant (No. DP200101328). Ivor W. Tsang was also supported by A∗STAR Centre for Frontier AI Research. 2024-09-02T02:27:10Z 2024-09-02T02:27:10Z 2024 Journal Article Xing, B. & Tsang, I. W. (2024). Exploiting contextual target attributes for target sentiment classification. Journal of Artificial Intelligence Research, 80, 419-439. https://dx.doi.org/10.1613/jair.1.14947 1076-9757 https://hdl.handle.net/10356/179897 10.1613/jair.1.14947 2-s2.0-85196053620 80 419 439 en Journal of Artificial Intelligence Research © 2024 The Authors. Published by AI Access Foundation under Creative Commons Attribution License CC BY 4.0 . application/pdf |
| spellingShingle | Computer and Information Science Classification tasks Sentiment classification Xing, Bowen Tsang, Ivor Wai-Hung Exploiting contextual target attributes for target sentiment classification |
| title | Exploiting contextual target attributes for target sentiment classification |
| title_full | Exploiting contextual target attributes for target sentiment classification |
| title_fullStr | Exploiting contextual target attributes for target sentiment classification |
| title_full_unstemmed | Exploiting contextual target attributes for target sentiment classification |
| title_short | Exploiting contextual target attributes for target sentiment classification |
| title_sort | exploiting contextual target attributes for target sentiment classification |
| topic | Computer and Information Science Classification tasks Sentiment classification |
| url | https://hdl.handle.net/10356/179897 |
| work_keys_str_mv | AT xingbowen exploitingcontextualtargetattributesfortargetsentimentclassification AT tsangivorwaihung exploitingcontextualtargetattributesfortargetsentimentclassification |