A Semi-Supervised Paraphrase Identification Model Based on Multi-Granularity Interaction Reasoning
Conventional paraphrase identification (PI) models based on deep learning usually focus on text representation and ignore the mining and matching of multi-granular interaction features. In addition, supervised learning relies on a large labeled data. However, labeled training set for PI is small in...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9050515/ |
| _version_ | 1819276400290430976 |
|---|---|
| author | Xu Li Fanxu Zeng Chunlong Yao |
| author_facet | Xu Li Fanxu Zeng Chunlong Yao |
| author_sort | Xu Li |
| collection | DOAJ |
| description | Conventional paraphrase identification (PI) models based on deep learning usually focus on text representation and ignore the mining and matching of multi-granular interaction features. In addition, supervised learning relies on a large labeled data. However, labeled training set for PI is small in comparison with the high complexity of the task. To solve the problems, we propose a semi-supervised deep learning framework for PI. We use a neural encoder with word-by-word attention mechanism to reason equivalence or contradiction over pairs of words, phrases and sentences. We employ a two-stage training procedure. First, we use a language modeling objective to learn the initial parameters on the unlabeled corpora of more than one million pairs of sentences. This is followed by a supervised training, where we adapt these parameters to a specific classification task with labeled data. Experimental results on MRPC (Microsoft Research Paraphrase Corpus) and SICK (Sentences Involving Compositional Knowledge) datasets demonstrate the effectiveness of our approach. Compared with the previous neural network models, we achieve absolute improvements in accuracy of 7.6% and F1 of 5.4% on MRPC, Pearson's r of 4.5% and Spearman's ρ of 5.1% on SICK. |
| first_indexed | 2024-12-23T23:39:37Z |
| format | Article |
| id | doaj.art-62cf5a9e927a43e3800160a865b2e89d |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-23T23:39:37Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-62cf5a9e927a43e3800160a865b2e89d2022-12-21T17:25:44ZengIEEEIEEE Access2169-35362020-01-018607906080010.1109/ACCESS.2020.29840099050515A Semi-Supervised Paraphrase Identification Model Based on Multi-Granularity Interaction ReasoningXu Li0https://orcid.org/0000-0002-4450-5305Fanxu Zeng1Chunlong Yao2School of Information Science and Engineering, Dalian Polytechnic University, Dalian, ChinaSchool of Information Science and Engineering, Dalian Polytechnic University, Dalian, ChinaSchool of Information Science and Engineering, Dalian Polytechnic University, Dalian, ChinaConventional paraphrase identification (PI) models based on deep learning usually focus on text representation and ignore the mining and matching of multi-granular interaction features. In addition, supervised learning relies on a large labeled data. However, labeled training set for PI is small in comparison with the high complexity of the task. To solve the problems, we propose a semi-supervised deep learning framework for PI. We use a neural encoder with word-by-word attention mechanism to reason equivalence or contradiction over pairs of words, phrases and sentences. We employ a two-stage training procedure. First, we use a language modeling objective to learn the initial parameters on the unlabeled corpora of more than one million pairs of sentences. This is followed by a supervised training, where we adapt these parameters to a specific classification task with labeled data. Experimental results on MRPC (Microsoft Research Paraphrase Corpus) and SICK (Sentences Involving Compositional Knowledge) datasets demonstrate the effectiveness of our approach. Compared with the previous neural network models, we achieve absolute improvements in accuracy of 7.6% and F1 of 5.4% on MRPC, Pearson's r of 4.5% and Spearman's ρ of 5.1% on SICK.https://ieeexplore.ieee.org/document/9050515/Natural language processingparaphrase identificationdeep learningrecurrent neural network (RNN)attention mechanismunsupervised pre-training |
| spellingShingle | Xu Li Fanxu Zeng Chunlong Yao A Semi-Supervised Paraphrase Identification Model Based on Multi-Granularity Interaction Reasoning IEEE Access Natural language processing paraphrase identification deep learning recurrent neural network (RNN) attention mechanism unsupervised pre-training |
| title | A Semi-Supervised Paraphrase Identification Model Based on Multi-Granularity Interaction Reasoning |
| title_full | A Semi-Supervised Paraphrase Identification Model Based on Multi-Granularity Interaction Reasoning |
| title_fullStr | A Semi-Supervised Paraphrase Identification Model Based on Multi-Granularity Interaction Reasoning |
| title_full_unstemmed | A Semi-Supervised Paraphrase Identification Model Based on Multi-Granularity Interaction Reasoning |
| title_short | A Semi-Supervised Paraphrase Identification Model Based on Multi-Granularity Interaction Reasoning |
| title_sort | semi supervised paraphrase identification model based on multi granularity interaction reasoning |
| topic | Natural language processing paraphrase identification deep learning recurrent neural network (RNN) attention mechanism unsupervised pre-training |
| url | https://ieeexplore.ieee.org/document/9050515/ |
| work_keys_str_mv | AT xuli asemisupervisedparaphraseidentificationmodelbasedonmultigranularityinteractionreasoning AT fanxuzeng asemisupervisedparaphraseidentificationmodelbasedonmultigranularityinteractionreasoning AT chunlongyao asemisupervisedparaphraseidentificationmodelbasedonmultigranularityinteractionreasoning AT xuli semisupervisedparaphraseidentificationmodelbasedonmultigranularityinteractionreasoning AT fanxuzeng semisupervisedparaphraseidentificationmodelbasedonmultigranularityinteractionreasoning AT chunlongyao semisupervisedparaphraseidentificationmodelbasedonmultigranularityinteractionreasoning |