Syntax Vector Learning Using Correspondence for Natural Language Understanding
Natural language understanding (NLU) is a core technique for implementing natural user interfaces. In this study, we propose a neural network architecture to learn syntax vector representation by employing the correspondence between texts and syntactic structures. For representing the syntactic stru...
| Κύριοι συγγραφείς: | , , , , |
|---|---|
| Μορφή: | Άρθρο |
| Γλώσσα: | English |
| Έκδοση: |
IEEE
2021-01-01
|
| Σειρά: | IEEE Access |
| Θέματα: | |
| Διαθέσιμο Online: | https://ieeexplore.ieee.org/document/9448015/ |
| _version_ | 1828735862698934272 |
|---|---|
| author | Hyein Seo Sangkeun Jung Taewook Hwang Hyunji Kim Yoon-Hyung Roh |
| author_facet | Hyein Seo Sangkeun Jung Taewook Hwang Hyunji Kim Yoon-Hyung Roh |
| author_sort | Hyein Seo |
| collection | DOAJ |
| description | Natural language understanding (NLU) is a core technique for implementing natural user interfaces. In this study, we propose a neural network architecture to learn syntax vector representation by employing the correspondence between texts and syntactic structures. For representing the syntactic structures of sentences, we used three methods: dependency trees, phrase structure trees, and part of speech tagging. A pretrained transformer is used to propose text-to-vector and syntax-to-vector projection approaches. The texts and syntactic structures are projected onto a common vector space, and the distances between the two vectors are minimized according to the correspondence property to learn the syntax representation. We conducted massive experiments to verify the effectiveness of the proposed methodology using Korean corpora, i.e., Weather, Navi, and Rest, and English corpora, i.e., the ATIS, SNIPS, Simulated Dialogue-Movie, Simulated Dialogue-Restaurant, and NLU-Evaluation datasets. Through the experiments, we concluded that our model is quite effective in capturing a syntactic representation and the learned syntax vector representations are useful. |
| first_indexed | 2024-04-12T23:14:51Z |
| format | Article |
| id | doaj.art-4f3cf30c20264bb9aa3b8e69cde2234a |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T23:14:51Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-4f3cf30c20264bb9aa3b8e69cde2234a2022-12-22T03:12:43ZengIEEEIEEE Access2169-35362021-01-019840678407810.1109/ACCESS.2021.30872719448015Syntax Vector Learning Using Correspondence for Natural Language UnderstandingHyein Seo0https://orcid.org/0000-0003-3107-0880Sangkeun Jung1https://orcid.org/0000-0003-3531-0618Taewook Hwang2https://orcid.org/0000-0003-2440-4707Hyunji Kim3https://orcid.org/0000-0002-1255-6212Yoon-Hyung Roh4Department of Computer Science and Engineering, Chungnam National University, Daejeon, South KoreaDepartment of Computer Science and Engineering, Chungnam National University, Daejeon, South KoreaDepartment of Computer Science and Engineering, Chungnam National University, Daejeon, South KoreaDepartment of Computer Science and Engineering, Chungnam National University, Daejeon, South KoreaElectronics and Telecommunications Research Institute, Daejeon, South KoreaNatural language understanding (NLU) is a core technique for implementing natural user interfaces. In this study, we propose a neural network architecture to learn syntax vector representation by employing the correspondence between texts and syntactic structures. For representing the syntactic structures of sentences, we used three methods: dependency trees, phrase structure trees, and part of speech tagging. A pretrained transformer is used to propose text-to-vector and syntax-to-vector projection approaches. The texts and syntactic structures are projected onto a common vector space, and the distances between the two vectors are minimized according to the correspondence property to learn the syntax representation. We conducted massive experiments to verify the effectiveness of the proposed methodology using Korean corpora, i.e., Weather, Navi, and Rest, and English corpora, i.e., the ATIS, SNIPS, Simulated Dialogue-Movie, Simulated Dialogue-Restaurant, and NLU-Evaluation datasets. Through the experiments, we concluded that our model is quite effective in capturing a syntactic representation and the learned syntax vector representations are useful.https://ieeexplore.ieee.org/document/9448015/Syntax vector learningsyntax similaritysyntax representationnatural language understandingtransformerdeep learning |
| spellingShingle | Hyein Seo Sangkeun Jung Taewook Hwang Hyunji Kim Yoon-Hyung Roh Syntax Vector Learning Using Correspondence for Natural Language Understanding IEEE Access Syntax vector learning syntax similarity syntax representation natural language understanding transformer deep learning |
| title | Syntax Vector Learning Using Correspondence for Natural Language Understanding |
| title_full | Syntax Vector Learning Using Correspondence for Natural Language Understanding |
| title_fullStr | Syntax Vector Learning Using Correspondence for Natural Language Understanding |
| title_full_unstemmed | Syntax Vector Learning Using Correspondence for Natural Language Understanding |
| title_short | Syntax Vector Learning Using Correspondence for Natural Language Understanding |
| title_sort | syntax vector learning using correspondence for natural language understanding |
| topic | Syntax vector learning syntax similarity syntax representation natural language understanding transformer deep learning |
| url | https://ieeexplore.ieee.org/document/9448015/ |
| work_keys_str_mv | AT hyeinseo syntaxvectorlearningusingcorrespondencefornaturallanguageunderstanding AT sangkeunjung syntaxvectorlearningusingcorrespondencefornaturallanguageunderstanding AT taewookhwang syntaxvectorlearningusingcorrespondencefornaturallanguageunderstanding AT hyunjikim syntaxvectorlearningusingcorrespondencefornaturallanguageunderstanding AT yoonhyungroh syntaxvectorlearningusingcorrespondencefornaturallanguageunderstanding |