Variational Deep Logic Network for Joint Inference of Entities and Relations
AbstractCurrently, deep learning models have been widely adopted and achieved promising results on various application domains. Despite their intriguing performance, most deep learning models function as black boxes, lacking explicit reasoning capabilities and explanations, which are...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
The MIT Press
2021-12-01
|
| Series: | Computational Linguistics |
| Online Access: | https://direct.mit.edu/coli/article/47/4/775/106773/Variational-Deep-Logic-Network-for-Joint-Inference |
| _version_ | 1811255387659173888 |
|---|---|
| author | Wenya Wang Sinno Jialin Pan |
| author_facet | Wenya Wang Sinno Jialin Pan |
| author_sort | Wenya Wang |
| collection | DOAJ |
| description |
AbstractCurrently, deep learning models have been widely adopted and achieved promising results on various application domains. Despite their intriguing performance, most deep learning models function as black boxes, lacking explicit reasoning capabilities and explanations, which are usually essential for complex problems. Take joint inference in information extraction as an example. This task requires the identification of multiple structured knowledge from texts, which is inter-correlated, including entities, events, and the relationships between them. Various deep neural networks have been proposed to jointly perform entity extraction and relation prediction, which only propagate information implicitly via representation learning. However, they fail to encode the intensive correlations between entity types and relations to enforce their coexistence. On the other hand, some approaches adopt rules to explicitly constrain certain relational facts, although the separation of rules with representation learning usually restrains the approaches with error propagation. Moreover, the predefined rules are inflexible and might result in negative effects when data is noisy. To address these limitations, we propose a variational deep logic network that incorporates both representation learning and relational reasoning via the variational EM algorithm. The model consists of a deep neural network to learn high-level features with implicit interactions via the self-attention mechanism and a relational logic network to explicitly exploit target interactions. These two components are trained interactively to bring the best of both worlds. We conduct extensive experiments ranging from fine-grained sentiment terms extraction, end-to-end relation prediction, to end-to-end event extraction to demonstrate the effectiveness of our proposed method. |
| first_indexed | 2024-04-12T17:23:07Z |
| format | Article |
| id | doaj.art-fca56ad859a74aa49f63d97797aae3d4 |
| institution | Directory Open Access Journal |
| issn | 0891-2017 1530-9312 |
| language | English |
| last_indexed | 2024-04-12T17:23:07Z |
| publishDate | 2021-12-01 |
| publisher | The MIT Press |
| record_format | Article |
| series | Computational Linguistics |
| spelling | doaj.art-fca56ad859a74aa49f63d97797aae3d42022-12-22T03:23:24ZengThe MIT PressComputational Linguistics0891-20171530-93122021-12-0147477581210.1162/coli_a_00415Variational Deep Logic Network for Joint Inference of Entities and RelationsWenya Wang0Sinno Jialin Pan1School of Computer Science and Engineering, Nanyang Technological University, Singapore. wangwy@ntu.edu.sgSchool of Computer Science and Engineering, Nanyang Technological University, Singapore. sinnopan@ntu.edu.sg AbstractCurrently, deep learning models have been widely adopted and achieved promising results on various application domains. Despite their intriguing performance, most deep learning models function as black boxes, lacking explicit reasoning capabilities and explanations, which are usually essential for complex problems. Take joint inference in information extraction as an example. This task requires the identification of multiple structured knowledge from texts, which is inter-correlated, including entities, events, and the relationships between them. Various deep neural networks have been proposed to jointly perform entity extraction and relation prediction, which only propagate information implicitly via representation learning. However, they fail to encode the intensive correlations between entity types and relations to enforce their coexistence. On the other hand, some approaches adopt rules to explicitly constrain certain relational facts, although the separation of rules with representation learning usually restrains the approaches with error propagation. Moreover, the predefined rules are inflexible and might result in negative effects when data is noisy. To address these limitations, we propose a variational deep logic network that incorporates both representation learning and relational reasoning via the variational EM algorithm. The model consists of a deep neural network to learn high-level features with implicit interactions via the self-attention mechanism and a relational logic network to explicitly exploit target interactions. These two components are trained interactively to bring the best of both worlds. We conduct extensive experiments ranging from fine-grained sentiment terms extraction, end-to-end relation prediction, to end-to-end event extraction to demonstrate the effectiveness of our proposed method.https://direct.mit.edu/coli/article/47/4/775/106773/Variational-Deep-Logic-Network-for-Joint-Inference |
| spellingShingle | Wenya Wang Sinno Jialin Pan Variational Deep Logic Network for Joint Inference of Entities and Relations Computational Linguistics |
| title | Variational Deep Logic Network for Joint Inference of Entities and Relations |
| title_full | Variational Deep Logic Network for Joint Inference of Entities and Relations |
| title_fullStr | Variational Deep Logic Network for Joint Inference of Entities and Relations |
| title_full_unstemmed | Variational Deep Logic Network for Joint Inference of Entities and Relations |
| title_short | Variational Deep Logic Network for Joint Inference of Entities and Relations |
| title_sort | variational deep logic network for joint inference of entities and relations |
| url | https://direct.mit.edu/coli/article/47/4/775/106773/Variational-Deep-Logic-Network-for-Joint-Inference |
| work_keys_str_mv | AT wenyawang variationaldeeplogicnetworkforjointinferenceofentitiesandrelations AT sinnojialinpan variationaldeeplogicnetworkforjointinferenceofentitiesandrelations |