Generative Transformer with Knowledge-Guided Decoding for Academic Knowledge Graph Completion
Academic knowledge graphs are essential resources and can be beneficial in widespread real-world applications. Most of the existing academic knowledge graphs are far from completion; thus, knowledge graph completion—the task of extending a knowledge graph with missing entities and relations—attracts...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-02-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/11/5/1073 |
| _version_ | 1827752570547863552 |
|---|---|
| author | Xiangwen Liu Shengyu Mao Xiaohan Wang Jiajun Bu |
| author_facet | Xiangwen Liu Shengyu Mao Xiaohan Wang Jiajun Bu |
| author_sort | Xiangwen Liu |
| collection | DOAJ |
| description | Academic knowledge graphs are essential resources and can be beneficial in widespread real-world applications. Most of the existing academic knowledge graphs are far from completion; thus, knowledge graph completion—the task of extending a knowledge graph with missing entities and relations—attracts many researchers. Most existing methods utilize low-dimensional embeddings to represent entities and relations and follow the discrimination paradigm for link prediction. However, discrimination approaches may suffer from the scaling issue during inference with large-scale academic knowledge graphs. In this paper, we propose a novel approach of a generative transformer with knowledge-guided decoding for academic knowledge graph completion. Specifically, we introduce generative academic knowledge graph pre-training with a transformer. Then, we propose knowledge-guided decoding, which leverages relevant knowledge in the training corpus as guidance for help. We conducted experiments on benchmark datasets for knowledge graph completion. The experimental results show that the proposed approach can achieve performance gains of 30 units of the MRR score over the baselines on the academic knowledge graph AIDA. |
| first_indexed | 2024-03-11T07:18:16Z |
| format | Article |
| id | doaj.art-1e5b7be76f2142df9710a8cdf7a1319e |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-11T07:18:16Z |
| publishDate | 2023-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-1e5b7be76f2142df9710a8cdf7a1319e2023-11-17T08:07:50ZengMDPI AGMathematics2227-73902023-02-01115107310.3390/math11051073Generative Transformer with Knowledge-Guided Decoding for Academic Knowledge Graph CompletionXiangwen Liu0Shengyu Mao1Xiaohan Wang2Jiajun Bu3College of Computer Science and Technology, Zhejiang University, Hangzhou 310007, ChinaCollege of Computer Science and Technology, Zhejiang University, Hangzhou 310007, ChinaCollege of Computer Science and Technology, Zhejiang University, Hangzhou 310007, ChinaCollege of Computer Science and Technology, Zhejiang University, Hangzhou 310007, ChinaAcademic knowledge graphs are essential resources and can be beneficial in widespread real-world applications. Most of the existing academic knowledge graphs are far from completion; thus, knowledge graph completion—the task of extending a knowledge graph with missing entities and relations—attracts many researchers. Most existing methods utilize low-dimensional embeddings to represent entities and relations and follow the discrimination paradigm for link prediction. However, discrimination approaches may suffer from the scaling issue during inference with large-scale academic knowledge graphs. In this paper, we propose a novel approach of a generative transformer with knowledge-guided decoding for academic knowledge graph completion. Specifically, we introduce generative academic knowledge graph pre-training with a transformer. Then, we propose knowledge-guided decoding, which leverages relevant knowledge in the training corpus as guidance for help. We conducted experiments on benchmark datasets for knowledge graph completion. The experimental results show that the proposed approach can achieve performance gains of 30 units of the MRR score over the baselines on the academic knowledge graph AIDA.https://www.mdpi.com/2227-7390/11/5/1073knowledge graphtransformergeneration |
| spellingShingle | Xiangwen Liu Shengyu Mao Xiaohan Wang Jiajun Bu Generative Transformer with Knowledge-Guided Decoding for Academic Knowledge Graph Completion Mathematics knowledge graph transformer generation |
| title | Generative Transformer with Knowledge-Guided Decoding for Academic Knowledge Graph Completion |
| title_full | Generative Transformer with Knowledge-Guided Decoding for Academic Knowledge Graph Completion |
| title_fullStr | Generative Transformer with Knowledge-Guided Decoding for Academic Knowledge Graph Completion |
| title_full_unstemmed | Generative Transformer with Knowledge-Guided Decoding for Academic Knowledge Graph Completion |
| title_short | Generative Transformer with Knowledge-Guided Decoding for Academic Knowledge Graph Completion |
| title_sort | generative transformer with knowledge guided decoding for academic knowledge graph completion |
| topic | knowledge graph transformer generation |
| url | https://www.mdpi.com/2227-7390/11/5/1073 |
| work_keys_str_mv | AT xiangwenliu generativetransformerwithknowledgeguideddecodingforacademicknowledgegraphcompletion AT shengyumao generativetransformerwithknowledgeguideddecodingforacademicknowledgegraphcompletion AT xiaohanwang generativetransformerwithknowledgeguideddecodingforacademicknowledgegraphcompletion AT jiajunbu generativetransformerwithknowledgeguideddecodingforacademicknowledgegraphcompletion |