Graph contrastive learning
Heterogeneous graph is a natural way to model complex relationships and interactions among entities in the real world, such as social networks or user--product relations. Learning good representations for heterogeneous graphs is a crucial step in deploying large-scale graph-based systems in an effic...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Final Year Project (FYP) |
| Language: | English |
| Published: |
Nanyang Technological University
2023
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/167024 |
| _version_ | 1826127744836042752 |
|---|---|
| author | Tran, Nguyen Manh Thien |
| author2 | Lihui Chen |
| author_facet | Lihui Chen Tran, Nguyen Manh Thien |
| author_sort | Tran, Nguyen Manh Thien |
| collection | NTU |
| description | Heterogeneous graph is a natural way to model complex relationships and interactions among entities in the real world, such as social networks or user--product relations. Learning good representations for heterogeneous graphs is a crucial step in deploying large-scale graph-based systems in an efficient and effective manner. Despite many great breakthroughs in self-supervised learning for Computer Vision and Natural Language Processing applications, similar efforts for graph data often pale in comparison, especially for heterogeneous graphs. Graph augmentation methods are limited and are weaker those for image data, limiting the potential of contrastive learning on graphs. Node dropping or edge perturbations are typically not suitable for heterogeneous graphs as they may result in invalid structures. Motivated by this, we propose to improve HeCo, the current state-of-the-art method for heterogeneous graph representation learning, with intra-view contrastive learning to obtain extra supervision signal. To maintain a graph's structural integrity, only Dropout is used to generate augmented views. To ensure the contrastive learning objective remain challenging, we further apply modified ArcFace loss to encourage more discriminative embeddings. We call our method HeCo-drop. HeCo-drop enhances HeCo consistently on various datasets, with up to +1% improvements in AUC scores. In addition, we analyse the key differences between graph and image/text data, thus outlining the challenges in adapting existing self-supervised methods to graphs. |
| first_indexed | 2024-10-01T07:13:53Z |
| format | Final Year Project (FYP) |
| id | ntu-10356/167024 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T07:13:53Z |
| publishDate | 2023 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1670242023-07-07T17:33:21Z Graph contrastive learning Tran, Nguyen Manh Thien Lihui Chen School of Electrical and Electronic Engineering ELHCHEN@ntu.edu.sg Engineering::Electrical and electronic engineering::Computer hardware, software and systems Heterogeneous graph is a natural way to model complex relationships and interactions among entities in the real world, such as social networks or user--product relations. Learning good representations for heterogeneous graphs is a crucial step in deploying large-scale graph-based systems in an efficient and effective manner. Despite many great breakthroughs in self-supervised learning for Computer Vision and Natural Language Processing applications, similar efforts for graph data often pale in comparison, especially for heterogeneous graphs. Graph augmentation methods are limited and are weaker those for image data, limiting the potential of contrastive learning on graphs. Node dropping or edge perturbations are typically not suitable for heterogeneous graphs as they may result in invalid structures. Motivated by this, we propose to improve HeCo, the current state-of-the-art method for heterogeneous graph representation learning, with intra-view contrastive learning to obtain extra supervision signal. To maintain a graph's structural integrity, only Dropout is used to generate augmented views. To ensure the contrastive learning objective remain challenging, we further apply modified ArcFace loss to encourage more discriminative embeddings. We call our method HeCo-drop. HeCo-drop enhances HeCo consistently on various datasets, with up to +1% improvements in AUC scores. In addition, we analyse the key differences between graph and image/text data, thus outlining the challenges in adapting existing self-supervised methods to graphs. Bachelor of Engineering (Electrical and Electronic Engineering) 2023-05-15T02:22:32Z 2023-05-15T02:22:32Z 2023 Final Year Project (FYP) Tran, N. M. T. (2023). Graph contrastive learning. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/167024 https://hdl.handle.net/10356/167024 en application/pdf Nanyang Technological University |
| spellingShingle | Engineering::Electrical and electronic engineering::Computer hardware, software and systems Tran, Nguyen Manh Thien Graph contrastive learning |
| title | Graph contrastive learning |
| title_full | Graph contrastive learning |
| title_fullStr | Graph contrastive learning |
| title_full_unstemmed | Graph contrastive learning |
| title_short | Graph contrastive learning |
| title_sort | graph contrastive learning |
| topic | Engineering::Electrical and electronic engineering::Computer hardware, software and systems |
| url | https://hdl.handle.net/10356/167024 |
| work_keys_str_mv | AT trannguyenmanhthien graphcontrastivelearning |