Graphs, Entities, and Step Mixture for Enriching Graph Representation
Graph neural networks have shown promising results on representing and analyzing diverse graph-structured data, such as social networks, traffic flow, drug discovery, and recommendation systems. Existing approaches for graph neural networks typically suffer from the oversmoothing issue that results...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9583282/ |
| _version_ | 1811209803351982080 |
|---|---|
| author | Kyuyong Shin Wonyoung Shin Jung-Woo Ha Sunyoung Kwon |
| author_facet | Kyuyong Shin Wonyoung Shin Jung-Woo Ha Sunyoung Kwon |
| author_sort | Kyuyong Shin |
| collection | DOAJ |
| description | Graph neural networks have shown promising results on representing and analyzing diverse graph-structured data, such as social networks, traffic flow, drug discovery, and recommendation systems. Existing approaches for graph neural networks typically suffer from the oversmoothing issue that results in indistinguishable node representation, as recursive and simultaneous neighborhood aggregation deepens. Also, most methods focus on transductive scenarios that are limited to fixed graphs, which do not generalize properly to unseen graphs. To address these issues, we propose a novel graph neural network that considers both edge-based neighborhood relationships and node-based entity features with multiple steps, i.e. <bold>G</bold>raph <bold>E</bold>ntities with <bold>S</bold>tep <bold>M</bold>ixture via <italic>random walk</italic> (GESM). GESM employs a mixture of various steps through random walk to alleviate the oversmoothing problem, an attention mechanism to dynamically reflect interrelations depending on node information, and structure-based regularization to enhance embedding representation. With intensive experiments, we show that our proposed GESM achieves state-of-the-art or comparable performances on eight benchmark graph datasets in both transductive and inductive learning tasks. Furthermore, we empirically demonstrate the superiority of our method on the oversmoothing issue with rich graph representations. Our source code is available at <uri>https://github.com/ShinKyuY/GESM</uri>. |
| first_indexed | 2024-04-12T04:45:05Z |
| format | Article |
| id | doaj.art-cce2ed80764148d7a7daa0e5f461e51e |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T04:45:05Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-cce2ed80764148d7a7daa0e5f461e51e2022-12-22T03:47:31ZengIEEEIEEE Access2169-35362021-01-01914402514403410.1109/ACCESS.2021.31217089583282Graphs, Entities, and Step Mixture for Enriching Graph RepresentationKyuyong Shin0https://orcid.org/0000-0002-4985-175XWonyoung Shin1https://orcid.org/0000-0002-9298-3280Jung-Woo Ha2https://orcid.org/0000-0002-7400-7681Sunyoung Kwon3https://orcid.org/0000-0003-3433-1409NAVER AI Laboratory, NAVER CLOVA, Seongnam, South KoreaNAVER Shopping, Naver Corporation, Seongnam, South KoreaNAVER AI Laboratory, NAVER CLOVA, Seongnam, South KoreaSchool of Biomedical Convergence Engineering, Pusan National University, Yangsan, South KoreaGraph neural networks have shown promising results on representing and analyzing diverse graph-structured data, such as social networks, traffic flow, drug discovery, and recommendation systems. Existing approaches for graph neural networks typically suffer from the oversmoothing issue that results in indistinguishable node representation, as recursive and simultaneous neighborhood aggregation deepens. Also, most methods focus on transductive scenarios that are limited to fixed graphs, which do not generalize properly to unseen graphs. To address these issues, we propose a novel graph neural network that considers both edge-based neighborhood relationships and node-based entity features with multiple steps, i.e. <bold>G</bold>raph <bold>E</bold>ntities with <bold>S</bold>tep <bold>M</bold>ixture via <italic>random walk</italic> (GESM). GESM employs a mixture of various steps through random walk to alleviate the oversmoothing problem, an attention mechanism to dynamically reflect interrelations depending on node information, and structure-based regularization to enhance embedding representation. With intensive experiments, we show that our proposed GESM achieves state-of-the-art or comparable performances on eight benchmark graph datasets in both transductive and inductive learning tasks. Furthermore, we empirically demonstrate the superiority of our method on the oversmoothing issue with rich graph representations. Our source code is available at <uri>https://github.com/ShinKyuY/GESM</uri>.https://ieeexplore.ieee.org/document/9583282/Graph neural networkrandom walkoversmoothingaggregationmixtureattention |
| spellingShingle | Kyuyong Shin Wonyoung Shin Jung-Woo Ha Sunyoung Kwon Graphs, Entities, and Step Mixture for Enriching Graph Representation IEEE Access Graph neural network random walk oversmoothing aggregation mixture attention |
| title | Graphs, Entities, and Step Mixture for Enriching Graph Representation |
| title_full | Graphs, Entities, and Step Mixture for Enriching Graph Representation |
| title_fullStr | Graphs, Entities, and Step Mixture for Enriching Graph Representation |
| title_full_unstemmed | Graphs, Entities, and Step Mixture for Enriching Graph Representation |
| title_short | Graphs, Entities, and Step Mixture for Enriching Graph Representation |
| title_sort | graphs entities and step mixture for enriching graph representation |
| topic | Graph neural network random walk oversmoothing aggregation mixture attention |
| url | https://ieeexplore.ieee.org/document/9583282/ |
| work_keys_str_mv | AT kyuyongshin graphsentitiesandstepmixtureforenrichinggraphrepresentation AT wonyoungshin graphsentitiesandstepmixtureforenrichinggraphrepresentation AT jungwooha graphsentitiesandstepmixtureforenrichinggraphrepresentation AT sunyoungkwon graphsentitiesandstepmixtureforenrichinggraphrepresentation |