Bayesian Graph Convolutional Neural Networks via Tempered MCMC
Deep learning models, such as convolutional neural networks, have long been applied to image and multi-media tasks, particularly those with structured data. More recently, there has been more attention to unstructured data that can be represented via graphs. These types of data are often found in he...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9535500/ |
| _version_ | 1818722452500381696 |
|---|---|
| author | Rohitash Chandra Ayush Bhagat Manavendra Maharana Pavel N. Krivitsky |
| author_facet | Rohitash Chandra Ayush Bhagat Manavendra Maharana Pavel N. Krivitsky |
| author_sort | Rohitash Chandra |
| collection | DOAJ |
| description | Deep learning models, such as convolutional neural networks, have long been applied to image and multi-media tasks, particularly those with structured data. More recently, there has been more attention to unstructured data that can be represented via graphs. These types of data are often found in health and medicine, social networks, and research data repositories. Graph convolutional neural networks have recently gained attention in the field of deep learning that takes advantage of graph-based data representation with automatic feature extraction via convolutions. Given the popularity of these methods in a wide range of applications, robust uncertainty quantification is vital. This remains a challenge for large models and unstructured datasets. Bayesian inference provides a principled approach to uncertainty quantification of model parameters for deep learning models. Although Bayesian inference has been used extensively elsewhere, its application to deep learning remains limited due to the computational requirements of the Markov Chain Monte Carlo (MCMC) methods. Recent advances in parallel computing and advanced proposal schemes in MCMC sampling methods has opened the path for Bayesian deep learning. In this paper, we present Bayesian graph convolutional neural networks that employ tempered MCMC sampling with Langevin-gradient proposal distribution implemented via parallel computing. Our results show that the proposed method can provide accuracy similar to advanced optimisers while providing uncertainty quantification for key benchmark problems. |
| first_indexed | 2024-12-17T20:54:51Z |
| format | Article |
| id | doaj.art-5a3e1b8cd08e435db246890db55e6705 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-17T20:54:51Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-5a3e1b8cd08e435db246890db55e67052022-12-21T21:32:54ZengIEEEIEEE Access2169-35362021-01-01913035313036510.1109/ACCESS.2021.31118989535500Bayesian Graph Convolutional Neural Networks via Tempered MCMCRohitash Chandra0https://orcid.org/0000-0001-6353-1464Ayush Bhagat1https://orcid.org/0000-0003-4299-7788Manavendra Maharana2https://orcid.org/0000-0001-5153-5214Pavel N. Krivitsky3School of Mathematics and Statistics, UNSW Sydney, Kensington, NSW, AustraliaDepartment of Computer Science Engineering, Manipal Institute of Technology, Manipal, Karnataka, IndiaDepartment of Computer Science Engineering, Manipal Institute of Technology, Manipal, Karnataka, IndiaSchool of Mathematics and Statistics, UNSW Sydney, Kensington, NSW, AustraliaDeep learning models, such as convolutional neural networks, have long been applied to image and multi-media tasks, particularly those with structured data. More recently, there has been more attention to unstructured data that can be represented via graphs. These types of data are often found in health and medicine, social networks, and research data repositories. Graph convolutional neural networks have recently gained attention in the field of deep learning that takes advantage of graph-based data representation with automatic feature extraction via convolutions. Given the popularity of these methods in a wide range of applications, robust uncertainty quantification is vital. This remains a challenge for large models and unstructured datasets. Bayesian inference provides a principled approach to uncertainty quantification of model parameters for deep learning models. Although Bayesian inference has been used extensively elsewhere, its application to deep learning remains limited due to the computational requirements of the Markov Chain Monte Carlo (MCMC) methods. Recent advances in parallel computing and advanced proposal schemes in MCMC sampling methods has opened the path for Bayesian deep learning. In this paper, we present Bayesian graph convolutional neural networks that employ tempered MCMC sampling with Langevin-gradient proposal distribution implemented via parallel computing. Our results show that the proposed method can provide accuracy similar to advanced optimisers while providing uncertainty quantification for key benchmark problems.https://ieeexplore.ieee.org/document/9535500/Bayesian neural networksMCMCLangevin dynamicsBayesian deep learninggraph neural networks |
| spellingShingle | Rohitash Chandra Ayush Bhagat Manavendra Maharana Pavel N. Krivitsky Bayesian Graph Convolutional Neural Networks via Tempered MCMC IEEE Access Bayesian neural networks MCMC Langevin dynamics Bayesian deep learning graph neural networks |
| title | Bayesian Graph Convolutional Neural Networks via Tempered MCMC |
| title_full | Bayesian Graph Convolutional Neural Networks via Tempered MCMC |
| title_fullStr | Bayesian Graph Convolutional Neural Networks via Tempered MCMC |
| title_full_unstemmed | Bayesian Graph Convolutional Neural Networks via Tempered MCMC |
| title_short | Bayesian Graph Convolutional Neural Networks via Tempered MCMC |
| title_sort | bayesian graph convolutional neural networks via tempered mcmc |
| topic | Bayesian neural networks MCMC Langevin dynamics Bayesian deep learning graph neural networks |
| url | https://ieeexplore.ieee.org/document/9535500/ |
| work_keys_str_mv | AT rohitashchandra bayesiangraphconvolutionalneuralnetworksviatemperedmcmc AT ayushbhagat bayesiangraphconvolutionalneuralnetworksviatemperedmcmc AT manavendramaharana bayesiangraphconvolutionalneuralnetworksviatemperedmcmc AT pavelnkrivitsky bayesiangraphconvolutionalneuralnetworksviatemperedmcmc |