Uncertainty Quantification for MLP-Mixer Using Bayesian Deep Learning
Convolutional neural networks (CNNs) have become a popular choice for various image classification applications. However, the multi-layer perceptron mixer (MLP-Mixer) architecture has been proposed as a promising alternative, particularly for large datasets. Despite its advantages in handling large...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-04-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/7/4547 |
| _version_ | 1797608319487049728 |
|---|---|
| author | Abdullah A. Abdullah Masoud M. Hassan Yaseen T. Mustafa |
| author_facet | Abdullah A. Abdullah Masoud M. Hassan Yaseen T. Mustafa |
| author_sort | Abdullah A. Abdullah |
| collection | DOAJ |
| description | Convolutional neural networks (CNNs) have become a popular choice for various image classification applications. However, the multi-layer perceptron mixer (MLP-Mixer) architecture has been proposed as a promising alternative, particularly for large datasets. Despite its advantages in handling large datasets and models, MLP-Mixer models have limitations when dealing with small datasets. This study aimed to quantify and evaluate the uncertainty associated with MLP-Mixer models for small datasets using Bayesian deep learning (BDL) methods to quantify uncertainty and compare the results to existing CNN models. In particular, we examined the use of variational inference and Monte Carlo dropout methods. The results indicated that BDL can improve the performance of MLP-Mixer models by 9.2 to 17.4% in term of accuracy across different mixer models. On the other hand, the results suggest that CNN models tend to have limited improvement or even decreased performance in some cases when using BDL. These findings suggest that BDL is a promising approach to improve the performance of MLP-Mixer models, especially for small datasets. |
| first_indexed | 2024-03-11T05:41:49Z |
| format | Article |
| id | doaj.art-54397031a3e14e2a923c94fc58bd5eda |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-11T05:41:49Z |
| publishDate | 2023-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-54397031a3e14e2a923c94fc58bd5eda2023-11-17T16:21:53ZengMDPI AGApplied Sciences2076-34172023-04-01137454710.3390/app13074547Uncertainty Quantification for MLP-Mixer Using Bayesian Deep LearningAbdullah A. Abdullah0Masoud M. Hassan1Yaseen T. Mustafa2Computer Science Department, Faculty of Science, University of Zakho, Duhok 42002, Kurdistan Region, IraqComputer Science Department, Faculty of Science, University of Zakho, Duhok 42002, Kurdistan Region, IraqComputer Science Department, College of Science, Nawroz University, Duhok 42001, Kurdistan Region, IraqConvolutional neural networks (CNNs) have become a popular choice for various image classification applications. However, the multi-layer perceptron mixer (MLP-Mixer) architecture has been proposed as a promising alternative, particularly for large datasets. Despite its advantages in handling large datasets and models, MLP-Mixer models have limitations when dealing with small datasets. This study aimed to quantify and evaluate the uncertainty associated with MLP-Mixer models for small datasets using Bayesian deep learning (BDL) methods to quantify uncertainty and compare the results to existing CNN models. In particular, we examined the use of variational inference and Monte Carlo dropout methods. The results indicated that BDL can improve the performance of MLP-Mixer models by 9.2 to 17.4% in term of accuracy across different mixer models. On the other hand, the results suggest that CNN models tend to have limited improvement or even decreased performance in some cases when using BDL. These findings suggest that BDL is a promising approach to improve the performance of MLP-Mixer models, especially for small datasets.https://www.mdpi.com/2076-3417/13/7/4547uncertainty quantificationBayesian deep learningMLP-Mixervariational inference (VI)MC-dropout |
| spellingShingle | Abdullah A. Abdullah Masoud M. Hassan Yaseen T. Mustafa Uncertainty Quantification for MLP-Mixer Using Bayesian Deep Learning Applied Sciences uncertainty quantification Bayesian deep learning MLP-Mixer variational inference (VI) MC-dropout |
| title | Uncertainty Quantification for MLP-Mixer Using Bayesian Deep Learning |
| title_full | Uncertainty Quantification for MLP-Mixer Using Bayesian Deep Learning |
| title_fullStr | Uncertainty Quantification for MLP-Mixer Using Bayesian Deep Learning |
| title_full_unstemmed | Uncertainty Quantification for MLP-Mixer Using Bayesian Deep Learning |
| title_short | Uncertainty Quantification for MLP-Mixer Using Bayesian Deep Learning |
| title_sort | uncertainty quantification for mlp mixer using bayesian deep learning |
| topic | uncertainty quantification Bayesian deep learning MLP-Mixer variational inference (VI) MC-dropout |
| url | https://www.mdpi.com/2076-3417/13/7/4547 |
| work_keys_str_mv | AT abdullahaabdullah uncertaintyquantificationformlpmixerusingbayesiandeeplearning AT masoudmhassan uncertaintyquantificationformlpmixerusingbayesiandeeplearning AT yaseentmustafa uncertaintyquantificationformlpmixerusingbayesiandeeplearning |