Variational Bayesian Inference in High-Dimensional Linear Mixed Models
In high-dimensional regression models, the Bayesian lasso with the Gaussian spike and slab priors is widely adopted to select variables and estimate unknown parameters. However, it involves large matrix computations in a standard Gibbs sampler. To solve this issue, the Skinny Gibbs sampler is employ...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-01-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/10/3/463 |
| _version_ | 1827659969220050944 |
|---|---|
| author | Jieyi Yi Niansheng Tang |
| author_facet | Jieyi Yi Niansheng Tang |
| author_sort | Jieyi Yi |
| collection | DOAJ |
| description | In high-dimensional regression models, the Bayesian lasso with the Gaussian spike and slab priors is widely adopted to select variables and estimate unknown parameters. However, it involves large matrix computations in a standard Gibbs sampler. To solve this issue, the Skinny Gibbs sampler is employed to draw observations required for Bayesian variable selection. However, when the sample size is much smaller than the number of variables, the computation is rather time-consuming. As an alternative to the Skinny Gibbs sampler, we develop a variational Bayesian approach to simultaneously select variables and estimate parameters in high-dimensional linear mixed models under the Gaussian spike and slab priors of population-specific fixed-effects regression coefficients, which are reformulated as a mixture of a normal distribution and an exponential distribution. The coordinate ascent algorithm, which can be implemented efficiently, is proposed to optimize the evidence lower bound. The Bayes factor, which can be computed with the path sampling technique, is presented to compare two competing models in the variational Bayesian framework. Simulation studies are conducted to assess the performance of the proposed variational Bayesian method. An empirical example is analyzed by the proposed methodologies. |
| first_indexed | 2024-03-09T23:32:39Z |
| format | Article |
| id | doaj.art-e7e8968958e6482e87ebfbcb09e00fb1 |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-09T23:32:39Z |
| publishDate | 2022-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-e7e8968958e6482e87ebfbcb09e00fb12023-11-23T17:07:55ZengMDPI AGMathematics2227-73902022-01-0110346310.3390/math10030463Variational Bayesian Inference in High-Dimensional Linear Mixed ModelsJieyi Yi0Niansheng Tang1Yunnan Key Laboratory of Statistical Modeling and Data Analysis, Yunnan University, Kunming 650091, ChinaYunnan Key Laboratory of Statistical Modeling and Data Analysis, Yunnan University, Kunming 650091, ChinaIn high-dimensional regression models, the Bayesian lasso with the Gaussian spike and slab priors is widely adopted to select variables and estimate unknown parameters. However, it involves large matrix computations in a standard Gibbs sampler. To solve this issue, the Skinny Gibbs sampler is employed to draw observations required for Bayesian variable selection. However, when the sample size is much smaller than the number of variables, the computation is rather time-consuming. As an alternative to the Skinny Gibbs sampler, we develop a variational Bayesian approach to simultaneously select variables and estimate parameters in high-dimensional linear mixed models under the Gaussian spike and slab priors of population-specific fixed-effects regression coefficients, which are reformulated as a mixture of a normal distribution and an exponential distribution. The coordinate ascent algorithm, which can be implemented efficiently, is proposed to optimize the evidence lower bound. The Bayes factor, which can be computed with the path sampling technique, is presented to compare two competing models in the variational Bayesian framework. Simulation studies are conducted to assess the performance of the proposed variational Bayesian method. An empirical example is analyzed by the proposed methodologies.https://www.mdpi.com/2227-7390/10/3/463Bayesian lassoevidence lower boundhigh-dimensional linear mixed modelspike and slab priorsvariational Bayesian inference |
| spellingShingle | Jieyi Yi Niansheng Tang Variational Bayesian Inference in High-Dimensional Linear Mixed Models Mathematics Bayesian lasso evidence lower bound high-dimensional linear mixed model spike and slab priors variational Bayesian inference |
| title | Variational Bayesian Inference in High-Dimensional Linear Mixed Models |
| title_full | Variational Bayesian Inference in High-Dimensional Linear Mixed Models |
| title_fullStr | Variational Bayesian Inference in High-Dimensional Linear Mixed Models |
| title_full_unstemmed | Variational Bayesian Inference in High-Dimensional Linear Mixed Models |
| title_short | Variational Bayesian Inference in High-Dimensional Linear Mixed Models |
| title_sort | variational bayesian inference in high dimensional linear mixed models |
| topic | Bayesian lasso evidence lower bound high-dimensional linear mixed model spike and slab priors variational Bayesian inference |
| url | https://www.mdpi.com/2227-7390/10/3/463 |
| work_keys_str_mv | AT jieyiyi variationalbayesianinferenceinhighdimensionallinearmixedmodels AT nianshengtang variationalbayesianinferenceinhighdimensionallinearmixedmodels |