Robust full Bayesian learning for radial basis networks.

We propose a hierarchical full Bayesian model for radial basis networks. This model treats the model dimension (number of neurons), model parameters, regularization parameters, and noise parameters as unknown random variables. We develop a reversible-jump Markov chain Monte Carlo (MCMC) method to pe...

Cijeli opis

Bibliografski detalji
Glavni autori: Andrieu, C, de Freitas, N, Doucet, A
Format: Journal article
Jezik:English
Izdano: 2001
_version_ 1826287761252941824
author Andrieu, C
de Freitas, N
Doucet, A
author_facet Andrieu, C
de Freitas, N
Doucet, A
author_sort Andrieu, C
collection OXFORD
description We propose a hierarchical full Bayesian model for radial basis networks. This model treats the model dimension (number of neurons), model parameters, regularization parameters, and noise parameters as unknown random variables. We develop a reversible-jump Markov chain Monte Carlo (MCMC) method to perform the Bayesian computation. We find that the results obtained using this method are not only better than the ones reported previously, but also appear to be robust with respect to the prior specification. In addition, we propose a novel and computationally efficient reversible-jump MCMC simulated annealing algorithm to optimize neural networks. This algorithm enables us to maximize the joint posterior distribution of the network parameters and the number of basis function. It performs a global search in the joint space of the parameters and number of parameters, thereby surmounting the problem of local minima to a large extent. We show that by calibrating the full hierarchical Bayesian prior, we can obtain the classical Akaike information criterion, Bayesian information criterion, and minimum description length model selection criteria within a penalized likelihood framework. Finally, we present a geometric convergence theorem for the algorithm with homogeneous transition kernel and a convergence theorem for the reversible-jump MCMC simulated annealing method.
first_indexed 2024-03-07T02:03:29Z
format Journal article
id oxford-uuid:9e2ed5f3-9898-449f-bc8b-2b51aa94d632
institution University of Oxford
language English
last_indexed 2024-03-07T02:03:29Z
publishDate 2001
record_format dspace
spelling oxford-uuid:9e2ed5f3-9898-449f-bc8b-2b51aa94d6322022-03-27T00:48:18ZRobust full Bayesian learning for radial basis networks.Journal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:9e2ed5f3-9898-449f-bc8b-2b51aa94d632EnglishSymplectic Elements at Oxford2001Andrieu, Cde Freitas, NDoucet, AWe propose a hierarchical full Bayesian model for radial basis networks. This model treats the model dimension (number of neurons), model parameters, regularization parameters, and noise parameters as unknown random variables. We develop a reversible-jump Markov chain Monte Carlo (MCMC) method to perform the Bayesian computation. We find that the results obtained using this method are not only better than the ones reported previously, but also appear to be robust with respect to the prior specification. In addition, we propose a novel and computationally efficient reversible-jump MCMC simulated annealing algorithm to optimize neural networks. This algorithm enables us to maximize the joint posterior distribution of the network parameters and the number of basis function. It performs a global search in the joint space of the parameters and number of parameters, thereby surmounting the problem of local minima to a large extent. We show that by calibrating the full hierarchical Bayesian prior, we can obtain the classical Akaike information criterion, Bayesian information criterion, and minimum description length model selection criteria within a penalized likelihood framework. Finally, we present a geometric convergence theorem for the algorithm with homogeneous transition kernel and a convergence theorem for the reversible-jump MCMC simulated annealing method.
spellingShingle Andrieu, C
de Freitas, N
Doucet, A
Robust full Bayesian learning for radial basis networks.
title Robust full Bayesian learning for radial basis networks.
title_full Robust full Bayesian learning for radial basis networks.
title_fullStr Robust full Bayesian learning for radial basis networks.
title_full_unstemmed Robust full Bayesian learning for radial basis networks.
title_short Robust full Bayesian learning for radial basis networks.
title_sort robust full bayesian learning for radial basis networks
work_keys_str_mv AT andrieuc robustfullbayesianlearningforradialbasisnetworks
AT defreitasn robustfullbayesianlearningforradialbasisnetworks
AT douceta robustfullbayesianlearningforradialbasisnetworks