Theoretical and methodological developments for Markov chain Monte Carlo algorithms for Bayesian regression

by Roy, Vivekananda, Ph.D., UNIVERSITY OF FLORIDA, 2008, 94 pages; 3425539


I develop theoretical and methodological results for Markov chain Monte Carlo (MCMC) algorithms for two different Bayesian regression models. First, I consider a probit regression problem in which Y1,…, Yn are independent Bernoulli random variables such that Pr(Yi = 1) = Φ([special characters omitted]β) where xi is a p-dimensional vector of known covariates associated with Yi, β is a p-dimensional vector of unknown regression coefficients and Φ(·) denotes the standard normal distribution function. I study two frequently used MCMC algorithms for exploring the intractable posterior density that results when the probit regression likelihood is combined with a flat prior on β. These algorithms are Albert and Chib’s data augmentation algorithm and Liu and Wu’s PX-DA algorithm. I prove that both of these algorithms converge at a geometric rate, which ensures the existence of central limit theorems (CLTs) for ergodic averages under a second moment condition. While these two algorithms are essentially equivalent in terms of computational complexity, I show that the PX-DA algorithm is theoretically more efficient in the sense that the asymptotic variance in the CLT under the PX-DA algorithm is no larger than that under Albert and Chib’s algorithm. A simple, consistent estimator of the asymptotic variance in the CLT is constructed using regeneration. As an illustration, I apply my results to van Dyk and Meng’s lupus data. In this particular example, the estimated asymptotic relative efficiency of the PX-DA algorithm with respect to Albert and Chib’s algorithm is about 65, which demonstrates that huge gains in efficiency are possible by using PX-DA.

Second, I consider multivariate regression models where the distribution of the errors is a scale mixture of normals. Let π denote the posterior density that results when the likelihood of n observations from the corresponding regression model is combined with the standard non-informative prior. I provide necessary and sufficient condition for the propriety of the posterior distribution, π. I develop two MCMC algorithms that can be used to explore the intractable density π. These algorithms are the data augmentation algorithm and the Haar PX-DA algorithm. I compare the two algorithms in terms of efficiency ordering. I establish drift and minorization conditions to study the convergence rates of these algorithms.

AdviserJames P. Hobert
Source TypeDissertation
Publication Number3425539

About ProQuest Dissertations & Theses
With nearly 4 million records, the ProQuest Dissertations & Theses (PQDT) Global database is the most comprehensive collection of dissertations and theses in the world. It is the database of record for graduate research.

PQDT Global combines content from a range of the world's premier universities - from the Ivy League to the Russell Group. Of the nearly 4 million graduate works included in the database, ProQuest offers more than 2.5 million in full text formats. Of those, over 1.7 million are available in PDF format. More than 90,000 dissertations and theses are added to the database each year.

If you have questions, please feel free to visit the ProQuest Web site - - or contact ProQuest Support.