Metropolis-Hastings Algorithm: An Introduction to MCMC Sampling

MCMC convergence diagnostics are essential tools for assessing whether Markov Chain Monte Carlo algorithms have converged to their stationary (target) distribution, enabling accurate Bayesian posterior estimation. Key diagnostic methods include: (1) Trace plots that visually examine whether multiple chains converge to the same distribution and exhibit adequate exploration; (2) Autocorrelation plots showing how quickly correlation between successive samples decays, with faster decay indicating more efficient sampling; (3) Effective sample size calculations estimating how many independent samples the correlated MCMC output is equivalent to; (4) The Gelman-Rubin diagnostic (R-hat) comparing within-chain and between-chain variability across multiple chains, with values near 1 indicating convergence; (5) The Heidelberger-Welch diagnostic performing sequential stationarity tests and half-width tests for mean estimation accuracy; (6) The Raftery-Lewis diagnostic determining required chain length to estimate quantiles with specified precision; and (7) The Geweke diagnostic comparing means of early and late chain portions to test for stationarity equality. These diagnostics help ensure that MCMC samples are sufficiently representative of the target posterior distribution for reliable statistical inference.

MCMC convergence diagnostics include: (1) visual inspection of trace plots for the log-likelihood, (2) checking effective sample sizes (ESS) - values below 1000 indicate insufficient mixing, (3) examining autocorrelation coefficients, and (4) comparing results from multiple chains. ESS measures the number of effectively independent samples; for 10,000 iterations, an ESS of 6,157 indicates strong autocorrelation requiring longer chains or thinning.

The Gelman-Rubin statistic (R̂) is a convergence diagnostic for MCMC chains. Run M chains from over-dispersed starting points, collect N samples from each, and compare within-chain variance to between-chain variance. Within-chain variance S_j² = (1/(N-1)) Σ_i (X_ji - X̄_j)² estimates variance within each chain. Between-chain variance B = (1/(M-1)) Σ_j (X̄_j - X̄)² estimates variance between chain means. The potential scale reduction factor R̂ = √(V̂_within / S_within²) ≥ 1 always, and R̂ → 1 as chains converge. For high-dimensional chains (e.g., 30,000 dimensions), convergence can be visualized as a heat map where blue cells indicate R̂ ≈ 1 (converged) and red cells indicate R̂ >> 1 (not converged).

MCMC convergence requires two diagnostics: trace plots and R-hat statistic. Trace plots visualize chain samples over iterations, showing if all chains explore the same parameter space region. Initial iterations (burn-in) are discarded as chains find the distribution. R-hat compares variance between chains to variance within chains; values near 1 indicate convergence. Common thresholds are 1.01, 1.05, or 1.1. False convergence occurs when chains appear converged but have only explored easy regions. Running chains longer reveals this pathology.

Validating Bayesian models requires checking both inference quality and model fit. Convergence diagnostics include trace plots showing 'fuzzy caterpillars' for good mixing, R-hat statistics comparing within-chain and between-chain variances (target ≈ 1), and energy plots examining transition energy coverage. Posterior predictive checking validates model fit by comparing observed data to draws from the posterior predictive distribution. Divergences indicate sampler failure, often resolved by increasing target acceptance probability or reparameterizing the model. These diagnostics ensure reliable inference before drawing conclusions.

Hamiltonian Monte Carlo (HMC) samples from target probability measures by enlarging them to phase space through the Boltzmann distribution e^(-H), where H combines potential energy U and kinetic energy from a Gaussian velocity. The leapfrog integrator approximates Hamiltonian dynamics with second-order accuracy while preserving volume (Jacobian determinant = 1), though it introduces energy errors. NUTS improves upon HMC by adding path length adaptivity, eliminating manual tuning requirements. The algorithm iteratively expands orbits forward/backward in time, doubling at each step, until a U-turn condition is met, making it particularly effective for high-dimensional problems where target measures concentrate in small regions.

NUTS (No-U-Turn Sampler) is an auto-tuning HMC variant that automatically determines optimal leapfrog step counts. The no-u-turn criterion prevents doubling back, eliminating manual tuning requirements. NUTS uses gradient information to simulate Hamiltonian dynamics, following posterior contours efficiently. The algorithm balances computational cost against exploration quality, taking as few steps as possible while adequately exploring the typical set. This represents state-of-the-art Bayesian inference, handling complex high-dimensional posteriors that Metropolis cannot efficiently explore.

Hamiltonian Monte Carlo (HMC) improves upon naive random walk samplers by exploiting gradient information to explore high-dimensional posteriors efficiently. HMC treats the posterior as a physical system where particles move according to Hamiltonian dynamics, following contours naturally without random walk inefficiencies. The algorithm uses leapfrog integration to discretely simulate these dynamics, with step size determining accuracy versus efficiency. The No-U-Turn Sampler (NUTS) automatically tunes step size and path length by detecting when trajectories would curve back upon themselves, eliminating manual tuning requirements. This approach addresses the curse of dimensionality by targeting the typical set where most probability mass resides, rather than getting stuck near modes with negligible volume.

The No-U-Turn Sampler (NUTS) is an adaptive extension of Hamiltonian Monte Carlo (HMC) that automatically determines optimal trajectory lengths by stopping the simulation when the trajectory begins to double back on itself, eliminating the need for manual tuning of HMC's step size and trajectory length parameters while retaining or improving sampling efficiency.

Hamiltonian Monte Carlo (HMC) and its adaptive variant NUTS (No-U-Turn Sampler) represent a major advancement in Bayesian inference by enabling efficient sampling from complex posterior distributions. These samplers use gradient-based dynamics to explore the parameter space more effectively than traditional Metropolis-Hastings methods. In PyMC3, after specifying a model with priors and likelihood functions, one instantiates an HMC/NUTS sampler and draws thousands of samples. The result is a posterior distribution over parameters, not point estimates, which naturally encodes uncertainty—narrow distributions indicate high confidence, while wide distributions reflect limited data or high uncertainty.

Probabilistic programming is a paradigm that combines structural knowledge of how the world works with probabilistic uncertainty to model human cognition, enabling flexible inference where we can define forward models of causal processes and then query them to answer questions about unobserved causes, such as inferring individual strengths from tug-of-war game outcomes.

Probabilistic programming treats models as computer programs that generate data rather than sets of equations. A universal probabilistic programming language can express any computable probability distribution. A universal inference engine performs automatic Bayesian inference over hidden variables and parameters given observed data. This is analogous to running computer programs backward to infer parameters—similar to how automatic differentiation computes derivatives through programs. Examples include expressing Bayesian hidden Markov models concisely in languages like Turing. This approach could revolutionize scientific modeling by combining with deep learning.

Probabilistic programming combines probabilistic modeling with programming languages to enable automated Bayesian inference, allowing researchers to encode complex probabilistic models and perform uncertainty-aware reasoning across diverse applications such as time series forecasting, experimental design, and scientific simulations like gravitational lensing analysis.

Probabilistic programming represents a paradigm shift toward making uncertainty and random variables first-class citizens in programming languages. The instructor describes three essential components: (1) random variables with probability distributions, (2) constraints relating variables, and (3) inference commands that compute posterior distributions. This approach enables direct programming with uncertainty, where code expresses complex probabilistic relationships concisely. The compiler automatically generates efficient inference code tailored to the specific model structure, dramatically reducing the effort required to implement sophisticated probabilistic models compared to manually deriving and coding message-passing algorithms.

Probabilistic programming is a technique that models real-world uncertainty by embedding randomness into AI models, allowing developers to represent and reason about probability distributions directly within their code; unlike traditional programming where the focus is on deterministic outcomes, probabilistic programming languages like Uber's Pyro (built on Python and PyTorch) treat probability distributions as fundamental program elements, enabling Bayesian inference where models can incorporate prior knowledge and update predictions based on observations.