Normal Prior Normal Likelihood Posterior Derivation

Added:

Robust Bayesian Inference and Non-Conjugate Priors

While the Normal-Normal conjugate model is mathematically convenient because the posterior mean is a simple weighted average of the prior and sample means, it suffers from a lack of robustness to prior-data conflict. If the prior belief is far from the observed data (due to an outlier or an incorrect assumption), the Normal posterior compromises between them, resulting in an estimate that represents neither well. An alternative perspective is Robust Bayesian Analysis, which advocates for heavy-tailed priors, such as the Student-t or Cauchy distribution. Under these non-conjugate models, if a severe conflict arises between the prior and the data, the influence of the prior is automatically discounted, allowing the posterior to align closely with the data. This approach sacrifices the analytical simplicity of the Normal-Normal derivation—requiring numerical methods like Markov Chain Monte Carlo (MCMC)—in exchange for model robustness and resistance to outliers.

Understanding of Bayes' Theorem, specifically the relationship where the Posterior is proportional to the Likelihood multiplied by the Prior.
Familiarity with the Probability Density Function (PDF) of the Normal (Gaussian) distribution and its parameters (mean and variance).
Algebraic proficiency in 'completing the square' within the exponent of exponential functions.
The concept of 'precision' in statistics, defined as the reciprocal of the variance.
Exploring the Multivariate Normal conjugate update, extending this univariate derivation to multi-dimensional data using matrix algebra.
Applying the Normal-Normal model to Bayesian Linear Regression, where priors are placed on the regression coefficients.
Studying Conjugate Priors and the Exponential Family to understand which other distribution pairs share this convenient property.
Implementing Sequential Bayesian Updating, demonstrating how a posterior distribution becomes the prior when new data is observed.
57.2K views397likes11:04@deetoherOriginal Release: 2013-03-08

When a normal distribution serves as the prior for a normal likelihood function, the resulting posterior distribution is also normal, with its precision equal to the sum of the prior precision and the likelihood precision, and its mean being a weighted average of the prior mean and the sample mean, where weights are proportional to their respective precisions.