0.0.1 What is Bayesian Inference?
Bayesian inference is a method of statistical inference in which Bayes' Theorem is used to update the probability for a hypothesis as more evidence or information becomes available. Fundamentally, Bayesian inference uses prior knowledge, in the form of a prior distribution in order to estimate posterior probabilities.
0.0.2 How does it work?
The structure of Bayesian inference is as follows:
You begin with some initial information about a system, mathematically describes by the prior distribution. This is often some probability distribution for the parameter \(\theta\).
You then conduct an experiment and obtain further information in the form of the sampling distribution, with which we can update our prior distribution using Bayes' Theorem to form a posterior distribution.
Bayes’ Theorem is a fundamental result in Bayesian inference where is is used to express the posterior distribution in terms of the prior distribution and the sampling distribution \[p(\theta|y)=\frac{p(\theta,y)}{p(y)}=\frac{p(\theta)p(y|\theta)}{p(y)}\propto p(\theta)p(y|\theta).\]