Bayesian Inference

Bayesian inference is a method of statistical inference in which Bayes' theorem is used to update the probability for a hypothesis as more evidence or information becomes available.

The Core Equation

The fundamental equation involves the Posterior, Likelihood, Prior, and Marginal Likelihood. Unlike frequentist statistics, we treat parameters as random variables.

• •P(A): The Prior probability (what we believe before seeing data).
• •P(B|A): The Likelihood (how probable is the data given our hypothesis).
• •P(A|B): The Posterior (our updated belief).

Python Implementation

Let's simulate a simple coin flip scenario to update our belief about the fairness of a coin using a grid approximation method.

import numpy as np

# Define prior: Uniform distribution
prior = np.ones(100) / 100
p_grid = np.linspace(0, 1, 100)

def update(prior, heads, total):
    # Calculate likelihood
    likelihood = p_grid**heads * (1 - p_grid)**(total - heads)
    
    # Update beliefs
    posterior = likelihood * prior
    
    # Normalize posterior
    posterior /= posterior.sum() 
    return posterior