Core Concept
Bayesian inference involves updating the probability estimate for a hypothesis as additional evidence is acquired. It's distinct from frequentist inference because it treats probability as a measure of belief rather than a frequency of occurrence.
The Theorem
At the heart of this approach lies Bayes' Theorem, a mathematical formula for determining conditional probability. It describes the probability of an event, based on prior knowledge of conditions that might be related to the event.
Where:
- P(A|B): The posterior probability of hypothesis A given data B.
- P(B|A): The likelihood of observing data B given hypothesis A is true.
- P(A): The prior probability of hypothesis A (before seeing data).
- P(B): The marginal likelihood (evidence) of the data B.
Python Implementation
Here is a basic implementation of a Bayesian update function using pure Python. In a real-world scenario (like building a spam filter), we would use NumPy for vectorized operations.
def bayesian_update(prior, likelihood):
"""
Updates beliefs based on new evidence.
"""
# Calculate the numerator (Joint Probability)
numerator = likelihood * prior
# For this binary example, let's assume P(B) is normalized
# In practice, P(B) is the sum of P(B|H)*P(H) for all H
posterior = numerator # Simplified
return posterior
# Initial Belief: 50% chance it rains
prior_belief = 0.5
# New Evidence: Clouds are dark (80% likely if it's going to rain)
new_evidence_likelihood = 0.8
posterior = bayesian_update(prior_belief, new_evidence_likelihood)
print(f"Updated Probability: {posterior:.2f}")Key Takeaways
The power of Bayesian inference allows us to inject domain knowledge (via the Prior) into our models. This is particularly useful in Data Science scenarios where data is scarce or expensive to collect.