Bayesian Updating with Matplotlib: Visualize Probability Updates

Introduction

Bayesian updating is a statistical approach that allows us to update the probability of a hypothesis as new data becomes available. In this lab, we will use Matplotlib to create an animation that shows how Bayesian updating works. Specifically, we will simulate a coin toss experiment and use Bayesian updating to estimate the probability of the coin landing heads up.

VM Tips

After the VM startup is done, click the top left corner to switch to the Notebook tab to access Jupyter Notebook for practice.

Sometimes, you may need to wait a few seconds for Jupyter Notebook to finish loading. The validation of operations cannot be automated because of limitations in Jupyter Notebook.

If you face issues during learning, feel free to ask Labby. Provide feedback after the session, and we will promptly resolve the problem for you.

Import Required Libraries

We begin by importing the libraries we will use in the lab. Specifically, we will use matplotlib.pyplot for visualization, numpy for numerical computing, and math for mathematical functions.

import math

import matplotlib.pyplot as plt
import numpy as np

Define Beta Distribution PDF

The beta distribution is a continuous probability distribution that is often used to represent the distribution of probabilities. In Bayesian updating, we use the beta distribution as a prior distribution to represent our beliefs about the probability of a hypothesis before observing any data. We then update the beta distribution as we observe new data.

To simulate Bayesian updating, we need to define a function that calculates the probability density function (PDF) of the beta distribution. We can use the math.gamma function to compute the gamma function, which is used in the beta distribution PDF.

def beta_pdf(x, a, b):
    return (x**(a-1) * (1-x)**(b-1) * math.gamma(a + b)
            / (math.gamma(a) * math.gamma(b)))

Define UpdateDist Class

Next, we define a class called UpdateDist that will be used to update the beta distribution as new data is observed. The UpdateDist class takes two arguments: the Matplotlib axis object and the initial probability of success.

class UpdateDist:
    def __init__(self, ax, prob=0.5):
        self.success = 0
        self.prob = prob
        self.line, = ax.plot([], [], 'k-')
        self.x = np.linspace(0, 1, 200)
        self.ax = ax

        ## Set up plot parameters
        self.ax.set_xlim(0, 1)
        self.ax.set_ylim(0, 10)
        self.ax.grid(True)

        ## This vertical line represents the theoretical value, to
        ## which the plotted distribution should converge.
        self.ax.axvline(prob, linestyle='--', color='black')

The __init__ method initializes the class instance by setting the initial number of successes to zero (self.success = 0) and the initial probability of success to the value passed as an argument (self.prob = prob). We also create a line object to represent the beta distribution and set up the plot parameters.

The __call__ method is called every time the animation updates. It simulates a coin toss experiment and updates the beta distribution accordingly.

def __call__(self, i):
        ## This way the plot can continuously run and we just keep
        ## watching new realizations of the process
        if i == 0:
            self.success = 0
            self.line.set_data([], [])
            return self.line,

        ## Choose success based on exceed a threshold with a uniform pick
        if np.random.rand() < self.prob:
            self.success += 1
        y = beta_pdf(self.x, self.success + 1, (i - self.success) + 1)
        self.line.set_data(self.x, y)
        return self.line,

If this is the first frame of the animation (if i == 0), we reset the number of successes to zero and clear the line object. Otherwise, we simulate a coin toss experiment by generating a random number between 0 and 1 (np.random.rand()) and comparing it to the probability of success (self.prob). If the random number is less than the probability of success, we count it as a success and update the beta distribution using the beta_pdf function. Finally, we update the line object with the new data and return it.

Create Animation

Now that we have defined the UpdateDist class, we can create the animation using Matplotlib's FuncAnimation class. We create a figure object and an axis object and pass the axis object to the UpdateDist class to create a new instance of the class.

fig, ax = plt.subplots()
ud = UpdateDist(ax, prob=0.7)
anim = FuncAnimation(fig, ud, frames=100, interval=100, blit=True)
plt.show()

The FuncAnimation class takes several arguments:

fig: the figure object
ud: the UpdateDist instance
frames: the number of frames to animate
interval: the time between frames in milliseconds
blit: whether to update only the parts of the plot that have changed

Interpret Results

The animation shows how the beta distribution is updated as new data is observed. The black dashed line represents the true probability of success (i.e., the probability of the coin landing heads up). As the animation progresses, we see that the beta distribution starts out with a peak at the prior probability of success (0.7) and gradually shifts towards the true probability of success as more data is observed.

Summary

In this lab, we used Matplotlib to create an animation that demonstrates Bayesian updating. We defined a function to calculate the beta distribution PDF and a class to update the beta distribution as new data is observed. We then used Matplotlib's FuncAnimation class to create the animation and interpret the results.

Bayesian Updating with Matplotlib