Unlock the Power of Multiprocessing and Matplotlib

Introduction

In this lab, you will learn how to use the multiprocessing library and Matplotlib to plot data generated from a separate process. We will create two classes - ProcessPlotter and NBPlot - to handle the plotting and data generation, respectively. The NBPlot class will generate random data and send it to the ProcessPlotter class through a pipe, which will then plot the data in real-time.

VM Tips

After the VM startup is done, click the top left corner to switch to the Notebook tab to access Jupyter Notebook for practice.

Sometimes, you may need to wait a few seconds for Jupyter Notebook to finish loading. The validation of operations cannot be automated because of limitations in Jupyter Notebook.

If you face issues during learning, feel free to ask Labby. Provide feedback after the session, and we will promptly resolve the problem for you.

Skills Graph

%%%%{init: {'theme':'neutral'}}%%%% flowchart RL matplotlib(("`Matplotlib`")) -.-> matplotlib/BasicConceptsGroup(["`Basic Concepts`"]) matplotlib(("`Matplotlib`")) -.-> matplotlib/PlottingDataGroup(["`Plotting Data`"]) matplotlib(("`Matplotlib`")) -.-> matplotlib/AdvancedTopicsGroup(["`Advanced Topics`"]) python(("`Python`")) -.-> python/BasicConceptsGroup(["`Basic Concepts`"]) python(("`Python`")) -.-> python/ControlFlowGroup(["`Control Flow`"]) python(("`Python`")) -.-> python/DataStructuresGroup(["`Data Structures`"]) python(("`Python`")) -.-> python/FunctionsGroup(["`Functions`"]) python(("`Python`")) -.-> python/ModulesandPackagesGroup(["`Modules and Packages`"]) python(("`Python`")) -.-> python/ObjectOrientedProgrammingGroup(["`Object-Oriented Programming`"]) python(("`Python`")) -.-> python/AdvancedTopicsGroup(["`Advanced Topics`"]) python(("`Python`")) -.-> python/PythonStandardLibraryGroup(["`Python Standard Library`"]) python(("`Python`")) -.-> python/DataScienceandMachineLearningGroup(["`Data Science and Machine Learning`"]) matplotlib/BasicConceptsGroup -.-> matplotlib/importing_matplotlib("`Importing Matplotlib`") matplotlib/BasicConceptsGroup -.-> matplotlib/figures_axes("`Understanding Figures and Axes`") matplotlib/PlottingDataGroup -.-> matplotlib/line_plots("`Line Plots`") matplotlib/AdvancedTopicsGroup -.-> matplotlib/custom_backends("`Custom Backends`") python/BasicConceptsGroup -.-> python/booleans("`Booleans`") python/ControlFlowGroup -.-> python/conditional_statements("`Conditional Statements`") python/ControlFlowGroup -.-> python/for_loops("`For Loops`") python/ControlFlowGroup -.-> python/while_loops("`While Loops`") python/DataStructuresGroup -.-> python/lists("`Lists`") python/DataStructuresGroup -.-> python/tuples("`Tuples`") python/FunctionsGroup -.-> python/function_definition("`Function Definition`") python/FunctionsGroup -.-> python/default_arguments("`Default Arguments`") python/ModulesandPackagesGroup -.-> python/importing_modules("`Importing Modules`") python/ModulesandPackagesGroup -.-> python/standard_libraries("`Common Standard Libraries`") python/ObjectOrientedProgrammingGroup -.-> python/classes_objects("`Classes and Objects`") python/ObjectOrientedProgrammingGroup -.-> python/constructor("`Constructor`") python/ObjectOrientedProgrammingGroup -.-> python/polymorphism("`Polymorphism`") python/ObjectOrientedProgrammingGroup -.-> python/encapsulation("`Encapsulation`") python/AdvancedTopicsGroup -.-> python/threading_multiprocessing("`Multithreading and Multiprocessing`") python/PythonStandardLibraryGroup -.-> python/math_random("`Math and Random`") python/DataScienceandMachineLearningGroup -.-> python/numerical_computing("`Numerical Computing`") python/DataScienceandMachineLearningGroup -.-> python/data_visualization("`Data Visualization`") python/FunctionsGroup -.-> python/build_in_functions("`Build-in Functions`") subgraph Lab Skills matplotlib/importing_matplotlib -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} matplotlib/figures_axes -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} matplotlib/line_plots -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} matplotlib/custom_backends -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/booleans -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/conditional_statements -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/for_loops -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/while_loops -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/lists -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/tuples -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/function_definition -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/default_arguments -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/importing_modules -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/standard_libraries -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/classes_objects -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/constructor -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/polymorphism -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/encapsulation -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/threading_multiprocessing -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/math_random -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/numerical_computing -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/data_visualization -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} python/build_in_functions -.-> lab-48844{{"`Multiprocessing with Matplotlib`"}} end

Import Libraries

We begin by importing the necessary libraries. We will use multiprocessing to handle the separate processes, time for the time delay, numpy to generate random data, and matplotlib for the plotting.

import multiprocessing as mp
import time
import numpy as np
import matplotlib.pyplot as plt

Define the `ProcessPlotter` Class

The ProcessPlotter class will handle the plotting of the data sent through the pipe. It will continuously check the pipe for new data and plot it in real-time.

class ProcessPlotter:
    def __init__(self):
        self.x = []
        self.y = []

    def terminate(self):
        plt.close('all')

    def call_back(self):
        while self.pipe.poll():
            command = self.pipe.recv()
            if command is None:
                self.terminate()
                return False
            else:
                self.x.append(command[0])
                self.y.append(command[1])
                self.ax.plot(self.x, self.y, 'ro')
        self.fig.canvas.draw()
        return True

    def __call__(self, pipe):
        print('starting plotter...')

        self.pipe = pipe
        self.fig, self.ax = plt.subplots()
        timer = self.fig.canvas.new_timer(interval=1000)
        timer.add_callback(self.call_back)
        timer.start()

        print('...done')
        plt.show()

Define the `NBPlot` Class

The NBPlot class will generate random data and send it to the ProcessPlotter class through a pipe.

class NBPlot:
    def __init__(self):
        self.plot_pipe, plotter_pipe = mp.Pipe()
        self.plotter = ProcessPlotter()
        self.plot_process = mp.Process(
            target=self.plotter, args=(plotter_pipe,), daemon=True)
        self.plot_process.start()

    def plot(self, finished=False):
        send = self.plot_pipe.send
        if finished:
            send(None)
        else:
            data = np.random.random(2)
            send(data)

Create an instance of `NBPlot` and send data to `ProcessPlotter`

Create an instance of the NBPlot class and send random data to the ProcessPlotter class. We will send 10 sets of data, with a 0.5 second delay between each set.

def main():
    pl = NBPlot()
    for _ in range(10):
        pl.plot()
        time.sleep(0.5)
    pl.plot(finished=True)

if __name__ == '__main__':
    if plt.get_backend() == "MacOSX":
        mp.set_start_method("forkserver")
    main()

Summary

In this lab, we learned how to use the multiprocessing library and Matplotlib to plot data generated from a separate process. We created two classes - ProcessPlotter and NBPlot - to handle the plotting and data generation, respectively. The NBPlot class generated random data and sent it to the ProcessPlotter class through a pipe, which then plotted the data in real-time.

Multiprocessing with Matplotlib

Introduction

VM Tips

Skills Graph

Import Libraries

Define the ProcessPlotter Class

Define the NBPlot Class

Create an instance of NBPlot and send data to ProcessPlotter

Summary

Other Python Tutorials you may like

Define the `ProcessPlotter` Class

Define the `NBPlot` Class

Create an instance of `NBPlot` and send data to `ProcessPlotter`