Python Multithreading and Multiprocessing

Introduction

Step into a world where the industrial revolution has reshaped society, and the hustle and bustle of 19th-century Victorian-era life patterns the cobblestone streets. Here, amidst the foggy alleyways lined with the clatter of horse-drawn carriages and the distant hum of factory machinery, lives a character whose talent captivates the thronging crowds – a street performer known for his incredible ability to multitask.

Our performer, named Oliver, is renowned for juggling objects while balancing on a unicycle, playing the harmonica, and solving complex riddles simultaneously. To the amazement of his audience, Oliver's performances are seamless and efficient, a testament to his mastery of concurrent activities. This Lab aims to mirror Oliver’s multitasking prowess by diving into Python's multithreading and multiprocessing capabilities, ensuring that coders can manage multiple tasks simultaneously without a hitch.

Skills Graph

Understanding Threads

In this step, you will learn the basics of threading in Python. Threading allows you to run multiple operations concurrently, making it seem as if your program is doing more than one thing at once, just like Oliver's ability to juggle while cycling.

Open a file named simple_threads.py in the ~/project directory with the following content:

import threading
import time

def print_numbers():
    for i in range(1, 6):
        time.sleep(1)
        print(i)

## Create two threads
thread1 = threading.Thread(target=print_numbers)
thread2 = threading.Thread(target=print_numbers)

## Start both threads
thread1.start()
thread2.start()

## Wait for both threads to complete
thread1.join()
thread2.join()

print("Done with numbers!")

Run the above code in the terminal with:

python simple_threads.py

You should see numbers being printed interleaved from both threads, followed by "Done with numbers!":

1
1
2
2
3
3
4
4
5
5
Done with numbers!

Harnessing Multiprocessing

Now, let's use multiprocessing to speed up computations. Multiprocessing in Python allows the execution of multiple processes, which can run on different CPU cores, reducing the total time required for CPU-bound tasks.

Open a file named process_prime.py in the ~/project directory with the following content:

from multiprocessing import Process
import math

def is_prime(num):
    """
    Check if a number is prime.
    """
    if num <= 1:
        return False
    for i in range(2, int(math.sqrt(num)) + 1):
        if num % i == 0:
            return False
    return True

def compute_primes(start, end):
    """
    Compute prime numbers within a given range.
    """
    prime_numbers = [num for num in range(start, end) if is_prime(num)]
    print(f"Primes in range {start}-{end}: {prime_numbers}")

processes = []
## Creating two processes
for i in range(0, 20000, 10000):
    ## Create a new process targeting the compute_primes function and passing the range as arguments
    p = Process(target=compute_primes, args=(i, i+10000))
    processes.append(p)
    ## Start the process
    p.start()

## Wait for all processes to finish
for p in processes:
    p.join()

print("Done with prime computation!")

Execute the script in the terminal:

python process_prime.py

This will output prime numbers found in the specified ranges and display a completion message:

Primes in range 0-10000: [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97, 101, 103, 107, 109, 113, 127, 131, 137, 139, 149, 151, 157, 163, 167, 173, 179, 181, 191, 193, 197, 199, 211, 223, 227, 229, 233, 239, 241,
 ... ...
9851, 9857, 9859, 9871, 9883, 9887, 9901, 9907, 9923, 9929, 9931, 9941, 9949, 9967, 9973]
Primes in range 10000-20000: [10007, 10009, 10037, 10039, 10061, 10067, 10069, 10079, 10091, 10093, 10099, 10103, 10111, 10133, 10139, 10141, 10151, 10159, 10163, 10169, 10177, 10181, 10193,
 ... ...
19739, 19751, 19753, 19759, 19763, 19777, 19793, 19801, 19813, 19819, 19841, 19843, 19853, 19861, 19867, 19889, 19891, 19913, 19919, 19927, 19937, 19949, 19961, 19963, 19973, 19979, 19991, 19993, 19997]
Done with prime computation!

Summary

In this lab, we embarked on an engaging journey to unpack the nuances of threading and multiprocessing in Python, akin to the multi-layered performances of Victorian street artists. We started by juggling tasks with threads and then escalated to processing hefty operations across CPU cores. This exercise not only demonstrated the parallel capabilities of Python but also revealed how these techniques have real-world significance akin to the multifaceted displays of skill by our character, Oliver.

Your takeaway from this lab should include a clear understanding of Python threads for I/O-bound tasks, and processes for CPU-bound tasks, effectively elevating your coding performance and making your applications more efficient.