3.1. Using the multiprocessing Module

When performance is critical and tasks are CPU-bound, multiprocessing is the way to go. Each process runs in its own Python interpreter and has its own memory space.

Multiprocessing in Python is utilized when you need to handle tasks that demand high computational power, specifically when these tasks are CPU-bound. That means they require a lot of processing power rather than waiting for input or output operations (I/O-bound tasks). Unlike threads, which share memory space, each process created by the multiprocessing module has its own separate memory area. This separation allows each process to run without being affected by the Global Interpreter Lock (GIL), effectively bypassing this limitation.

Think of a restaurant kitchen where multiple chefs are working on individual meals. Each chef (process) works independently at their own station, using their own set of ingredients (memory). Even if they are preparing similar dishes (tasks), they don’t interfere with each other’s work. This allows for faster service, as each chef can focus on their dish without waiting for the others.

from multiprocessing import Process
import os
def compute():
    print(f"Running on process ID: {os.getpid()}")
p1 = Process(target=compute)
p2 = Process(target=compute)
p1.start()
p2.start()
p1.join()
p2.join()

In this Python example, we import the Process class from the multiprocessing module. We define a function named 'compute' that, when executed, prints the process ID, which uniquely identifies the running process. We then create two instances of Process, p1 and p2, both set to target the 'compute' function. By calling start() on each process, they begin executing the 'compute' function simultaneously. The join() method is called on both processes to ensure that the main program waits for these processes to finish executing before it continues or exits.

Imagine two delivery drivers (processes) who are tasked with delivering packages. Each driver drives independently to their destination. When both drivers have completed their deliveries, the dispatcher (main program) waits for both to return before closing the office for the day.

✅ True parallelism using multiple CPU cores ✅ Bypasses the GIL ❌ Higher overhead than threads ❌ Data must be serialized for communication between processes

Multiprocessing provides significant advantages when it comes to performance. One key benefit is true parallelism, which allows multiple processes to run on different CPU cores simultaneously, leading to faster computations. It also bypasses the limitations imposed by the GIL, meaning that you can utilize the full power of a multi-core CPU for CPU-bound tasks. However, there are downsides. Multiprocessing involves greater overhead due to the need to start and manage separate processes and because of the need for inter-process communication. If processes need to exchange data, that data must be serialized (converted into a format suitable for transfer), which can add additional complexity and potential performance penalties.

Consider a factory that produces toys. If the factory has multiple assembly lines (CPU cores) running different production processes (multiprocessing), it can produce a lot of toys at once (true parallelism). However, if managers need to share information between lines, they might have to fill out report forms (serialization) to ensure everyone is on the same page, which adds extra work and can slow things down.

--