Python Multiprocessing Tutorial

What are processes in Python?

Understanding the concept of processes and threads is extremely useful to comprehend better how an operating system manages programs through the various execution stages. A process is merely a reference to a computer program. Each program has one process associated with it.

  • On Mac: open Spotlight Search on Mac and type “Activity Monitor,” then hit Return
  • On Linux: click on Application Menu and search for System Monitor.

Multiprocessing vs. Threading

To be more specific, one instance of Python’s interpreter — the tool that converts the code written in Python to the language a computer can understand — is equal to one process. A process will consist of at least one thread, called the “main thread” in Python, although other threads may be created within the same process — all other threads created within a process will belong to that process.

  • Thread: A representation of how a Python program is executed within a Python process.

Similarities

#1 Concurrency

Differences

#1 Functionality

The benefits of Python Multiprocessing

Think of a processor as an entrepreneur. As the entrepreneur’s business grows, there are more tasks that must be managed to keep up with the growth of the business. If the entrepreneur decides to take on all of these tasks alone (i.e., accounting, sales, marketing, innovation, etc.), she risks hampering the overall efficiency and performance of the business since one person can only do so much at any one time. For example, before moving on to innovation tasks, she must stop the sales tasks — this is known as running tasks “sequentially.”

The basics of Python’s Multiprocessing module

In the “What are processes in Python?” section, we mentioned that the lifecycle of a Python process consists of three stages: the new process, the running process, and the terminated process. This section will delve deeper into each phase of the lifecycle and provide coded examples.

The new process

A new process may be defined as the process that has been created by instantiating an instance of the Process class. A child process is spawned when we assign the Process object to a variable.

from multiprocessing import Process# Create a new process
process = Process()
# Create a new process with a specified function to execute.
def example_function():
pass
new_process = Process(target=example_function)
# Create a new process with specific function to execute with args.
def example(args):
pass
process = Process(target=example, args=("Hi",))

The running process

Running a new process is quite straightforward: simply call the start() method of the process instance.

# Run the new process
process.start()
# Run the new process
process.start()
# Check process is running
process.is_alive()
"""
True
"""

The terminated process

When the run() function returns or exits, the process is terminated: we don’t necessarily have to do anything explicitly. In other words, you can expect a process to be terminated after the instructions you set as the target function is complete. Thus, the is_alive() method would return false.

# Check process is terminated - should return false.
process.is_alive()
"""
False
"""
# Create a new process with a specific function that has an error
# to execute with args.
def example(args):
split_args = list(args.split())
# "name" variable is not in the function namespace - should raise error
return name
# New process
process = Process(target=example, args=("Hi",))
# Running the new process
process.start()
# Check process is running
process.is_alive()
"""
True
Process Process-15:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
"""
# Create a new process with a specific function to execute with args.
def example(args):
split_args = list(args.split())
# "name" variable is not in the function namespace - should raise error
return split_args
# New process
process = Process(target=example, args=("Hi",))
# Running the new process
process.start()
if process.is_alive():
process.terminate() # You can also use process.kill()
print("Process terminated forcefully")
"""
Process terminated forcefully
"""

Python Multiprocessing Tutorial

Now that you understand the basics of multiprocessing, let’s work on an example to demonstrate how to do concurrent programming in Python.

import timedef do_something():
print("I'm going to sleep")
time.sleep(1)
print("I'm awake")
# Create new child processprocess_1 = Process(target=do_something)
process_2 = Process(target=do_something)
%%time# Starts both processes
process_1.start()
process_2.start()
"""
I'm going to sleep
CPU times: user 810 µs, sys: 7.34 ms, total: 8.15 ms
Wall time: 6.04 ms
I'm going to sleep
"""
%%time

# Create new child process (Cannot run a process more than once)
new_process_1 = Process(target=do_something)
new_process_2 = Process(target=do_something)

# Starts both processes
new_process_1.start()
new_process_2.start()

new_process_1.join()
new_process_2.join()
"""
I'm going to sleep
I'm going to sleep
I'm awake
I'm awake
CPU times: user 0 ns, sys: 14 ms, total: 14 ms
Wall time: 1.01 s
"""
POWERED BY DATACAMP WORKSPACECOPY CODE

Wrap up

In this tutorial, you learned about how to make Python programs more efficient by running them concurrently. Specifically, you learned:

  • The similarities and differences between Python’s multiprocessing and threading modules.
  • The basics of the multiprocessing module and how to run a Python program concurrently using multiprocessing.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Itexamtools

Itexamtools

At ITExamtools.com we help IT students and Professionals by providing important info. about latest IT Trends & for selecting various Academic Training courses.