Python3 Multithreading

Multithreading is similar to running multiple different programs simultaneously. Multithreading has the following advantages:

• Using threads can put tasks from long-running programs into the background for processing.
• User interfaces can be more attractive, for example, when a user clicks a button to trigger some event processing, a progress bar can pop up to show the processing progress.
• The program’s running speed may be accelerated.
• In some waiting tasks such as user input, file reading/writing, and network data sending/receiving, threads are more useful. In such cases, we can release some precious resources such as memory usage, etc.

Each independent thread has a program entry point, a sequential execution sequence, and a program exit. However, threads cannot execute independently; they must exist within an application, which provides multiple threads with execution control.

Each thread has its own set of CPU registers, called the thread’s context, which reflects the state of the CPU registers the last time the thread ran.

The instruction pointer and stack pointer registers are the two most important registers in the thread context. Threads always run within the context of a process, and these addresses are used to mark memory within the process’s address space that owns the thread.

• Threads can be preempted (interrupted).
• While other threads are running, a thread can be temporarily suspended (also called sleeping) – this is thread yielding.

Threads can be divided into:

• Kernel threads: Created and destroyed by the operating system kernel.
• User threads: Threads implemented in user programs without kernel support.

Two commonly used modules for threads in Python3 are:

• _thread
• threading (recommended)

The thread module has been deprecated. Users can use the threading module instead. Therefore, in Python3, you can no longer use the “thread” module. For compatibility, Python3 renamed thread to “_thread”.

Getting Started with Python Threads

There are two ways to use threads in Python: using functions or wrapping thread objects with classes.

Functional: Call the start_new_thread() function in the _thread module to spawn a new thread. The syntax is as follows:

_thread.start_new_thread ( function, args[, kwargs] )

Parameter description:

• function - The thread function.
• args - Arguments passed to the thread function; it must be a tuple type.
• kwargs - Optional parameters.

Example

#!/usr/bin/python3

import _thread
import time

# Define a function for the thread
def print_time( threadName, delay):
   count = 0
   while count < 5:
      time.sleep(delay)
      count += 1
      print ("%s: %s" % ( threadName, time.ctime(time.time()) ))

# Create two threads
try:
   _thread.start_new_thread( print_time, ("Thread-1", 2, ) )
   _thread.start_new_thread( print_time, ("Thread-2", 4, ) )
except:
   print ("Error: Unable to start thread")

while 1:
   pass

The output of the above program is:

Thread-1: Wed Jan  5 17:38:08 2022
Thread-2: Wed Jan  5 17:38:10 2022
Thread-1: Wed Jan  5 17:38:10 2022
Thread-1: Wed Jan  5 17:38:12 2022
Thread-2: Wed Jan  5 17:38:14 2022
Thread-1: Wed Jan  5 17:38:14 2022
Thread-1: Wed Jan  5 17:38:16 2022
Thread-2: Wed Jan  5 17:38:18 2022
Thread-2: Wed Jan  5 17:38:22 2022
Thread-2: Wed Jan  5 17:38:26 2022

After executing the above program, you can press ctrl-c to exit.

---

Threading Module

Python3 provides support for threads through two standard libraries: _thread and threading.

_thread provides low-level, primitive threads and a simple lock. Compared to the threading module, its functionality is relatively limited.

In addition to containing all methods in the _thread module, the threading module also provides other methods:

• threading.current_thread(): Returns the current thread variable.
• threading.enumerate(): Returns a list containing all running threads. “Running” means after the thread has started and before it has finished, excluding threads before start and after termination.
• threading.active_count(): Returns the number of running threads, which has the same result as len(threading.enumerate()).
• threading.Thread(target, args=(), kwargs={}, daemon=None):
  • Creates an instance of the Thread class.
  • target: The target function to be executed by the thread.
  • args: Arguments to the target function, passed as a tuple.
  • kwargs: Keyword arguments to the target function, passed as a dictionary.
  • daemon: Specifies whether the thread is a daemon thread.

The threading.Thread class provides the following methods and properties:

1. __init__(self, group=None, target=None, name=None, args=(), kwargs={}, *, daemon=None)：

   • Initializes the Thread object.
   • group: Thread group, currently unused, reserved for future extension.
   • target: The target function to be executed by the thread.
   • name: The name of the thread.
   • args: Arguments to the target function, passed as a tuple.
   • kwargs: Keyword arguments to the target function, passed as a dictionary.
   • daemon: Specifies whether the thread is a daemon thread.

2. start(self)：

   • Starts the thread. Will call the thread’s run() method.

3. run(self)：

   • The thread defines the code to be executed in this method.

4. join(self, timeout=None)：

   • Waits for the thread to terminate. By default, join() blocks until the called thread terminates. If the timeout parameter is specified, it waits at most timeout seconds.

5. is_alive(self)：

   • Returns whether the thread is running. Returns True if the thread has started and not yet terminated, otherwise returns False.

6. getName(self)：

   • Returns the name of the thread.

7. setName(self, name)：

   • Sets the name of the thread.

8. ident property：

   • The unique identifier of the thread.

9. daemon property：

   • The daemon flag of the thread, used to indicate whether it is a daemon thread.

10. isDaemon() method：

A simple thread example:

Example

import threading
import time

def print_numbers():
    for i in range(5):
        time.sleep(1)
        print(i)

# Create a thread
thread = threading.Thread(target=print_numbers)

# Start the thread
thread.start()

# Wait for the thread to finish
thread.join()

Output:

0
1
2
3
4

---

Creating Threads Using the threading Module

We can create a new subclass by directly inheriting from threading.Thread, instantiate it, and call the start() method to start the new thread, which calls the thread’s run() method:

Example

#!/usr/bin/python3

import threading
import time

exitFlag = 0

class myThread (threading.Thread):
    def __init__(self, threadID, name, delay):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.delay = delay
    def run(self):
        print ("Starting thread: " + self.name)
        print_time(self.name, self.delay, 5)
        print ("Exiting thread: " + self.name)

def print_time(threadName, delay, counter):
    while counter:
        if exitFlag:
            threadName.exit()
        time.sleep(delay)
        print ("%s: %s" % (threadName, time.ctime(time.time())))
        counter -= 1

# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)

# Start new threads
thread1.start()
thread2.start()
thread1.join()
thread2.join()
print ("Exiting main thread")

The output of the above program is:

Starting thread: Thread-1
Starting thread: Thread-2
Thread-1: Wed Jan  5 17:34:54 2022
Thread-2: Wed Jan  5 17:34:55 2022
Thread-1: Wed Jan  5 17:34:55 2022
Thread-1: Wed Jan  5 17:34:56 2022
Thread-2: Wed Jan  5 17:34:57 2022
Thread-1: Wed Jan  5 17:34:57 2022
Thread-1: Wed Jan  5 17:34:58 2022
Exiting thread: Thread-1
Thread-2: Wed Jan  5 17:34:59 2022
Thread-2: Wed Jan  5 17:35:01 2022
Thread-2: Wed Jan  5 17:35:03 2022
Exiting thread: Thread-2
Exiting main thread

---

Thread Synchronization

If multiple threads modify the same data, unpredictable results may occur. To ensure data correctness, multiple threads need to be synchronized.

Using the Lock and Rlock of the Thread object can achieve simple thread synchronization. Both objects have acquire and release methods. For data that needs to be operated on by only one thread at a time, put its operations between the acquire and release methods. As follows:

The advantage of multithreading is that multiple tasks can run simultaneously (or at least feel that way). However, when threads need to share data, data synchronization issues may arise.

Consider this scenario: all elements in a list are 0. The “set” thread changes all elements from back to front to 1, while the “print” thread reads the list from front to back and prints it.

Then, it’s possible that when the “set” thread starts modifying, the “print” thread starts printing the list. The output would be half 0s and half 1s. This is data desynchronization. To avoid this situation, the concept of locks was introduced.

A lock has two states – locked and unlocked. Whenever a thread, such as “set”, wants to access shared data, it must first acquire the lock. If another thread, such as “print”, has already acquired the lock, then the “set” thread is suspended, which is synchronous blocking. Once the “print” thread finishes accessing and releases the lock, the “set” thread can continue.

After this processing, when printing the list, it will output either all 0s or all 1s, avoiding the awkward situation of half 0s and half 1s.

Example

#!/usr/bin/python3

import threading
import time

class myThread (threading.Thread):
    def __init__(self, threadID, name, delay):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.delay = delay
    def run(self):
        print ("Starting thread: " + self.name)
        # Acquire lock for thread synchronization
        threadLock.acquire()
        print_time(self.name, self.delay, 3)
        # Release lock, start the next thread
        threadLock.release()

def print_time(threadName, delay, counter):
    while counter:
        time.sleep(delay)
        print ("%s: %s" % (threadName, time.ctime(time.time())))
        counter -= 1

threadLock = threading.Lock()
threads = []

# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)

# Start new threads
thread1.start()
thread2.start()

# Add threads to the thread list
threads.append(thread1)
threads.append(thread2)

# Wait for all threads to complete
for t in threads:
    t.join()
print ("Exiting main thread")

The output of the above program is:

Starting thread: Thread-1
Starting thread: Thread-2
Thread-1: Wed Jan  5 17:36:50 2022
Thread-1: Wed Jan  5 17:36:51 2022
Thread-1: Wed Jan  5 17:36:52 2022
Thread-2: Wed Jan  5 17:36:54 2022
Thread-2: Wed Jan  5 17:36:56 2022
Thread-2: Wed Jan  5 17:36:58 2022
Exiting main thread

---

Thread Priority Queue (Queue)

Python’s Queue module provides synchronized, thread-safe queue classes, including FIFO (First In First Out) queue Queue, LIFO (Last In First Out) queue LifoQueue, and priority queue PriorityQueue.

These queues implement lock primitives and can be used directly in multithreading. Queues can be used to implement synchronization between threads.

Common methods in the Queue module:

• Queue.qsize() Returns the size of the queue
• Queue.empty() If the queue is empty, returns True, otherwise False
• Queue.full() If the queue is full, returns True, otherwise False
• Queue.full Corresponds to the maxsize size
• Queue.get([block[, timeout]]) Gets from the queue, timeout wait time
• Queue.get_nowait() Equivalent to Queue.get(False)
• Queue.put(item) Writes to the queue, timeout wait time
• Queue.put_nowait(item) Equivalent to Queue.put(item, False)
• Queue.task_done() After completing a task, Queue.task_done() sends a signal to the queue that the task has been completed
• Queue.join() Actually means to wait until the queue is empty before performing other operations

Example

#!/usr/bin/python3

import queue
import threading
import time

exitFlag = 0

class myThread (threading.Thread):
    def __init__(self, threadID, name, q):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.q = q
    def run(self):
        print ("Starting thread: " + self.name)
        process_data(self.name, self.q)
        print ("Exiting thread: " + self.name)

def process_data(threadName, q):
    while not exitFlag:
        queueLock.acquire()
        if not workQueue.empty():
            data = q.get()
            queueLock.release()
            print ("%s processing %s" % (threadName, data))
        else:
            queueLock.release()
        time.sleep(1)

threadList = ["Thread-1", "Thread-2", "Thread-3"]
nameList = ["One", "Two", "Three", "Four", "Five"]
queueLock = threading.Lock()
workQueue = queue.Queue(10)
threads = []
threadID = 1

# Create new threads
for tName in threadList:
    thread = myThread(threadID, tName, workQueue)
    thread.start()
    threads.append(thread)
    threadID += 1

# Fill the queue
queueLock.acquire()
for word in nameList:
    workQueue.put(word)
queueLock.release()

# Wait for the queue to empty
while not workQueue.empty():
    pass

# Notify threads that it's time to exit
exitFlag = 1

# Wait for all threads to complete
for t in threads:
    t.join()
print ("Exiting main thread")

The output of the above program is:

Starting thread: Thread-1
Starting thread: Thread-2
Starting thread: Thread-3
Thread-3 processing One
Thread-1 processing Two
Thread-2 processing Three
Thread-3 processing Four
Thread-1 processing Five
Exiting thread: Thread-3
Exiting thread: Thread-2
Exiting thread: Thread-1
Exiting main thread