Python queue Module

In Python, the queue module provides a thread-safe queue implementation for safely passing data in multi-threaded programming.

A queue is a first-in-first-out (FIFO) data structure. The queue module provides multiple queue types, including Queue, LifoQueue, and PriorityQueue, to meet different needs.

---

Queue Types

1. Queue

Queue is the most commonly used queue type in the queue module. It implements a standard first-in-first-out (FIFO) queue. Here is the basic usage of Queue:

Example

import queue

# Create a queue
q = queue.Queue()

# Add elements to the queue
q.put(1)
q.put(2)
q.put(3)

# Retrieve elements from the queue
print(q.get())  # Output: 1
print(q.get())  # Output: 2
print(q.get())  # Output: 3

2. LifoQueue

LifoQueue is a last-in-first-out (LIFO) queue, similar to a stack. Here is the basic usage of LifoQueue:

Example

import queue

# Create a LIFO queue
q = queue.LifoQueue()

# Add elements to the queue
q.put(1)
q.put(2)
q.put(3)

# Retrieve elements from the queue
print(q.get())  # Output: 3
print(q.get())  # Output: 2
print(q.get())  # Output: 1

3. PriorityQueue

PriorityQueue is a priority queue where elements are retrieved in order of priority. Here is the basic usage of PriorityQueue:

Example

import queue

# Create a priority queue
q = queue.PriorityQueue()

# Add elements to the queue, elements are tuples (priority, data)
q.put((3, 'Low priority'))
q.put((1, 'High priority'))
q.put((2, 'Medium priority'))

# Retrieve elements from the queue
print(q.get())  # Output: (1, 'High priority')
print(q.get())  # Output: (2, 'Medium priority')
print(q.get())  # Output: (3, 'Low priority')

---

Common Methods

1. put(item, block=True, timeout=None)

Puts item into the queue. If block is True and the queue is full, wait for timeout seconds until the queue has free space. If timeout is None, wait indefinitely.

2. get(block=True, timeout=None)

Retrieves and removes an element from the queue. If block is True and the queue is empty, wait for timeout seconds until the queue has an element. If timeout is None, wait indefinitely.

3. qsize()

Returns the number of elements in the queue.

4. empty()

Returns True if the queue is empty, otherwise False.

5. full()

Returns True if the queue is full, otherwise False.

---

Thread Safety

All queue types in the queue module are thread-safe, meaning multiple threads can safely operate on the same queue simultaneously without needing additional synchronization mechanisms. This makes the queue module an ideal choice for passing data in multi-threaded programming.

---

Example: Multi-Threaded Queue

Below is an example of using Queue to pass data between multiple threads:

Example

import queue
import threading
import time

# Create a queue
q = queue.Queue()

# Producer thread
def producer():
    for i in range(5):
        print(f'Producing {i}')
        q.put(i)
        time.sleep(1)

# Consumer thread
def consumer():
    while True:
        item = q.get()
        if item is None:
            break
        print(f'Consuming {item}')
        q.task_done()

# Start producer thread
producer_thread = threading.Thread(target=producer)
producer_thread.start()

# Start consumer thread
consumer_thread = threading.Thread(target=consumer)
consumer_thread.start()

# Wait for producer thread to finish
producer_thread.join()

# Wait for all tasks in the queue to complete
q.join()

# Send termination signal
q.put(None)
consumer_thread.join()

---

Common Attributes and Methods

Below is a tabular description of the common classes, methods, and attributes of the Python queue module (thread-safe queue), including descriptions and examples:

Core Classes of the queue Module

Class	Description	Use Case
queue.Queue	First-in-first-out (FIFO) queue	General task queue
queue.LifoQueue	Last-in-first-out (LIFO) queue (like a stack)	Scenarios requiring LIFO order
queue.PriorityQueue	Priority queue (min-heap implementation)	Processing tasks by priority
queue.SimpleQueue	Simpler FIFO queue (Python 3.7+)	Scenarios not needing advanced features

Common Methods (supported by all queue classes)

Method	Description	Example	Return Value
put(item)	Put an element	q.put("task1")	None
get()	Retrieve and remove an element	item = q.get()	Queue element
empty()	Check if the queue is empty	if q.empty():	True/False
full()	Check if the queue is full	if q.full():	True/False
qsize()	Return the current size of the queue	size = q.qsize()	Integer
task_done()	Mark a task as completed (used with join())	q.task_done()	None
join()	Block until all tasks are completed	q.join()	None

Blocking Control Parameters

Parameter	Description	Default	Example
block	Whether to block when the queue is empty/full	True	q.get(block=False)
timeout	Blocking timeout duration (seconds)	None	q.put(x, timeout=5)

PriorityQueue-Specific Usage

Element format: (priority, data), lower priority values are dequeued first

Example

pq = queue.PriorityQueue()
pq.put((1, "low"))
pq.put((0, "high"))
print(pq.get()[1])  # Output: "high"

Examples

Producer-consumer model:

Example

import queue, threading

q = queue.Queue(maxsize=3)  # Queue with capacity of 3

def producer():
    for i in range(5):
        q.put(f"Task-{i}")
        print(f"Produced: Task-{i}")

def consumer():
    while True:
        item = q.get()
        print(f"Consumed: {item}")
        q.task_done()

threading.Thread(target=producer, daemon=True).start()
threading.Thread(target=consumer, daemon=True).start()
q.join()  # Wait for all tasks to complete

Priority task processing:

Example

pq = queue.PriorityQueue()
pq.put((3, "Scan"))
pq.put((1, "Emergency"))
pq.put((2, "Log"))

while not pq.empty():
    print(pq.get()[1])  # Output order: Emergency → Log → Scan

Non-blocking retrieval (avoiding deadlocks):

Example

try:
    item = q.get_nowait()  # Equivalent to q.get(block=False)
except queue.Empty:
    print("Queue is empty")