Queue Implementation in Python: Basic Operations and Examples

Queue Implementation in Python: Basic Operations and Examples

6 mins read613 Views Comment
Vikram
Vikram Singh
Assistant Manager - Content
Updated on May 14, 2024 14:50 IST

Learn about queues in Python, a data structure that allows you to store and manipulate collections of items in a first-in, first-out (FIFO) order. Discover the different types of queues available in Python, advanced queue operations, best practices for using queues in multi-threaded environments, and more.

2023_02_MicrosoftTeams-image-194.jpg

Table of Content

Introduction to Queue in Python

What is Python Queue?

In Python, a queue is a data structure that allows adding and removing elements in a first-in, first-out (FIFO) order, i.e., the first element added to the queue will be the first one to be removed. It is used for task scheduling, job processing, and event handling.
Must Read: Queue Data Structure

Why use Python Queue?

Queue in Python is useful for:

Ordering

  • It helps to maintain the order of elements (that need to be added) in the queue.
  • The order in which elements are processed is to be FIFO (First-In, First-Out)

Task Scheduling and Event Handling

  • It helps to schedule and prioritize tasks.
    • i.e., you can ensure that high-priority tasks are processed first by adding tasks to a queue and processing them in order.
  • It is useful in event handling, such as GUI programming.

Multi-Threading

  • Used to coordinate data flow between threads in multi-threaded environments.
  • With a thread-safe implementation, we can safely add or remove elements from the queue across multiple threads.

Asynchronous programming:

  • It helps to manage the data flow between different program parts.
  • Adding data to a queue and processing asynchronously ensures that your program is efficient and responsive.
What is Programming What is Python
What is Data Science What is Machine Learning

What are the different types of Queues in Python?

Python has several types of queues, each with different features and uses cases. Here are a few examples:

Simple Queue

  • A basic data structure supporting the enqueue and dequeue operations.
  • Elements are added to the back of the queue and removed from the front of the queue in a FIFO (first-in, first-out) order. 
  • Python implements it using lists, collections.deque, and queue.Queue.

Priority Queue

  • A queue where the elements are processed based on their priority level.
    • i.e., an element with a higher priority will be processed before the element with a lower priority.
  • In Python, it is implemented using heapq module or the queue.PriorityQueue class.

Must Read: What is Priority Queue?

Circular Queue

  • A queue that uses a circular buffer to store elements, and once the buffer is full new elements overwrite the oldest element in the buffer. 
  • Python implements it using a fixed-size list or collections.deque.

Must Read: What is Circular Queue?

Deque

  • Deque or Double Ended Queue is a data structure that allows adding and removing elements from both ends.
  • In Python, it is implemented using collections.deque.

Must Read: What is Deque?

LIFO Queue

  • LIFO (or Last-In, First-Out) queue is a data structure in which the last added element will be the first to be out (or removed).
  • In Python, it is implemented using a queue.LifoQueue class.

How to Implement Queues in Python?

Using a List

A simple way to implement a queue is to use a list. Elements can be added to the end of the list using the append() method and removed from the beginning using the pop() method with an index of 0. For example:

 
# implement a queue using a list
queue = []
queue.append(1) # enqueue
queue.append(2)
queue.append(3)
print(queue) # [1, 2, 3]
x = queue.pop(0) # dequeue
print(x) # 1
print(queue) # [2, 3]
Copy code

Output

Must Read: Python List Practice Program for Beginners 

Must Read: How to use the append() method in Python

Using collections.deque

Python’s collections module provides a deque (double-ended queue) data structure, which can be used as a queue by appending to the right and popping from the left. For example:

 
# implement queue using collections.deque
from collections import deque
queue = deque()
queue.append(1) # enqueue
queue.append(2)
queue.append(3)
print(queue) # deque([1, 2, 3])
x = queue.popleft() # dequeue
print(x) # 1
print(queue) # deque([2, 3])
Copy code

Output

Using queue.Queue

Python’s built-in queue module provides the Queue class, which is a thread-safe queue that can be used in a multi-threaded environment. Elements can be added to the queue using the put() method and removed using the get() method. For example:

 
# implement queue using queue.Qeque
import queue
queue = queue.Queue()
queue.put(1) # enqueue
queue.put(2)
queue.put(3)
print(queue.queue) # [1, 2, 3]
x = queue.get() # dequeue
print(x) # 1
print(queue.queue) # [2, 3]
Copy code

Output

Advanced Queue Operations

Blocking operations: 

  • Python queue module provides blocking operations: get() and put().
    • These operations block the thread until an item is available or until there is a space in the queue.
  • It is useful in multi-threaded programs where you want to ensure that threads do not consume resources unnecessarily.

Queue size limits: 

  • Any queue size can be set using the maxsize parameter (or maxsize() method).
    • Once the queue reaches its maximum size, further put() operations will block until space becomes available. 
  • It is useful for preventing the queue from consuming too much memory.

Joining queues:

  • Two or more queues can be added using queue.join() methods, which blocks until all items in the queue have been processed.

What are synchronized queues in Python?

Synchronized queues in Python thread-safe allow multiple threads to access the queue simultaneously without causing conflicts or race conditions.

In a synchronized queue, all access to the queue is managed through locks and synchronization primitives, which ensure that only one thread can access the queue at a time. It prevents data corruption, lost data, or race conditions when multiple threads try to access the same data simultaneously.

Python’s queue module provides several synchronized queue classes, including Queue, LifoQueue, and PriorityQueue, used in multi-threaded environments. These classes provide blocking and non-blocking methods for adding and removing items from the queue, as well as methods for getting the current size of the queue, waiting for the queue to become non-empty, and waiting for the queue to become empty.

Best Practice and Tips for using queues in Python

Here are some best practices and tips for using queues in Python:

  • Use thread-safe queue implementations: When working with queues in multi-threaded environments, always use queue implementations designed to be thread-safe, such as Queue, LifoQueue, and PriorityQueue from Python’s queue module.
  • Use bounded queues: If you’re working with a queue that may receive an unbounded amount of data, use a bounded queue implementation and set a reasonable maximum size to prevent the queue from consuming too much memory.
  • Use blocking operations: When accessing a queue from multiple threads, use blocking operations such as put() and get() to prevent threads from spinning in a tight loop, consuming CPU resources.
  • Use a timeout when blocking: When using blocking operations such as get() with a timeout, set a reasonable timeout value to prevent the threads from waiting indefinitely.
  • Use priority queues: If you need to process items in a specific order, consider implementing a priority queue, as it processes items based on their priority level, allowing you to process higher-priority items first.
  • Use separate threads for producers and consumers: When designing a multi-threaded application that uses queues, use separate threads for producing and consuming items to prevent contention for the queue.
  • Avoid busy-waiting: When waiting for a queue to become non-empty or non-full, avoid busy-waiting and use blocking operations with a timeout instead.
  • Avoid unnecessary locking: When accessing a synchronized queue, lock only the critical sections of the code that need to be protected, and release the lock as soon as possible to avoid unnecessary contention.

Conclusion

In conclusion, queues are an essential data structure in Python for managing collections of items in a FIFO order. Python provides several queue implementations, including synchronized and priority queues, that can be used in multi-threaded environments to ensure thread safety and prevent race conditions. With the help of advanced queue operations, such as blocking and non-blocking operations, timeouts, and priority levels, you can easily manipulate and process items in a queue. By following best practices and tips for using queues in Python, you can ensure that your code is efficient, thread-safe, and reliable. Whether you’re working on a small script or a large-scale application, queues in Python can help you manage your data in a structured and efficient manner.

About the Author
author-image
Vikram Singh
Assistant Manager - Content

Vikram has a Postgraduate degree in Applied Mathematics, with a keen interest in Data Science and Machine Learning. He has experience of 2+ years in content creation in Mathematics, Statistics, Data Science, and Mac... Read Full Bio