Concurrent Queue Communication with Producer and Consumer Threads

This challenge focuses on implementing a robust queue-based communication system using Python's threading capabilities. You'll design a system where one or more "producer" threads add items to a queue, and one or more "consumer" threads process those items. This is a fundamental pattern in concurrent programming, useful for tasks like asynchronous processing, message passing, and decoupling components.

Problem Description

You are tasked with creating a Python program that simulates a producer-consumer scenario using a queue. The program should consist of:

1. A Queue: Utilize Python's queue.Queue class to manage the items being passed between producers and consumers.
2. Producer Threads: Implement one or more producer threads. Each producer thread should generate a sequence of numbers (starting from a given start value and incrementing by 1 for each item) and add them to the queue. Producers should run for a specified number of iterations.
3. Consumer Threads: Implement one or more consumer threads. Each consumer thread should continuously retrieve items from the queue and process them. Processing, in this case, simply means printing the received item to the console. Consumers should gracefully terminate when the queue is empty and all producers have finished adding items.
4. Synchronization: Ensure proper synchronization between producers and consumers to prevent race conditions and ensure data integrity. The queue.Queue class handles much of this internally, but you need to manage thread termination correctly.

Key Requirements:

• The program must be thread-safe.
• Producers and consumers must operate concurrently.
• Consumers must wait for producers to finish before terminating.
• The queue should have a maximum size to prevent unbounded growth.
• The program should handle potential exceptions gracefully (e.g., queue is empty when a consumer tries to get an item).

Expected Behavior:

The program should print the numbers added to the queue by the producers, in the order they were produced, as consumed by the consumers. The output should be interleaved, reflecting the concurrent nature of the threads. The program should terminate cleanly when all producers have finished and all items in the queue have been consumed.

Edge Cases to Consider:

• What happens if the queue is full and a producer tries to add an item? (The producer should block until space becomes available).
• What happens if the queue is empty and a consumer tries to retrieve an item? (The consumer should block until an item is available).
• How do you ensure consumers terminate gracefully when there are no more items to process?
• What happens if a producer encounters an error? How does it signal the consumer to stop? (For simplicity, assume producers don't encounter errors in this challenge).

Examples

Example 1:

Input:
- Number of producers: 2
- Number of consumers: 1
- Queue size: 5
- Producer start value: 1
- Producer iterations: 3

Output: (Order may vary due to concurrency, but all numbers should be present)
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5

Explanation: Two producers add numbers 1-3 to the queue. The single consumer processes them and prints them. The queue size limits the number of items in the queue at any given time.

Example 2:

Input:
- Number of producers: 1
- Number of consumers: 2
- Queue size: 3
- Producer start value: 10
- Producer iterations: 5

Output: (Order may vary due to concurrency, but all numbers should be present)
10
11
12
13
14
15
16
17
18
19

Explanation: One producer adds numbers 10-14 to the queue. Two consumers process them concurrently.

Constraints

• The number of producers and consumers must be between 1 and 5 (inclusive).
• The queue size must be between 1 and 10 (inclusive).
• The producer start value must be a non-negative integer.
• The producer iterations must be a positive integer.
• The program should complete within 10 seconds. (This is a soft constraint; the goal is correctness first).

Notes

• Use Python's queue.Queue class for the queue implementation.
• Use Python's threading module for thread management.
• Consider using a threading.Event to signal consumers when producers have finished.
• Think carefully about how to handle the termination of consumer threads when the queue is empty and all producers are done. Avoid busy-waiting.
• Focus on clear, readable code. Proper commenting is encouraged.
• Error handling is not explicitly required for this challenge, but consider how you might handle errors in a real-world scenario.