Producer-Consumer with Condition Variables in Python

This challenge focuses on implementing the classic Producer-Consumer problem using Python's threading and condition variables. You'll build a system where multiple producer threads generate data and multiple consumer threads process it, ensuring efficient and safe data exchange between them. This exercise is crucial for understanding concurrent programming patterns and inter-thread communication.

Problem Description

You need to implement a producer-consumer system using Python's threading module, specifically leveraging threading.Condition for synchronization. The system will consist of:

A Shared Buffer: A fixed-size queue (e.g., a collections.deque or a list) where producers place items and consumers retrieve them.
Producer Threads: Threads that generate data items and add them to the buffer. Producers should wait if the buffer is full and resume when there's space.
Consumer Threads: Threads that retrieve data items from the buffer and process them. Consumers should wait if the buffer is empty and resume when there are items available.
Synchronization: Use threading.Condition to manage access to the shared buffer and signal producers and consumers when the buffer's state changes (i.e., becomes not full or not empty).

Key Requirements:

Thread Safety: Access to the shared buffer must be protected to prevent race conditions.
Blocking Behavior: Producers must block when the buffer is full, and consumers must block when the buffer is empty.
Signaling: Use threading.Condition.wait() and threading.Condition.notify() (or notify_all()) to manage thread blocking and waking.
Graceful Termination: Implement a mechanism to signal threads to terminate cleanly.

Expected Behavior:

Producers should continuously produce items (e.g., integers) until a termination signal is received.
Consumers should continuously consume items until a termination signal is received and the buffer is empty.
The buffer size should be respected, preventing overfilling or under-consumption.
When the buffer is full, producers will block. When an item is consumed, a producer should be notified and resume.
When the buffer is empty, consumers will block. When an item is produced, a consumer should be notified and resume.

Edge Cases:

Zero buffer size: While unlikely in a practical scenario, consider how your code would behave (though constraints will likely prevent this).
One producer, multiple consumers: Or vice versa.
No producers or no consumers: How does the system behave?
Termination during blocking: How do threads exit their wait() state?

Examples

Example 1:

Scenario: A single producer adds 5 items to a buffer of size 3, and a single consumer consumes them.

# Conceptual input:
buffer_size = 3
num_producers = 1
num_consumers = 1
items_to_produce_per_producer = 5

# Conceptual output:
# (Sequence of produced items might vary in order due to threading,
# but all items should be produced and consumed. Let's represent consumption)
Consumed: 0
Consumed: 1
Consumed: 2
Consumed: 3
Consumed: 4

Explanation: The producer creates items 0 through 4. The buffer has a size of 3. The producer will add items, and the consumer will remove them. If the producer tries to add an item when the buffer is full, it will block until the consumer removes an item. Similarly, if the consumer tries to remove an item when the buffer is empty, it will block until the producer adds an item.

Example 2:

Scenario: Two producers add 3 items each to a buffer of size 2, and two consumers consume them.

# Conceptual input:
buffer_size = 2
num_producers = 2
num_consumers = 2
items_to_produce_per_producer = 3

# Conceptual output:
# (Again, consumption order is flexible. This shows *all* items consumed)
Consumed: 0 (produced by producer 1)
Consumed: 1 (produced by producer 1)
Consumed: 2 (produced by producer 2)
Consumed: 3 (produced by producer 2)
Consumed: 4 (produced by producer 1)
Consumed: 5 (produced by producer 2)

Explanation: With a smaller buffer size and multiple producers/consumers, contention for the buffer is higher. Producers will block more frequently if consumers are slow, and consumers will block if producers are slow to fill the buffer. The Condition variable ensures that when space becomes available or an item is added, waiting threads are woken up appropriately. The termination signal would need to be handled to stop all threads after all items are produced and consumed.

Constraints

The shared buffer size will be between 2 and 10 (inclusive).
The number of producer threads will be between 1 and 5 (inclusive).
The number of consumer threads will be between 1 and 5 (inclusive).
Each producer will produce a fixed number of items, between 5 and 20 (inclusive), per producer.
Items produced will be non-negative integers.
The implementation should be in Python 3.
The solution must use threading.Condition.

Notes

Consider how to elegantly signal the producers and consumers to stop. A common pattern is to use a special "sentinel" value in the buffer or a separate shared flag.
When using threading.Condition, remember that wait() must be called within a with condition: block.
The notify() method wakes up a single waiting thread, while notify_all() wakes up all waiting threads. Choose the appropriate one for your logic.
Think about the order in which wait() and notify() calls are made to ensure correct state transitions of the buffer.
For termination, ensure that all produced items are consumed before the main program exits and all threads have joined.
You'll likely need to manage a shared counter or index for producing unique items, which also needs to be protected by the lock associated with the condition variable.