Concurrent Task Processing with Worker Pools in Go
This challenge asks you to implement a worker pool system in Go to efficiently process a queue of tasks concurrently. Worker pools are a common pattern for managing concurrent operations, allowing you to limit the number of concurrent workers and prevent resource exhaustion while maximizing throughput. This is useful in scenarios like image processing, data analysis, or any task that can be broken down into smaller, independent units of work.
Problem Description
You are tasked with creating a worker pool that processes tasks from a channel. The worker pool should consist of a configurable number of worker goroutines. Each worker continuously reads tasks from the task channel, performs a simulated "work" operation on each task, and then signals completion. The main goroutine should wait for all tasks to be completed before exiting.
What needs to be achieved:
- Create a
WorkerPoolstruct that manages a pool of worker goroutines. - Implement a
Runmethod on theWorkerPoolthat accepts a task channel and starts the worker goroutines. - Each worker goroutine should continuously read tasks from the provided channel.
- Simulate work being done on each task (e.g., by sleeping for a short duration).
- Ensure that the main goroutine waits for all tasks to be processed before exiting.
Key Requirements:
- The number of worker goroutines should be configurable when creating the
WorkerPool. - The
Runmethod should accept a channel of tasks as input. - The worker pool should gracefully handle the completion of all tasks.
- The simulated work should be represented by a sleep.
- The code should be well-structured, readable, and idiomatic Go.
Expected Behavior:
When the Run method is called with a task channel, the worker pool should:
- Start the specified number of worker goroutines.
- Each worker should continuously read tasks from the channel.
- Each worker should simulate work by sleeping for a short duration (e.g., 100 milliseconds).
- Once the channel is closed and all tasks have been processed, the worker pool should exit gracefully.
- The main goroutine should wait for all workers to finish before exiting.
Edge Cases to Consider:
- What happens if the task channel is closed before all workers have started?
- What happens if the number of tasks is less than the number of workers?
- How does the system handle errors during task processing (for simplicity, you can ignore error handling in this challenge)?
Examples
Example 1:
Input: numWorkers = 3, numTasks = 5, taskDuration = 100ms
Output: All tasks processed, worker pool exits.
Explanation: 3 worker goroutines are started. Each worker processes tasks until the channel is closed. The main goroutine waits for all workers to finish.
Example 2:
Input: numWorkers = 5, numTasks = 2, taskDuration = 500ms
Output: All tasks processed, worker pool exits.
Explanation: 5 worker goroutines are started, but only 2 tasks are available. The workers process the tasks and then exit.
Example 3: (Edge Case)
Input: numWorkers = 2, numTasks = 0, taskDuration = 100ms
Output: Worker pool exits immediately.
Explanation: No tasks are provided, so the workers never receive any work and the pool exits.
Constraints
numWorkersmust be a positive integer (greater than 0).numTaskscan be any non-negative integer (0 or greater).taskDurationshould be a reasonable sleep duration (e.g., between 10ms and 500ms).- The solution should be efficient and avoid unnecessary resource consumption.
- The code should compile and run without errors.
Notes
- Consider using a
sync.WaitGroupto synchronize the worker goroutines. - The task channel should be buffered to prevent blocking.
- The simulated work can be implemented using
time.Sleep. - Focus on the core logic of the worker pool; error handling and more complex task processing can be omitted for this challenge.
- Think about how to signal the workers to exit gracefully when the task channel is closed.