Go Goroutine Pool
This challenge requires you to implement a goroutine pool in Go. A goroutine pool is a concurrency pattern that limits the number of goroutines that can run concurrently. This is useful for managing system resources, preventing excessive memory usage, and avoiding performance degradation when dealing with a large number of concurrent tasks.
Problem Description
You need to create a GoroutinePool struct in Go that manages a fixed number of worker goroutines. This pool should be able to accept tasks (functions) and distribute them to available worker goroutines for execution.
Key Requirements:
- Fixed Worker Count: The pool should be initialized with a specific number of worker goroutines.
- Task Submission: Provide a method to submit tasks to the pool. A task is a function that takes no arguments and returns no values (
func()). - Task Distribution: The pool should manage a queue of submitted tasks and dispatch them to idle worker goroutines.
- Concurrency Control: Ensure that no more than the configured number of worker goroutines are active at any given time.
- Graceful Shutdown: Implement a mechanism to gracefully shut down the pool, ensuring all submitted tasks are completed before the workers exit.
- Error Handling: Consider how to handle potential panics within submitted tasks. For this challenge, panics should be recovered, logged (or printed to stderr), and the worker should continue processing other tasks.
Expected Behavior:
When tasks are submitted, they should be picked up by available workers and executed concurrently. If all workers are busy, the tasks should wait in a queue until a worker becomes free. The Shutdown method should signal the pool to stop accepting new tasks and wait for all ongoing and queued tasks to finish.
Edge Cases:
- Submitting tasks after
Shutdownhas been called. - Submitting a large number of tasks very quickly.
- Tasks that panic.
- Submitting zero tasks.
Examples
Example 1:
Input:
- Pool size: 2
- Tasks: 5 simple tasks that print their ID and sleep for a short duration.
Output:
(Order of task execution may vary, but all tasks should complete)
Task 0 started
Task 1 started
Task 2 started
Task 3 started
Task 4 started
Task 0 finished
Task 1 finished
Task 2 finished
Task 3 finished
Task 4 finished
Explanation: With a pool size of 2, at most two tasks will run concurrently. As tasks finish, new tasks from the queue are picked up by the available workers.
Example 2:
Input:
- Pool size: 3
- Tasks: 10 tasks, some of which panic.
- Task function for non-panicking tasks: print task ID and sleep.
- Task function for panicking tasks: print task ID, panic, and then print a message indicating recovery.
Output:
(Order of task execution may vary)
Task 0 started
Task 1 started
Task 2 started
Task 0 finished
Task 3 started
Task 1 finished
Task 4 started
Task 2 finished
Task 5 started (panicking task)
... (panic message and recovery message for Task 5)
Task 6 started
...
All tasks completed.
Explanation:
Even when tasks panic, the pool should recover, log the panic, and continue processing other tasks. The Shutdown method should wait for all tasks, including the panicking ones, to finish their execution or recovery.
Constraints
- The pool size must be a positive integer.
- Tasks submitted to the pool are of type
func(). - The
Shutdownmethod should block until all submitted tasks have been processed. - The implementation should be memory efficient and avoid unbounded growth of internal queues if possible (consider using buffered channels).
- Panics within tasks must be recovered, logged to
stderr, and should not bring down the worker goroutine.
Notes
- Consider using channels for task distribution and worker communication.
- A common pattern involves a central dispatcher and multiple worker goroutines.
- Think about how to signal the workers to stop when
Shutdownis called. - A way to track the number of active tasks can be helpful for the
Shutdownprocess. - You might want to use
sync.WaitGroupto coordinate the shutdown.