Implement a Basic Executor Service in Rust
This challenge asks you to build a foundational component of concurrent programming: an executor. An executor is responsible for managing and executing tasks (often represented as futures or closures) across a set of worker threads. Implementing this from scratch will deepen your understanding of threading, synchronization, and asynchronous programming primitives in Rust.
Problem Description
You need to implement a Executor struct in Rust that can accept and run tasks. The executor should maintain a pool of worker threads, each of which will pick up tasks from a shared queue and execute them.
Key Requirements:
- Task Queue: A thread-safe queue to hold tasks waiting to be executed.
- Worker Threads: A fixed number of worker threads that continuously poll the task queue.
- Task Execution: Each worker thread should be able to take a task from the queue and execute it.
- Graceful Shutdown: The executor should support a way to signal all worker threads to shut down and wait for them to finish their current tasks.
- Task Definition: Tasks will be represented by closures that return
(). They should beSendand'staticto be safely passed between threads.
Expected Behavior:
When tasks are submitted to the executor, they should be placed in the queue. Worker threads will dequeue tasks and execute them. If the executor is asked to shut down, workers should stop accepting new tasks and exit after finishing any tasks already in their local processing pipeline.
Edge Cases:
- Submitting a large number of tasks.
- Submitting tasks when the executor is shutting down.
- Zero worker threads (though this might be disallowed by constraints).
Examples
Example 1: Simple Task Execution
// Assume we have a way to create an executor and submit tasks.
// This is conceptual for illustration.
// Executor created with 2 worker threads
let mut executor = Executor::new(2);
// Submit a few tasks
executor.spawn(|| {
println!("Task 1 running on thread: {:?}", std::thread::current().id());
std::thread::sleep(std::time::Duration::from_millis(100));
println!("Task 1 finished.");
});
executor.spawn(|| {
println!("Task 2 running on thread: {:?}", std::thread::current().id());
std::thread::sleep(std::time::Duration::from_millis(50));
println!("Task 2 finished.");
});
// Wait for tasks to complete and then shut down
executor.shutdown();
Expected Output (order of "running" messages may vary):
Task 1 running on thread: ThreadId(X)
Task 2 running on thread: ThreadId(Y)
Task 2 finished.
Task 1 finished.
(Note: X and Y will be actual thread IDs and may be the same or different depending on scheduling. The important part is that both tasks execute and finish.)
Example 2: Multiple Tasks on the Same Thread
// Executor with 1 worker thread
let mut executor = Executor::new(1);
// Submit several tasks that will be processed sequentially by the single thread
for i in 0..5 {
executor.spawn(move || {
println!("Task {} running.", i);
std::thread::sleep(std::time::Duration::from_millis(20));
println!("Task {} finished.", i);
});
}
// Shut down the executor
executor.shutdown();
Expected Output (tasks will execute in order):
Task 0 running.
Task 0 finished.
Task 1 running.
Task 1 finished.
Task 2 running.
Task 2 finished.
Task 3 running.
Task 3 finished.
Task 4 running.
Task 4 finished.
Example 3: Shutdown Behavior
// Executor with 2 worker threads
let mut executor = Executor::new(2);
// Submit a long-running task
executor.spawn(|| {
println!("Long task started.");
std::thread::sleep(std::time::Duration::from_secs(2));
println!("Long task finished.");
});
// Submit a short task
executor.spawn(|| {
println!("Short task started.");
std::thread::sleep(std::time::Duration::from_millis(50));
println!("Short task finished.");
});
// Initiate shutdown immediately after submitting tasks
// The shutdown should wait for the long task to complete.
executor.shutdown();
Expected Output (order of "started" may vary):
Long task started.
Short task started.
Short task finished.
Long task finished.
(Note: The shutdown call should block until both tasks, including the long one, are completed.)
Constraints
- The
Executormust be instantiated with a positive integer specifying the number of worker threads (e.g.,Executor::new(num_threads)wherenum_threads >= 1). - Tasks submitted via
spawnmust be closures that areSendand'static. - The
shutdownmethod must block until all submitted tasks have completed execution. - The executor should not panic if
spawnis called aftershutdownhas been initiated, though tasks submitted after initiation may not be processed.
Notes
- You will likely need to use standard Rust concurrency primitives such as
std::thread,std::sync::mpsc(for communication between threads),std::sync::Arc, andstd::sync::Mutex. - Consider how to signal worker threads to exit their loop when
shutdownis called. A common pattern involves using a shared flag or a special message in the communication channel. - Think about how to handle the case where a task panics during execution. For this challenge, you can let the panic propagate, but in a real-world executor, you'd want more robust error handling.
- The
Arc<Mutex<...>>pattern is very common for shared mutable state across threads. - Consider using
std::sync::mpsc::channelfor passing tasks to the worker threads. - When shutting down, ensure that worker threads don't deadlock waiting for tasks that will never arrive.