Python Service Discovery: The Microservice Navigator

In a microservices architecture, services need to find and communicate with each other dynamically. This challenge asks you to implement a basic service discovery mechanism in Python. You will build a system that allows services to register themselves and other services to query for available instances of a specific service. This is crucial for building resilient and scalable distributed systems.

Problem Description

Your task is to create a Python-based service discovery system. This system will consist of two main components:

Service Registry: A central component that maintains a list of registered services and their network addresses (e.g., IP address and port).
Service Clients: Components that can query the registry to find the network addresses of available service instances.

You will implement a simplified version of this. You do not need to build a full-fledged network service for the registry. Instead, you can simulate its behavior using in-memory data structures.

Key Requirements:

Service Registration: A service should be able to register itself with the discovery system, providing its name and its address (host and port).
Service De-registration: A service should be able to de-register itself.
Service Discovery: A client should be able to query the discovery system for all available instances of a given service name.
Heartbeat (Optional but Recommended): Implement a mechanism where registered services periodically "check in" with the registry to indicate they are still alive. If a service fails to check in within a certain timeout, it should be automatically de-registered.

Expected Behavior:

When a service registers, its address should be added to the registry associated with its service name.
When a client requests a service, the registry should return a list of addresses for all active instances of that service.
If a service is not registered or has been de-registered, the discovery system should return an empty list or an appropriate indicator.
If heartbeats are implemented, services that stop "checking in" should be removed from the registry.

Edge Cases to Consider:

Attempting to register a service with an already registered address (consider if this should overwrite or be an error).
Attempting to de-register a service that is not registered.
Querying for a service that has no registered instances.
Handling concurrent registrations and discoveries (though for this in-memory simulation, standard Python threading/concurrency primitives might suffice).

Examples

Example 1: Basic Registration and Discovery

# Simulating the Service Registry
registry = ServiceRegistry()

# Service A registers
registry.register_service("UserService", "localhost", 8080)
registry.register_service("UserService", "localhost", 8081)
registry.register_service("ProductService", "192.168.1.10", 9000)

# Client queries for UserService
user_services = registry.discover_service("UserService")
print(user_services)

# Client queries for ProductService
product_services = registry.discover_service("ProductService")
print(product_services)

# Client queries for a non-existent service
auth_services = registry.discover_service("AuthService")
print(auth_services)

Output:
[('localhost', 8080), ('localhost', 8081)]
[('192.168.1.10', 9000)]
[]

Explanation:

The UserService was registered twice with different ports. The discovery system correctly returns both instances. The ProductService has one instance. AuthService has no registered instances, so an empty list is returned.

Example 2: De-registration

# Continuing from Example 1
registry = ServiceRegistry()
registry.register_service("UserService", "localhost", 8080)
registry.register_service("UserService", "localhost", 8081)

print("Before de-registration:", registry.discover_service("UserService"))

# De-register one instance of UserService
registry.deregister_service("UserService", "localhost", 8080)

print("After de-registration:", registry.discover_service("UserService"))

# Attempt to de-register a non-existent instance
registry.deregister_service("UserService", "localhost", 8082) # No error, no change
print("After invalid de-registration:", registry.discover_service("UserService"))

Output:
Before de-registration: [('localhost', 8080), ('localhost', 8081)]
After de-registration: [('localhost', 8081)]
After invalid de-registration: [('localhost', 8081)]

Explanation:

One instance of UserService was successfully de-registered. Attempting to de-register an instance that wasn't registered had no effect.

Example 3: Heartbeat Simulation (if implemented)

# Assuming a ServiceRegistry with heartbeat functionality
registry = ServiceRegistry(heartbeat_timeout=5) # 5 seconds timeout

# Service A registers and sends a heartbeat
registry.register_service("WorkerService", "10.0.0.5", 7000)
registry.record_heartbeat("WorkerService", "10.0.0.5", 7000)

print("Services before timeout:", registry.discover_service("WorkerService"))

# Simulate time passing without a heartbeat
import time
time.sleep(6)

# The service should be automatically de-registered
print("Services after timeout:", registry.discover_service("WorkerService"))

# Another instance registers and sends heartbeats
registry.register_service("WorkerService", "10.0.0.6", 7001)
registry.record_heartbeat("WorkerService", "10.0.0.6", 7001)
time.sleep(1) # Short delay
registry.record_heartbeat("WorkerService", "10.0.0.6", 7001) # Another heartbeat

print("Services after new registration:", registry.discover_service("WorkerService"))

Output:
Services before timeout: [('10.0.0.5', 7000)]
Services after timeout: []
Services after new registration: [('10.0.0.6', 7001)]

Explanation:

The first WorkerService instance timed out because it didn't send a heartbeat. The second instance, which continuously sent heartbeats, remained registered.

Constraints

The service registry should be implemented in a single Python module.
For simplicity, assume the registry uses an in-memory dictionary or similar data structure.
Network addresses will be represented as tuples of (host: str, port: int).
If implementing heartbeats, the timeout duration should be configurable and measured in seconds.
The solution should be thread-safe if concurrency is a concern (e.g., using threading.Lock).

Notes

Consider how you will store the service information. A dictionary where keys are service names and values are lists of addresses is a good starting point.
For heartbeats, you'll need a way to track the last seen time for each service instance. A background thread or periodic checks within the ServiceRegistry class could manage the timeouts.
Think about the data structures needed to support efficient registration, de-registration, and discovery.
This challenge focuses on the core logic of service discovery. In a real-world scenario, you would likely use a dedicated service discovery tool like Consul, etcd, or ZooKeeper.

Python Service Discovery: The Microservice Navigator

Problem Description

Your task is to create a Python-based service discovery system. This system will consist of two main components:

Service Registry: A central component that maintains a list of registered services and their network addresses (e.g., IP address and port).
Service Clients: Components that can query the registry to find the network addresses of available service instances.

You will implement a simplified version of this. You do not need to build a full-fledged network service for the registry. Instead, you can simulate its behavior using in-memory data structures.

Key Requirements:

Service Registration: A service should be able to register itself with the discovery system, providing its name and its address (host and port).
Service De-registration: A service should be able to de-register itself.
Service Discovery: A client should be able to query the discovery system for all available instances of a given service name.
Heartbeat (Optional but Recommended): Implement a mechanism where registered services periodically "check in" with the registry to indicate they are still alive. If a service fails to check in within a certain timeout, it should be automatically de-registered.

Expected Behavior:

When a service registers, its address should be added to the registry associated with its service name.
When a client requests a service, the registry should return a list of addresses for all active instances of that service.
If a service is not registered or has been de-registered, the discovery system should return an empty list or an appropriate indicator.
If heartbeats are implemented, services that stop "checking in" should be removed from the registry.

Edge Cases to Consider:

Attempting to register a service with an already registered address (consider if this should overwrite or be an error).
Attempting to de-register a service that is not registered.
Querying for a service that has no registered instances.
Handling concurrent registrations and discoveries (though for this in-memory simulation, standard Python threading/concurrency primitives might suffice).

Examples

Example 1: Basic Registration and Discovery

# Simulating the Service Registry
registry = ServiceRegistry()

# Service A registers
registry.register_service("UserService", "localhost", 8080)
registry.register_service("UserService", "localhost", 8081)
registry.register_service("ProductService", "192.168.1.10", 9000)

# Client queries for UserService
user_services = registry.discover_service("UserService")
print(user_services)

# Client queries for ProductService
product_services = registry.discover_service("ProductService")
print(product_services)

# Client queries for a non-existent service
auth_services = registry.discover_service("AuthService")
print(auth_services)

Output:
[('localhost', 8080), ('localhost', 8081)]
[('192.168.1.10', 9000)]
[]

Explanation:

Example 2: De-registration

# Continuing from Example 1
registry = ServiceRegistry()
registry.register_service("UserService", "localhost", 8080)
registry.register_service("UserService", "localhost", 8081)

print("Before de-registration:", registry.discover_service("UserService"))

# De-register one instance of UserService
registry.deregister_service("UserService", "localhost", 8080)

print("After de-registration:", registry.discover_service("UserService"))

# Attempt to de-register a non-existent instance
registry.deregister_service("UserService", "localhost", 8082) # No error, no change
print("After invalid de-registration:", registry.discover_service("UserService"))

Output:
Before de-registration: [('localhost', 8080), ('localhost', 8081)]
After de-registration: [('localhost', 8081)]
After invalid de-registration: [('localhost', 8081)]

Explanation:

One instance of UserService was successfully de-registered. Attempting to de-register an instance that wasn't registered had no effect.

Example 3: Heartbeat Simulation (if implemented)

# Assuming a ServiceRegistry with heartbeat functionality
registry = ServiceRegistry(heartbeat_timeout=5) # 5 seconds timeout

# Service A registers and sends a heartbeat
registry.register_service("WorkerService", "10.0.0.5", 7000)
registry.record_heartbeat("WorkerService", "10.0.0.5", 7000)

print("Services before timeout:", registry.discover_service("WorkerService"))

# Simulate time passing without a heartbeat
import time
time.sleep(6)

# The service should be automatically de-registered
print("Services after timeout:", registry.discover_service("WorkerService"))

# Another instance registers and sends heartbeats
registry.register_service("WorkerService", "10.0.0.6", 7001)
registry.record_heartbeat("WorkerService", "10.0.0.6", 7001)
time.sleep(1) # Short delay
registry.record_heartbeat("WorkerService", "10.0.0.6", 7001) # Another heartbeat

print("Services after new registration:", registry.discover_service("WorkerService"))

Output:
Services before timeout: [('10.0.0.5', 7000)]
Services after timeout: []
Services after new registration: [('10.0.0.6', 7001)]

Explanation:

The first WorkerService instance timed out because it didn't send a heartbeat. The second instance, which continuously sent heartbeats, remained registered.

Constraints

The service registry should be implemented in a single Python module.
For simplicity, assume the registry uses an in-memory dictionary or similar data structure.
Network addresses will be represented as tuples of (host: str, port: int).
If implementing heartbeats, the timeout duration should be configurable and measured in seconds.
The solution should be thread-safe if concurrency is a concern (e.g., using threading.Lock).

Notes

Consider how you will store the service information. A dictionary where keys are service names and values are lists of addresses is a good starting point.
For heartbeats, you'll need a way to track the last seen time for each service instance. A background thread or periodic checks within the ServiceRegistry class could manage the timeouts.
Think about the data structures needed to support efficient registration, de-registration, and discovery.
This challenge focuses on the core logic of service discovery. In a real-world scenario, you would likely use a dedicated service discovery tool like Consul, etcd, or ZooKeeper.