Building a Simple Service Discovery System in Go

In distributed systems, services often need to find and communicate with each other. Manually configuring network addresses for every service can be brittle and difficult to manage as services scale or move. Service discovery addresses this by providing a mechanism for services to register themselves and for other services to query for their locations. This challenge will guide you in building a basic service discovery system in Go.

Problem Description

Your task is to implement a simple service discovery system that allows services to register their network addresses and enables clients to discover these addresses. The system should be able to handle multiple instances of the same service and provide up-to-date information.

Key Requirements:

1. Service Registration: A "service" should be able to register itself with the discovery system, providing its name and its network address (e.g., host:port).
2. Service Discovery: A "client" should be able to query the discovery system for a specific service name and receive a list of available network addresses for that service.
3. Heartbeats/Health Checks (Optional but Recommended): Implement a mechanism for services to periodically send heartbeats to the discovery system to indicate they are still alive. The discovery system should automatically remove services that stop sending heartbeats within a certain grace period.
4. Data Storage: The discovery system needs a way to store the registered service information. For this challenge, an in-memory store is sufficient.
5. API: Expose a simple API (e.g., HTTP endpoints) for registration and discovery.

Expected Behavior:

• When a service registers, its address should be stored under its service name.
• When a client queries for a service, it should receive all registered addresses for that service.
• If a service fails to send heartbeats, it should eventually be removed from the discovery system and no longer be discoverable.

Edge Cases to Consider:

• Registering the same service name multiple times with different addresses.
• Querying for a service that has not been registered.
• Network partitions (though for this in-memory version, this is less of a concern for client-server interaction within a single process, but good to keep in mind for distributed scenarios).

Examples

Example 1: Basic Registration and Discovery

• Service A registers:

  • Service Name: auth-service
  • Address: localhost:8080

• Service B registers:

  • Service Name: user-service
  • Address: localhost:8081

• Service A registers again:

  • Service Name: auth-service
  • Address: localhost:8082

• Client queries for auth-service:

  • Expected Output: A list containing localhost:8080 and localhost:8082.

• Client queries for user-service:

  • Expected Output: A list containing localhost:8081.

• Client queries for payment-service:

  • Expected Output: An empty list or an indication that the service was not found.

Example 2: Heartbeat and Service Removal

Assume a heartbeat interval of 10 seconds and a grace period of 20 seconds.

• Service C registers:
  • Service Name: data-service
  • Address: localhost:8083
• Service C sends heartbeats at t=0s, t=10s, t=20s.
• Service C stops sending heartbeats.
• At t=30s, a client queries for data-service.
  • Expected Output: An empty list or indication of not found, as Service C has been removed due to inactivity.

Constraints

• The service discovery system and the services/clients can all run within the same Go program for this challenge.
• The registration and discovery API should be accessible via HTTP.
• The in-memory data store should be thread-safe.
• Heartbeat interval and grace period should be configurable.

Notes

• Consider using Go's net/http package for building the API.
• For managing concurrent access to the in-memory store, consider using sync.Mutex or other synchronization primitives.
• For heartbeats, you might want to use Go routines and time.Ticker.
• Think about how you will represent service data (e.g., a struct with service name, address, and last heartbeat timestamp).
• A good starting point for the API endpoints could be:
  • POST /register: To register a service.
  • GET /discover/{service_name}: To discover services.
  • POST /heartbeat/{service_name}/{address}: For services to send heartbeats. (Alternatively, heartbeats could be part of the registration endpoint or a separate background process).