Robust Audit Logging System in Python
Audit logging is crucial for tracking changes and activities within a system, enabling debugging, security analysis, and compliance. This challenge asks you to design and implement a flexible audit logging system in Python that records events with relevant details, allowing for filtering and analysis. The system should be extensible to handle different types of events and log levels.
Problem Description
You are tasked with creating a Python-based audit logging system. The system should be able to record events with the following information:
- Timestamp: The date and time the event occurred.
- Event Type: A string describing the type of event (e.g., "user_login", "data_update", "file_access").
- User: The user associated with the event (can be None if not applicable).
- Details: A dictionary containing additional details specific to the event. This could include things like the data that was updated, the file that was accessed, or the IP address of the user.
- Log Level: A string representing the severity of the event (e.g., "INFO", "WARNING", "ERROR").
The system should provide the following functionalities:
- Log Event: A function to record a new event with the specified information.
- Retrieve Logs: A function to retrieve logs based on various criteria (e.g., event type, user, log level, date range). The retrieval should return a list of events matching the criteria.
- Persistence: The logs should be stored persistently to a file (e.g., a JSON file). The system should be able to load logs from the file on startup and save logs to the file after each new event is logged.
Key Requirements:
- The system should be modular and extensible, allowing for easy addition of new event types and log levels.
- The log retrieval function should support multiple filtering criteria.
- Error handling should be implemented to gracefully handle potential issues like file access errors.
- The system should be reasonably efficient for storing and retrieving logs.
Expected Behavior:
- When a new event is logged, it should be appended to the persistent storage.
- When logs are retrieved, they should be returned in chronological order (most recent first).
- The system should handle cases where no logs match the specified criteria.
- The system should handle cases where the log file does not exist.
Edge Cases to Consider:
- Empty details dictionary.
- Invalid log levels.
- Large number of logs.
- File corruption.
- Concurrent access to the log file (consider thread safety if applicable, though not strictly required for this challenge).
Examples
Example 1:
Input: log_event(timestamp="2023-10-27 10:00:00", event_type="user_login", user="john.doe", details={"ip_address": "192.168.1.100"}, log_level="INFO")
Output: (No direct output, but the log is saved to the file)
Explanation: A new user login event is recorded with the provided details.
Example 2:
Input: retrieve_logs(event_type="data_update", user="jane.doe")
Output: [
{'timestamp': '2023-10-27 10:15:00', 'event_type': 'data_update', 'user': 'jane.doe', 'details': {'field': 'name', 'old_value': 'Old Name', 'new_value': 'New Name'}, 'log_level': 'INFO'}
]
Explanation: Retrieves all data_update events associated with user jane.doe.
Example 3:
Input: retrieve_logs(event_type="nonexistent_event")
Output: []
Explanation: Returns an empty list because there are no events of type "nonexistent_event".
Constraints
- The log file should be stored in a file named "audit.log" in the same directory as the Python script.
- The log file should be in JSON format.
- The timestamp should be a string in the format "YYYY-MM-DD HH:MM:SS".
- The log level should be one of: "INFO", "WARNING", "ERROR".
- The system should be able to handle at least 1000 log entries without significant performance degradation.
Notes
- Consider using the
jsonmodule for handling JSON data. - Think about how to structure your code to make it easy to add new event types and log levels in the future.
- You can use a simple list to store the logs in memory before writing them to the file.
- Error handling is important – consider what might go wrong and how to handle it gracefully.
- While thread safety isn't a strict requirement, thinking about it can lead to a more robust design. If you choose to address it, use appropriate locking mechanisms.