Memory Profiling Tool for Python Functions
This challenge asks you to build a Python decorator that measures the memory usage of functions. Memory profiling is crucial for identifying performance bottlenecks and optimizing resource consumption in Python applications, especially for data-intensive tasks or long-running processes.
Problem Description
You need to create a Python decorator named memory_profiler. This decorator will wrap around any Python function and record the peak memory usage of that function during its execution. The decorator should then return the peak memory usage in a human-readable format (e.g., KB, MB, GB).
Key Requirements:
- Decorator Functionality: Implement
memory_profileras a decorator that can be applied to any function. - Memory Measurement: The decorator must accurately measure the peak memory consumed by the decorated function while it is running.
- Return Value: The decorator should not alter the original return value of the decorated function. Instead, it should print the peak memory usage to the console.
- Human-Readable Output: The memory usage should be presented in a user-friendly format (e.g., "Peak Memory Usage: 1.25 MB").
- Compatibility: The solution should be compatible with standard Python installations.
Expected Behavior:
When a function decorated with @memory_profiler is called, it should execute normally, and after execution, a message indicating the peak memory usage during that execution should be printed to standard output.
Edge Cases:
- Functions that return large objects.
- Functions with recursive calls.
- Functions that perform I/O operations.
- Functions that consume very little memory.
Examples
Example 1: Simple Function
import time
@memory_profiler
def create_list(n):
"""Creates a list of n integers."""
data = [i for i in range(n)]
time.sleep(0.1) # Simulate some work
return data
# Call the function
my_list = create_list(1000000)
Expected Output (Memory usage will vary based on system):
Peak Memory Usage: 38.15 MB
Explanation: The create_list function allocates memory to store one million integers. The decorator measures the peak memory consumed during its execution and prints it. The function also returns the list as intended.
Example 2: Function with No Explicit Return
@memory_profiler
def print_message(msg):
"""Prints a message and allocates some temporary memory."""
temp_data = "a" * 10000000
print(msg)
del temp_data # Explicitly free memory to see effect
# Call the function
print_message("Hello, Memory Profiler!")
Expected Output (Memory usage will vary):
Hello, Memory Profiler!
Peak Memory Usage: 9.54 MB
Explanation: Even though print_message doesn't explicitly return a value, the decorator still measures the peak memory used by the temporary string temp_data.
Example 3: Function with Large Data Structures
import numpy as np
@memory_profiler
def create_large_array(rows, cols):
"""Creates a large NumPy array."""
array = np.random.rand(rows, cols)
return array
# Call the function
large_array = create_large_array(1000, 1000)
Expected Output (Memory usage will vary):
Peak Memory Usage: 7.63 MB
Explanation: This example demonstrates profiling a function that uses a third-party library (NumPy) to create a large data structure. The decorator will capture the memory allocation for this array.
Constraints
- The solution must be implemented purely in Python without relying on external profiling libraries like
memory_profilerorpsutil. You should use standard Python libraries. - The decorator should handle functions with any number of arguments (positional and keyword).
- The memory usage should be reported in bytes internally, then converted to KB, MB, or GB for display.
- The decorator should not introduce significant overhead to the function's execution time beyond the memory measurement itself.
Notes
- You will likely need to use modules from the
sysorosmodule to get memory information. - Consider how to access the peak memory usage, not just the memory usage at the start or end.
- Think about how to ensure the decorator correctly passes arguments to the decorated function and captures its return value.
- For human-readable output, you might want to define a helper function to convert bytes into KB, MB, or GB.