Go Concurrent File Downloader

This challenge focuses on implementing an asynchronous file downloader in Go. You will leverage Go's powerful concurrency primitives to download multiple files simultaneously, significantly improving performance compared to sequential downloading. This is a common and practical task in many applications that need to fetch resources from the web.

Problem Description

Your task is to create a Go program that can download multiple files from a given list of URLs concurrently. The program should accept a list of URLs and save each downloaded file to a specified directory.

Key Requirements:

• Concurrency: Implement the downloading process using goroutines to achieve asynchronous I/O.
• Error Handling: Gracefully handle potential errors during the download process (e.g., network issues, invalid URLs, file writing errors). For each failed download, report the URL and the error.
• Output Directory: Allow the user to specify a directory where all downloaded files should be saved. If the directory doesn't exist, it should be created.
• File Naming: Files should be named based on their original URL (e.g., if the URL is http://example.com/images/logo.png, the file should be saved as logo.png).
• Progress Reporting (Optional but Recommended): Implement a way to show the progress of downloads, either globally (total files downloaded) or per-file.

Expected Behavior:

The program should start multiple download goroutines, each responsible for downloading a single file. Once all downloads are initiated, the program should wait for all goroutines to complete. Upon completion, it should print a summary of any files that failed to download, along with the reason for failure.

Edge Cases:

• Empty list of URLs.
• Invalid URL formats.
• URLs pointing to non-existent files.
• Network connectivity issues.
• Permissions issues when creating the output directory or writing files.
• Very large files that might take a long time to download.

Examples

Example 1:

Input: URLs: ["http://example.com/file1.txt", "http://example.com/image.jpg"] Output Directory: ./downloads

Output (assuming successful downloads):

Successfully downloaded http://example.com/file1.txt to ./downloads/file1.txt
Successfully downloaded http://example.com/image.jpg to ./downloads/image.jpg

(No error messages would be printed if all downloads are successful)

Example 2:

Input: URLs: ["http://example.com/file1.txt", "http://invalid-url", "http://example.com/nonexistent.pdf"] Output Directory: ./my_files

Output (example, assuming first download succeeds, second fails due to invalid URL, third fails due to 404):

Successfully downloaded http://example.com/file1.txt to ./my_files/file1.txt
Failed to download http://invalid-url: invalid URL format
Failed to download http://example.com/nonexistent.pdf: received status code 404 Not Found

Example 3:

Input: URLs: [] Output Directory: ./empty_downloads

Output:

No URLs provided to download.

Constraints

• The number of URLs to download can range from 0 to 100.
• The output directory path will be a valid string.
• The program should be reasonably efficient, with download times not significantly exceeding sequential download times for a small number of files.
• Each individual file download should not exceed 5 minutes for the purpose of testing timeout scenarios (though your solution should handle longer downloads correctly).

Notes

• You'll likely need to use the net/http package for making HTTP requests and the os package for file operations.
• Consider using a sync.WaitGroup to ensure all goroutines have completed before the main function exits.
• For parsing the filename from a URL, the path.Base function from the path package can be helpful.
• Think about how to collect and report errors from multiple goroutines back to the main goroutine. Channels are a good candidate for this.
• For optional progress reporting, consider how to safely update a shared counter or display information without race conditions.