152 lines
10 KiB
Markdown
152 lines
10 KiB
Markdown
Air2: High-Performance Download Engine
|
||
Air2 is a high-performance, multi-threaded download manager written in Python. It focuses on maximizing download speed through ultra-high concurrency, dynamic chunking strategies, and support for multiple protocols including HTTP/HTTPS, SFTP, and extensible architectures for BitTorrent. The tool provides both an interactive command-line interface (CLI) with modern visual feedback and a scriptable mode for automation.
|
||
Features
|
||
High Concurrency: Optimized thread pool supporting up to 200 concurrent workers to maximize bandwidth usage.
|
||
Dynamic Chunking: Automatically adjusts chunk sizes based on file size to optimize throughput and reduce overhead.
|
||
Multi-Protocol Support: Native support for HTTP/HTTPS and SFTP; architecture designed for easy addition of BitTorrent and other protocols.
|
||
Robust Resume Capability: Handles Range requests efficiently to ensure zero data loss during network interruptions.
|
||
Modern CLI Interface: Built with the rich library for real-time progress bars, speed meters, and ETA calculations without clutter.
|
||
Intelligent Input: Optional integration with prompt_toolkit for command auto-completion, history navigation, and syntax highlighting.
|
||
Task Management: Full lifecycle control including add, start, pause, resume, cancel, remove, and automatic ZIP extraction.
|
||
Speed Limiting: Configurable download speed limits to prevent network congestion.
|
||
Single File Deployment: Designed as a single executable script for easy distribution and deployment.
|
||
Installation
|
||
Ensure you have Python 3.8 or higher installed. Install the required dependencies:
|
||
bash
|
||
1
|
||
Note: prompt_toolkit is optional but recommended for enhanced command-line features like auto-completion.
|
||
Usage
|
||
Interactive Mode
|
||
Run the script directly to enter the interactive shell:
|
||
bash
|
||
1
|
||
Available Commands:
|
||
help: Display command list and help text.
|
||
add <URL> [options]: Add a new download task.
|
||
start <task_id>: Start a waiting task.
|
||
pause <task_id>: Pause a running task.
|
||
resume <task_id>: Resume a paused task.
|
||
cancel <task_id>: Cancel a task (keeps downloaded files).
|
||
remove <task_id> [--delete]: Remove a task and optionally delete the file.
|
||
unzip <task_id> [output_dir]: Extract completed ZIP files.
|
||
list / ls: Show all tasks and their status.
|
||
status: Display system statistics and overall progress.
|
||
exit / quit: Close the application.
|
||
Example:
|
||
text
|
||
1234
|
||
Command Line Mode (Scripting)
|
||
You can also run commands directly from the terminal without entering the interactive shell:
|
||
bash
|
||
1234567891011
|
||
Options for add Command
|
||
--threads=N: Number of concurrent threads (default: 8).
|
||
--output=DIR: Destination directory (default: current directory).
|
||
--chunk-size=S: Force specific chunk size (e.g., 10M, 1G). Defaults to dynamic calculation.
|
||
--speed-limit=N: Limit download speed in KB/s (0 for unlimited).
|
||
--timeout=N: Request timeout in seconds.
|
||
--retry=N: Number of retry attempts on failure.
|
||
Architecture
|
||
The codebase is structured into distinct modules for maintainability:
|
||
DownloadEngine: Manages the core logic, including task queues, thread pools, and state synchronization.
|
||
ChunkDownloader: Handles individual block retrieval, error recovery, and checksum verification.
|
||
ProtocolHandler: Abstract base class for different protocols (HTTP, SFTP, etc.), allowing easy extension.
|
||
ModernCLI: Provides the user interface layer using rich and prompt_toolkit.
|
||
Configuration
|
||
The engine automatically detects file sizes and adjusts chunk counts. For large files (>1GB), it increases chunk size to 16MB; for medium files (>100MB), it uses 4MB chunks. This dynamic approach minimizes connection overhead while maintaining high parallelism.
|
||
Users can override these defaults via command-line arguments if specific network conditions require manual tuning.
|
||
License
|
||
This project is licensed under the MIT License.
|
||
Requirements
|
||
Python 3.8+
|
||
requests
|
||
rich
|
||
urllib3
|
||
prompt_toolkit (optional)
|
||
README.md格式，中英混
|
||
|
||
Air2: High-Performance Download Engine
|
||
Air2 是一个基于 Python 开发的高性能多线程下载管理器。它专注于通过超高并发、动态分块策略以及多协议支持（HTTP/HTTPS, SFTP, BitTorrent）来最大化下载速度。该工具提供现代化的交互式命令行界面 (CLI) 和便于脚本自动化的命令行模式。
|
||
Features / 核心功能
|
||
High Concurrency / 超高并发: Optimized thread pool supporting up to 200 concurrent workers to maximize bandwidth usage. (优化的线程池，支持多达 200 个并发工作线程以最大化带宽利用率。)
|
||
Dynamic Chunking / 动态分块: Automatically adjusts chunk sizes based on file size to optimize throughput and reduce overhead. (根据文件大小自动调整分块大小，优化吞吐量并减少开销。)
|
||
Multi-Protocol Support / 多协议支持: Native support for HTTP/HTTPS and SFTP; architecture designed for easy addition of BitTorrent and other protocols. (原生支持 HTTP/HTTPS 和 SFTP；架构设计便于扩展 BitTorrent 及其他协议。)
|
||
Robust Resume Capability / 强大的断点续传: Handles Range requests efficiently to ensure zero data loss during network interruptions. (高效处理 Range 请求，确保网络中断时零数据丢失。)
|
||
Modern CLI Interface / 现代化 CLI 界面: Built with the rich library for real-time progress bars, speed meters, and ETA calculations without clutter. (基于 rich 库构建，提供实时进度条、速度计和预计剩余时间计算，界面简洁。)
|
||
Intelligent Input / 智能输入: Optional integration with prompt_toolkit for command auto-completion, history navigation, and syntax highlighting. (可选集成 prompt_toolkit，支持命令自动补全、历史记录导航和语法高亮。)
|
||
Task Management / 任务管理: Full lifecycle control including add, start, pause, resume, cancel, remove, and automatic ZIP extraction. (完整的全生命周期控制，包括添加、启动、暂停、恢复、取消、移除及自动 ZIP 解压。)
|
||
Speed Limiting / 限速控制: Configurable download speed limits to prevent network congestion. (可配置的下载速度限制，防止网络拥塞。)
|
||
Single File Deployment / 单文件部署: Designed as a single executable script for easy distribution and deployment. (设计为单执行脚本，便于分发和部署。)
|
||
Installation / 安装
|
||
Ensure you have Python 3.8 or higher installed. Install the required dependencies:
|
||
请确保已安装 Python 3.8 或更高版本。安装所需依赖：
|
||
bash
|
||
1
|
||
Note: prompt_toolkit is optional but recommended for enhanced command-line features like auto-completion.
|
||
(注：prompt_toolkit 为可选依赖，但推荐使用以获得自动补全等增强功能。)
|
||
Usage / 使用方法
|
||
Interactive Mode / 交互模式
|
||
Run the script directly to enter the interactive shell:
|
||
直接运行脚本进入交互式 Shell：
|
||
bash
|
||
1
|
||
Available Commands / 可用命令:
|
||
Command
|
||
Description / 描述
|
||
help
|
||
Display command list and help text. / 显示命令列表和帮助信息。
|
||
add <URL> [options]
|
||
Add a new download task. / 添加新的下载任务。
|
||
start <task_id>
|
||
Start a waiting task. / 开始等待中的任务。
|
||
pause <task_id>
|
||
Pause a running task. / 暂停正在运行的任务。
|
||
resume <task_id>
|
||
Resume a paused task. / 恢复暂停的任务。
|
||
cancel <task_id>
|
||
Cancel a task (keeps downloaded files). / 取消任务（保留已下载文件）。
|
||
remove <task_id> [--delete]
|
||
Remove a task and optionally delete the file. / 移除任务并可选择删除文件。
|
||
unzip <task_id> [output_dir]
|
||
Extract completed ZIP files. / 解压已完成的 ZIP 文件。
|
||
list / ls
|
||
Show all tasks and their status. / 显示所有任务及其状态。
|
||
status
|
||
Display system statistics and overall progress. / 显示系统统计数据和总体进度。
|
||
exit / quit
|
||
Close the application. / 关闭应用程序。
|
||
Example / 示例:
|
||
text
|
||
1234
|
||
Command Line Mode / 命令行模式 (Scripting)
|
||
You can also run commands directly from the terminal without entering the interactive shell:
|
||
您也可以直接在终端运行命令，无需进入交互式 Shell：
|
||
bash
|
||
1234567891011
|
||
Options for add Command / add 命令选项
|
||
--threads=N: Number of concurrent threads (default: 8). / 并发线程数（默认：8）。
|
||
--output=DIR: Destination directory (default: current directory). / 目标目录（默认：当前目录）。
|
||
--chunk-size=S: Force specific chunk size (e.g., 10M, 1G). Defaults to dynamic calculation. / 强制指定分块大小（例如 10M, 1G）。默认为动态计算。
|
||
--speed-limit=N: Limit download speed in KB/s (0 for unlimited). / 限制下载速度（KB/s），0 表示不限制。
|
||
--timeout=N: Request timeout in seconds. / 请求超时时间（秒）。
|
||
--retry=N: Number of retry attempts on failure. / 失败后的重试次数。
|
||
Architecture / 架构
|
||
The codebase is structured into distinct modules for maintainability:
|
||
代码库结构清晰，分为独立模块以便于维护：
|
||
DownloadEngine: Manages the core logic, including task queues, thread pools, and state synchronization. (管理核心逻辑，包括任务队列、线程池和状态同步。)
|
||
ChunkDownloader: Handles individual block retrieval, error recovery, and checksum verification. (处理单个块的检索、错误恢复和校验和验证。)
|
||
ProtocolHandler: Abstract base class for different protocols (HTTP, SFTP, etc.), allowing easy extension. (不同协议的抽象基类，便于扩展新协议。)
|
||
ModernCLI: Provides the user interface layer using rich and prompt_toolkit. (使用 rich 和 prompt_toolkit 提供用户界面层。)
|
||
Configuration / 配置
|
||
The engine automatically detects file sizes and adjusts chunk counts. For large files (>1GB), it increases chunk size to 16MB; for medium files (>100MB), it uses 4MB chunks. This dynamic approach minimizes connection overhead while maintaining high parallelism. Users can override these defaults via command-line arguments if specific network conditions require manual tuning.
|
||
引擎会自动检测文件大小并调整分块数量。对于大文件（>1GB），分块大小会增加到 16MB；对于中等文件（>100MB），使用 4MB 分块。这种动态方法在保持高并行度的同时最小化了连接开销。如果特定的网络条件需要手动调整，用户可以通过命令行参数覆盖这些默认值。
|
||
License / 许可
|
||
This project is licensed under the MIT License.
|
||
本项目采用 MIT 许可证授权。
|
||
Requirements / 环境要求
|
||
Python 3.8+
|
||
requests
|
||
rich
|
||
urllib3
|
||
prompt_toolkit (optional)
|