Skip to content

Code overview and cleanup #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
LHAMNS wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Code overview and cleanup #8

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions CODE_OVERVIEW.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# 关键函数和API关系概览

## main.py
- `main()` - 程序入口，解析命令行参数并调用 `run_server` 启动 Web 服务。
- `setup_enhanced_logging(level)` - 配置日志系统，同时记录 FFmpeg 日志。
- `check_dependencies()` - 检查 FFmpeg 及硬件加速（NVENC/QSV）。
- `check_environment()` - 创建并验证 `cache/` 与 `output/` 目录。

## converter 模块
- 全局常量定义于 `converter/__init__.py`，并导出核心类：
- `FrameGenerator`/`OptimizedFrameGenerator`
- `VideoEncoder`(别名 `DirectAVIEncoder`)、`StreamingVideoEncoder`、`BatchVideoEncoder`
- `ReedSolomonEncoder`
- `utils.py` 提供:
- `expand_pixels_9x1()`、`bytes_to_color_indices()`：Numba 加速的像素映射
- `CacheManager`：文件分块缓存管理
- `calculate_video_params()`：根据文件大小计算帧数等参数
- `verify_video_file()`：利用 FFmpeg/ffprobe 检查输出视频
- `is_nvenc_available()`、`is_qsv_available()`：检测硬件编码支持
- `frame_generator.py`:
- `FrameGenerator.generate_frames_from_data()` 生成帧序列，可回调进度
- `OptimizedFrameGenerator` 进一步利用预分配缓冲区提高性能
- `encoder.py`:
- `DirectAVIEncoder`/`StreamingDirectAVIEncoder`/`ParallelDirectAVIEncoder`
- `get_optimal_encoder()` 根据 CPU 核心数选择合适的编码器实现
- `decoder.py`:
- `VideoDecoder.extract_data()` 从视频还原字节流
- `ParallelVideoDecoder` 通过线程池并行解码
- `error_correction.py`:
- `ReedSolomonEncoder`、`XORInterleaver`、`HybridErrorCorrection`
- `gpu_error_correction.py`:
- `GPUReedSolomonEncoder` 使用 CUDA 进行 RS 编码
- `gpu_frame_generator.py`:
- `GPUFrameGenerator` 在 GPU 上生成帧
- `avi_writer.py`:
- `SimpleAVIWriter` 直接写入未压缩 AVI
- `convert_bytes_to_avi()` 使用给定帧生成器批量写入
- `pipeline.py`:
- `ConversionPipeline` 组织 `ErrorCorrectionStage`、`FrameGenerationStage`、`VideoEncodingStage`
- `video_raptor_encoder.py`:
- `VideoRaptorEncoder` 生成包含元数据和校准图的帧

## web_ui/server.py
- Flask + Socket.IO Web 服务，关键路由：
- `/api/upload` 上传文件并缓存
- `/api/start-conversion` 创建并启动 `ConversionTask`
- `/api/stop-conversion` 停止指定任务
- `/api/download/<task_id>` 下载输出文件
- `ConversionTask` 内部调用 `FrameGenerator`, `VideoEncoder`, `ReedSolomonEncoder` 等执行完整流程。

## 其他
- 测试位于 `tests/test_frame_generator.py`，验证帧生成逻辑。

39 changes: 0 additions & 39 deletions main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,43 +21,6 @@
from converter.utils import is_nvenc_available, is_qsv_available


def setup_enhanced_logging(level=logging.INFO):
"""
Set up enhanced logging with file handler and more detailed formatting
"""
# Create logs directory if it doesn't exist
log_dir = BASE_DIR / "logs"
log_dir.mkdir(exist_ok=True)

# Create log file with timestamp
log_file = log_dir / f"app_{time.strftime('%Y%m%d_%H%M%S')}.log"

# Configure log format
log_format = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
formatter = logging.Formatter(log_format)

# Set up file handler
file_handler = logging.FileHandler(log_file)
file_handler.setFormatter(formatter)

# Set up console handler with the same format
console_handler = logging.StreamHandler()
console_handler.setFormatter(formatter)

# Configure root logger
root_logger = logging.getLogger()
root_logger.setLevel(level)
root_logger.handlers = [] # Remove any existing handlers
root_logger.addHandler(file_handler)
root_logger.addHandler(console_handler)

# Log system information
logger = logging.getLogger(__name__)
logger.info(f"System: {sys.platform}")
logger.info(f"Python: {sys.version}")
logger.info(f"Log file: {log_file}")

return log_file


def check_dependencies():
Expand Down Expand Up @@ -195,8 +158,6 @@ def main():
except Exception as e:
logger.error(f"服务器运行错误: {e}", exc_info=True)

logger.info("文件到视频转换系统已关闭")
# Add this to setup_enhanced_logging in main.py

def setup_enhanced_logging(level=logging.INFO):
"""
Expand Down
3 changes: 0 additions & 3 deletions requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# Core dependencies
numpy>=1.21.0
numba>=0.55.1
opencv-python>=4.5.5.64
flask>=2.0.1
flask-socketio>=5.1.1
Pillow>=8.3.1
Expand All @@ -13,7 +11,6 @@ python-magic>=0.4.24
# GPU acceleration (optional but recommended)
# Install appropriate version based on CUDA version:
# For CUDA 12.x:
cupy-cuda12x>=12.0.0
# For CUDA 11.x (comment above and uncomment below):
# cupy-cuda11x>=11.0.0

Expand Down