Fix: prevent fontconfig deadlock in forked processes#14
Merged
Conversation
Implements lazy fontconfig initialization to fix deadlock issues when using PixelRendererProcessor in forked worker processes (e.g., PyTorch DataLoader with fork multiprocessing). The issue occurred because fontconfig's internal mutexes were initialized in the parent process, then inherited by child processes during fork(). This caused child processes to deadlock when trying to acquire locks owned by threads that don't exist in the child. The fix ensures fontconfig is initialized independently in each process: - Parent process stores configuration only, never initializes fontconfig - Each child process initializes fontconfig on first render_text() call - Detects forks via PID tracking and re-initializes if needed Fixes #13 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add python3-dev dependency to GitHub Actions setup - Create apt cache directory before setting permissions - Increase DataLoader test timeouts from 3-5s to 10s for more reliable testing 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PixelRendererProcessorto prevent deadlock when using fork-based multiprocessingProblem
When using
PixelRendererProcessorwith PyTorch'sDataLoader(num_workers > 0) or TRL'spack_dataset(), processes would deadlock and hang indefinitely.Root Cause:
fork()creates a child process, it copies the mutex state but not the threadsReproduction: Double fork scenario (most common in practice)
PixelRendererProcessorin main process (fontconfig initialized)pack_dataset()which forks → First child inherits locked mutexesDataLoaderwithnum_workers>0→ Second fork inherits locked mutexesSolution
Implement lazy initialization of fontconfig:
render_text()callChanges
pixel_renderer/processor.py:__init___ensure_fontconfig_initialized()method called before renderingfontconfig_pathto property with lazy initializationtests/pixel_renderer/test_no_deadlock.py:test_no_deadlock_with_dataloader_single_fork: Basic single fork testtest_no_deadlock_with_dataloader_double_fork: Reproduces the issue scenario (main test)test_no_deadlock_with_dataloader_spawn: Validates spawn still works (baseline)Test Plan
pytest tests/pixel_renderer/test_no_deadlock.py -vReferences
Fixes #13
🤖 Generated with Claude Code