Skip to content

ENH: Add OptimizedPreprocessing algorithm from Calmon et al. (fixes #9) - #12

Open
divyanshbhai wants to merge 4 commits into
usarfoss:mainfrom
divyanshbhai:fix-issue-9
Open

ENH: Add OptimizedPreprocessing algorithm from Calmon et al. (fixes #9)#12
divyanshbhai wants to merge 4 commits into
usarfoss:mainfrom
divyanshbhai:fix-issue-9

Conversation

@divyanshbhai

@divyanshbhai divyanshbhai commented Oct 15, 2025

Copy link
Copy Markdown

Summary

Implements the preprocessing algorithm from "Optimized Pre-Processing for Discrimination Prevention" by Calmon et al. as requested in issue #9.

Changes

  • Algorithm Implementation: Added OptimizedPreprocessing class in fairlearn.preprocessing
  • Core Features: Probabilistic transformation for demographic parity with configurable epsilon parameter
  • Tests: Comprehensive unit tests in test.unit.preprocessing.test_optimized_preprocessing
  • Documentation: Added detailed section in user guide with mathematical formulation and usage examples
  • API Reference: Complete docstring with parameter descriptions and references

Implementation Details

  • Follows sklearn TransformerMixin interface with fit/transform methods
  • Solves constrained optimization problem to minimize distortion while ensuring fairness
  • Supports binary sensitive features and multi-class targets
  • Includes parameter validation and comprehensive error handling

Testing

  • 15+ unit tests covering functionality, edge cases, and error conditions
  • Parameter validation, input validation, and sklearn compatibility
  • Reproducibility tests with random state control

Documentation

  • Mathematical formulation in user guide
  • Practical usage examples
  • Added Calmon et al. reference to bibliography

Fix #9

Divyansh added 4 commits October 15, 2025 19:11
Divyansh Divyansh
Add basic OptimizedPreprocessing class structure
aec5844 
- Create _optimized_preprocessing.py with class skeleton
- Add OptimizedPreprocessing to preprocessing module exports
- Implement basic fit/transform interface following sklearn conventions
- Add comprehensive docstring with algorithm description and references

Part of implementing Calmon et al. algorithm for issue usarfoss#9
Divyansh Divyansh
Implement core optimization algorithm for OptimizedPreprocessing
468a7d9 
- Add probabilistic transformation learning via constrained optimization
- Implement feature discretization for tractable optimization
- Add demographic parity constraints with epsilon tolerance
- Implement distortion minimization objective function
- Add probabilistic data transformation with noise injection

Follows Calmon et al. methodology for discrimination prevention
Divyansh Divyansh
Add comprehensive unit tests for OptimizedPreprocessing
e195f4c 
- Create test_optimized_preprocessing.py with 15+ test cases
- Test initialization, fit/transform functionality, and edge cases
- Add parameter validation and input validation
- Test reproducibility, different epsilon values, and sklearn compatibility
- Verify error handling for invalid inputs and parameters

Ensures robust implementation following fairlearn testing standards
Divyansh Divyansh
Add comprehensive documentation for OptimizedPreprocessing
69489c8 
- Add detailed section in preprocessing.rst user guide
- Include mathematical formulation and algorithm description
- Add practical usage example with synthetic data
- Include Calmon et al. reference in bibliography
- Document fairness-utility trade-off via epsilon parameter

Completes documentation requirements for issue usarfoss#9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

ENH Add mitigation algorithm from "Optimized Pre-Processing for Discrimination Prevention" by Calmon et al.

1 participant

@divyanshbhai