Skip to content

refactor: decompose monolithic MEA pipeline scripts into focused subpackages#155

Draft
mandarmp with Copilot wants to merge 3 commits into
mainfrom
copilot/propose-modularity-improvements
Draft

refactor: decompose monolithic MEA pipeline scripts into focused subpackages#155
mandarmp with Copilot wants to merge 3 commits into
mainfrom
copilot/propose-modularity-improvements

Conversation

Copilot AI commented Mar 13, 2026

Copy link
Copy Markdown
Contributor

mea_analysis_routine.py (1,783 lines) and run_pipeline_driver.py (385 lines) were single-file monoliths with no internal modularity, making navigation, testing, and extension painful.

New IPNAnalysis/pipeline/ package

Each pipeline phase is now a dedicated mixin; MEAPipeline in core.py assembles them via Python MRO:

Module Responsibility
stages.py ProcessingStage enum, NpEncoder, schema version
defaults.py Default kwarg factory functions
_setup_mixin.py Logger, runtime controls, metadata parsing
_checkpoint_mixin.py Checkpoint load / save / skip
_preprocessing_mixin.py Phase 1: filter chain, binary save
_sorting_mixin.py Phase 2: Kilosort4 + spike-detection-only path
_merge_mixin.py Phase 2.5: UnitMatch + SI auto-merge
_analyzer_mixin.py Phase 3: SortingAnalyzer + extensions
_reports_mixin.py Phase 4: curation, probe/waveform plots
_burst_mixin.py Network burst analysis, raster sort
core.py MEAPipeline (mixin assembly)
options.py MEARunOptions / MEARunResult frozen dataclasses
runner.py run_mea_pipeline() + stage-rewind helpers
cli.py Argument parser + main()

New IPNAnalysis/driver/ package

run_pipeline_driver.py split into file_scanner.py (H5/NWB/RAW discovery), launcher.py (subprocess + logger), and cli.py (argument parser + main()).

Backward compatibility

Both original files are kept as thin re-export facades — zero breaking changes for existing callers:

# Still works unchanged
from mea_analysis_routine import MEAPipeline, run_mea_pipeline, MEARunOptions

setup.py updated to register the two new subpackages.

Bug fix

--unitmatch-oversplit-min-probability CLI help text incorrectly advertised 0.80; corrected to 0.99 to match the actual default in defaults.py.


🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

Copilot AI and others added 2 commits March 13, 2026 20:21
…ular pipeline/ and driver/ packages

Co-authored-by: mandarmp <33731455+mandarmp@users.noreply.github.com>
…ult (0.99)

Co-authored-by: mandarmp <33731455+mandarmp@users.noreply.github.com>
Copilot AI changed the title [WIP] Propose better architecture for analysis routines refactor: decompose monolithic MEA pipeline scripts into focused subpackages Mar 13, 2026
Copilot AI requested a review from mandarmp March 13, 2026 20:25
@mandarmp mandarmp force-pushed the copilot/propose-modularity-improvements branch from 702c47d to f7e6f43 Compare April 28, 2026 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants