Feature/ndit 821 address ddb dc performance issues#41

Open

georgeRobertson wants to merge 9 commits intodevelop_v06from

feature/ndit-821_address_ddb_dc_performance_issues

Commits on Jan 26, 2026

feat: Change how error messages are generated (by writing in batches). Duckdb no long relies on pandas- use of pyarrow, multiprocessing and background thread batch writing to avoid memory pressure
stevenhsd
committed

Commits on Jan 27, 2026

style: run black, isort and resolve linting issues
stevenhsd
committed

Commits on Jan 28, 2026

Commits on Feb 3, 2026

refactor: merge in main and resolve conflicts and linting issues
stevenhsd
committed

Commits on Feb 6, 2026

Commits on Feb 10, 2026

refactor: merging develop v06
stevenhsd
committed