Problem:
When _process_delimited_file is registered as a custom handler via register_handler, it gets called with the 3-arg signature (file_path, spark, options) instead of the 4-arg signature (file_path, spark, options, metadata_path). This silently skips CSVW metadata discovery.
# file_processor.py — the dispatch logic:
if handler == _process_delimited_file:
result = handler(file_path, spark, options, metadata_path) # ← 4 args, CSVW works
else:
result = handler(file_path, spark, options) # ← 3 args, CSVW skipped
If a consumer registers the built-in delimited handler for a new extension:
from file_processing.delimited_files import _process_delimited_file
register_handler(".myext", _process_delimited_file)
The identity check handler == _process_delimited_file fails because get_handler returns the function from the registry, but the comparison is against the module-level reference — and the registered handler goes through the custom path (3-arg call), losing CSVW support.
Expected behaviour:
Registering _process_delimited_file (or any delimited handler) for a custom extension should not lose CSVW metadata discovery.
Acceptance criteria:
- [] Custom-registered handlers that support CSVW receive
metadata_path
- [] No silent loss of functionality when re-registering built-in handlers for new extensions
- [] Existing custom handlers with 3-arg signatures continue to work
Problem:
When
_process_delimited_fileis registered as a custom handler viaregister_handler, it gets called with the 3-arg signature(file_path, spark, options)instead of the 4-arg signature(file_path, spark, options, metadata_path). This silently skips CSVW metadata discovery.If a consumer registers the built-in delimited handler for a new extension:
The identity check
handler == _process_delimited_filefails becauseget_handlerreturns the function from the registry, but the comparison is against the module-level reference — and the registered handler goes through the custom path (3-arg call), losing CSVW support.Expected behaviour:
Registering
_process_delimited_file(or any delimited handler) for a custom extension should not lose CSVW metadata discovery.Acceptance criteria:
metadata_path