Skip to content

Custom-registered handlers bypass CSVW metadata discovery #8

Description

@Iwan-Dyke

Problem:
When _process_delimited_file is registered as a custom handler via register_handler, it gets called with the 3-arg signature (file_path, spark, options) instead of the 4-arg signature (file_path, spark, options, metadata_path). This silently skips CSVW metadata discovery.

# file_processor.py — the dispatch logic:
if handler == _process_delimited_file:
    result = handler(file_path, spark, options, metadata_path)  # ← 4 args, CSVW works
else:
    result = handler(file_path, spark, options)  # ← 3 args, CSVW skipped

If a consumer registers the built-in delimited handler for a new extension:

from file_processing.delimited_files import _process_delimited_file
register_handler(".myext", _process_delimited_file)

The identity check handler == _process_delimited_file fails because get_handler returns the function from the registry, but the comparison is against the module-level reference — and the registered handler goes through the custom path (3-arg call), losing CSVW support.

Expected behaviour:
Registering _process_delimited_file (or any delimited handler) for a custom extension should not lose CSVW metadata discovery.

Acceptance criteria:

  • [] Custom-registered handlers that support CSVW receive metadata_path
  • [] No silent loss of functionality when re-registering built-in handlers for new extensions
  • [] Existing custom handlers with 3-arg signatures continue to work

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions