adapt detector config and detectors + write and adapt tests

[viktorbeck98]

This pull request introduces a major refactor and enhancement of the configuration system for detectors, focusing on improved support for event-based variable configurations, round-trip YAML compatibility, and programmatic config generation. It also updates the default pipeline YAML to use the new structure and adds utility methods for converting configs to and from dictionary/YAML format. Additionally, it introduces a helper for extracting configured variables from parsed logs and improves the internal structure and serialization of event and variable configs.

Key changes include:

Configuration System Refactor and Serialization:

• Introduced new classes (EventsConfig, _EventConfig, _EventInstance, Variable, Header) to represent event-based detector configurations, with to_dict methods for round-trip serialization to YAML-compatible dictionaries. (src/detectmatelibrary/common/_config/_formats.py [1] [2]
• Added a to_dict method to the main config class to enable converting configs back to YAML format, ensuring preservation of structure and compatibility. (src/detectmatelibrary/common/_config/__init__.py src/detectmatelibrary/common/_config/init.pyR41-R92)
• Updated the import/export interface in __init__.py to expose new config-related utilities and types. (src/detectmatelibrary/common/_config/__init__.py src/detectmatelibrary/common/_config/init.pyL1-R4)

Detector Configuration Generation and Processing:

• Added a generate_detector_config function to programmatically build detector configs from variable selections, supporting both single variables and variable combinations, and handling both positional and header variables. (src/detectmatelibrary/common/_config/_compile.py [1] [2]
• Refactored the internal config processing logic to support the new events structure and improved handling of missing or auto-configured parameters, including new warnings for edge cases. (src/detectmatelibrary/common/_config/_compile.py [1] [2]

YAML Pipeline Config Update:

• Updated the default pipeline YAML (config/pipeline_config_default.yaml) to use the new events-based structure for detectors, removing legacy log_variables and all_log_variables fields and updating parameters accordingly. (config/pipeline_config_default.yaml [1] [2] [3] [4]

Detector Utility Improvements:

• Added a get_configured_variables helper to extract only those variables from parsed logs that are defined in the detector's config, supporting both positional and header variables. (src/detectmatelibrary/common/detector.py [1] [2]
• Updated the detector run output to use lists for logIDs and extractedTimestamps instead of extending, aligning with expected output structure. (src/detectmatelibrary/common/detector.py src/detectmatelibrary/common/detector.pyL69-R107)

These changes collectively modernize and modularize the detector configuration system, making it more flexible, maintainable, and compatible with both code and YAML workflows.

---

Configuration System Refactor and Serialization

• Introduced new event-based config classes (EventsConfig, _EventConfig, _EventInstance, etc.) with to_dict methods for YAML round-trip compatibility. [1] [2]
• Added to_dict to the main config class for YAML serialization, and updated __all__ to expose new utilities. [1] [2]

Detector Configuration Generation and Processing

• Implemented generate_detector_config for programmatic config creation from variable selections, supporting both single and tuple variables. [1] [2]
• Refactored config processing logic to handle new events structure and improved parameter validation/warnings. [1] [2]

YAML Pipeline Config Update

• Updated pipeline_config_default.yaml to use events for detector variable definitions, removing legacy fields and updating parameters. [1] [2] [3] [4]

Detector Utility Improvements

• Added get_configured_variables to extract only configured variables from parser output, supporting both positional and header variables. [1] [2]
• Changed detector output to assign rather than extend logIDs and extractedTimestamps fields.

[ipmach]

@viktorbeck98 Looks good but you have a conflict merge issue :) solve it and I think the PR is ready

[ipmach]

@viktorbeck98 also.. I know you want to have the script comment it in the combo detector. But that doesnt look good in a production code :D

• I suggest that you create a folder in testing call smoke tests and add it there as a normal script