Skip to content

Comments

[feat] Support custom indexed and external references#3630

Open
vkarak wants to merge 2 commits intoreframe-hpc:developfrom
vkarak:feat/multikeyref
Open

[feat] Support custom indexed and external references#3630
vkarak wants to merge 2 commits intoreframe-hpc:developfrom
vkarak:feat/multikeyref

Conversation

@vkarak
Copy link
Contributor

@vkarak vkarak commented Feb 24, 2026

This PR extends the test's reference syntax to support arbitrary indexing on test attributes as well as other configuration parameters, such as the current partition, environment, processor and/or device details. If no special indexing is requested, the reference dictionary behaves exactly as before.

Support for external reference files is also added, where the reference dict of a test can be in an external YAML file.

Check the docs for all the details of the new features.

Implementation details

The test's prefix is now passed as a metaclass parameter and is not set during the test's initialization. This was needed to support the default external reference prefix, which coincides with the test's prefix, so we needed the test's prefix to be available during class creation and when the reference is defined in the test class body.

Closes #3463

@codecov
Copy link

codecov bot commented Feb 24, 2026

Codecov Report

❌ Patch coverage is 95.85492% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.45%. Comparing base (998bf08) to head (b0ac411).

Files with missing lines Patch % Lines
reframe/core/pipeline.py 95.83% 7 Missing ⚠️
reframe/core/meta.py 93.33% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3630      +/-   ##
===========================================
+ Coverage    91.38%   91.45%   +0.06%     
===========================================
  Files           62       62              
  Lines        13520    13675     +155     
===========================================
+ Hits         12355    12506     +151     
- Misses        1165     1169       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

Allow references to be indexed arbitrarily

1 participant