Skip to content

Implement two-stage test execution with resolved dependencies - #20

Draft
Subramanian-K812 wants to merge 33 commits into
mainfrom
Subramanian-K812_test_execution_upgrade
Draft

Implement two-stage test execution with resolved dependencies#20
Subramanian-K812 wants to merge 33 commits into
mainfrom
Subramanian-K812_test_execution_upgrade

Conversation

@Subramanian-K812

@Subramanian-K812 Subramanian-K812 commented Jul 1, 2026

Copy link
Copy Markdown

DR-008 Option 4: two-stage test-execution workflow (PR 2 of 2)

Closes eclipse-score#264. This is the second of two PRs. It is stacked on PR 1
(Subramanian-K812_resolve_override_mechanism, "resolved-dependency resolve + override
mechanism") — please review/merge that one first. Until PR 1 merges, this PR's base branch
is the PR-1 branch, so the diff shown here is only the test-execution delta.

What this PR does

Rewires the quality workflow to the DR-008 Option 4 flow, using the resolve + override
mechanism added in PR 1:

  1. Stage 1 — integration: builds the platform, runs Feature Integration Tests, and
    exports the resolved dependency set as the stage1-resolved-deps artifact
    (resolved_versions.json).
  2. prepare_matrix: derives the Stage-2 module list dynamically from
    known_good.json (target_sw) — never hardcoded.
  3. Stage 2 — per module: checks the module out at its known_good commit, overrides
    its MODULE.bazel with the Stage-1 resolved set (PR 1's mechanism), and runs the
    module's own unit tests + coverage as the Bazel root (//...).
  4. aggregate: consolidates Stage 1 + Stage 2 into one quality report (and the
    release-tag test-report ZIP).

Files

Workflow & runners

  • .github/workflows/test_and_docs.yml — two-stage restructure (+ parallel docs build)
  • scripts/quality_runners.py — module-context mode (--module-dir / --resolved-deps); calls ResolvedDependencies.overwrite
  • scripts/aggregate_quality_report.py — new, consolidated report
  • scripts/known_good/list_modules.py — new, dynamic Stage-2 matrix from known_good.json
  • scripts/integration_test.py — derive build targets from known_good.json (drop build_config.json)

Resolved pins + their config ripple (kept together so the tree is self-consistent)

  • known_good.json — pin bumps (baselibs, lifecycle) + per-module bazel_config + lifecycle code_root_path/extra_test_config
  • bazel_common/score_basic_bazel.MODULE.bazel — flatbuffers bump
  • bazel_common/score_modules_target_sw.MODULE.bazel — regenerated (baselibs, lifecycle)
  • MODULE.bazel.lock — updated lock
  • rust_coverage/BUILD — lifecycle query //src/... → //score/... (layout moved by the bump)
  • showcases/simple_lifecycle/BUILD — lifecycle bin path //src/... → //score/launch_manager
  • .bazelrc — coverage atomic-gcov / -no-coverage; eb-aarch64 outline-atomics link fix

Known Stage 2 integration findings (surfaced, not fixed here)

The workflow correctly surfaces two module-owned integration failures caused by the
baselibs bump (score_logging CheckSizeValid stale uint8_t assumption;
score_persistency to_string on std::string_view).

@Subramanian-K812
Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch from c9462ea to fd3df76 Compare July 7, 2026 08:49
@PiotrKorkus
Release notes from v0.8 on main (eclipse-score#297)
cbf818a 
* include release notes from v0.8 to main

* update overall-status skill refs
@Subramanian-K812
Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch 2 times, most recently from d293277 to b6df34d Compare July 24, 2026 06:56
@Subramanian-K812
Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch 2 times, most recently from e51b340 to 640dfee Compare August 3, 2026 03:26
@Subramanian-K812
Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch from b960a5a to a1b3db2 Compare August 6, 2026 13:14
@Subramanian-K812
Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch from 46125ba to 15090a8 Compare August 9, 2026 14:36
@Subramanian-K812
Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch from 75e4c57 to cb3fbcc Compare August 10, 2026 07:26
@Subramanian-K812
Subramanian-K812 changed the base branch from main to Subramanian-K812_resolve_override_mechanism August 10, 2026 08:59
@Subramanian-K812
Subramanian-K812 changed the base branch from Subramanian-K812_resolve_override_mechanism to main August 11, 2026 10:15
@Subramanian-K812
Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch from 04a7a73 to e816620 Compare August 11, 2026 10:22
@Chahult
feat(deps): add score_config_management to reference integration (ecl…
9143efa 
…ipse-score#299)

* feat(deps): add score_config_management to reference integration

Pin score_config_management to 49f26b74 and add a patch (sourced from
config_management PR#37) fixing the ProxyFuture API migration and
removing SOCAL/component_tests targets not present in the OSS build.

* ci: re-trigger CI

* ci: re-trigger CI

* feat: add config_management showcase

* docs: mark config_management showcase as non functional yet

* ci: re-trigger CI
@Subramanian-K812
Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch from 1a9a26e to 913bc97 Compare August 18, 2026 11:02
@Subramanian-K812
Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch 2 times, most recently from b22c037 to a992474 Compare August 21, 2026 09:58
antonkri and others added 5 commits August 21, 2026 16:10
@antonkri
chore(deps): bump score_itf to v0.5.0 (eclipse-score#307)
8ec9f3d 
* chore(deps): bump score_itf to v0.5.0

Update score_itf commit from bf5294d (v0.3.0) to 802e018 (v0.5.0)
in known_good.json and regenerate the derived files via
scripts/known_good/update_module_from_known_good.py.

The consumed itf API (py_itf_test, plugins=, @score_itf//score/itf/plugins:*)
is already on the 0.2+ layout, so no BUILD or Python changes are required.

* feature_integration_tests/itf: pin dlt_config to container gateway

score_itf 0.5.0 (commit bb3a788 "docker: Create per-container bridge
network") makes the Docker plugin create a dedicated bridge network
score_itf_<rand> per container instead of joining the default docker0
bridge. As a consequence the hardcoded 172.17.0.1 host_ip/target_ip in
feature_integration_tests/configs/dlt_config_x86_64.json no longer
matches the container's actual gateway, so DLT UDP multicast (239.255.
42.99) is never delivered to the host-side dlt-receive and the two DLT
tests fail with 'DLT TRACE FILE IS EMPTY'.

Add a local conftest.py that overrides the session-scoped dlt_config
fixture and pins host_ip/target_ip to target.get_gateway() when the
target exposes it (Docker). QEMU targets (QNX) do not expose
get_gateway; the static IPs from dlt_config_qnx_x86_64.json continue
to be used unchanged there.

Wire the new conftest.py into the py_itf_test data so it is present in
the Bazel test sandbox next to the test sources.

Refs: eclipse-score#307
@Subramanian-K812
Add resolved-dependency override mechanism for DR-008 (eclipse-score#278
535763f 
)
@Subramanian-K812
feat: two-stage DR-008 test workflow with per-module Stage 2 execution
4624b81
@Subramanian-K812
ci: run new two-stage workflow alongside unchanged old job during tra…
715f615 
…nsition
@Subramanian-K812
fix: give central-mode test runs their own exclusion list
35d83cd
@Subramanian-K812
Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch from 587dcbb to 35d83cd Compare August 25, 2026 04:05
Subramanian-K812 and others added 21 commits August 25, 2026 06:51
@Subramanian-K812
ci: split DR-008 test execution into its own workflow file, keep old …
38dd214 
…job unchanged
@Subramanian-K812
fix: restore itf conftest.py
05184ee
@Subramanian-K812
ci(stage2): isolate score_communication under a dedicated Stage-2 rc
e24f86e
@lurtz
chore: Bump devcontainer to 1.11.0, use KVM and docker-in-docker feat…
ef78894 
…ure (eclipse-score#312)

With that `bazel test --lockfile_mode=error --config=linux-x86_64 //feature_integration_tests/itf` works.

`bazel --output_base=build/linux-x86_64 run --config linux-x86_64 //images/linux_x86_64:run` for which the bind mount of the Docker socket was added, works as well.
@Subramanian-K812
fix(stage2): isolate ferrocene_report's nested bazel calls from a mod…
743c730 
…ule's own stale .bazelrc
@Subramanian-K812
ci(stage2): drop the .bazelrc runtime rewrite for ferrocene_report
3969ba1
@Subramanian-K812
fix(stage2): transport dependency patches into Stage 2 injection
f3929d1
@Subramanian-K812
style: apply ruff formatting
e8d4ba5
@lurtz
ci: Retry SDK downloads via bazel (eclipse-score#316)
7652538 
Especially the flaky QNX download fails a lot in CI and causes builds to fail. Let bazel retry a few times before failing the build.

This is already in use at e.g. https://github.com/eclipse-score/communication/blob/f5a5612c870781c87d786a77d2361d3eaa80574b/.bazelrc#L16-L22
@lurtz
chore: Bump to Ebclfsa to 2.0.0-beta2 SDK - (eclipse-score#315)
ed79483 
There is a new release. Lets make use of that.
@pawelrutkaq
Make timeouts more stable for QNX (eclipse-score#322)
a1f0980
@Subramanian-K812
fix(stage2): add score_config_management to ref_int's GCC toolchain
4fc2c62
@pawelrutkaq
Remove Orchestrator module (eclipse-score#325)
10e431d 
Orchestrator was scheduled for removal in 0.8
@fortunaLim
chore: use Bazel label for config_management patch reference (eclipse…
e312aa0 
…-score#324)
@Subramanian-K812
Merge upstream/main into DR-008 branch; drop score_orchestrator (remo…
99ec4d1 
…ved in eclipse-score#325)
@kgraeper
extract bazel_cpp_toolchains to known_good (eclipse-score#326)
8f652f9
@RSingh1511
[Test] score_logging integration - Bumps score_logging to v0.2.3 (ecl…
c1e93f1 
…ipse-score#327)
@antonkri
Fix docs build: '//:docs_combo_experimental' no longer exists in the …
0315030 
…pinned score_docs_as_code version; use '//:docs' instead (same CLI, still accepts --github_user/--github_repo). (eclipse-score#329)
@Subramanian-K812
fix(stage2): isolate ferrocene_report's nested bazel calls via a PATH…
5bc1b5e 
… shim
@Subramanian-K812
Merge upstream/main into DR-008 branch
8c69c36
@Subramanian-K812
Widen score_communication and score_logging code_root_path to //...
891e471
@Subramanian-K812
Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch from 8b5251f to 891e471 Compare September 4, 2026 03:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pawelrutkaq pawelrutkaq Awaiting requested review from pawelrutkaq pawelrutkaq will be requested when the pull request is marked ready for review pawelrutkaq is a code owner
@PiotrKorkus PiotrKorkus Awaiting requested review from PiotrKorkus PiotrKorkus will be requested when the pull request is marked ready for review PiotrKorkus is a code owner
@dcalavrezo-qorix dcalavrezo-qorix Awaiting requested review from dcalavrezo-qorix dcalavrezo-qorix will be requested when the pull request is marked ready for review dcalavrezo-qorix is a code owner
@MaximilianSoerenPollak MaximilianSoerenPollak Awaiting requested review from MaximilianSoerenPollak MaximilianSoerenPollak will be requested when the pull request is marked ready for review MaximilianSoerenPollak is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

ref_int - update test workflow according to DR8