Skip to content

test-matrix.yml's continue-on-error: true matrix field is never applied to the job, so it has no effect #134

Description

@morcen

Problem

The Laravel 12 include entry in the test matrix sets a custom continue-on-error: true field:

include:
  - laravel: 11.*
    testbench: 9.*
    ...
  - php: 8.3
    laravel: 12.*
    testbench: 10.*
    ...
    continue-on-error: true

(.github/workflows/test-matrix.yml:18-33)

This reads as an attempt to mark the Laravel 12 / PHP 8.3 combination as allowed to fail without failing the overall workflow — a common pattern for testing a newer/less-stable dependency combination. However, GitHub Actions only honors continue-on-error when it's set on the job or a step, e.g. continue-on-error: ${{ matrix.continue-on-error }}. Here it's just an arbitrary custom variable inside matrix.include — nothing in the job (.github/workflows/test-matrix.yml:11-45) or any step references matrix.continue-on-error.

Where

  • .github/workflows/test-matrix.yml:24-33 (the continue-on-error: true field, defined but never consumed)

Why it matters

As written, this configuration is dead — it has zero effect on workflow behavior. If the Laravel 12/PHP 8.3 combination fails, the job fails exactly like any other matrix entry, contradicting the evident intent behind adding the field. Combined with fail-fast: false at the strategy level, a failure here won't cancel sibling jobs, but it will still mark the overall workflow run as failed/red — which is presumably not what was intended when this "experimental" combination was carved out into its own include entry.

Suggested fix

Add continue-on-error: ${{ matrix.continue-on-error == true }} at the job level in test-matrix.yml (job-level continue-on-error does accept an expression referencing matrix.*), so the flag actually takes effect for this combination.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtech-debtFrom the technical debt register

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions