Skip to content

Consider run concurrency for integration test CI #17095

Description

@janbrasna

Description

For reasons explained in #16800 there can be changes pushed while a test is running, making its results completely unreliable, both false positives and false negatives. While the linked ticket aims at surfacing that visibly, there's also the option for a newer run enqueued to cancel out the previous one. I originally wanted to avoid this to always have the results of the previous run, even if not completely reliable — under the condition it's annotated so, but the ticket below shows the disparity in the (fast subsequent) deployment, and running older specs (or is it actually newer? gotta verify how SHAs are used from the webhooks) for the integration test CI as it was back when the former deployment went to roll out is probably not worth it.


Success Criteria

  • Eliminate overlapping runs: Redirect tests regression #17077
  • Verify the way the canceled runs end up visualized on #www-notify is understandable
  • Check whether the triggering SHA is thoroughly pulled for matching test specs, instead of the implicit ref or branch tip

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions