Use case: artifact composition / system releases

[alexgnq]

We use CDEvents to calculate DORA metrics across independently released components.

A concrete example is a Helm umbrella chart. Each component produces an independently versioned artifact/release. Publishing a component may subsequently produce a new version of a system artifact containing that component together with unchanged versions of other components:

component-a@1.5 ──┐
component-b@2.8 ──┼──> system@43 ──> deployment
component-c@4.1 ──┘

The individual events originate from isolated systems/webhooks, so the relationship is not necessarily known when the component event is emitted. We currently retain the events and project RELATION links later, once the system release is known. This lets us traverse component change → system release → deployment and calculate DORA metrics.

The Links proposal already seems well suited to the late-binding aspect: links can be emitted separately, fan-in is an explicit use case, and domainId now allows relationships to artifact resources identified by PURL.

What seems to be missing is standardized semantics for artifact composition.

There is also an important distinction between causality and composition:

component-a@1.5 --triggered/causedBy--> system@43
component-a@1.5 --componentOf--> system@43
component-b@2.8 --componentOf--> system@43
component-c@4.1 --componentOf--> system@43

Only component-a@1.5 may have triggered creation of system@43, but all three artifacts are constituents of that immutable system release.

Would it make sense for CDEvents Links to define a standard relationship such as componentOf / contains for artifact-to-artifact composition?

This would not need to be Helm-specific. The same relationship occurs with application bundles, manifests, release trains, multi-component products, container/image indexes, and other composite artifacts.

I think this could potentially be handled by defining semantics for a RELATION.linkKind, rather than introducing a new event type or subject. It would also provide an interoperable way to reconstruct change → component artifact → system artifact → deployment graphs for metrics such as DORA Lead Time for Changes.

[xibz]

@alexgnq - This is a use case that #329 help solve. It introduces deployment and targetrollout events.

And yes, one thing I am planning on doing is introducing a broader set of standardized semantics for link relations: GROUP, REFERENCE, ARTIFACT will be the three I start with. If you have any other, let's talk about them.

GROUP means this is part of the same chain. REFERENCE means not in the same chain but some reference to something that may be useful to know in the graph. ARTIFACT is special type of reference.

Please take a look at the #329. I'll have a follow up PR for links for standardizing (put them in the specification rather than markdown) some default link kinds.

[xibz]

I think I understand the use case better now.

For LTTD specifically, do you actually need CDEvents to understand the full artifact composition, or do you just need a late-bound link from the component artifact to the system artifact that eventually gets deployed?

For example:

component-a@1.5 > system@43 > deployment

Since Links can be asserted later, it seems like the system that discovers the promotion relationship could emit that link once system@43 exists.

That would give you the traversal needed for per-component LTTD without requiring CDEvents to model that system@43 is composed of a@1.5, b@2.8, c@4.1, etc.

What does the composition semantic itself give you for the LTTD calculation that a late-bound REFERENCE/ARTIFACT link would not?

So I have a suspicion about what the underlying issue may be. Is the problem that your producers are completely independent and correlation context is not propagated across the lifecycle?

Even with stable domain IDs, you still need enough information to associate the relevant PR, CI execution, system release, and deployment. If those systems emit independently without carrying that relationship forward, then the consumer has to reconstruct it later.

Is that essentially the gap you’re trying to solve?

[alexgnq]

Yeah, you're right. I think I was mixing two different problems. Sorry about that.

For LTTD, we only need the late-bound path:

component-a@1.5 → system@43 → deployment

That is actually more precise than composition for this purpose. b@2.8 and c@4.1 may also be present in system@43, but neither caused that system release. Knowing the full composition would not help identify which change started the clock.

Composition would answer different questions, such as which component versions were deployed in system@43, where a particular version is running, or which systems are affected by a component issue. Those are useful questions, but separate from LTTD.

Your point about correlation is also right. Our events come from separate GitHub webhooks across repositories, and we do not currently carry the relationship all the way through the promotion flow. Purls give us stable artifact identities, but not the relationship between them, so we reconstruct that later from the promotion data.

For our immediate LTTD use case, a late-bound REFERENCE/ARTIFACT link should be enough. Composition is still useful, but it is a separate requirement.

[xibz]

Okay, that's good to hear. It sounds like the follow-up semantics PR will make this clearer, which should satisfy your DORA needs.

For composition, though, it’s still not clear to me what CDEvents use cases we have for it. I’m trying to think of cases where we would actually want to represent resource topology, and one example that comes to mind is commits within a PR.

Links give us a way to create topology between CDEvents and their related resources. Composition sounds different: we would be describing a resource and its granular constituents.

For example, a PR can contain commits. Would we want that topology represented in CDEvents? Probably not.

I’m trying to find the boundary where resource composition becomes something CDEvents should actually model.

@afrittoli - You seem to already have some thoughts on this. Was there a particular use case that you have that could help me understand the boundary that CDEvents would represent?

[alexgnq]

I can see one operational use case, although I’m not sure it means CDEvents should own the composition itself.

If an incident is associated with component-b@2.8, we may want to find every deployed system artifact containing that version. The causal link from the component that triggered a system release would not cover unchanged constituents.

However, the composition is already represented more naturally by the system artifact’s SBOM or package metadata. CDEvents could link the incident to the affected component and the deployment to the deployed system artifact, while a consumer joins those event relationships through the SBOM.

So a possible boundry could be: CDEvents describes occurrences and links them to stable resources, while an SBOM or package manifest remains authoritative for a resource’s internal topology. What I’m less sure about is whether referencing that external metadata is enough for interoperable impact analysis, or whether CDEvents needs a normalized way to expose the operationally relevant artifact-to-artifact relationships. Or if that is even something CDEvents should do...?

[afrittoli]

@alexgnq That's a good point. The general approach we take with CDEvents is to keep them as thin as possible - compatibly with the use cases. In that spirit, the artifact subject supports an SBOM field already.

[xibz]

Then to your other point of Helm charts. This to me is a gap in CDEvents. Once I have my other PRs merged, there's another area in CD that isn't modeled, rendering.

This is basically helm charts flow, we render to a k8s manifest. This could be a separate event that links to deployment events.

[afrittoli]

Thanks @alexgnq, this is very welcome. We've been postponing the discussion on composition for a while but it's been in our roadmap for quite some time, so definitely something we should have.

I agree that causality and composition are not the same, and we may need a composition mechanism.
Create a generic composition mechanism that works with existing subjects would be great because it means that we could continue to use existing events with compositions e.g. a deployment that references an artifact could be then reference and artifact that is made of multiple artifacts.

[alexgnq]

Agreed. For our umbrella-chart case, composition associated with the artifact subject sounds like a good fit. The composite producer knows the exact component versions from the release notes or lock file, so system@43 could reference those components by purl/domainId. A deployment can then continue to reference system@43, while consumers walk its composition when needed.

This also works for constituents that never emit CDEvents themselves, which is important for vendor charts and other external artifacts.