Bug 2034765: Expose glean telemetry dictionary in a standardized format

[michaelvanstraten]

Description

Bugzilla: Bug-2034765

To support client side telemetry reporting to generic Open Telemetry compatible endpoint we need to convert the glean telemetry dictionary information into a schema that is standardized and well understood.

This patch sets up a task that generates a semconv schema from all metrics.yaml files and uploads it as an artifact.

You can run the generator manually by running:

mach glean semconv

[michaelvanstraten]

Hi @jonathanmendez, I heard that you started to get familiar with task cluster, this might be the perfect PR to review for you :)?

[jonathanmendez]

I haven't had time to review all of the python, but I have some comments on the rest so far.

I don't feel like I am enough to review this on my own. I would add @ahal for extra taskcluster coverage at least.

[jonathanmendez]

I would think this validation should happen before writing the file(s).

I would expect either:

if validate_resolved():
  print errors
  exit
write_resolved()
print("Wrote resolved")

if validate_manifest():
  print errors
  exit
write_manifest()
print("Wrote manifest")

or:

if validate_resolved():
  print errors
  exit
if validate_manifest():
  print errors
  exit

write_resolved()
write_manifest()

print("Wrote resolved")
print("Wrote manifest")

[michaelvanstraten]

That is a very good point, thank you.

[jonathanmendez]

Why is this marked shippable?

[jonathanmendez]

As above: why is this marked shippable?

[jonathanmendez]

Is there a way to provide better context for what any of this means? I barely know what OTel means (using "OpenTelemetry" would be nicer), but would have no idea what a "resolved semantic convention registry schema" is nor what the moa generate release tool is or why we want to use the moa generate release tool.

Here's an attempt:

Suggested change

	Tasks that run ``./mach glean`` utilities. The ``semconv`` task produces the
	resolved OTel semantic convention registry schema from Firefox Glean metric
	definitions, consumed by the ``moa generate release``
	tool.
	Tasks that run ``./mach glean`` utilities.
	The ``semconv`` task transforms Firefox Glean metric definitions into
	a standardized OpenTelemetry format known as "Semantic Conventions".
	This format can be consumed by the [[tooling explained by purpose]].

[jonathanmendez]

Same concerns on the description as above in kinds.rst

[jonathanmendez]

There's a process for vendoring components: https://firefox-source-docs.mozilla.org/mozbuild/vendor/index.html

I don't know if it works differently when there's only a couple of schema files, but it does feel odd for them to just be here in the source without any indication that they are third party.

[michaelvanstraten]

Okay, valid point. I will add a moz.yaml