[1/n] Add generalized event types and GPU Performance Monitoring counter event support

[briancoutinho]

Overview

Originally, this was part 1 of splitting PR #1148. It supports a new kind of GPU Counter events that will be published to the timeline as a time series. In the process I realized we should add more generic event types for accelerators rather than being tied to CUDA specific naming. This has historically lead to each new accelerator adding it's own events which is maintenance burden.

1. Generalized Accelerator Event types: Add generic event types in the ActivityType enum. There are now aliases in the enum class definition for older events like CUDA_RUNTIME, MTIA_RUNTIME to name a few. The dynamic plugin will leverage generic events so upstream source code changes will not be required for new accelerators.
2. GPU PM Counter Event Type: This enables supporting a performance counter event stream. Counter events can be serialized in ChromeTrace format and will be rendered as time series in the UI.

Details

Accelerator-Agnostic Event Types

The change reorganizes the ActivityType enum class to introduce generic, accelerator-agnostic event types that work across all hardware backends (CUDA, MTIA, HPU, XPU, etc.). Device-specific types are now deprecated aliases pointing to their generic counterparts. There are few corner case exceptions like MTIA_INSIGHT, CUDA_SYNC, see the header.

New Generic Event Types

Event Type	Description	Replaces (Deprecated Aliases)
RUNTIME	Host-side runtime events from any accelerator backend	CUDA_RUNTIME, MTIA_RUNTIME, GLOW_RUNTIME, XPU_RUNTIME, PRIVATEUSE1_RUNTIME, HPU_OP
DRIVER	Host-side driver events from any accelerator backend	CUDA_DRIVER, PRIVATEUSE1_DRIVER
CONCURRENT_KERNEL	On-device kernel execution across all accelerators	MTIA_CCP_EVENTS
GPU_PM_COUNTER	Performance monitoring counters for hardware profiling	(new)

Guidance for future use of ActivityTypes

Existing code using deprecated aliases will continue to work, but new code should use the generic types:

// ❌ Deprecated (still works for backward compatibility)
ActivityType::CUDA_RUNTIME
ActivityType::MTIA_RUNTIME

// ✅ Preferred
ActivityType::RUNTIME

I have not changed the usage of these types in the code base yet. That can happen in a follow up change.

Notes

• Old string names like "cuda_runtime" still parse correctly via aliasMap, and old enum values like ActivityType::CUDA_RUNTIME still compile (as aliases).
• defaultActivityTypesArray is now constexpr, enabling compile-time evaluation and eliminating runtime overhead.

GPU PM Counter Events

This is a straightforward change to emit Chrome Trace counter events] for counters obtained from the GPU. The event can be leveraged by any accelerator backend.

The values of the counters are embedded as key/val pairs in the output json

{..., "name": "ctr", "ph": "C", "ts":  0, "args": {"my_counter":  42}},

Testing

make test
// or
ctest -R ActivityTypes

The ParseTest.ActivityTypes validates that aliases in the Config string are correctly converted to the underlying base type. Since the enum class uses aliases all existing code in kineto that uses the original activity types continues to compile and work as expected.
The test above also checks default activity types are unchanged
https://github.com/pytorch/kineto/blob/main/libkineto/test/ConfigTest.cpp#L89-L108

[KarhouTam]

Hi @briancoutinho, this update is absolutely fantastic! As an out-of-tree accelerator developer, I’m genuinely excited about these advancements. If possible, could you kindly share an approximate timeline for the upcoming splitting changes? Can't wait to begin integrating our accelerator profiler with these impressive updates.

Additionally, may I ask if Kineto is still open to accepting contributions for third-party accelerator profiler as a plugin after these changes merged? Thank you!

At the end, happy new year to you and the community (maybe a little bit late, hah)!

[briancoutinho]

cc @sraikund16 @malfet @valentinandrei could you help review this PR. This is the first split of my original longer PR but I also introduced generic events that came in the last PR discussion.
This should certainly help with modularizing Kineto as well.

[briancoutinho]

@KarhouTam thanks for the feedback :)

If possible, could you kindly share an approximate timeline for the upcoming splitting changes?

It should not take too long for me to develop the changes like 1-2 weeks. There is one more somewhat larger PR that should enable the dynamic plugin interface.
But I think the main bottleneck will be to review the changes one by one. So I cannot create stacked PRs because I do not have write access to repo. Could take between 1-2 months I think.

Additionally, may I ask if Kineto is still open to accepting contributions for third-party accelerator profiler as a plugin after these changes merged? Thank you!

Actually, that is what our proposal was #1121
There were some concerns from Meta maintainers on dynamic loading, so we still need some clarification if this approach is fundamentally ok with PyTorch.

@malfet / @valentinandrei could we actually meet on a VC to discuss this? It is totally reasonable if we need to address security and code issues in the PRs. But if dynamic loading itself is an issue then it would be good to know that before putting some effort on this.

Also see: pytorch/pytorch#166205 (comment)

[malfet]

@briancoutinho can you please coordinate with @eqy to get added to say Tue meeting series and we can discuss it there

And giving you a write permissions to the repo is not a problem imo

[fffrog]

Hi @briancoutinho, thank you so much for introducing this significant work; it's incredibly valuable for external accelerators.

There were some concerns from Meta maintainers on dynamic loading

Could you mind to share those concerns here after discussing with the Meta maintainers later? As far as I know, the ONNX runtime framework has been using dynamic loading for years to support multiple accelerators, which really helps in integrating new accelerators

[briancoutinho]

Thanks @malfet and @eqy! Please also include @yisitu in the discussion.
Our emails are bcoutinho@nvidia.com and ysitu@nvida.com

[briancoutinho]

ping @eqy can you please include Situ and me in the sync above
Our emails are bcoutinho@nvidia.com and ysitu@nvida.com
You can find us on slack too