Feat/devalshah/generic splitter

[devalshahamd]

Making some changes to simplify the code and remove incorrect/redundant logic

Pull Request Template

Note to AMDers:
This is a public repository. Please do not upload any confidential or customer data. Make sure all such data has been anonymized or removed before making this PR. If you need to attach any private files or links, please insert a Internal OneDrive Link or a Jira Ticket Link instead.

[kyle-hoffmeyer]

@devalshahamd

• Removed promote_bookends_iterations -> periodic_candidates and grouped_candidates are able to find all iterations.
• Simplified identification of gpu paths by relying exclusively on non_gpu_path field.
• Added tests for steady state identification and phase division:
  • Runs steady state identification + phase division on LLM inference traces- with annotations then again with annotations stripped
  • Passes if:
    • Number of splits is the same between each steady state category / phase
    • Number of kernels per split is the same within 5% tolerance
• Modified batch size identification to only consider non-memory ops when extracting the most common input dimension. This prevents false positives where KV cache initialization and cleanup operations are included in the calculation.
• Fixed steady state identification for LLM inference without annotations to not require minimum number of steady state splits
• Steady state identification adds prefill iterations at the end of the window rather than just the ones within the window

[kyle-hoffmeyer]

Given list of candidates:
Iter 0 (prefill): 0 run_batch [GPU] 3 pop_and_process [GPU] 9 get_next_batch_to_run [GPU]
Iter 1: 11 run_batch [GPU] 14 pop_and_process [ ] 20 get_next_batch_to_run [GPU]
Iter 2: 22 run_batch [GPU] 25 pop_and_process [ ] 31 get_next_batch_to_run [GPU]
Iter 3: 33 run_batch [GPU] 36 pop_and_process [ ] 42 get_next_batch_to_run [GPU]
Iter 4: 44 run_batch [GPU] 47 pop_and_process [ ] 53 get_next_batch_to_run [GPU]
Iter 5: 55 run_batch [GPU] 58 pop_and_process [GPU] 64 get_next_batch_to_run [ ]

periodic_candidate in feat/khoffmey filters out the non gpu path events. Meaning iter 0's pop_and_process is included but the iter's pop_and_process gets filtered out. This means the pattern it finds is get_next_batch_to_run -> run_batch and iter 0 is left out (since it starts with run_batch, not get_next_batch_to_run).

periodic_candidate in feat/devalshah branch however works. It only filters out events that are never gpu paths. So since pop_and_process is sometimes a gpu path, all invocations are included, meaning the recurring pattern is very clean: run_batch -> pop_and_process -> get_next_batch_to_run. So grouped_candidate actually isn't needed for this example.
However, lets say we remove pop_and_process altogether for iters 1-5. In this case, periodic_candidate fails but grouped_candidate identifies run_batch as the gpu bearing event and all 6 iters are picked up.

So either way, the bookend enhancement isn't needed. periodic_candidate finds everything and if grouped_candidate works correctly if periodic_candidate were to fail.

[kyle-hoffmeyer]

Regarding the prune_nongpu_path flag, it just adds a non_gpu_path flag like we discussed. But nothing in trace splitter actually checks this. gpu_bearing func just checks if gpu_events is set / kernel_bearing is set (a field assigned in reattach_worker_threads I should delete). It doesn't check non_gpu_path.

So we can just simplify this by making reattach_worker_threads assign non_gpu_path arg to events if prune_nongpu_path is true (instead of kernel_bearing field), and then in trace splitter we just check non_gpu_path arg and nothing else.