Skip to content

Generic trace splitter shortcomings #1018

Description

@kyle-hoffmeyer

Documenting issues that generic trace splitter is not able to solve yet.

  1. Trace has an annotation family for every model layer. In this case, since annotation's take precedence over tree traversal, the annotation family will be chosen as the splitting group, creating a split for every model layer (too fine a granularity).
  2. Trace has irregular pattern in events.
    Ex:
get_next_batch (gpu_path with very few kernels)
run_batch (main gpu path)
get_next_batch
run_batch
get_next_batch
idle_step (non gpu path)
get_next_batch
run_batch

idle_step breaks pattern. Ideally, the trace splitter should have some tolerance for this and still consider this to be a relevant pattern.
3. Trace contains one major root with most of the GPU work and two minor roots with some GPU work. The call stack of the major root is a superset of the call stack of the two minor roots - it just has more wrapper nodes at the start. Sibling root detection would ideally traverse down each root and find a recurring pattern of events connecting the roots, but because the two minor roots start at a lower level than the main root, sibling root detection fails. Need some way of recognizing that the two minor roots start at a lower level in call stack than the major root.
4. Split window extension extends into next iters cpu ops.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions