feat: EP registration + monitoring — subprocess isolation, structured failures, universal op-tracing dispatch, lazy CLI startup

docs/commands/export.md

@@ -22,7 +22,7 @@ $ winml export [options]
 | `--output` | `-o` | path | *(required)* | Output ONNX file path (e.g., `model.onnx`). |
 | `--with-report/--no-with-report` | | flag | `false` | Generate full export reports: Markdown, JSON, and a console tree. |
 | `--hierarchy/--no-hierarchy` | | flag | `true` | Preserve `hierarchy_tag` metadata in ONNX nodes (use `--no-hierarchy` for a clean ONNX file). |
-| `--dynamo/--no-dynamo` | | flag | `false` | Enable PyTorch 2.9+ dynamo export for richer node metadata. (Experimental — currently logs a warning.) |
+| `--dynamo/--no-dynamo` | | flag | `true` | Use PyTorch's TorchDynamo ONNX exporter (default) for richer per-node module metadata. Pass `--no-dynamo` for the legacy TorchScript exporter, whose opset-17 op decomposition is the validated path for QNN/NPU compilation today. |
 | `--torch-module` | | string | `None` | Comma-separated list of `torch.nn` module types to include in hierarchy (e.g., `LayerNorm,Embedding`). (Experimental — currently logs a warning.) |
 | `--input-specs` | | path | `None` | JSON file with explicit input tensor specifications. Auto-generated when omitted. |
 | `--task` | `-t` | string | `None` | Override auto-detected Hugging Face task (e.g., `image-feature-extraction`). |
@@ -37,9 +37,12 @@ $ winml export [options]
 
 `winml export` loads the model via Hugging Face `transformers`, then runs the
 eight-step Hierarchy-preserving Tags Protocol (HTP): model preparation, input
-generation, module-hierarchy tracing, TorchScript ONNX export, node-tagger
+generation, module-hierarchy recovery, ONNX export, node-tagger
 creation, per-node tagging, tag injection into ONNX `metadata_props`, and
-optional report generation. The hierarchy metadata allows downstream tools to
+optional report generation. By default the hierarchy is reconstructed from the
+TorchDynamo exporter's per-node module metadata; with `--no-dynamo` it is
+captured by tracing the model's forward pass under the legacy TorchScript
+exporter instead. The hierarchy metadata allows downstream tools to
 reason about operators grouped by their originating module rather than flat
 graph position. When `--no-hierarchy` is specified, hierarchy steps are bypassed
 and a bare ONNX file is written, useful for third-party tools that do not
@@ -79,6 +82,15 @@ winml export -m bert-base-uncased -o bert.onnx \
 winml export -m bert-base-uncased -o bert.onnx --input-specs inputs.json
 ```
 
+```bash
+# Export a fully static-shaped model (the default) for NPU/QNN compilation.
+# The default dummy inputs already produce static shapes; use --shape-config to
+# pin any symbolic dimensions to concrete sizes, or --input-specs to fully
+# specify every input tensor.
+winml export -m bert-base-uncased -o bert.onnx --shape-config shape_config.json
+# shape_config.json: {"sequence_length": 128}
+```
+
 ```bash
 # Export with dynamic batch and sequence dimensions
 winml export -m bert-base-uncased -o bert.onnx --dynamic-axes dynamic_axes.json
@@ -114,9 +126,26 @@ winml export -m microsoft/resnet-50 -o resnet50_clean.onnx --no-hierarchy
 - **Dynamic dimensions can reduce QNN optimization coverage.** Static batch and
   static shapes remain the default because some QNN fusions require them. Use
   `--dynamic-axes` only when downstream runtime scenarios need variable sizes.
-- **`--dynamo` and `--torch-module` are experimental.** Both flags emit a
-  warning and have no effect in the current release. Do not rely on them in
-  automated pipelines yet.
+- **Dynamo is the default exporter.** `winml export` uses PyTorch's TorchDynamo
+  ONNX exporter, which records rich per-node module metadata that drives the
+  hierarchy tags. Pass `--no-dynamo` to select the legacy TorchScript exporter.
+  Shape staticness is independent of the exporter: both default to static shapes
+  and only emit dynamic axes when you ask for them. The QNN-relevant
+  difference is opset and op decomposition: torch's dynamo op library is built
+  against a minimum opset of 18. When you request a lower opset (17 by default),
+  dynamo attempts to down-convert the graph to it; this can succeed (the graph is
+  saved at the requested opset) or fail (the graph stays at opset 18). `winml
+  export` reports the opset actually produced and warns when it differs from the
+  one you requested, so pass `--no-dynamo` if you need a graph natively at opset
+  17. The TorchScript path always exports at the configured opset (17 by
+  default). Dynamo also
+  lowers some ops differently -- for example ResNet's classification head becomes
+  `ReduceMean` + `Reshape` under dynamo but `GlobalAveragePool` + `Flatten` under
+  TorchScript -- which can change or reduce QNN fusion coverage. Prefer
+  `--no-dynamo` for hand exports targeting QNN/NPU compilation until the dynamo
+  graph is validated for your model.
+- **`--torch-module` is experimental.** The flag emits a warning and has no
+  effect in the current release. Do not rely on it in automated pipelines yet.
 - **Output directory must be writable.** The command creates parent directories
   automatically, but will fail with a permission error on read-only paths.
 - **Model weights are downloaded to the Hugging Face cache.** Set `HF_HOME` or

docs/concepts/load-and-export.md

@@ -2,23 +2,23 @@
 
 The first stage of the winml-cli pipeline is the most deterministic: bring a model into memory and convert it to ONNX. Everything that follows — optimization, quantization, compilation — operates on that ONNX artifact. A well-exported graph with accurate metadata travels cleanly through the rest of the pipeline without requiring patching or re-export.
 
-Loading is an internal operation: the loader module resolves model provenance, selects the right HuggingFace model class, and prepares the weights for tracing. The `winml export` command is the surface users interact with directly.
+Loading is an internal operation: the loader module resolves model provenance, selects the right HuggingFace model class, and prepares the weights for export. The `winml export` command is the surface users interact with directly.
 
 ## Loading a model
 
 When you point winml-cli at a model identifier, the internal loader resolves it in one of two ways. If the identifier looks like a HuggingFace Hub path (e.g., `prajjwal1/bert-tiny`), the loader downloads the model weights and configuration to the standard HuggingFace cache at `~/.cache/huggingface`. Subsequent runs are served from that cache without re-downloading. If the identifier is a path to a local PyTorch checkpoint directory, the loader reads it directly without network access.
 
-In both cases the loader auto-detects the task — image classification, text feature extraction, and so on — and selects a corresponding HuggingFace model class. The result is a PyTorch model object ready for tracing.
+In both cases the loader auto-detects the task — image classification, text feature extraction, and so on — and selects a corresponding HuggingFace model class. The result is a PyTorch model object ready for export.
 
 Before committing to a full export you can verify that the loader resolved everything correctly with `winml inspect`. It prints the detected task, the HuggingFace model class, the export configuration, and the WinML inference class — all without downloading weights. Add `--hierarchy` to reconstruct the PyTorch module tree from random-weight tracing.
 
 Some community models host custom Python code in their repositories. The loader refuses to execute it by default. Pass `--trust-remote-code` to `winml config` when generating a build configuration for such a model.
 
 ## Exporting to ONNX
 
-`winml export` converts the loaded model to ONNX. The conversion uses TorchScript tracing by default, which follows actual execution paths and tends to produce compact, inference-oriented graphs. A `--dynamo` flag exists for the PyTorch 2.x dynamo exporter; however, **Note:** the `--dynamo` flag is reserved for the PyTorch 2.x dynamo exporter but is **not yet functional** in the current release — passing it logs a warning and the flag is ignored.
+`winml export` converts the loaded model to ONNX. By default it uses PyTorch's TorchDynamo ONNX exporter (`torch.onnx.export(dynamo=True)`), which records rich per-node module metadata that is used to derive the `winml.hierarchy.*` node tags. Pass `--no-dynamo` to fall back to the legacy TorchScript exporter, which follows actual execution paths and tends to produce compact inference-oriented graphs. Both exporters default to static shapes; the QNN-relevant difference is opset and op decomposition: torch's dynamo op library is built against a minimum opset of 18. When a lower opset is requested (17 by default), dynamo attempts to down-convert to it, which may succeed (the graph is saved at the requested opset) or fail (it stays at opset 18); `winml export` reports the opset actually produced and warns when it differs from the requested value. The TorchScript path always exports at the configured opset (17 by default) and lowers some ops differently (e.g. ResNet's head becomes `ReduceMean` + `Reshape` under dynamo but `GlobalAveragePool` + `Flatten` under TorchScript). Because the opset-17 TorchScript graph is what the QNN/NPU toolchain has been validated against, `--no-dynamo` remains the validated choice for QNN/NPU hand exports today.
 
-By default the exporter runs an eight-step process that includes hierarchy tracing and tag injection. The result is an ONNX file enriched with structural metadata that powers downstream features such as per-module benchmarking, inspector views, and optimizer scoping.
+By default the exporter runs an eight-step process that includes hierarchy recovery and tag injection. The result is an ONNX file enriched with structural metadata that powers downstream features such as per-module benchmarking, inspector views, and optimizer scoping.
 
 ### Hierarchy tagging in detail
 
@@ -31,7 +31,9 @@ During export the HTP (Hierarchy-preserving Tags Protocol) exporter attaches two
 
 #### How tags are built
 
-The exporter registers PyTorch forward hooks on each module. When a module executes, a pre-hook pushes its class name onto a tag stack; the post-hook pops it. This produces hierarchical paths that mirror the PyTorch module tree:
+The module path attached to each node is obtained differently depending on the exporter. Under the default TorchDynamo exporter, dynamo records originating-module information as ONNX metadata, and the exporter reconstructs the hierarchy directly from that metadata after export — no forward hooks are involved. Under `--no-dynamo`, the TorchScript path instead registers PyTorch forward hooks on each module: when a module executes, a pre-hook pushes its class name onto a tag stack and the post-hook pops it. Either way the result is hierarchical paths that mirror the PyTorch module tree.
+
+The hook flow below applies only to the `--no-dynamo` TorchScript path:
 
 ```mermaid
 flowchart LR
@@ -42,11 +44,11 @@ flowchart LR
     E --> F[Tag stack → path]
 ```
 
-Only modules that are actually executed during tracing receive tags — unused modules are excluded. For example, `prajjwal1/bert-tiny` has 48 registered modules but only 18 are reached during a forward pass.
+Under `--no-dynamo`, only modules that are actually executed during tracing receive tags — unused modules are excluded. For example, `prajjwal1/bert-tiny` has 48 registered modules but only 18 are reached during a forward pass. (Under the default dynamo path the hierarchy instead reflects the modules present in the exported graph's metadata.)
 
 #### Concrete example: BERT-tiny
 
-Running `winml export -m prajjwal1/bert-tiny -o model.onnx -v` produces the following hierarchy tree (18 traced modules, 132 ONNX nodes, 100 % coverage):
+Running `winml export -m prajjwal1/bert-tiny -o model.onnx -v --no-dynamo` produces the following hierarchy tree (18 traced modules, 132 ONNX nodes, 100 % coverage) — the numbers below are from the TorchScript trace path:
 
 ```
 BertModel (132 nodes)
@@ -75,7 +77,7 @@ Each ONNX node gets its tag from the module it belongs to. Here are a few exampl
 
 #### Node-to-module mapping
 
-After the ONNX graph is produced by `torch.onnx.export`, a 4-priority system assigns each ONNX node to the closest matching module:
+Under the default dynamo path, each node is assigned directly from its `pkg.torch.onnx.name_scopes` and `pkg.torch.onnx.class_hierarchy` metadata; nodes without usable metadata receive the model-root fallback. Under `--no-dynamo`, after the ONNX graph is produced by `torch.onnx.export`, the legacy tagger instead uses a 4-priority system to assign each ONNX node to the closest traced module:
 
 1. **Direct match** (61 %) — the node's scope name maps exactly to a traced module.
 2. **Parent match** (24 %) — walk up the scope hierarchy until a traced module is found.
@@ -100,7 +102,7 @@ These I/O specs enable tools like `winml perf` to generate correct dummy inputs
 Alongside the `.onnx` file, the exporter writes a `*_htp_metadata.json` sidecar containing:
 
 - **`nodes`** — complete mapping of every ONNX node name → hierarchy tag
-- **`modules`** — traced module information (class name, tag, execution order)
+- **`modules`** — hierarchy module information (class name, tag, discovery order)
 - **`statistics`** — export time, node counts, coverage percentage
 - **`outputs`** — I/O tensor specifications
 

docs/concepts/perf-and-monitoring.md

@@ -139,12 +139,13 @@ In JSON output (`-f json`), these metrics appear under the `hw_monitor` key:
   "adapter_luid": null,
   "cpu": { "mean_pct": 15.8, "peak_pct": 16.71, "sample_count": 2 },
   "ram": { "used_mb": 640.21, "peak_mb": 640.21 },
+  "gpu": { "mean_pct": 0.0, "peak_pct": 0.0, "sample_count": 0, "luids": [] },
   "device_memory": { "local_peak_mb": 0.0, "shared_peak_mb": 0.0 },
   "running_time_ns": 0
 }
 ```
 
-When a hardware accelerator is active, `device_kind` will be `"npu"` or `"gpu"`, and an additional key (e.g. `"npu"`) appears with device utilisation:
+The `gpu` block is always present because GPU telemetry is collected independently of the selected inference adapter. When an accelerator is active, `device_kind` will be `"npu"` or `"gpu"` and the selected-adapter metrics appear under the stable `adapter` block. For compatibility, an additional dynamic block is only added when the selected kind does not already have a reserved aggregate block — today that means NPU selections still include `npu`, while GPU selections keep `gpu` reserved for aggregate telemetry.
 
 ```json
 "hw_monitor": {
@@ -153,12 +154,16 @@ When a hardware accelerator is active, `device_kind` will be `"npu"` or `"gpu"`,
   "adapter_luid": "0x0000abcd12340000",
   "cpu": { "mean_pct": 12.1, "peak_pct": 34.5, "sample_count": 50 },
   "ram": { "used_mb": 1842.0, "peak_mb": 1910.0 },
+  "gpu": { "mean_pct": 3.2, "peak_pct": 9.7, "sample_count": 50, "luids": ["0x00001234abcd0000"] },
+  "adapter": { "mean_pct": 87.3, "peak_pct": 100.0, "sample_count": 50 },
   "device_memory": { "local_peak_mb": 245.0, "shared_peak_mb": 0.0 },
   "npu": { "mean_pct": 87.3, "peak_pct": 100.0, "sample_count": 50 },
   "running_time_ns": 4820000000
 }
 ```
 
+When the selected device is a GPU, `adapter` contains the selected GPU's live metrics while `gpu` continues to report the aggregate GPU view (including `luids`).
+
 This makes it straightforward to track memory consumption across model revisions or compare devices programmatically.
 
 ## Per-module benchmarking

docs/naming-convention.md

@@ -11,7 +11,7 @@ Domain acronyms in PascalCase class names **retain their uppercase form**, excep
 | Acronym | Meaning | Class Casing | Example |
 |---------|---------|--------------|---------|
 | ONNX | Open Neural Network Exchange | `ONNX` | `ONNXStaticAnalyzer`, `ONNXLoader` |
-| EP | Execution Provider | `EP` | `EPChecker`, `EPConfig`, `EPMonitor` |
+| EP | Execution Provider | `EP` | `EPChecker`, `EPConfig`, `WinMLEPMonitor` |
 | QDQ | Quantize-Dequantize | `QDQ` | `QDQParameterConfig`, `QDQGenerator` |
 | QNN | Qualcomm Neural Network | `QNN` | `QNNMonitor` |
 | Op | Operator (2-letter prefix) | `Op` | `OpUnsupportedError` |

docs/reference/index.md

@@ -54,7 +54,7 @@ stages based on the target device and precision.
 | `export_params` | `bool` | `true` | Include model parameters in ONNX. |
 | `do_constant_folding` | `bool` | `true` | Fold constants during export. |
 | `verbose` | `bool` | `false` | Verbose export logging. |
-| `dynamo` | `bool` | `false` | Use PyTorch 2.x Dynamo exporter. |
+| `dynamo` | `bool` | `true` | Use PyTorch's TorchDynamo ONNX exporter (default); set `false` for the legacy TorchScript exporter. |
 | `enable_hierarchy_tags` | `bool` | `true` | Add module hierarchy tags to ONNX nodes. |
 | `clean_onnx` | `bool` | `false` | Strip hierarchy tags after export. |
 | `hierarchy_tag_format` | `"full" \| "module_only"` | `"full"` | Tag detail level. |

examples/recipes/MIT_ast-finetuned-audioset-10-10-0.4593/cpu/cpu/audio-classification_fp16_config.json

@@ -0,0 +1,60 @@
+{
+  "export": {
+    "opset_version": 17,
+    "batch_size": 1,
+    "export_params": true,
+    "do_constant_folding": true,
+    "verbose": false,
+    "dynamo": false,
+    "enable_hierarchy_tags": true,
+    "clean_onnx": false,
+    "hierarchy_tag_format": "full",
+    "input_tensors": [
+      {
+        "name": "input_values",
+        "dtype": "float32",
+        "shape": [
+          1,
+          1024,
+          128
+        ],
+        "value_range": [
+          -1,
+          1
+        ]
+      }
+    ],
+    "output_tensors": [
+      {
+        "name": "logits"
+      }
+    ]
+  },
+  "optim": {},
+  "quant": {
+    "mode": "fp16",
+    "samples": 10,
+    "calibration_method": "minmax",
+    "weight_type": "uint8",
+    "activation_type": "uint8",
+    "per_channel": false,
+    "symmetric": false,
+    "weight_symmetric": null,
+    "activation_symmetric": null,
+    "save_calibration": false,
+    "distribution": "uniform",
+    "seed": null,
+    "calibration_load_path": null,
+    "calibration_save_path": null,
+    "op_types_to_quantize": null,
+    "nodes_to_exclude": null,
+    "fp16_keep_io_types": true,
+    "fp16_op_block_list": null
+  },
+  "compile": null,
+  "loader": {
+    "task": "audio-classification",
+    "model_class": "AutoModelForAudioClassification",
+    "model_type": "audio-spectrogram-transformer"
+  }
+}

examples/recipes/MIT_ast-finetuned-audioset-10-10-0.4593/cpu/cpu/audio-classification_fp32_config.json

@@ -0,0 +1,41 @@
+{
+  "export": {
+    "opset_version": 17,
+    "batch_size": 1,
+    "export_params": true,
+    "do_constant_folding": true,
+    "verbose": false,
+    "dynamo": false,
+    "enable_hierarchy_tags": true,
+    "clean_onnx": false,
+    "hierarchy_tag_format": "full",
+    "input_tensors": [
+      {
+        "name": "input_values",
+        "dtype": "float32",
+        "shape": [
+          1,
+          1024,
+          128
+        ],
+        "value_range": [
+          -1,
+          1
+        ]
+      }
+    ],
+    "output_tensors": [
+      {
+        "name": "logits"
+      }
+    ]
+  },
+  "optim": {},
+  "quant": null,
+  "compile": null,
+  "loader": {
+    "task": "audio-classification",
+    "model_class": "AutoModelForAudioClassification",
+    "model_type": "audio-spectrogram-transformer"
+  }
+}