NXP backend: Improve test result gathering

[roman-janik-nxp]

Summary

Feature for creating improved test results for reporting when test log-level is set to DEBUG. Adds IR models to .outputs/test_dir, result tensor differences, text variant of input and result tensors, summary with test info.

Test plan

Test provided. All tests that use NSYS.

cc @robert-kalmar

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20024

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 9 Cancelled Jobs, 5 Pending, 6 Unrelated Failures, 2 Unclassified Failures

As of commit 8bdbaf7 with merge base 0da9ca3 ():

NEW FAILURES - The following jobs have failed:

• pull / test-lora-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 8cf21a01faa0373df953fc0b873c5d43c615182bfd96e9da4fafaef762f8cd1e /exec failed with exit code 1
• pull / test-lora-multimethod-linux / linux-job (gh)
  RuntimeError: Command docker exec -t be0c28f9d269a4780d3e91f78cc82b32eb0be1df3e4a299e36d9c0ad4bfae546 /exec failed with exit code 1
• pull / test-voxtral-realtime-xnnpack-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 3bfd51642834efc515d5e6d8df1bf772569056b316310b7d6e6cf6868f731a0e /exec failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 91e083f000132de7f63ab072558f5fa650ad9363477f0e18d839ed122755970d /exec failed with exit code 1
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t b5300e150418664d66c06665d3c2496583e73d105e914998426c634cd03f508c /exec failed with exit code 1
• trunk / unittest-release / linux / linux-job (gh)
  RuntimeError: Command docker exec -t e804d25f47c05fca5aa701283f972591df83ce8e5aa7d28ff6819fe5926fdf3d /exec failed with exit code 1

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

• trunk / test-llama-runner-mac (fp32, xnnpack+custom+quantize_kv) / macos-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 92
• trunk / test-models-macos-coreml (mv2) / macos-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ##[error]The operation was canceled.

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• pull / test-models-linux (ic4, portable, linux.4xlarge.memory) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (ic4, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (mobilebert, portable, linux.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (phi_4_mini, portable, linux.4xlarge.memory) / linux-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-linux-aarch64 (ic4, portable, linux.arm64.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-linux-aarch64 (mobilebert, portable, linux.arm64.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-linux-aarch64 (mobilebert, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-linux-aarch64 (phi_4_mini, portable, linux.arm64.m7g.4xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-linux-aarch64 (qwen2_5_1_5b, portable, linux.arm64.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• pull / unittest-editable / windows / windows-job (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh) (trunk failure)
• pull / unittest / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• pull / unittest-editable / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• trunk / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh) (trunk failure)
• trunk / unittest-release / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[linux-foundation-easycla]

• ❌ - login: @roman-janik-nxp / name: roman-janik-nxp. The commit (8bdbaf7, 9f86c89) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please visit our EasyCLA portal and chat with our support bot.

[roman-janik-nxp]

@novak-vaclav

[novak-vaclav]

Provided additional information to some of my comments. Otherwise LGTM

[roman-janik-nxp]

Internal CI: https://bamboo3.sw.nxp.com/browse/MLTECE-EXIGH183-13.
CI is passing please review again.

[roman-janik-nxp]

Resolved the test_dir -> intermediates_dir and export condition in nxp_backend. Added parallel run for pytest NXP backend test.

[roman-janik-nxp]

The failing checks seem unrelated and the internal build is passing. Merging.