[Feature] Use pinned HTTPS-only PTO-ISA checkout under build/pto-isa

[ChaoZheng109]

Summary

Simplify PTO-ISA dependency resolution to one deterministic managed/manual checkout path:

• Clone PTO-ISA over HTTPS only.
• Use <repo>/build/pto-isa as the only checkout location.
• Always build/run against the commit recorded in pto_isa.pin.
• If users need a different PTO-ISA commit, they must update pto_isa.pin.
• If automatic clone fails, users may manually clone PTO-ISA into build/pto-isa; simpler should still checkout/reset that repository to the pinned commit before building or compiling kernels.
• Remove the old build-time vs run-time PTO-ISA mismatch enforcement path, because build and run should no longer choose independent ISA revisions.
• Keep a small build metadata JSON as runtime-binary provenance: it records what the current build artifacts were actually built with and can help detect stale prebuilt binaries after pto_isa.pin changes.

This removes protocol selection, arbitrary checkout location selection, ad hoc CLI/env commit selection, and compatibility enforcement that only existed to guard those independent choices. The remaining metadata JSON is not configuration and must not become a second source of truth.

Motivation / Use Case

The current PTO-ISA setup has too many independent knobs:

• SSH vs HTTPS clone protocol.
• PTO_ISA_ROOT as an arbitrary user-managed path.
• --clone-protocol across pytest, scene-test, and build-runtimes.
• SIMPLER_PTO_CLONE_PROTOCOL at CMake/install time.
• --pto-isa-commit / SIMPLER_PTO_ISA_COMMIT as a separate runtime/install commit selector.
• Runtime lookup validation comparing the runtime build PTO-ISA commit against the run-time PTO-ISA commit.

That makes install-time runtime builds and run-time kernel compilation easy to drift apart. The repo already has pto_isa.pin; make it the single source of truth. Developers who want another ISA revision should change the pin file, so the selected revision is visible in the repo diff and applies consistently to install/build and run/kernel compile flows.

Once both phases use the same pinned checkout, the old mismatch detector becomes redundant complexity: there should be no separate run-time commit source to compare against. We still want artifact provenance, because build/lib binaries can be stale after a pin change. A JSON manifest can answer: which ISA commit were these runtime binaries actually built with?

HTTPS-only auto-clone also avoids first-run SSH key failures such as Permission denied (publickey) on fresh developer machines, GitHub-hosted CI runners, and containers.

Proposed API / Behavior

• Managed checkout path is always <repo>/build/pto-isa.

• Auto-clone command uses only:
  https://github.com/hw-native-sys/pto-isa.git

• Resolution flow:

  1. Read the required PTO-ISA commit from pto_isa.pin.
  2. If build/pto-isa is missing, clone it over HTTPS.
  3. If auto-clone fails, error message should tell the user they can manually run:
     git clone https://github.com/hw-native-sys/pto-isa.git build/pto-isa
  4. Once build/pto-isa exists, fetch if needed and checkout/reset to the commit in pto_isa.pin.
  5. Verify the checkout HEAD equals the pinned commit; if it cannot be resolved, fail before building or compiling kernels.
  6. Build runtime binaries and compile kernels using that checkout.

• Remove or deprecate protocol/path/commit selection surfaces that conflict with the pin-file model:

  • pytest/scene-test --clone-protocol
  • build-runtimes --clone-protocol
  • CMake SIMPLER_PTO_CLONE_PROTOCOL
  • Python ensure_pto_isa_root(clone_protocol=...)
  • PTO_ISA_ROOT as an arbitrary checkout override
  • pytest/scene-test/build --pto-isa-commit overrides
  • SIMPLER_PTO_ISA_COMMIT
  • internal SIMPLER_RUN_PTO_ISA_COMMIT / SIMPLER_RUN_PTO_ISA_ROOT tracking, if only used for the old mismatch check

• Keep build provenance metadata:

  • write a JSON file in the runtime build output, e.g. build/lib/pto_isa_build.json
  • JSON is output-only; it must not be read as an ISA selection input
  • record at least:
    • schema_version
    • required_commit_from_pin
    • actual_checkout_commit
    • optionally pin_file, checkout_path, and build timestamp/source metadata if useful
  • build should fail if required_commit_from_pin and actual_checkout_commit differ
  • runtime lookup may optionally compare required_commit_from_pin in the JSON with the current pto_isa.pin to report stale runtime binaries and ask for reinstall/rebuild
  • this optional stale-artifact check is not the old build-vs-run ISA mismatch check; it only verifies that prebuilt binaries correspond to the current source pin

• Remove the old a2a3 onboard runtime PTO-ISA mismatch enforcement:

  • remove validate_runtime_pto_isa_compatible() as a comparison against an independently selected run-time commit
  • remove tests/docs that describe build-time vs run-time PTO-ISA drift as a supported condition
  • replace any remaining safety checks with two simpler invariants:
    1. all PTO-ISA consumers resolve through pto_isa.pin and build/pto-isa
    2. prebuilt runtime metadata, if present, describes artifacts built for the current pin

CI Impact

CI should be updated to stop passing protocol and commit override flags once the pin file is the single source of truth:

• Remove --clone-protocol https and --clone-protocol ssh from workflows.
• Remove CMake SIMPLER_PTO_CLONE_PROTOCOL usage.
• Replace --pto-isa-commit $PTO_ISA_COMMIT / SIMPLER_PTO_ISA_COMMIT=... plumbing with reading pto_isa.pin inside the code path, unless a specific CI job is intentionally testing pin changes.
• Keep or update expectations around the metadata JSON as an informational record of the pinned commit used for the runtime build.
• Self-hosted runners that cannot auto-clone can pre-populate build/pto-isa; simpler should still enforce checkout to pto_isa.pin before use.

Alternatives Considered

Keep PTO_ISA_ROOT, --pto-isa-commit, or the mismatch detector as escape hatches. That keeps flexibility but also keeps the core mismatch risk: runtime binaries can be built with one ISA revision while kernels compile against another. A single pto_isa.pin source of truth is more reproducible. A build metadata JSON is still useful, but only as artifact provenance and stale-build diagnostics.

Additional Context

This affects the managed checkout used by runtime builds and kernel compilation. Manual recovery from clone failure remains possible by cloning PTO-ISA into the standard build/pto-isa location, but the selected revision is still controlled by pto_isa.pin.