Skip to content

NagaYu/FoundationRadar

Repository files navigation

FoundationRadar

Measure what Apple's Foundation Models stack can actually do on this device — availability, real context size, real latency, Private Cloud Compute signals — instead of trusting a boolean.

CI Swift versions Platforms License: MIT

The numbers you memorized are not constants

Apple says so directly — WWDC 2026, What's new in the Foundation Models framework:

"In iOS 26.4, we released new APIs for inspecting the model's context size and counting the tokens in instructions, prompts, and transcripts. You'll want to use these going forward to adapt your app to the hardware it's running on."

contextSize varies across device and OS generations. It gets subtler: on OS 26.0–26.3 the property is back-deployed, and the back-deployment shim in the SDK returns a fixed fallback value — so code "querying" it there may be reading a compiled-in constant. Latency differs meaningfully across silicon, between a cold session and a prewarm()ed one. And Private Cloud Compute arrives (OS 27) with its own runtime preconditions — network, a daily quota, a managed entitlement — that a well-behaved app should check, not assume.

None of this can be looked up once and hardcoded. It has to be measured, on real devices, and re-measured as OS versions change. That is all this package does.

Quickstart

dependencies: [
    .package(url: "https://github.com/NagaYu/FoundationRadar.git", from: "0.1.0")
]
import FoundationRadar

// One call. Latency benchmarking is opt-in (it runs real inference);
// the history log is a plain JSON file you own.
let report = await FoundationRadar.generateReport(
    latency: .quick,
    appendingTo: CapabilityHistory(fileURL: logURL)
)

print(report.markdown())   // paste-ready for a GitHub issue
let json = try report.json()  // machine-readable, stable-ordered

Real output from a real device (M2 MacBook Air, macOS 26.5.2 — measured, not typed in):

| Backend | Status |
|---|---|
| On-device | ✅ available |
| Private Cloud Compute | ❌ requiresNewerSDK |

### Context size
- **4096 tokens** (measured at run time)

### Latency (measured, not asserted)
| Measurement | Seconds |
|---|---|
| Fresh session, first response (no prewarm) | 1.796 |
| Follow-up response 1 (same session) | 0.251 |
| Fresh session, first response after `prewarm()` | 0.354 |

Or skip the code entirely and run the bundled CLI on any device you care about:

$ swift run radar-cli --latency quick --history my-devices.json

What it measures

Signal How
On-device availability SystemLanguageModel.availability, wrapped in one defensive type — Apple has added unavailability reasons before (systemNotReady arrived with PCC) and future ones land in a typed unrecognized(description:), never a crash or a lie
PCC availability PrivateCloudComputeLanguageModel.availability where the SDK/OS allow, honest typed reasons (osVersionTooOld, requiresNewerSDK) where they don't
Context size contextSize queried at run time, flagged when the OS is old enough that the value may be the back-deployment fallback rather than a live answer
Latency Opt-in micro-benchmark: fresh-session first response, same-session follow-ups, and first response after prewarm() — greedy sampling, fixed trivial prompt, so runs are comparable
PCC signals Network path (passive NWPathMonitor), com.apple.developer.private-cloud-compute entitlement presence (runtime-checkable on macOS), quota status where the OS 27 API exists
History Every run appends a timestamped, device/OS-identified record to a local plain-JSON file you own — your own compatibility matrix across your real test devices and OS betas

Three principles, enforced throughout:

  • No number in this package is asserted from memory or marketing. Everything reported was measured on the device at run time, or is honestly reported as unmeasurable and why.
  • No telemetry. The only file writes go to the history URL you pass; the only network activity is whatever Apple's own availability/PCC checks perform. grep the sources.
  • Typed everything. No Any, no force-unwraps in the public API; errors are a single RadarError enum; unknown future states are typed, not fatal.

Private Cloud Compute: what works with which toolchain

PCC's API surface (PrivateCloudComputeLanguageModel, quotaUsage) exists only in the OS 27 SDKs shipped with Xcode 27 (Swift 6.4). FoundationRadar compiles everywhere and tells the truth at each tier:

Built with Running on PCC availability reports
Xcode 26.x anything requiresNewerSDK (the symbols aren't in the binary)
Xcode 27+ OS 26.x osVersionTooOld
Xcode 27+ OS 27+ live answer from Apple's API

Network reachability and entitlement presence are SDK-independent and always captured. The Xcode 27 branch is written against Apple's current documentation and is marked experimental until compile-verified against a stable Xcode 27 — see the note in PCCSignals.swift and CONTRIBUTING.md.

How this differs from FoundationModelsKit

FoundationModelsKit (by @rryam, actively maintained) is the layer for building features on Foundation Models: conversation management with context recovery, JSON Schema conversion, token accounting and context budgeting, error projection, and a suite of ready-made Tool implementations. It also includes runtime readiness checks as part of that toolkit.

FoundationRadar deliberately covers a narrower, different layer: empirical measurement as data. It doesn't manage conversations, convert schemas, or ship tools — it measures availability, context size, latency, and PCC signals, records them with device/OS identity into a history you own, and renders shareable reports. If you're building a chat feature, you likely want FoundationModelsKit; if you're deciding what your app can rely on across your device fleet — or filing a "this model behaves differently on X" issue with evidence — that's FoundationRadar. Using both together is entirely reasonable.

RadarCLI

Examples/RadarCLI is the practical verification path and demo: a dependency-free CLI that produces a real report on the device it runs on.

$ swift run radar-cli --help
$ swift run radar-cli                           # availability + context + PCC signals
$ swift run radar-cli --latency thorough        # opt into the benchmark
$ swift run radar-cli --format json             # machine-readable
$ swift run radar-cli --history radar-log.json  # build your device matrix over time

Requirements

  • Swift 6.1+ / Xcode 26+ to build.
  • The package's deployment floor is deliberately low (macOS 14, iOS 17, Mac Catalyst 17, visionOS 1): apps that support older OS versions are exactly the apps that need runtime capability measurement. On such systems every live check degrades to a typed answer (osVersionTooOld, etc.).
  • Live measurement requires a device with Apple Intelligence available and enabled (Foundation Models floor: iOS/macOS/visionOS 26.0; live contextSize/tokenCount: 26.4; PCC: 27.0 + entitlement).

License

MIT — see LICENSE.

About

Measure what Apple's Foundation Models stack can actually do on this device — availability, real context size, real latency, PCC signals — instead of trusting a boolean.

Topics

Resources

License

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Packages

 
 
 

Contributors

Languages