Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -273,10 +273,12 @@ A live single-line progress indicator runs on stderr during interactive sessions

## Requirements

- macOS 27 or newer (Apple's `fm` CLI is preinstalled).
- macOS 27.0 or newer (Apple's `fm` CLI is preinstalled there).
- Node.js 20 or newer.
- Apple Intelligence enabled on the device.

Benchmark commands refuse to start on older macOS versions and report the detected version plus the latest supported macOS — see [docs/supported-platforms.md](docs/supported-platforms.md).

`pcc` (Private Cloud Compute) availability depends on Apple's current eligibility. `fm-bench` shows it as skipped if `fm available --model pcc` reports unavailable.

See [docs/methodology.md](docs/methodology.md) for benchmark methodology and metric references.
Expand Down
3 changes: 2 additions & 1 deletion docs/releasing.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
## Option B — Local

```sh
npm ci && npm test && npm run lint
npm ci && npm test && npm run lint && npm run check:pack
npm version patch # or minor / major
git push --follow-tags
```
Expand All @@ -37,6 +37,7 @@ gh release edit v0.6.0 --notes-file /tmp/notes.md --repo devinoldenburg/fm-bench

```sh
npm run publish:dry-run
npm run check:pack
```

CI runs the same dry-run on every push to `main` and on pull requests.
26 changes: 26 additions & 0 deletions docs/supported-platforms.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Supported platforms

`fm-bench` benchmarks Apple's `fm` command, so support follows the platforms where Apple ships that CLI.

## Requirements

- **macOS 27.0 or newer** — Apple's `fm` CLI is preinstalled starting with macOS 27. Older macOS releases do not ship `fm`, so `fm-bench` cannot run there.
- **Node.js 20 or newer**.
- **Apple Intelligence enabled** on the device.

`pcc` (Private Cloud Compute) availability additionally depends on Apple's current eligibility. `fm-bench` reports it as skipped when `fm available --model pcc` reports unavailable.

## Version enforcement

Commands that launch `fm` benchmarks — the default `run` command and `models` — check the macOS version first and refuse to start on anything older than macOS 27:

```text
fm-bench: unsupported macOS: detected macOS 26.1, but fm-bench requires macOS 27 or newer (Apple's fm CLI is preinstalled there).
Latest supported: macOS 27.0 or newer (fm is not available on older macOS releases).
```

The process exits with code `2`. This is deliberate: running on an unsupported macOS could never produce a valid benchmark, so the CLI fails fast with the exact version it found and the latest supported macOS version.

Commands that only read local report files — `compare`, `history`, `validate`, `export`, and `legend` — are not gated and work anywhere Node.js runs.

`fm-bench doctor` still runs on unsupported hosts so you can diagnose the environment: it prints the detected macOS version, an explicit `macOS support` line, and the latest supported version.
10 changes: 7 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,12 @@
"scripts": {
"test": "node --test",
"lint": "node --check bin/fm-bench.js && find src test -name '*.js' -print0 | xargs -0 -n1 node --check",
"prepack": "npm test && npm run lint",
"prepack": "npm test && npm run lint && npm run check:pack",
"release:patch": "npm version patch && git push --follow-tags",
"release:minor": "npm version minor && git push --follow-tags",
"release:major": "npm version major && git push --follow-tags",
"publish:dry-run": "npm publish --dry-run --access public"
"publish:dry-run": "npm publish --dry-run --access public",
"check:pack": "node scripts/check-package.mjs"
},
"keywords": [
"apple",
Expand All @@ -46,5 +47,8 @@
"publishConfig": {
"access": "public",
"registry": "https://registry.npmjs.org/"
}
},
"os": [
"darwin"
]
}
49 changes: 49 additions & 0 deletions scripts/check-package.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
#!/usr/bin/env node
/**
* Verify the packed npm tarball includes every runtime file.
* Runtime imports outside bin/, src/, and package.json are refused so a
* missing "files" entry cannot ship a broken release.
*/
import { execFileSync } from 'node:child_process';
import { readFileSync } from 'node:fs';
import { join } from 'node:path';

const root = join(import.meta.dirname, '..');
const pkg = JSON.parse(readFileSync(join(root, 'package.json'), 'utf8'));

// --ignore-scripts: this script runs from `prepack`, so a nested pack that
// ran lifecycle scripts would recurse forever.
const pack = JSON.parse(execFileSync('npm', ['pack', '--dry-run', '--json', '--ignore-scripts'], { cwd: root, encoding: 'utf8' }));
const files = new Set(pack[0].files.map((file) => file.path));

const missing = [];
const check = (path, reason) => {
if (!files.has(path)) missing.push(`${path} (${reason})`);
};

const entry = pkg.bin['fm-bench'];
check(entry, 'bin entry');

const seen = new Set();
const queue = [entry];
const importRe = /from\s+['"](\.\.?\/[^'"]+)['"]/g;
while (queue.length > 0) {
const current = queue.pop();
if (seen.has(current)) continue;
seen.add(current);
const source = readFileSync(join(root, current), 'utf8');
for (const match of source.matchAll(importRe)) {
const resolved = join(current, '..', match[1]).replace(/\\/g, '/');
queue.push(resolved);
check(resolved, `imported by ${current}`);
}
}

if (!files.has('package.json')) missing.push('package.json');

if (missing.length > 0) {
console.error(`Packed tarball is missing runtime files:\n ${missing.join('\n ')}`);
process.exit(1);
}

console.log(`check:pack ok — ${seen.size} runtime module(s) all present in tarball (${files.size} files total).`);