Skip to content

catalogue: add io.pilot.firecrawl v0.1.0, drop io.pilot.insforge, fix agentphone metadata pin - #447

Closed
Alexgodoroja wants to merge 2 commits into
mainfrom
catalogue/firecrawl-v0.1.0
Closed

Alexgodoroja wants to merge 2 commits into
mainfrom
catalogue/firecrawl-v0.1.0

Conversation

@Alexgodoroja

Copy link
Copy Markdown
Collaborator

Three catalogue changes, one re-sign.

1. Add io.pilot.firecrawl v0.1.0

The complete Firecrawl v2 API — all 50 operations generated 1:1 from Firecrawl's published OpenAPI spec, plus firecrawl.help and the auto firecrawl.balance (51 methods). Keyless/managed: the broker holds one partner key, verifies each caller's ed25519 identity, meters per user, and enforces per-user resource ownership.

Submission merged as pilot-protocol/app-template#95. Bundle released at pilot-protocol/catalog@firecrawl-v0.1.0.

Verified end to end against this exact bundle, via a locally-signed catalogue pointing at the released URL:

  • install fetched the released tarball, bundle_sha256 matched, product_demo printed at install;
  • the daemon supervised it to ready and registered all 51 methods;
  • real calls succeeded (scrape, map, search, crawl, crawl_status, research_papers, docs_search, balance);
  • cross-tenant isolation holds: a second Pilot identity gets 404 on another user's job id, and on a cancel attempt;
  • the next_steps graph renders correctly, including a match-on-success-body edge ("status":"scraping" → keep polling).

Note: publish-on-merge could not do any of this automatically — it failed with HTTP 401 against pilot-protocol/catalog, so the CATALOG_PUBLISH_TOKEN secret needs rotating. The release and this PR were produced by hand following scripts/publish-submission.sh. The same 401 killed agentphone's publish on 2026-07-29.

2. Remove io.pilot.insforge

Dropped from apps[] as requested.

3. Fix io.pilot.agentphone's metadata pin

io.pilot.agentphone is currently broken for every client:

$ pilotctl appstore view io.pilot.agentphone
warn: could not load detail metadata: metadata sha256 mismatch for io.pilot.agentphone:
      want=d1b5fea717be1a3c21fc3fcf38c23362b8a727502e822c23dd3c831dab950823
      got=452d988a4814c3e92f2af095e7a02466a150f088478dda819fec2d6c724fbb9a

catalogue.json and catalogue/apps/io.pilot.agentphone/metadata.json were changed in the same commit (#438, 216bcfc), but the recorded metadata_sha256 was computed from a different revision of the file than the one committed. The served bytes have hashed to 452d988a… ever since, so the store page fails its integrity check.

Fixed by pinning the sha of the metadata actually being served. No metadata content change.

Signature

catalogue.json re-signed with CATALOG_SIGN_KEY; verified against the embedded trust anchor (iHdBWayA/hYjkwUOZopTXY70qOlR90d6ii/hin0ZMdI=) before pushing. Still version: 2.

🤖 Generated with Claude Code

… agentphone metadata pin

- add io.pilot.firecrawl v0.1.0 (full Firecrawl v2 API, 51 methods, managed/keyless)
- remove io.pilot.insforge
- fix io.pilot.agentphone metadata_sha256: the pin has been wrong since #438, so
  every client fails the integrity check on its store page
- catalogue.json re-signed with CATALOG_SIGN_KEY

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The store page and the install-time demo were publishing our cost basis with
Firecrawl: the micro-USD ledger unit, the 1 credit ~= 830 micro-USD conversion,
per-operation dollar prices, and that Pilot runs one shared Firecrawl team.

Restated as the published per-user limits: 1000 credits, 2 concurrent calls.
metadata_sha256 recomputed and catalogue.json re-signed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Alexgodoroja pushed a commit that referenced this pull request Aug 29, 2026
The signature is over the exact bytes of catalogue.json, so the listing and
index changes on this branch invalidated it. pilotctl fail-closes on a bad
signature — it rejects the whole catalogue rather than one entry — so the
detached signature has to travel with the data, the same way #447 did it.

Signed with the catalogue key; sign-catalogue refuses any key that does not
match the public key embedded in internal/catalogtrust, so acceptance is itself
proof the right key was used.

Verified: cmd/pilotctl TestLoadCatalogue_VerifiesSignedRepoCatalogue passes
again, along with FailsClosedWithoutSignature and FailsOnTamper; and a released
v1.13.9 pilotctl loads the re-signed catalogue from disk and lists all 29 apps.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Alexgodoroja added a commit that referenced this pull request Sep 1, 2026
…l index fields (#458)

* catalogue: align published method lists with shipped bundles; backfill index fields

Audited all 29 catalogue apps against three published surfaces (catalogue.json,
apps/<id>/metadata.json, the appstore-meta API) and against ground truth: each
bundle's signed manifest `exposes` list, cross-checked with the installed app's
own `<app>.help` on a clean Linux node.

Twelve apps published a method list that did not match the app they ship.

Methods
- io.pilot.orthogonal: drop orthogonal.check / .transactions / .usage. The
  v0.1.1 bundle exposes 7 methods and none of these three. orthogonal is
  metered, so these were exactly the calls an agent would make to check its own
  spend — the method that does exist is orthogonal.balance.
- Document methods the bundles expose but the listing omitted:
  agentphone.setup, agentphone.status; bowmark.balance; kinetic.balance;
  rentahuman.balance; ideon-free.generate, ideon-free.poll; and <app>.help for
  didit, insforge, mysql, postgres, tldr.
  Summaries are each app's own runtime help text, not new copy.
  wallet.hookPreSendMessage / .hookPostRecvMessage are deliberately left
  undocumented — they are daemon lifecycle hooks, not user-callable methods,
  and no other app documents its hooks.

Names, so the same app is called the same thing everywhere
- mysql:      "Mysql"      -> "MySQL"
- tldr:       "Tldr"       -> "tldr"
- ideon-free: "Ideon-free" -> "Ideon (Free)", and the index display_name was
  empty, so `pilotctl appstore catalogue` printed its whole description
  paragraph in the name column.

Index fields
- metadata_sha256 for io.pilot.agentphone was stale, not wrong: the pin said
  d1b5fea7 while both the repo file and raw.githubusercontent served 452d988a.
  `pilotctl appstore view io.pilot.agentphone` therefore warned and fell back
  to the teaser. Re-pinned, along with every metadata.json touched here.
- categories were empty for agentphone, miren, orthogonal, otto, plainweb and
  postgres; filled from each listing.
- license was empty for orthogonal; set to MIT to match its listing.
- bundle_size was 0 for agentphone, ideon-free, miren, otto, plainweb and
  postgres, and wrong for sixtyfour (4898153 declared, 5058693 served). All set
  from the bytes actually served at each bundle_url.
- changelog head trailed the released version for agentphone (v0.3.0 vs v0.3.1)
  and orthogonal (v0.1.0 vs v0.1.1).

catalogue.json stays "version": 2 — `loadCatalogue` fail-closes on any other
value. io.telepat.ideon-free/metadata.json is emitted by Go, whose <
escaping a Python round-trip cannot reproduce, so it was edited textually to
keep the diff to the intended lines.

Verified: all 28 metadata.json parse; every metadata_sha256 matches its file;
zero remaining differences between documented methods and shipped `exposes`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* catalogue: backfill license and categories from each app's submission listing

Four more index entries had both fields empty on every surface, so the earlier
catalogue-vs-listing drift check could not see them — nothing disagreed because
nothing was set anywhere.

Values come from the `listing` block of submissions/<id>/submission.json in
pilot-protocol/app-template, which is what the publish pipeline reads:

  io.pilot.insforge  Apache-2.0   developer-tools, data, storage
  io.pilot.mysql     GPL-2.0      database, data, sql
  io.pilot.didit     Proprietary  identity, verification, compliance, security, kyc
  io.pilot.tldr      MIT          developer-tools, documentation, cli

insforge's submission spells its categories "Developer Tools" / "Data & Storage";
normalised here to the catalogue's existing all-lowercase vocabulary, which
already contains developer-tools, data and storage.

io.telepat.ideon-free is left as-is: it is third-party, has no submissions/
directory in app-template, and its licence is not ours to assert.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* catalogue: re-sign catalogue.json

The signature is over the exact bytes of catalogue.json, so the listing and
index changes on this branch invalidated it. pilotctl fail-closes on a bad
signature — it rejects the whole catalogue rather than one entry — so the
detached signature has to travel with the data, the same way #447 did it.

Signed with the catalogue key; sign-catalogue refuses any key that does not
match the public key embedded in internal/catalogtrust, so acceptance is itself
proof the right key was used.

Verified: cmd/pilotctl TestLoadCatalogue_VerifiesSignedRepoCatalogue passes
again, along with FailsClosedWithoutSignature and FailsOnTamper; and a released
v1.13.9 pilotctl loads the re-signed catalogue from disk and lists all 29 apps.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* go.mod: bump to 1.25.13 for the August stdlib security fixes

govulncheck resolves the toolchain from go-version-file: go.mod, so the 1.25.12
pin meant CI scanned against a stdlib with five disclosed vulnerabilities and
failed every PR opened against the repo today, whether or not it touched Go:

  GO-2026-6218  net/url        GO-2026-6090  crypto/tls
  GO-2026-6088  encoding/xml   GO-2026-5972  encoding/asn1
  GO-2026-5026  net/http

All five are standard-library only and all five are fixed in go1.25.13, so the
patch bump clears the whole set. main has not been scanned since 2026-08-09,
before these were published, which is why it still shows green.

This is separable from the catalogue work in the rest of the branch and can be
cherry-picked out or landed on its own; it is here because the branch cannot go
green without it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Alexgodoroja <alex@vulturelabs.io>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@TeoSlayer

Copy link
Copy Markdown
Collaborator

Closing as obsolete. The shared-account Firecrawl entry is superseded by the isolated-account design, and the AgentPhone metadata pin correction is already on main. Any remaining removal should be a small newly signed catalogue change.

@TeoSlayer TeoSlayer closed this Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants