Conversation
|
Tested at I drove the actions through Run A: tor 0.4.9.11:5 → this PR, with bitcoind already on the released native 31.1:17.
Run B: released tor 0.4.9.12:5 parks it first, then this PR.
The untouched RPC and admin onions answered through Tor in both runs. One behavior to be aware of, which I don't think is a defect in this PR: the first connection to the reattached address, about a minute after it was rendered, failed with Not exercised: submitting Delete Unused, submitting Delete Onion Service, a fresh install, a restore, and the aarch64/riscv64 builds. |
|
Pushed b4add8a: This branch won't typecheck yet. It still pins start-sdk 2.0.9, which has neither End to end on a VM (0.3.5.1, then OTA, then a StartOS build from start-technologies#4069):
|
There was a problem hiding this comment.
Not ready to release at this head.
Correctness finding: exportUrls checks only that the internal binding is enabled, while renderTorrc also requires getBridgeAddress({…, ssl}) to resolve. If a service keeps its host/port but changes from a two-leg binding to TLS-only, an existing plaintext onion mapping is omitted from torrc but still exported on the interface. The export watcher also sees no change (its projection is still the same enabled-port list), so it can retain that dead URL indefinitely. The old reconciler handled a missing SSL mode; the rework removes that path. I exercised the two actual handlers with an enabled binding and a null matching bridge address: no HiddenServicePort is rendered, but one URL is exported. Make export eligibility use the same matching bridge-leg condition, watched with .const(), as rendering; keep the projection narrow enough not to react to the plugin's own exports.
Release blockers:
- This branch conflicts with master, which now ships Tor
0.4.9.13:0;0.4.9.12:7is behind it. Rebase and choose the next revision on the newer upstream line. - With a clean
npm ci, bothnpm run checkandnpm run buildfail on the threecallerreads. SDK 3.0.0 is still unpublished (npm latest is 2.0.9). - A pin bump alone no longer suffices. I built and packed SDK 3.0.0 from current monorepo master
3454c4f6f, installed it in a throwaway archive of this head, and found additional errors: removedsdk.setupDependencies, the new thirdbuildManifestargument, and the removed manifestdependenciesfield. The package needs the current SDK-3 migration as well. - CI is skipped because the PR is draft. Marking it ready alone will not trigger this package's current workflow; it needs a subsequent synchronize event or an explicit build dispatch.
Fleet-scope correction: I originally treated historical host-id changes as migration defects and proposed cross-host migration across the fleet. That conclusion did not establish the reason for those changes and is withdrawn. A deliberately replaced or split host need not inherit the old host's addresses; LND's REST/gRPC split does not need migration. Automatic repair remains appropriate for a confirmed port-renumbering defect such as Bitcoin's peer/8333 → peer/58333. Allowing an entirely unused address to be explicitly reused on another host of the same package is a separate capability, not a reason to move addresses automatically or a release blocker for this PR.
Two independent cleanup/preservation points:
- The renamed-host importer still does
if (!host) continueand then consumes the handoff file. A missing historical host must not prevent preserving its key and an unused store entry; otherwise no later action can recover it from the store. This is inherited from the old importer, not newly introduced by the rework. - The removed relay's
or-multihost is only disabled, not retired. Now that this package needs SDK 3 anyway, retire it explicitly in the migration rather than reserving its ports permanently. Disclose that retirement's effect on user-added domains in the release notes.
Checks in this review: clean installs, typecheck and ncc build on the committed SDK pin (both fail); current-SDK-3 consumer typecheck in a scratch tree (fails as above); Prettier on changed TypeScript (passes); mocked-handler render/export comparison. I did not repeat the earlier VM runs or complete fresh-install/restore/delete-action testing.
…utomatically; 0.4.9.12:6 → 0.4.9.12:7 torrc was generated from structured state and then parsed back as the source of truth, and most of the fixes since #19 were patches on that choice. The decisions this implements are recorded in #38. - Onion state moves to store.json on the startos volume. torrc is rendered from it one way, below a marker; the section above the marker is the user's and is carried over byte for byte. A forward target is resolved from the live binding at render time and never stored, so there is nothing to reconcile and no stale target to inherit another service's port. The reconcile pass, parked entries and the annotation parser go; the old two-way model is frozen at versions/legacy/torrc.ts for the migrations that read old volumes. - Relay and bridge mode are removed. The release notes say why. - Nothing deletes a key or a mapping automatically. torrc and the exported URLs hold only what resolves right now, an address whose target is gone becomes unused and returns if the target does, and restore order needs no special case. Delete Onion Addresses becomes Delete Unused Onion Addresses, the only thing that destroys a key; the interface-page delete detaches and keeps it. - Tor's DataDirectory moves to data/, so a reset deletes that directory instead of walking an allow-list. The migration carries state and the caches over so the update does not re-select entry nodes. - The recovery watchdog stays on by default and gains an Automatic Recovery action; off is fail closed. - Backups cover both volumes. down is IMPOSSIBLE: an older release would read the rendered torrc as its database and find no annotations in it. Helix-Harness: pi Helix-Model: openai-codex/gpt-6.1-sol
StartOS keeps a disabled binding's assigned port and bridge address, so getBridgeAddress still resolves for it. torrc kept forwarding the port to nothing, the URL stayed exported, and the address counted as in use only by that accident. - renderTorrc skips a port whose binding is disabled, watching the flag with its own .const(): toggling it leaves the bridge address unchanged, so the target's watch would not re-run the render. - exportUrls exports enabled bindings only, so the URLs match what Tor serves. - isServed counts a disabled binding as in use explicitly. Its service still holds the port and can enable it again, so Delete Unused Onion Addresses does not offer the address. Verified on the dev VM (StartOS 0.4.0.2): tsc passes, the three existing onions still render, and Delete Unused lists nothing. Not exercised: an onion on a disabled binding, since the VM has none and store.json is not reachable from any subcontainer. Helix-Harness: pi Helix-Model: openai-codex/gpt-6.1-sol
Add Onion Service and Delete Onion Service become access: 'public', so a service can run them through effects.action.run, for instance to move an address back onto a port it renumbered. requireOwner refuses any caller other than the user (null) or the service whose host the address is on; Add also checks in its input form, which otherwise lists that host's unused addresses. Delete Unused Onion Addresses, the only action that destroys a key, stays user-only. Needs start-sdk 3.0.0 for the action's caller (start-technologies#4045). Helix-Harness: pi Helix-Model: openai-codex/gpt-6.1-sol
b4add8a to
c7c4c5c
Compare
…31.1:18 The 0.3.5 package bound container port 8333 on the peer host. Every 0.4-native version binds 58333 there instead, and setupInterfaces disables what it no longer declares rather than deleting it, so a server migrated from 0.3.5 kept the 8333 record: enabled false, holding external port 8333, with Tor still forwarding the peer .onion to it. up() retires it with retirePort(8333). Under Start9Labs/tor-startos#39 the address becomes unused, key kept, and Add Onion Service on the Peer interface offers it back for 58333 with the same hostname. On a server that never carried the binding, retirePort resolves false and nothing happens. Needs start-sdk 3.0.0 and StartOS 0.4.0.2, neither published to npm yet. Helix-Harness: pi Helix-Model: openai-codex/gpt-6.1-sol
Helix-Harness: pi Helix-Model: openai-codex/gpt-6.1-sol
c7c4c5c to
fe3c2dc
Compare
helix-a
changed the title
|
Pushed the rebase and fixes in fe3c2dc: shared render/export eligibility, unused-address dropdown and explicit same-package cross-host reuse with stable keys, fail-closed lookup errors, absent-host import preservation/retry safety, relay-host retirement, SDK-3 API conversion, and an exact Tor APK pin. README/instructions/UPDATING are synced. Seven regression tests and all three architecture packs pass with the local SDK-3 build; live create/detach/reuse/restart/delete and a stopped released-version upgrade passed. The description now separates current verification from the earlier test record. Still draft pending published SDK-3 pins/locks; a running same-version sideload stalled on the alpha VM and remains an explicit, unattributed runtime gap. |
2 participants
Closes #38, which records the decisions this implements and why. Answers #36 (decision 4 there).
What changes
torrcis rendered one way. Onion state moves tostore.jsonon thestartosvolume.torrcis written from it below a marker line; everything above the marker is the user's and is preserved with trailing newlines normalized. A forward target is resolved from the live binding when the file is rendered and is never stored, so there is nothing to reconcile and no stale target left to inherit a port another service later claims. The reconcile pass, parked entries and the annotation parser are gone. The old two-way model is frozen atstartos/versions/legacy/torrc.ts, read only by the historical migration and the current layout migration.keys/is left in place.torrcand the exported URLs hold only what resolves right now. An address whose package was uninstalled, or whose host or port was retired, becomes unused and comes back if its target does, so restore order needs no special case. Delete Onion Addresses becomes Delete Unused Onion Addresses: every unused address, all selected by default, and the only thing that destroys a key. If a selected address has come into use by the time it runs, it deletes nothing and fails naming it. The interface-page delete detaches and keeps the key; Add Onion Service offers unused addresses from any host of the same package for explicit reuse; an address still in use stays on its existing host.DataDirectorymoves todata/, so Reset Tor Connection deletes that directory instead of walking an allow-list. The migration carriesstateand the caches over, so the update does not re-select anyone's entry nodes.instructions.mdrewritten;AGENTS.mdcut from ten bullets to three.down: IMPOSSIBLE: an older release would read the renderedtorrcas its database and find no annotations in it.Earlier implementation verification (historical)
x86_64 VM, StartOS 0.4.0.2, Tor updated from
0.4.9.11:5, so the whole migration chain ran. A VM snapshot was taken first..onionhostname is identical to before, and so is the guard set. The old file is set aside astorrc.legacy.*.startoscontainer hostnames. Both now resolve from the live binding; qbittorrent's is10.0.3.1:59231, its assigned port rather than the 8080 it advertises, which is Onion service entries are never reconciled against the binding they were derived from #17's drift.delete-onion-serviceas their remove action.SocksPortbeside 9050, and the line survived a full container rebuild.data/at the next start (57 guards to 20), left the onions untouched, and health returned to success.tor --verify-configaccepts the result; the lastDataDirectorywins whileSocksPortandControlSocketare additive; guards live only instate.tsc, Prettier,nccandmake x86pass.Noticed, not caused by this
On that box the
start-os/adminonion, an SSL one, fails its TLS handshake withunrecognized name, while the same onion and port with no SNI return HTTP 200. That is a StartOS regression onmaster, not this package: the vhost lookup became exact and plugin-exported hostnames were never given an entry. Fixed in Start9Labs/start-technologies#4044. Plaintext onions are unaffected.Current-head follow-up — 0.4.9.13:1
Rebased on current
master; this now targets0.4.9.13:1.keyId; disabled bindings count as in use. No automatic cross-host migration is introduced.or-multi, freeing its bindings. Custom domains assigned to that host must be reattached; release notes explain this in each locale.torVersionsupplies both the build argument and the package version's upstream component;UPDATING.mddocuments the exact APK selector.Verification for this follow-up
Using a locally packed SDK 3.0.0 from monorepo
3454c4f6f(no SDK/core-TS changes between that commit andd55cf47cf):make x86,make arm, andmake riscvpass with start-cli 2.1.0. The pinned x86 image reports Tor 0.4.9.13.6c16910: fresh Tor/Bitcoin installs start successfully; new onion creation agrees with rendered config and URL export; detaching an RPC onion and explicitly reusing it on Peer retains its hostname and key hash, exports public port 8333, survives a Tor restart, and deletes the original key directory when subsequently deleted as unused.0.4.9.13:0while stopped imports the existing mapping, preserves its hostname/key hash, retainstorrc.legacy, and retires a relay host configured while stopped.The VM tests above preceded the final APK pin (the cached daemon there was 0.4.9.12); the final pinned image was checked with Docker, not re-exercised on that VM. Full restore/install-order permutations and the complete 0.3.5 migration path were not rerun in this follow-up.
Release prerequisites / outstanding runtime check
Remain draft. npm still publishes SDK 2.0.9. The committed SDK pin/lockfile remain unchanged; this branch's SDK-3 calls do not compile against that pin. Publish SDK 3, refresh the normal dependency pin/lockfile, then rerun ordinary clean-install CI. No local tarball dependency is committed.
A later running, same-version sideload remained
updatingafter the client's 240-second timeout (Tor's process stayed alive). This is not established as a package, SDK, or alpha-runtime fault; stopped upgrade and restart tests passed. Hot-update behavior remains an explicit verification gap, not a green check.🤖 Generated with Claude Code