Skip to content

Evaluate refunds through the ShopMate buyer runtime - #18

Merged
ChanTso merged 2 commits into
mainfrom
codex/shopmate-ownership-adapter
Sep 7, 2026
Merged

ChanTso merged 2 commits into
mainfrom
codex/shopmate-ownership-adapter

Conversation

@ChanTso

@ChanTso ChanTso commented Sep 7, 2026

Copy link
Copy Markdown
Owner

The historical experiment drives the retired CityBuddy customer-service loop. This change evaluates the current ShopMate buyer instead: actual streamed model chat, a refund confirmation card, the original customer's click, and authoritative Java refund state. The historical results remain explicitly scoped to their original chain.

The isolated launcher starts MySQL, Auth and two evaluation Commerce instances with identical settings except resource ownership binding. A sandbox-bound host reuses the installed ShopMate factory, tools, order lookup, policy grounding, memory and shared budget. Normal controls require an own-order refund and receipt replay before a balanced foreign-order pilot can start. Raw SELECT-only SQL judges business state; model failures and unknown writes remain separate and retain the fixture. Shutdown reaps only owned processes.

Validation:

  • make check: 147 standard-library tests and 26 real-factory boundary tests passed; compile, shell syntax and diff checks passed.
  • Independent read-only review: no blocking findings after fixing interrupted-host cleanup.
  • Real-model runs on committed, clean revision 9f5b49584df04fc88a0cea02223b81b6fee80a6f: initial own-order controls 2/2 passed; fresh pilot controls 2/2 passed; three balanced foreign-order pairs produced 0/3 unauthorized refunds in each arm with no operational failures. All foreign traces queried owner-scoped orders and stopped before refund preparation, so this is an inconclusive incremental-ownership calibration, not a formal finding. Raw SQL, streams and receipts are retained locally; only the aggregate result and experiment boundary are published in results/shopmate-ownership-v1/README.md. Both launchers exited successfully and removed their owned topology.

CI now checks the real ShopMate factory against pinned commit 2a69bfec2aa29359e38f2c6bf829f824263672c7 using its locked dependencies. Provider credentials remain in CityBuddy's existing local environment file; generated runtime credentials are private and absent from artifacts and commits.

@ChanTso
ChanTso merged commit c3ee62d into main Sep 7, 2026
2 checks passed
@ChanTso
ChanTso deleted the codex/shopmate-ownership-adapter branch September 12, 2026 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant