Filing gate: ① a product defect, with its reach measured locally (cloud's real kernel factory on the hosted Turso face). The staging attribution is pending one log read (see Reach).
Filed by the repo:cloud seat (repo:cloud#1, session session_01Wxo1xhh2bU66T73q23jzE4, R44), from objectstack-ai/cloud#2637 (p1), round 2 (os-dev-report 6031983341 on that card). ⛔ Not a claim. The maintainer ranked cloud#2637 first today.
The step
bootstrapEffectiveEmailTemplates in packages/plugins/plugin-email/src/bootstrap-declared-email-templates.ts (loop at about :412, called from email-plugin.ts:1105 at EmailServicePlugin start) calls upsertDeclaredEmailTemplate (about :276-323) for every template of the effective list, on every boot. Each call:
- finds the row with
find(..., { where: { name, locale }, limit: 1 }). That is SCAN sys_email_template, because the declared unique index is (COALESCE(organization_id,'__global__'), name, locale) and cannot serve (name, locale);
UPDATEs it unconditionally, even when nothing changed;
- runs the engine's two read-backs by id.
That is 4 serial round trips per template per boot, and K × T rows visited.
08adfeade (#21818, Fixes #21785) reads the list through protocol.getMetaItems in tenancy.defaultOrgId() (about :234), instead of the registry. That is correct for #21785: an org-scoped template edit must survive a boot. But it means the default organization's email_template overlays now enter this per-template loop. The unconditional per-template rewrite itself predates that commit.
Reach (measured)
All figures are from cloud's ArtifactKernelFactory on the hosted Turso remote face, with byte-identical database copies per pin, steady-state boots, and 36 ms injected per round trip.
| shape |
7d078148 |
4e4e881427 (17.7.0) |
| HotCRM, 79 templates (every hosted HotCRM environment) |
26.2 s |
27.0 s |
| + 1,000 default-org template overlays |
26.2 s |
183.0 s |
| + 5,000 default-org template overlays |
26.3 s |
812.6 s (first build 814.5 s, past the 600 s hard timeout) |
- On the plain HotCRM shape, the sweep is 316 of the phase's ~830 serial round trips.
8832655a matches 4e4e881427 on every shape.
- Staging (cloud#2637). Since cloud moved to
8832655a, every build of the large environment 1ac85ba2 has passed the 600 s hard timeout, with a phase of about 14 minutes. Whether 1ac85ba2 holds thousands of default-org email templates is not yet known. One staging log line decides it, and the maintainer is asked for it on cloud#2637.
- Independent of that reading, every hosted HotCRM environment pays the rewrite on every boot.
Proposed fix (at the producer; keeps #21785's contract)
- Bulk-read the stored rows for the declared names, in
$in pages of 200, instead of one find per template. The first row by id wins, as before.
- Compare before write: a
managed_by: 'package' row that already holds the projected columns is not rewritten.
- Fall back: if the bulk read fails, use the per-template lookup.
The live doors keep their own lookup. One source file (+72/-1) and its test (+18/-1). No spec, schema, contract or migration change.
A rig-only trial at 4e4e881427 (reverted, never pushed) measured:
- 5,000 default-org templates: 812.6 s → 20.1 s at 36 ms; round trips 20,717 → 427 at 0 ms;
- the plain HotCRM shape: 27.0 s → 15.1 s at 36 ms (1.59 s → 1.28 s at 0 ms);
- plugin-email tests: 44 passed. The new test fails on the unpatched source:
expected { seeded: 450, skipped: 0 } to deeply equal { seeded: 0, skipped: 450 }.
The draft patch below is against 4e4e881427; the file pair is identical at 8832655a. It is a starting point for the implementer, not a reviewed change.
--- a/packages/plugins/plugin-email/src/bootstrap-declared-email-templates.ts
+++ b/packages/plugins/plugin-email/src/bootstrap-declared-email-templates.ts
@@ -266,4 +266,48 @@
}
+/** [cloud#2637] The sweep's row key: a template is unique per `(name, locale)`. */
+function templateKey(name: unknown, locale: unknown): string {
+ return JSON.stringify([String(name), String(locale)]);
+}
+
+/**
+ * [cloud#2637] True when every column the projection writes already holds the
+ * projected value: the same column set an update would write. Booleans are
+ * stored as 0/1 and an absent optional column reads as null, so both sides are
+ * normalized before the compare.
+ */
+function projectionHeld(row: { [column: string]: unknown }, projected: { [column: string]: unknown }): boolean {
+ const norm = (v: unknown) => (v === undefined ? null : typeof v === 'boolean' ? (v ? 1 : 0) : v);
+ return Object.entries(projected).every(([k, v]) => norm(row[k]) === norm(v));
+}
+
+/**
+ * [cloud#2637] The stored rows for the declared names, read in `$in` pages of
+ * 200 instead of one lookup per template. The first row by id wins, as the
+ * per-template `find(..., { limit: 1 })` it replaces resolved it. Undefined
+ * when the read fails: the sweep then looks up per template, as before.
+ */
+async function prefetchTemplateRows(engine: IDataEngine, declared: unknown[], object: string) {
+ const names = [...new Set(declared.map((d) => (d as { name?: unknown })?.name))].filter((n) => typeof n === 'string');
+ const byKey = new Map();
+ try {
+ for (let i = 0; names.length > i; i += 200) {
+ const found = await (engine as any).find(object, {
+ where: { name: { $in: names.slice(i, i + 200) } },
+ context: SYSTEM_CTX,
+ });
+ const rows: any[] = Array.isArray(found) ? found : ((found as any)?.data ?? []);
+ for (const row of rows) {
+ const key = templateKey(row?.name, row?.locale);
+ const held = byKey.get(key);
+ if (!held || String(held.id) > String(row.id)) byKey.set(key, row);
+ }
+ }
+ } catch {
+ return undefined;
+ }
+ return byKey;
+}
+
/**
* Materialize ONE declared template into `sys_email_template`, honouring
@@ -289,5 +333,22 @@
});
const row: any = Array.isArray(existing) ? existing[0] : (existing as any)?.data?.[0];
+ return writeTemplateRow(engine, tpl, row, now, object, logger);
+}
+/**
+ * [cloud#2637] Write one parsed template against the row already read for its
+ * `(name, locale)`: by the live doors' own lookup above, or by the boot sweep's
+ * bulk read. Seed-not-clobber as before, and a row that already holds the
+ * projection is not rewritten (before, every boot rewrote every template: one
+ * UPDATE plus its two read-backs each).
+ */
+async function writeTemplateRow(
+ engine: IDataEngine,
+ tpl: EmailTemplateDefinition,
+ row: any,
+ now: string,
+ object: string,
+ logger?: Logger,
+) {
if (row?.id) {
// Admin owns a same-named row, or has edited this seeded one — never
@@ -301,4 +362,5 @@
}
if (row.customized === true) return false;
+ if (row.managed_by === 'package' && projectionHeld(row, mapTemplateToRow(tpl))) return false;
await (engine as any).update(object, {
id: row.id,
@@ -410,7 +472,16 @@
let skipped = 0;
+ const prefetched = await prefetchTemplateRows(engine, declared, object);
+
for (const raw of declared) {
try {
- const written = await upsertDeclaredEmailTemplate(engine, raw, object, logger);
+ let written: boolean;
+ if (prefetched) {
+ const tpl = EmailTemplateDefinitionSchema.parse(raw);
+ const row = prefetched.get(templateKey(tpl.name, tpl.locale));
+ written = await writeTemplateRow(engine, tpl, row, new Date().toISOString(), object, logger);
+ } else {
+ written = await upsertDeclaredEmailTemplate(engine, raw, object, logger);
+ }
if (written) seeded += 1;
else skipped += 1;
--- a/packages/plugins/plugin-email/src/bootstrap-declared-email-templates.test.ts
+++ b/packages/plugins/plugin-email/src/bootstrap-declared-email-templates.test.ts
@@ -56,3 +56,5 @@
if (!cond) return true;
- return Object.entries(cond).every(([k, v]) => row[k] === v);
+ // `$in` as the real engine reads it: the bulk read the boot sweep issues.
+ return Object.entries(cond).every(([k, v]) =>
+ v && typeof v === 'object' && Array.isArray((v as any).$in) ? (v as any).$in.includes(row[k]) : row[k] === v);
}
@@ -173,2 +175,17 @@
});
+
+ it('[cloud#2637] a boot over unchanged templates writes nothing and reads in bulk, not per template', async () => {
+ const declared = Array.from({ length: 450 }, (_, i) => declaredTemplate({ name: `tpl.n${i}` }));
+ const engine = new FakeEngine({ declared: { email_template: declared } });
+ await bootstrapDeclaredEmailTemplates(engine as any, undefined);
+
+ const find = vi.spyOn(engine, 'find');
+ const update = vi.spyOn(engine, 'update');
+ const result = await bootstrapDeclaredEmailTemplates(engine as any, undefined);
+
+ expect(result).toEqual({ seeded: 0, skipped: 450 });
+ expect(update).not.toHaveBeenCalled();
+ expect(find).toHaveBeenCalledTimes(3);
+ expect(rowsOf(engine)).toHaveLength(450);
+ });
⛔ Not the fix: reverting or narrowing 08adfeade back to the registry. That reopens #21785: an org-scoped template edit lost on the next boot.
Timing
Cloud stays on 17.x until its own v18 ceremony (objectstack-ai/cloud#1979; ruling recorded on #15193). So this fix reaches cloud only if it lands on objectstack main before the v18 opening. Its size fits that window.
Done when
- A steady boot over unchanged templates issues no
sys_email_template write, and one read per 200 template names (the pin test above).
- If cloud#2637's staging reading attributes the time to this step:
1ac85ba2's first kernel build reaches kernel ready inside the 600 s hard timeout on a cloud pin that carries the fix. That half is verified on cloud#2637.
Also measured, not filed (costs, not defect classes)
18c2ddc1e adds one index-served sys_metadata read per stored active permission set per boot (overlayLockLayerAt beside getMetaItem's findOne): about +22 s at 36 ms for 600 sets.
backfillOrgAdminGrants (plugin-security) reconciles every member on every boot, 5 serial statements per member up to 5,000. That is about 25,900 round trips per boot at 20,000 members, at every pin.
Reader
The objectstack lane that owns plugin-email. The repo:cloud seat verifies the cloud half on cloud#2637 after the pin carries the fix.
Generated by Claude Code
Filing gate: ① a product defect, with its reach measured locally (cloud's real kernel factory on the hosted Turso face). The staging attribution is pending one log read (see Reach).
Filed by the
repo:cloudseat (repo:cloud#1, sessionsession_01Wxo1xhh2bU66T73q23jzE4, R44), from objectstack-ai/cloud#2637 (p1), round 2 (os-dev-report 6031983341 on that card). ⛔ Not a claim. The maintainer ranked cloud#2637 first today.The step
bootstrapEffectiveEmailTemplatesinpackages/plugins/plugin-email/src/bootstrap-declared-email-templates.ts(loop at about:412, called fromemail-plugin.ts:1105atEmailServicePluginstart) callsupsertDeclaredEmailTemplate(about:276-323) for every template of the effective list, on every boot. Each call:find(..., { where: { name, locale }, limit: 1 }). That isSCAN sys_email_template, because the declared unique index is(COALESCE(organization_id,'__global__'), name, locale)and cannot serve(name, locale);UPDATEs it unconditionally, even when nothing changed;That is 4 serial round trips per template per boot, and K × T rows visited.
08adfeade(#21818, Fixes #21785) reads the list throughprotocol.getMetaItemsintenancy.defaultOrgId()(about:234), instead of the registry. That is correct for #21785: an org-scoped template edit must survive a boot. But it means the default organization'semail_templateoverlays now enter this per-template loop. The unconditional per-template rewrite itself predates that commit.Reach (measured)
All figures are from cloud's
ArtifactKernelFactoryon the hosted Turso remote face, with byte-identical database copies per pin, steady-state boots, and 36 ms injected per round trip.7d0781484e4e881427(17.7.0)8832655amatches4e4e881427on every shape.8832655a, every build of the large environment1ac85ba2has passed the 600 s hard timeout, with a phase of about 14 minutes. Whether1ac85ba2holds thousands of default-org email templates is not yet known. One staging log line decides it, and the maintainer is asked for it on cloud#2637.Proposed fix (at the producer; keeps #21785's contract)
$inpages of 200, instead of onefindper template. The first row by id wins, as before.managed_by: 'package'row that already holds the projected columns is not rewritten.The live doors keep their own lookup. One source file (+72/-1) and its test (+18/-1). No spec, schema, contract or migration change.
A rig-only trial at
4e4e881427(reverted, never pushed) measured:expected { seeded: 450, skipped: 0 } to deeply equal { seeded: 0, skipped: 450 }.The draft patch below is against
4e4e881427; the file pair is identical at8832655a. It is a starting point for the implementer, not a reviewed change.⛔ Not the fix: reverting or narrowing
08adfeadeback to the registry. That reopens #21785: an org-scoped template edit lost on the next boot.Timing
Cloud stays on 17.x until its own v18 ceremony (objectstack-ai/cloud#1979; ruling recorded on #15193). So this fix reaches cloud only if it lands on objectstack
mainbefore the v18 opening. Its size fits that window.Done when
sys_email_templatewrite, and one read per 200 template names (the pin test above).1ac85ba2's first kernel build reacheskernel readyinside the 600 s hard timeout on a cloud pin that carries the fix. That half is verified on cloud#2637.Also measured, not filed (costs, not defect classes)
18c2ddc1eadds one index-servedsys_metadataread per stored active permission set per boot (overlayLockLayerAtbesidegetMetaItem'sfindOne): about +22 s at 36 ms for 600 sets.backfillOrgAdminGrants(plugin-security) reconciles every member on every boot, 5 serial statements per member up to 5,000. That is about 25,900 round trips per boot at 20,000 members, at every pin.Reader
The objectstack lane that owns
plugin-email. Therepo:cloudseat verifies the cloud half on cloud#2637 after the pin carries the fix.Generated by Claude Code