fix(metadata): TypeScriptSerializer annotates each item with its own metadata type's spec type, never ServiceObject for a non-object

.changeset/19852-typescript-serializer-per-type-annotation.md

@@ -0,0 +1,15 @@
+---
+"@objectstack/metadata": patch
+---
+
+`TypeScriptSerializer` no longer annotates every `typescript`-format file `ServiceObject` (#19852). A saved view, or any other item that is not an object, used to be written as `export const metadata: ServiceObject = { … }`: a false annotation, which `tsc` refused with TS2353 (for a view, `'"type"' does not exist in type …`).
+
+If you type-check the `.ts` files that `MetadataManager.save()` / `FilesystemLoader.save()` write:
+
+- An `object` file written by the built-in serializer the package wires in is byte-identical: `import type { ServiceObject } from '@objectstack/spec/data'` and `export const metadata: ServiceObject = …`.
+- Every other metadata type whose spec type is exactly the `z.input` type of its schema is now annotated with that type instead: `Flow` (`@objectstack/spec/automation`) for a `flow`, `Page` (`@objectstack/spec/ui`) for a `page`, `PermissionSet` (`@objectstack/spec/security`) for a `permission`, and so on: 28 annotated metadata types, `object` included. `tsc` now checks such a file against its own type instead of `ServiceObject`.
+- `view`, `book`, `external_catalog` and any other metadata type (a plugin's own, for example) are written with no annotation and no import: `export const metadata = { … };`. `ViewMetadata` is `unknown` and `Book` is narrower than `BookSchema`, so neither would be a true annotation.
+- A serializer you wire into `FilesystemLoader` by hand is called as it always was: a custom one, or a subclass that overrides `serialize()`, writes what its `serialize()` writes, and a `TypeScriptSerializer` taken from the package's other entry point (`.` versus `./node`) writes no annotation.
+- If you call `TypeScriptSerializer.serialize()` yourself, it now writes no annotation for any item, an object included. It used to write `ServiceObject` whatever the item was, and it cannot know the item's metadata type, so that annotation could be false. The loader picks the annotation through a package-internal function. The public API is unchanged: `SerializeOptions` and every declaration the package exports are as before.
+
+Reading is unchanged: `deserialize` still reads the first JSON block, so a file written before this fix, `ServiceObject` annotation and all, still reads back. The `javascript` format is unchanged.

packages/metadata/README.md

@@ -74,7 +74,7 @@ Serializers convert metadata objects to/from different file formats:
 
 - **JSONSerializer** — `.json` files with optional key sorting
 - **YAMLSerializer** — `.yaml`/`.yml` files (JSON_SCHEMA for security)
-- **TypeScriptSerializer** — the `typescript` / `javascript` formats (`.ts` / `.js`): a JSON document wrapped in a module, the file format `FilesystemLoader` writes and reads for those two formats (`typescript` is `FilesystemLoader.save()`'s default, so `MetadataManager.save()` routed to the filesystem loader writes `{rootDir}/{type}/{name}.ts`). It writes `export const metadata = { …JSON… };` then `export default metadata;` — the `typescript` format also imports the `ServiceObject` type and annotates the constant with it, whatever the item's metadata type — and reads back the first `{ … }` block after the first `export const` (or, failing that, `export default`), which must be JSON: double-quoted keys and strings, no comments, no trailing commas, no functions. It is **not** an authoring shape: authored metadata such as a `*.object.ts` is written `ObjectSchema.create({ … })` (or `defineView()`, …), which this serializer never emits, and an authored file with unquoted keys is refused rather than read.
+- **TypeScriptSerializer** — the `typescript` / `javascript` formats (`.ts` / `.js`): a JSON document wrapped in a module, the file format `FilesystemLoader` writes and reads for those two formats (`typescript` is `FilesystemLoader.save()`'s default, so `MetadataManager.save()` routed to the filesystem loader writes `{rootDir}/{type}/{name}.ts`). It writes `export const metadata = { …JSON… };` then `export default metadata;`. A `typescript`-format file that `FilesystemLoader.save()` writes with the built-in serializer the package wires in also annotates that constant with the spec type of the item's metadata type, and imports it: `ServiceObject` from `@objectstack/spec/data` for an `object`, `Flow` from `@objectstack/spec/automation` for a `flow`, and so on. Each is exactly the `z.input` type of the schema `getMetadataTypeSchema()` resolves for that metadata type. A metadata type with no such spec type is written with no annotation and no import: `view` (its `ViewMetadata` type is `unknown`), `book`, `external_catalog`, or a plugin's own type. Only the loader knows the metadata type, so it picks the annotation through a package-internal function, and `TypeScriptSerializer.serialize()` called directly writes no annotation. A serializer wired in by hand (a custom one, a subclass that overrides `serialize()`, or a `TypeScriptSerializer` from the package's other entry point) is called through its own `serialize()`. It reads back the first `{ … }` block after the first `export const` (or, failing that, `export default`), which must be JSON: double-quoted keys and strings, no comments, no trailing commas, no functions. It is **not** an authoring shape: authored metadata such as a `*.object.ts` is written `ObjectSchema.create({ … })` (or `defineView()`, …), which this serializer never emits, and an authored file with unquoted keys is refused rather than read.
 
 ### 4. Overlay / Customization System
 

packages/metadata/src/loaders/filesystem-loader.ts

@@ -22,6 +22,7 @@ import type {
 import type { Logger } from '@objectstack/core';
 import type { MetadataLoader, MetadataKeyedItem } from './loader-interface.js';
 import type { MetadataSerializer } from '../serializers/serializer-interface.js';
+import { TypeScriptSerializer, serializeTypeScriptForMetadataType } from '../serializers/typescript-serializer.js';
 import { AmbiguousMetadataStemError } from './ambiguous-metadata-stem.js';
 
 /**
@@ -459,12 +460,25 @@ export class FilesystemLoader implements MetadataLoader {
         }
       }
 
-      // Serialize data
-      const content = serializer.serialize(data, {
-        prettify,
-        indent,
-        sortKeys,
-      });
+      // Serialize data. The built-in `typescript` serializer is the one format
+      // that annotates, and the annotation depends on the metadata type, which
+      // only this call knows: it goes through the package-internal
+      // `serializeTypeScriptForMetadataType`, so the published
+      // `SerializeOptions` does not grow a key.
+      //
+      // The test is the METHOD, not the class: only when this module's own,
+      // un-overridden `TypeScriptSerializer.prototype.serialize` is the one
+      // that would run. `instanceof` also matched a subclass whose overridden
+      // `serialize()` must still be called. Any other serializer (a custom
+      // one, a subclass that overrides `serialize()`, or a `TypeScriptSerializer`
+      // from the package's other entry bundle, whose prototype is a different
+      // object) is called as it always was.
+      const serializeOptions = { prettify, indent, sortKeys };
+      const content =
+        serializer.serialize === TypeScriptSerializer.prototype.serialize &&
+        serializer.getFormat() === 'typescript'
+          ? serializeTypeScriptForMetadataType(data, type, serializeOptions)
+          : serializer.serialize(data, serializeOptions);
 
       // Write to disk (atomic or direct)
       if (atomic) {

packages/metadata/src/serializers/serializers.test.ts

@@ -1,7 +1,14 @@
-import { describe, it, expect } from 'vitest';
+import { describe, it, expect, vi } from 'vitest';
+import fs from 'node:fs/promises';
+import os from 'node:os';
+import path from 'node:path';
+import { NodeMetadataManager } from '../node-metadata-manager.js';
 import { JSONSerializer } from '../serializers/json-serializer.js';
 import { YAMLSerializer } from '../serializers/yaml-serializer.js';
-import { TypeScriptSerializer } from '../serializers/typescript-serializer.js';
+import { TypeScriptSerializer, serializeTypeScriptForMetadataType } from '../serializers/typescript-serializer.js';
+import type { MetadataFormat } from '@objectstack/spec/system';
+import type { MetadataSerializer, SerializeOptions } from '../serializers/serializer-interface.js';
+import { FilesystemLoader } from '../loaders/filesystem-loader.js';
 
 describe('Serializers', () => {
   describe('JSONSerializer', () => {
@@ -55,14 +62,143 @@ describe('Serializers', () => {
   describe('TypeScriptSerializer', () => {
     const serializer = new TypeScriptSerializer('typescript');
 
+    // The saved-view repro: a view used to be annotated `ServiceObject`, which
+    // has no top-level `type` or `columns` key, so `tsc` refused the file (TS2353).
+    const view = { name: 'all_accounts', type: 'grid', object: 'account', columns: ['name'] };
+    const object = { name: 'account', label: 'Account', fields: { name: { type: 'text', label: 'Name' } } };
+    const plain = (item: unknown) =>
+      `export const metadata = ${JSON.stringify(item, null, 2)};\n\nexport default metadata;\n`;
+    // Exactly what `FilesystemLoader.save('object', …)` wrote before this fix, and still writes.
+    const annotatedObject =
+      `import type { ServiceObject } from '@objectstack/spec/data';\n\n` +
+      `export const metadata: ServiceObject = ${JSON.stringify(object, null, 2)};\n\n` +
+      `export default metadata;\n`;
+
     it('should serialize to TypeScript module', () => {
       const data = { name: 'test', value: 42 };
       const result = serializer.serialize(data);
-      expect(result).toContain('import type');
       expect(result).toContain('export const metadata');
       expect(result).toContain('export default metadata');
     });
 
+    it('the public serialize() writes no annotation, even for an object: it does not know the metadata type', () => {
+      expect(serializer.serialize(object)).toBe(plain(object));
+      expect(serializer.serialize(view)).toBe(plain(view));
+    });
+
+    it('writes a view with no annotation: no ServiceObject, no import type', () => {
+      const result = serializeTypeScriptForMetadataType(view, 'view');
+      expect(result).not.toContain('ServiceObject');
+      expect(result).toBe(plain(view));
+    });
+
+    it('still annotates an object ServiceObject, byte-identical to the earlier output', () => {
+      expect(serializeTypeScriptForMetadataType(object, 'object')).toBe(annotatedObject);
+    });
+
+    it.each([
+      ['a metadata type the spec has no type for', 'external_catalog'],
+      ['a metadata type whose spec type is narrower than its schema', 'book'],
+      ['a plugin-registered metadata type', 'acme_widget'],
+      ['a plural spelling', 'objects'],
+    ])('writes no annotation for %s', (_label, metadataType) => {
+      expect(serializeTypeScriptForMetadataType({ name: 'x' }, metadataType)).toBe(plain({ name: 'x' }));
+    });
+
+    it('the javascript format never annotates', () => {
+      expect(new TypeScriptSerializer('javascript').serialize(object)).toBe(plain(object));
+    });
+
+    it('reads back a file written before this fix: a view annotated ServiceObject', () => {
+      const legacy =
+        `import type { ServiceObject } from '@objectstack/spec/data';\n\n` +
+        `export const metadata: ServiceObject = ${JSON.stringify(view, null, 2)};\n\n` +
+        `export default metadata;\n`;
+      expect(serializer.deserialize(legacy)).toEqual(view);
+    });
+
+    it('round-trips a view and an object through serialize and deserialize', () => {
+      for (const [metadataType, item] of [['view', view], ['object', object]] as const) {
+        expect(serializer.deserialize(serializeTypeScriptForMetadataType(item, metadataType))).toEqual(item);
+        expect(serializer.deserialize(serializer.serialize(item))).toEqual(item);
+      }
+    });
+
+    it('FilesystemLoader.save() annotates by metadata type: the view file is unannotated, the object file is ServiceObject', async () => {
+      const rootDir = await fs.mkdtemp(path.join(os.tmpdir(), 'objectstack-ts-serializer-'));
+      try {
+        const manager = new NodeMetadataManager({ rootDir, watch: false });
+        await manager.save('view', 'all_accounts', view);
+        await manager.save('object', 'account', object);
+        const viewFile = await fs.readFile(path.join(rootDir, 'view', 'all_accounts.ts'), 'utf-8');
+        const objectFile = await fs.readFile(path.join(rootDir, 'object', 'account.ts'), 'utf-8');
+        expect(viewFile).not.toContain('ServiceObject');
+        expect(viewFile).toBe(plain(view));
+        expect(objectFile).toBe(annotatedObject);
+      } finally {
+        await fs.rm(rootDir, { recursive: true, force: true });
+      }
+    });
+
+    it('FilesystemLoader.save() calls a custom serializer registered for typescript as it always did', async () => {
+      const rootDir = await fs.mkdtemp(path.join(os.tmpdir(), 'objectstack-ts-serializer-custom-'));
+      try {
+        const seen: unknown[] = [];
+        const custom: MetadataSerializer = {
+          serialize: (item, options) => {
+            seen.push(options);
+            return `// custom\nexport const metadata = ${JSON.stringify(item)};\n`;
+          },
+          deserialize: (content) => serializer.deserialize(content),
+          getExtension: () => '.ts',
+          canHandle: (format) => format === 'typescript',
+          getFormat: () => 'typescript',
+        };
+        const loader = new FilesystemLoader(rootDir, new Map<MetadataFormat, MetadataSerializer>([['typescript', custom]]));
+        await loader.save('object', 'account', object);
+        const file = await fs.readFile(path.join(rootDir, 'object', 'account.ts'), 'utf-8');
+        expect(file).toBe(`// custom\nexport const metadata = ${JSON.stringify(object)};\n`);
+        expect(seen).toEqual([{ prettify: true, indent: 2, sortKeys: false }]);
+      } finally {
+        await fs.rm(rootDir, { recursive: true, force: true });
+      }
+    });
+
+    // A FilesystemLoader wired by hand with one `typescript` serializer, saving `object`.
+    const saveObjectWith = async (tsSerializer: MetadataSerializer): Promise<string> => {
+      const rootDir = await fs.mkdtemp(path.join(os.tmpdir(), 'objectstack-ts-serializer-wired-'));
+      try {
+        const loader = new FilesystemLoader(rootDir, new Map<MetadataFormat, MetadataSerializer>([['typescript', tsSerializer]]));
+        await loader.save('object', 'account', object);
+        return await fs.readFile(path.join(rootDir, 'object', 'account.ts'), 'utf-8');
+      } finally {
+        await fs.rm(rootDir, { recursive: true, force: true });
+      }
+    };
+
+    it('FilesystemLoader.save() calls a subclass that overrides serialize(), as it always did', async () => {
+      class HeaderSerializer extends TypeScriptSerializer {
+        override serialize<T>(item: T, options?: SerializeOptions): string {
+          return '// header\n' + super.serialize(item, options);
+        }
+      }
+      expect(await saveObjectWith(new HeaderSerializer('typescript'))).toBe('// header\n' + plain(object));
+    });
+
+    it('FilesystemLoader.save() annotates through the built-in serialize(), inherited by a subclass or not', async () => {
+      class KeepsSerialize extends TypeScriptSerializer {}
+      expect(await saveObjectWith(new TypeScriptSerializer('typescript'))).toBe(annotatedObject);
+      expect(await saveObjectWith(new KeepsSerialize('typescript'))).toBe(annotatedObject);
+    });
+
+    it('FilesystemLoader.save() calls a TypeScriptSerializer from another module copy through its own serialize(): no annotation', async () => {
+      // The published `.` and `./node` entries are separate bundles, each with its own class copy.
+      vi.resetModules();
+      const other = await import('../serializers/typescript-serializer.js');
+      expect(other.TypeScriptSerializer).not.toBe(TypeScriptSerializer);
+      expect(await saveObjectWith(new other.TypeScriptSerializer('typescript'))).toBe(plain(object));
+    });
+
     it('should get correct extension', () => {
       const ts = new TypeScriptSerializer('typescript');
       expect(ts.getExtension()).toBe('.ts');