fix(driver-turso): the remote face refuses a missing table or column with the local face's code instead of answering []

.changeset/20424-turso-remote-missing-table-column-refused.md

@@ -0,0 +1,33 @@
+---
+"@objectstack/driver-turso": minor
+---
+
+`TursoDriver` in **remote** mode refuses a read over a missing table or a missing column with the same code the local mode answers, instead of answering "no rows" (#20424).
+
+Clause-②: no (narrowing)
+
+<!-- adr-0087: not-required (already-registered driver-sql-unresolvable-where-column-refused) That registered entry names `TursoDriver` among the `SqlDriver` subclasses whose reads now refuse an unresolvable column, and prescribes this change's remedy: name a column the table has, or run schema sync so a declared field exists as a column. The `aggregate` legs (a table that is really absent, a `groupBy` or aggregation column that is absent) are the same drifted-schema family with the same remedy, so no new migration entry is owed. -->
+
+**BREAKING** — an accept-set narrowing on the remote face of `TursoDriver`'s read doors, shipped as `minor` under the launch-window convention (`check-changeset-no-major` refuses `major` until GA; breaking-ness is carried by this banner and the ADR-0087 disposition above, not by the level). Remote-face users meet this refusal for the first time here.
+
+**FROM → TO.** A remote-face read (`aggregate`, `find`, `findOne`, `count`) that answered `[]` / `null` for a missing table or a missing column now refuses, as the local face does: `DATABASE_ERROR` / 500 for a table that is absent, `INVALID_FIELD` / 400 for a `groupBy` or aggregation column that is absent, `INVALID_FILTER` / 400 for a `where` column that is absent. **The fix:** run schema sync so the declared field has its column (or the object its table), or name a column the table has. `RemoteTransport.find` and `RemoteTransport.aggregate`, exported from the package root, now raise the backend's error where they answered `[]`.
+
+**What was wrong.** Two catches in `RemoteTransport` read a backend "no such table" or "no such column" as an empty result. `aggregate` answered `[]` for both. `find` (and `findOne` through it) answered `[]` (`null`) for a missing column once its projection retry was spent, or when there was no projection to drop. So on a remote Turso database a schema drift or a missing table read as "there is no data", while the local mode of the same driver, over the same file, refused it. Measured with a local driver over the same libSQL file as the control, for a federated and for a managed object alike:
+
+| read | local | remote before |
+|:--|:--|:--|
+| `aggregate` on a table that is really absent | `DATABASE_ERROR` / 500 | `[]` |
+| `aggregate` grouped by, or aggregating, a declared field whose column is absent | `INVALID_FIELD` / 400 | `[]` |
+| `aggregate` whose `where` names that field | `INVALID_FILTER` / 400 | `[]` |
+| `find` / `findOne` whose `where` names that field | `INVALID_FILTER` / 400 | `[]` / `null` |
+| `count` whose `where` names that field | `INVALID_FILTER` / 400 | `DATABASE_ERROR` / 500 |
+| `find` ordered by that field | the rows, unordered | `[]` |
+
+**What changes, on the remote face only:**
+
+- `aggregate`, `find`, `findOne` and `count` answer each row above the way the local face does. The backend's error is classified by the local face's own inherited seam, `SqlDriver.aggregateBackendFault`, and not by a second copy: an unresolvable column named by a `groupBy` or an aggregation is `INVALID_FIELD` / 400, one named by the `where` is `INVALID_FILTER` / 400, and anything else is `DATABASE_ERROR` / 500. The dialect text goes to the server log, never to the caller.
+- `find` keeps the local face's recovery ladder: a projection naming a column the table lacks is dropped first, then an ORDER BY on one, and the rows answer. A `where` is never dropped. Before, the ORDER BY rung was missing and the sort answered `[]`.
+- A refusal the transport raises while it compiles the statement (a filter or aggregate-vocabulary refusal, the timeout envelope) keeps its own code and status.
+- `RemoteTransport.find` and `RemoteTransport.aggregate`, used on their own, now raise the backend's error where they answered `[]`.
+
+This is the refusal the registered migration entry `driver-sql-unresolvable-where-column-refused` already names for `driver-sql` "and its `TursoDriver` / `SqliteWasmDriver` subclasses": the remote face of `TursoDriver` now delivers it. **If a read now refuses for you:** the table or column it names is missing from the remote database. Run schema sync so the declared field has its column (or the object its table), or correct the name the query uses.

packages/drivers/driver-turso/src/remote-transport-unknown-select.test.ts

@@ -51,12 +51,23 @@ describe('RemoteTransport unknown-$select column', () => {
     expect(calls[1].sql).toMatch(/SELECT \* FROM "product"/);
   });
 
-  it('still returns empty when even SELECT * fails (e.g. unknown table)', async () => {
-    const { t } = transportWithClient(async () => {
+  // [#20424] This case REPLACES the pin 'still returns empty when even SELECT *
+  // fails (e.g. unknown table)', which asserted `[]` for exactly this input.
+  // `[]` was the defect: a column the WHERE still names once the projection is
+  // gone is a predicate that never ran, and "no rows" is a false answer to it.
+  // The transport now raises the backend's error from the last rung, and
+  // `TursoDriver` classifies it (`INVALID_FILTER` / 400, the local face's
+  // answer). The same input, the opposite assertion.
+  it('raises the last rung\'s error when even SELECT * fails, instead of answering []', async () => {
+    const { t, calls } = transportWithClient(async () => {
       throw new Error('SQLITE_ERROR: no such column: status');
     });
-    const result = await t.find('ghost', { fields: ['id', 'status'], limit: 10 });
-    expect(result).toEqual([]);
+    await expect(t.find('ghost', { fields: ['id', 'status'], limit: 10 })).rejects.toThrow(
+      /no such column: status/,
+    );
+    // The projection attempt, then the one rung this query has.
+    expect(calls).toHaveLength(2);
+    expect(calls[1].sql).toMatch(/SELECT \* FROM "ghost"/);
   });
 
   it('propagates non-column errors instead of hiding them as empty', async () => {

packages/drivers/driver-turso/src/remote-transport.ts

@@ -1412,16 +1412,39 @@ export class RemoteTransport {
         // real rows still come back; the unknown field is simply absent from
         // each row (it never existed). Mirrors the SqlDriver backstop — the
         // remote Turso path overrides find(), so it needs its own copy.
+        //
+        // [#20424] The copy now has the local ladder's two rungs and its
+        // terminal. The rungs, in `SqlDriver.findRows`' order: the projection
+        // first, then the ORDER BY (#3821: rows matter more than their order),
+        // each rebuilt with the caller's WHERE, which neither rung may drop.
+        // The terminal was `return []` here, on both the no-rung path and a
+        // failed rung, so an unresolvable column in the WHERE, or an ORDER BY
+        // on a column the table lacks, read as "there are no rows". The local
+        // face answers the first with `INVALID_FILTER` / 400 (#8790) and the
+        // second with its rows, unordered. Now the last error leaves this
+        // method as the backend raised it, and `TursoDriver` classifies it
+        // with the local face's own read-exit seam. The refusal and its code
+        // are the driver's, not this transport's.
+        const rungs: any[] = [];
         if (query?.fields && Array.isArray(query.fields) && query.fields.length > 0) {
+          rungs.push({ ...query, fields: undefined });
+        }
+        if (Array.isArray(query?.orderBy) && query.orderBy.some((item: any) => item?.field)) {
+          rungs.push({ ...query, fields: undefined, orderBy: undefined });
+        }
+        let lastError: unknown = error;
+        for (const rung of rungs) {
           try {
-            const fallback = this.buildSelectSQL(object, { ...query, fields: undefined }, table);
+            const fallback = this.buildSelectSQL(object, rung, table);
             const result = await this.client!.execute({ sql: fallback.sql, args: fallback.args });
             return this.mapRows(result);
-          } catch {
-            return [];
+          } catch (rungError) {
+            // The next, broader rung. The last one to fail names the column
+            // the WHERE still holds, once the projection and the sort are gone.
+            lastError = rungError;
           }
         }
-        return [];
+        throw lastError;
       }
       throw error;
     }
@@ -1626,19 +1649,18 @@ export class RemoteTransport {
       sql += ` GROUP BY ${groupBy.map((g) => `"${g.field}"`).join(', ')}`;
     }
 
-    try {
-      const result = await this.client!.execute({ sql, args });
-      return this.foldEmptyAggregateAnswers(this.mapRows(result), foldedOutput);
-    } catch (error: any) {
-      if (
-        error.message &&
-        (error.message.includes('no such table') ||
-          error.message.includes('no such column'))
-      ) {
-        return [];
-      }
-      throw error;
-    }
+    // [#20424] The backend's error leaves this method as it was raised. A
+    // catch here used to answer `no such table` and `no such column` with `[]`,
+    // so a table that is really absent, or a groupBy, aggregation or WHERE
+    // naming a declared field whose column is absent, read as "no data". The
+    // local face of the same driver refuses all of them. `TursoDriver`
+    // classifies the error with the local face's own aggregate seam
+    // (`SqlDriver.aggregateBackendFault`): `INVALID_FIELD` / 400 for a groupBy
+    // or aggregation column, `INVALID_FILTER` / 400 for a WHERE column, and
+    // `DATABASE_ERROR` / 500 for the rest. The catch arrived with this method's
+    // first version, with no comment, no pin and no case it was written for.
+    const result = await this.client!.execute({ sql, args });
+    return this.foldEmptyAggregateAnswers(this.mapRows(result), foldedOutput);
   }
 
   /**
@@ -2883,11 +2905,12 @@ export class RemoteTransport {
       //
       // The first two are the silent empty set: SQLite's backwards-compatible
       // rule degrades a double-quoted name that resolves to no column into a
-      // STRING LITERAL, so the statement compiles, runs and matches nothing —
-      // and where a build disables that rule (`SQLITE_DQS=0`) `find()`'s own
-      // `no such column` backstop swallows the error into `[]` anyway. Two
-      // roads, one answer, and neither is distinguishable from "no rows
-      // matched".
+      // STRING LITERAL, so the statement compiles, runs and matches nothing,
+      // which is not distinguishable from "no rows matched". Where a build
+      // disables that rule (`SQLITE_DQS=0`), the statement fails with
+      // `no such column` instead, and `find()`'s backstop now refuses it, as
+      // the local face does (`INVALID_FILTER` / 400, see #20424). It used to
+      // swallow that error into `[]` too: a second road to the same answer.
       //
       // The third is the expensive direction, and it needs no dialect quirk at
       // all: a `{}` disjunct absorbs its `$or` to TRUE, the compiled clauses are

packages/drivers/driver-turso/src/turso-driver.ts

@@ -673,8 +673,10 @@ function refuseRemoteInheritedMember(
  * this face could reach the right table and still name columns it does not
  * have. Measured on the remote face with the table resolved and the map
  * ignored, a filter on a renamed field answered an empty list: the transport
- * reads the backend's `no such column` as "no rows". A write failed with the
- * backend's own error.
+ * then read the backend's `no such column` as "no rows". A write failed with the
+ * backend's own error. (Since #20424 that read is refused `INVALID_FILTER` /
+ * 400 instead, which is loud but still wrong for a field the author declared:
+ * the refusal below stays the answer.)
  *
  * Translating the map here would be a second copy of the local compiler's
  * column rule inside the transport's compiler. That is the second
@@ -1918,7 +1920,10 @@ export class TursoDriver extends SqlDriver {
     if (this.isRemote) {
       const table = this.remoteTableFor(object, 'find');
       const remoteQuery = this.toRemoteReadQuery(object, query);
-      return this.formatRemoteRows(object, await this.remoteReadExit(object, () => this.remoteTransport!.find(object, remoteQuery, table)));
+      return this.formatRemoteRows(
+        object,
+        await this.remoteReadExit(object, { where: query?.where }, () => this.remoteTransport!.find(object, remoteQuery, table)),
+      );
     }
     return super.find(object, query, options);
   }
@@ -1934,7 +1939,10 @@ export class TursoDriver extends SqlDriver {
     if (this.isRemote) {
       const table = this.remoteTableFor(object, 'findOne');
       const remoteQuery = this.toRemoteReadQuery(object, query, { singleRowLookup: true });
-      return this.formatRemoteRow(object, await this.remoteReadExit(object, () => this.remoteTransport!.findOne(object, remoteQuery, table)));
+      return this.formatRemoteRow(
+        object,
+        await this.remoteReadExit(object, { where: query?.where }, () => this.remoteTransport!.findOne(object, remoteQuery, table)),
+      );
     }
     return super.findOne(object, query, options);
   }
@@ -2061,22 +2069,84 @@ export class TursoDriver extends SqlDriver {
    * table is the one this face's statement named. It returns anything that
    * already declares a `status` unchanged: the transport's filter refusals and
    * the remote timeout envelope keep their own answers. `distinct` already
-   * reaches the same terminal through `distinctBackendFault`. `aggregate` is
-   * not wrapped, because nothing would reach a wrapper: the transport's own
-   * catch answers a missing table or column with `[]`, where the local face
-   * answers this envelope. That divergence predates this change and is not
-   * widened by it. The write doors are left alone exactly as the local face
-   * leaves them, because a write fault is classified at the REST boundary from
-   * its message.
+   * reaches the same terminal through `distinctBackendFault`. The write doors
+   * are left alone exactly as the local face leaves them, because a write
+   * fault is classified at the REST boundary from its message.
+   *
+   * [#20424] `aggregate` now ends here too, and every exit reaches the
+   * terminal through {@link remoteReadFault}, which adds the local face's
+   * unresolvable-column arms in front of it.
    */
-  private async remoteReadExit<T>(object: string, read: () => Promise<T>): Promise<T> {
+  private async remoteReadExit<T>(object: string, query: DriverQuery, read: () => Promise<T>): Promise<T> {
     try {
       return await read();
     } catch (error) {
-      throw this.backendStatementFault(object, error);
+      throw this.remoteReadFault(object, query, error);
     }
   }
 
+  /**
+   * [#20424] Which envelope a backend error leaving a remote read exit
+   * deserves: the local face's answer, from the local face's own seam.
+   *
+   * # The defect this closes
+   *
+   * `RemoteTransport` answered a missing table or column with `[]` in two
+   * catches: `aggregate` for `no such table` and `no such column`, and the
+   * terminal of `find`'s projection backstop for `no such column`. Measured at
+   * base `6e3e5462c` over one libSQL `file:` database, with a local driver over
+   * the same file as the control, for a federated object and for a managed one
+   * alike:
+   *
+   * ```
+   * aggregate, table really absent                     local DATABASE_ERROR 500  remote []
+   * aggregate, groupBy a declared field, column absent local INVALID_FIELD 400   remote []
+   * find, where names a declared field, column absent  local INVALID_FILTER 400  remote []
+   * findOne, the same where                            local INVALID_FILTER 400  remote null
+   * find, orderBy on that field                        local rows, unordered     remote []
+   * count, the same where                              local INVALID_FILTER 400  remote DATABASE_ERROR 500
+   * ```
+   *
+   * The transport now lets the backend's error out (its `find` keeps the local
+   * ladder's projection and ORDER BY rungs first), and this classifies it.
+   *
+   * # The seam: `SqlDriver.aggregateBackendFault`, called, not copied
+   *
+   * The local face decides AFTER its statement runs, from the backend's error:
+   * `count` and `findRows` send an unresolvable column to
+   * `unresolvableFilterColumnRefusal` (#8790) and everything else to
+   * `backendStatementFault` (#8931); `aggregate` attributes the column to the
+   * clause the caller's own query names it in first (#11541). All three
+   * compositions are protected members this driver inherits. The class
+   * predicate they share, `isUnresolvableColumnError`, is not exported from
+   * `@objectstack/driver-sql`, so `aggregateBackendFault` is the one inherited
+   * member that asks it. For `aggregate` it is the local exit verbatim. For
+   * `find`, `findOne` and `count` it is handed the WHERE alone, and then it is
+   * the local exit too: with no groupBy and no aggregation its first arm cannot
+   * fire, its second is `unresolvableFilterColumnRefusal` with the caller's
+   * `where`, and its terminal is `backendStatementFault`. The one place the two
+   * could differ, a recognised wording whose column name does not parse (the
+   * local `count` still answers `INVALID_FILTER` there, this answers
+   * `DATABASE_ERROR`), cannot arise on libSQL, whose only wording is
+   * `no such column: <name>`.
+   *
+   * # Anything that already declares a `status` passes unchanged
+   *
+   * The local face guards only the statement's EXECUTION, so its classifier
+   * never sees a refusal raised while the statement is built. A remote door
+   * compiles and executes inside one transport call, so the transport's own
+   * refusals (the filter compiler's `INVALID_FILTER`, the aggregate
+   * vocabulary's refusals, the timeout envelope) arrive here beside the
+   * backend's errors. They all declare a `status`; a libSQL error declares
+   * none. So the gate `backendStatementFault` applies first on its own terms
+   * ("is it already ours") is applied here before the classifier reads a
+   * message, and the classifier sees exactly what the local one sees.
+   */
+  private remoteReadFault(object: string, query: DriverQuery, error: unknown): Error {
+    if (typeof (error as { status?: unknown } | null | undefined)?.status === 'number') return error as Error;
+    return this.aggregateBackendFault(object, query, error);
+  }
+
   /**
    * [#6944] Refuse a remote write that would need a record number this face
    * cannot issue — see {@link refuseRemoteAutonumber} for the ruling and the
@@ -2274,7 +2344,7 @@ export class TursoDriver extends SqlDriver {
     if (this.isRemote) {
       const table = this.remoteTableFor(object, 'count');
       const remoteQuery = this.toRemoteQuery(object, query);
-      return this.remoteReadExit(object, () => this.remoteTransport!.count(object, remoteQuery, table));
+      return this.remoteReadExit(object, { where: query?.where }, () => this.remoteTransport!.count(object, remoteQuery, table));
     }
     return super.count(object, query, options);
   }
@@ -2305,7 +2375,12 @@ export class TursoDriver extends SqlDriver {
     this.assertRemoteTransactionUnsupported(options, 'aggregate');
     if (this.isRemote) {
       const table = this.remoteTableFor(object, 'aggregate');
-      return this.remoteTransport!.aggregate(object, this.toRemoteQuery(object, query), table);
+      // [#20424] The caller's own query is what the fault is attributed
+      // against, as `SqlDriver.aggregate` does locally: its groupBy and
+      // aggregation fields, and the `where` before `toRemoteFilter` rewrote it.
+      return this.remoteReadExit(object, query, () =>
+        this.remoteTransport!.aggregate(object, this.toRemoteQuery(object, query), table),
+      );
     }
     return super.aggregate(object, query, options);
   }