Skip to content

16-api-reference.md

Latest commit

 

History

History
427 lines (348 loc) · 12.4 KB

16-api-reference.md

File metadata and controls

427 lines (348 loc) · 12.4 KB

API Reference

This is a factual summary of the public API. See the topic docs for deeper behavior.

Imports

import { table, column, fromRows, ColQLError } from "@colql/colql";
import type {
  MutationResult,
  ObjectWherePredicate,
  Operator,
  QueryExplainPlan,
  QueryExplainReasonCode,
  QueryHook,
  QueryInfo,
  RowForSchema,
  RowPredicate,
  Schema,
  TableOptions,
} from "@colql/colql";

Table Creation

const users = table(schema);
const loaded = fromRows(schema, rows);
const instrumented = table(schema, { onQuery: (info) => console.log(info) });
const restored = table.deserialize(buffer);

table(schema) returns a Table instance. fromRows(schema, rows, options?) creates a table and inserts rows with insertMany. table(schema, options) accepts compatible table options such as onQuery. table.deserialize(input) accepts an ArrayBuffer or Uint8Array and returns a table.

onQuery is called by terminal query operations such as toArray, first, count, aggregations, and query mutations. Query construction itself is not instrumented.

type QueryInfo = {
  duration: number;
  durationMs?: number;
  rowsScanned: number;
  indexUsed: boolean;
  scanType?: "index" | "full";
  selectedIndex?: string;
  indexState?: "fresh" | "dirty" | "queued" | "rebuilding" | "failed";
  fallbackReason?:
    | "dirty-index"
    | "queued-index"
    | "rebuilding-index"
    | "failed-index"
    | "background-disabled"
    | "not-zero-copy-capable"
    | "memory-budget"
    | "no-usable-index";
  reasonCode?: QueryExplainReasonCode;
  backgroundRebuildScheduled?: boolean;
  backgroundRebuildState?: "queued" | "rebuilding" | "failed";
  candidateRows?: number;
  materializedRows?: number;
  resultCount?: number;
  projectionPushdown?: boolean;
  dirtyIndexRebuildPaid?: boolean;
  dirtyIndexReason?: "equality" | "sorted" | "unique";
};

type QueryHook = (info: QueryInfo) => void;

type TableOptions = {
  onQuery?: QueryHook;
};

Columns

column.int16();
column.int32();
column.uint8();
column.uint16();
column.uint32();
column.float32();
column.float64();
column.boolean();
column.dictionary(["active", "passive"] as const);

column.smallint();
column.integer();
column.real();
column.doublePrecision();

Insert and Read

users.insert(row);       // this
users.insertMany(rows);  // this
users.get(rowIndex);     // row
users.getSchema();       // schema
users.rowCount;          // number
users.capacity;          // number

Low-level typed reads are also exposed:

users.getValue(rowIndex, column);
users.getComparableValue(rowIndex, column);
users.getNumericValue(rowIndex, numericColumn);

These are mainly useful for advanced integrations and diagnostics. Row indexes are internal positions, not stable external IDs. Most application code should use query and row APIs with an explicit ID column when stable identity is required.

Query Construction

users.where(column, operator, value);
users.where(objectPredicate);
users.whereIn(column, values);
users.whereNotIn(column, values);
users.filter(callback);
users.firstWhere(predicate);
users.countWhere(predicate);
users.exists(predicate);
users.select(columns);
users.limit(n);
users.offset(n);
users.query();

query() creates an unfiltered query over the table. The table-level helpers above are the usual entrypoints for application code.

users.where({ age: { gt: 25 }, status: "active" });
users.filter((row) => row.age > 25);

where(objectPredicate) is structured predicate syntax and may use indexes. filter(callback) is a full-scan callback escape hatch, runs after structured predicates, and is not index-aware. firstWhere, countWhere, and exists are table-level wrappers over structured where(...) or callback filter(fn).

Operators:

type Operator = "=" | "!=" | ">" | ">=" | "<" | "<=" | "in" | "not in";

Query Execution

users.toArray();
users.first();
users.count();
users.size();
users.isEmpty();
users.forEach(callback);
users.stream();

for (const row of users) {
  // all rows
}

for (const row of users.where("status", "=", "active")) {
  // matching rows
}

Query objects also support toArray, first, count, size, isEmpty, forEach, stream, and iteration.

Query Objects

Queries returned by where, select, limit, offset, and query() support:

query.where(column, operator, value);
query.where(objectPredicate);
query.whereIn(column, values);
query.whereNotIn(column, values);
query.filter(callback);
query.select(columns);
query.limit(n);
query.offset(n);
query.explain();

query.first();
query.toArray();
query.forEach(callback);
query.count();
query.size();
query.isEmpty();
query.stream();

query.sum(numericColumn);
query.avg(numericColumn);
query.min(numericColumn);
query.max(numericColumn);
query.top(n, numericColumn);
query.bottom(n, numericColumn);

query.update(partialRow);
query.delete();

for (const row of query) {
  // matching rows
}

query.update() and query.delete() respect filters, offset, and limit. select() affects query output but does not restrict update payloads.

type ObjectWherePredicate<TSchema extends Schema> = {
  // column-specific object predicate shape
};

type RowPredicate<TSchema extends Schema> = (row: RowForSchema<TSchema>) => boolean;

Query Diagnostics

const plan = users
  .where({ status: "active", age: { gte: 18 } })
  .select(["id"])
  .explain();

query.explain() returns structured diagnostics for a query. It does not execute the query. It does not scan rows, materialize rows, call onQuery, rebuild dirty indexes, or schedule background rebuilds.

Example output:

{
  scanType: "index",
  indexesUsed: ["equality:status"],
  predicates: 2,
  predicateOrder: ["status =", "age >="],
  projectionPushdown: true,
  candidateRows: 42,
  indexState: "fresh"
}

Types:

type QueryExplainReasonCode =
  | "NO_PREDICATES"
  | "NO_INDEX_FOR_COLUMN"
  | "RANGE_QUERY_WITHOUT_SORTED_INDEX"
  | "INDEX_CANDIDATE_SET_TOO_LARGE"
  | "CALLBACK_PREDICATE_REQUIRES_FULL_SCAN"
  | "INDEX_DIRTY_WOULD_REBUILD_ON_EXECUTION"
  | "UNSUPPORTED_INDEX_OPERATOR";

type QueryExplainPlan = {
  scanType: "index" | "full";
  indexesUsed: readonly string[];
  selectedIndex?: string;
  predicates: number;
  predicateOrder: readonly string[];
  projectionPushdown: boolean;
  candidateRows?: number;
  indexState?: "fresh" | "dirty" | "queued" | "rebuilding" | "failed";
  fallbackReason?:
    | "dirty-index"
    | "queued-index"
    | "rebuilding-index"
    | "failed-index"
    | "background-disabled"
    | "not-zero-copy-capable"
    | "memory-budget"
    | "no-usable-index";
  reasonCode?: QueryExplainReasonCode;
  /** deprecated: use reasonCode and backgroundRebuildState */
  backgroundIndexing?: "sync";
  backgroundRebuildState?: "queued" | "rebuilding" | "failed";
  reason?: string;
};

Fields:

  • scanType: whether execution is expected to use an index or full scan.
  • indexesUsed: selected index labels such as equality:status or sorted:startedAt.
  • selectedIndex: the selected index label when an index plan is expected.
  • predicates: structured predicates plus callback predicates.
  • predicateOrder: structured predicate evaluation order after planner ordering.
  • projectionPushdown: true when select(...) limits materialized columns.
  • candidateRows: concrete indexed candidate count when it can be computed without scanning, materializing, or rebuilding.
  • indexState: lifecycle state for selected indexes: fresh, dirty, queued, rebuilding, or failed.
  • fallbackReason: low-cardinality reason when execution or diagnostics must fall back instead of using the selected index.
  • reasonCode: stable reason code for full scans, dirty-index diagnostics, and background fallback state.
  • backgroundIndexing: deprecated compatibility field; use reasonCode and backgroundRebuildState.
  • backgroundRebuildState: queued, rebuilding, or failed background state when available.
  • reason: human-readable explanation; prefer reasonCode for programmatic handling.

Dirty indexes are reported without being rebuilt:

users.updateMany({ status: "active" }, { status: "expired" });

console.log(users.where("status", "=", "expired").explain());
// {
//   scanType: "index",
//   indexesUsed: ["equality:status"],
//   indexState: "dirty",
//   reasonCode: "INDEX_DIRTY_WOULD_REBUILD_ON_EXECUTION",
//   ...
// }

Aggregations

users.sum(numericColumn);
users.avg(numericColumn);
users.min(numericColumn);
users.max(numericColumn);
users.top(n, numericColumn);
users.bottom(n, numericColumn);

Query objects support the same aggregation methods.

Mutations

users.delete(rowIndex); // this

users.update(rowIndex, partialRow); // MutationResult
users.updateWhere(column, operator, value, partialRow); // MutationResult
users.deleteWhere(column, operator, value); // MutationResult
users.updateMany(predicate, partialRow); // MutationResult
users.deleteMany(predicate); // MutationResult

users.where(...).update(partialRow); // MutationResult
users.where(...).delete(); // MutationResult
type MutationResult = {
  affectedRows: number;
};

Equality Indexes

users.createIndex(column);   // this
users.dropIndex(column);     // this
users.hasIndex(column);      // boolean
users.indexes();             // string[]
users.indexStats();          // EqualityIndexStats[]
users.rebuildIndex(column);  // this
users.rebuildIndexes();      // this

Equality indexes are derived performance structures. Unsupported predicates fall back to scan without changing query results. Dirty equality indexes currently prefer synchronous rebuilds; queued, rebuilding, or failed internal background states are not used for query results.

Sorted Indexes

users.createSortedIndex(numericColumn);   // this
users.dropSortedIndex(column);            // this
users.hasSortedIndex(column);             // boolean
users.sortedIndexes();                    // string[]
users.sortedIndexStats();                 // SortedIndexStats[]
users.rebuildSortedIndex(numericColumn);  // this
users.rebuildIndexes();                   // this

Sorted indexes are numeric range indexes. They are derived performance structures. Dirty sorted indexes rebuild synchronously on smaller tables; eligible large zero-copy sorted indexes may schedule an internal background rebuild and use scan fallback for the current query. Queued, rebuilding, or failed internal background states are not used for query results.

Unique Indexes

users.createUniqueIndex(column);       // this
users.dropUniqueIndex(column);         // this
users.hasUniqueIndex(column);          // boolean
users.uniqueIndexes();                 // string[]
users.uniqueIndexStats();              // UniqueIndexStats[]
users.rebuildUniqueIndex(column);      // this
users.rebuildUniqueIndexes();          // this

users.findBy(column, value);           // row | undefined
users.updateBy(column, value, partialRow); // MutationResult
users.deleteBy(column, value);         // MutationResult

Unique indexes support numeric and dictionary columns. They are derived structures, not serialized, and enforce uniqueness while present. By-key helpers require an existing unique index and do not scan when one is missing. Unique indexes remain synchronous and main-thread-only.

Serialization

const buffer = users.serialize();      // ArrayBuffer
const restored = table.deserialize(buffer);

deserialize accepts ArrayBuffer or Uint8Array. Indexes, index lifecycle state, and worker jobs are not serialized; recreate equality, sorted, and unique indexes after deserialization when indexed performance or uniqueness enforcement is needed.

Diagnostics

users.materializedRowCount;
users.resetMaterializationCounter();
users.scannedRowCount;
users.resetScanCounter();
users.getIndexedCandidatePlan(filters);
users.getIndexDebugPlan(filters);

Use query.explain() for stable public query diagnostics. The scan/materialization counters and typed reads are advanced diagnostics. getIndexedCandidatePlan(), getIndexDebugPlan(), and query __debugPlan() are unstable internal diagnostics retained for tests and low-level debugging; application code should not depend on them as stable planning contracts.

Errors

try {
  users.get(999);
} catch (error) {
  if (error instanceof ColQLError) {
    console.log(error.code);
    console.log(error.message);
    console.log(error.details);
  }
}