Support optional loadable extensions for DecentDB

[sphildreth]

As an advanced user or integrator,
I want DecentDB to support optional loadable extensions,
so that I can add specialized functionality (custom SQL functions, domain logic, optional features) without bloating the core engine.

This feature is not required for MVP and must not compromise DecentDB’s correctness, security, or stability guarantees.

---

Scope & Constraints

In Scope

• Ability to dynamically load a native extension module at runtime.
• Extensions may register:
  • Scalar SQL functions
  • Aggregate SQL functions
  • (Optional, future) table-valued functions or virtual tables
• Extensions are loaded via an explicit API or SQL command.
• Extensions run in-process with the database engine.

Explicitly Out of Scope (Initial Implementation)

• Network-distributed extensions
• Extension marketplaces or package managers
• Automatic extension loading
• Extension hot-reload/unload
• Sandboxed or interpreted extensions

---

Security & Safety Requirements (Critical)

• Loadable extensions are disabled by default.
• Extension loading requires explicit opt-in via:
  • Database open configuration or
  • Explicit runtime command.
• Extension loading must be auditable:
  • Log extension name, path, version, and load time.
• Extensions execute with full process privileges; this must be documented clearly.
• Failure to load an extension must:
  • Not crash the database
  • Return a clear error
  • Leave the database in a consistent state

---

API / UX Requirements

Programmatic API

• Provide a function such as:
  • db.loadExtension(path: string)
• The API must:
  • Return success or structured error
  • Prevent duplicate loading of the same extension instance

SQL Interface (Optional)

LOAD EXTENSION 'path/to/extension';

• SQL-based loading must respect the same safety checks as the API.

---

Extension Interface Requirements

• Extensions must expose a stable, versioned entry point.
• Extensions must receive:
  • A pointer/handle to the DecentDB extension API
  • Engine version information
• Extensions must explicitly register:
  • Functions
  • Aggregates
  • Collations (if supported)

Compatibility Rules

• DecentDB must reject extensions built for incompatible ABI versions.
• ABI version must be checked before any registration occurs.

---

Error Handling Requirements

• Extension load failures must be non-fatal.
• Registration failures must:
  • Roll back partial registrations
  • Leave engine state unchanged
• Errors must be surfaced clearly to the caller.

---

Testing Requirements

• Unit tests for:
  • Successful extension loading
  • ABI version mismatch rejection
  • Duplicate extension load prevention
• Integration test:
  • Load a test extension
  • Register a scalar function
  • Execute a query using that function
• Negative test:
  • Attempt to load extensions when disabled
  • Verify explicit failure

---

Documentation Requirements

• Clearly document:
  • Extensions execute native code
  • They are disabled by default
  • They should only be enabled for trusted code
• Provide a minimal “Hello Extension” example.

---

Acceptance Criteria

• Extensions load only when explicitly enabled.
• A test extension can register and execute a SQL function.
• Invalid or incompatible extensions are rejected safely.
• No impact to WAL, recovery, or core correctness.
• Feature is clearly marked Post-MVP in documentation.