diff --git a/docs/docs/Infrastructure/Web3-Adapter.md b/docs/docs/Infrastructure/Web3-Adapter.md index 145e86c70..21b629c72 100644 --- a/docs/docs/Infrastructure/Web3-Adapter.md +++ b/docs/docs/Infrastructure/Web3-Adapter.md @@ -41,7 +41,7 @@ Every piece of data has an "owner" — the [eName](/docs/W3DS%20Basics/W3ID) of ### 3. Bidirectional mapping and ID mapping -- **Field mapping**: `localToUniversalMap` defines how each local field maps to a global field (including relations and special functions like `__date`, `__calc`). The same map is used in both directions: `toGlobal` for outbound, `fromGlobal` for inbound. +- **Field mapping**: `localToUniversalMap` defines how each local field maps to a global field (including relations and special functions like `__date`, `__calc`, `__file`). The same map is used in both directions: `toGlobal` for outbound, `fromGlobal` for inbound. - **ID mapping**: A separate store (e.g. SQLite `MappingDatabase`) holds `(globalId, localId)`. When syncing out, after a successful `storeMetaEnvelope` the adapter stores the new global ID against the local ID. When a webhook arrives, the adapter looks up the global ID to decide whether to create or update the local entity and then stores or updates the mapping. Without this, the same logical entity could be duplicated or never linked across platforms. When consuming our TypeScript implementation of the Web3 Adapter, this is already taken care of. ### 4. Change detection on the platform side @@ -119,7 +119,7 @@ graph TB ### Mapping configuration (IMapping) -Mapping configs define how local fields map to the global ontology. For the full syntax (direct fields, relations, arrays, `__date`, `__calc`, owner path), see the [Mapping Rules](/docs/Post%20Platform%20Guide/mapping-rules) or the repository at `infrastructure/web3-adapter/MAPPING_RULES.md`. +Mapping configs define how local fields map to the global ontology. For the full syntax (direct fields, relations, arrays, `__date`, `__calc`, `__file`, owner path), see the [Mapping Rules](/docs/Post%20Platform%20Guide/mapping-rules) or the repository at `infrastructure/web3-adapter/MAPPING_RULES.md`. File fields use the `__file` directive backed by the [`w3ds://file` URI scheme](/docs/W3DS%20Protocol/File-URIs). Each mapping is a JSON file with: diff --git a/docs/docs/Post Platform Guide/mapping-rules.md b/docs/docs/Post Platform Guide/mapping-rules.md index 638287cee..c70fc6349 100644 --- a/docs/docs/Post Platform Guide/mapping-rules.md +++ b/docs/docs/Post Platform Guide/mapping-rules.md @@ -92,6 +92,34 @@ Performs mathematical calculations using field values. - Can reference other fields in the same entity - Automatically resolves field values before calculation +### File Referencing (`__file`) + +Uploads inline file payloads to the owner eVault's object storage and replaces +them with a stable [`w3ds://file`](/docs/W3DS%20Protocol/File-URIs) URI. On the +way back, the URI is dereferenced to the file's public URL. + +Same global field name as the local field: + +```json +"avatar": "__file(avatar)" +``` + +Different global field name (via a `,alias` suffix): + +```json +"avatar": "__file(avatar),avatarUri" +``` + +- The inner path (`avatar`) points to the field holding the file value. +- An optional `,alias` sets the global field name (defaults to the inner path). +- The value may be a single file or an **array** of files. Array paths such as + `__file(images[].src)` are supported and each item is referenced/dereferenced. +- **`toGlobal`**: a `data:` URI value is uploaded and replaced with a + `w3ds://file?id=@/` URI. Values that are already + `w3ds://file` URIs, plain URLs, or empty are passed through unchanged. +- **`fromGlobal`**: a `w3ds://file` URI is dereferenced to the file's public + object-storage URL; other values pass through unchanged. + ## Owner Path The `ownerEnamePath` defines how to determine which [eVault](/docs/Infrastructure/eVault) owns the data (via the owner's [eName](/docs/W3DS%20Basics/W3ID)): diff --git a/docs/docs/W3DS Protocol/File-URIs.md b/docs/docs/W3DS Protocol/File-URIs.md new file mode 100644 index 000000000..d4538171b --- /dev/null +++ b/docs/docs/W3DS Protocol/File-URIs.md @@ -0,0 +1,95 @@ +--- +sidebar_position: 5 +--- + +# File URIs + +This document explains the `w3ds://file` URI scheme — a standardised, +human-readable way to reference and dereference files across the MetaState +ecosystem. A file attached to or described by a [Meta Envelope](/docs/Infrastructure/eVault) +can be uniquely addressed and resolved with a consistent URI tied to a user's +entity name (`ename`) and the envelope's identifier. + +## Format + +```text +w3ds://file?id=@/ +``` + +| Component | Description | +| -------------------- | ------------------------------------------------------------ | +| `w3ds://` | The scheme. Always lowercase. | +| `file` | The resource host. Identifies the URI as addressing a file. | +| `id` | Required query parameter carrying the file's address. | +| `@` | The owning user's entity name (`ename`), always `@`-prefixed.| +| `` | The ID of the Meta Envelope describing the file. | + +Example: + +```text +w3ds://file?id=@alice/envelope-abc123 +``` + +## How files are stored + +A file uploaded to an [eVault](/docs/Infrastructure/eVault) is: + +1. Streamed to object storage (DigitalOcean Spaces, S3-compatible) as a + `public-read` object. +2. Recorded as a **File Meta Envelope** (ontology `w3ds-file-v1`) with payload: + `{ filename, contentType, size, blobKey, publicUrl, uploadedAt }`. +3. Addressed by a `w3ds://file` URI built from the owner `ename` and the + Meta Envelope ID. + +Uploads are performed through the eVault `uploadFile` GraphQL mutation, which +takes base64 content and returns the `w3ds://file` URI, the Meta Envelope ID, +and the public object-storage URL. + +## Resolving (dereferencing) a URI + +There are two dereferencers. + +### HTTP — eVault core + +```http +GET /files/:metaEnvelopeId (header: X-ENAME: @) +``` + +Resolves the File Meta Envelope and responds with a **302 redirect** to the +file's public object-storage URL. The redirect target is validated to be +`http(s)` only. + +- `400` — missing `X-ENAME` header, malformed ID, or an unsafe stored URL scheme. +- `404` — no File Meta Envelope for that ID, or it has no public URL. + +### Programmatic — web3-adapter + +```ts +import { dereferenceFileUri } from "@web3-adapter/w3ds/resolver"; + +const file = await dereferenceFileUri( + "w3ds://file?id=@alice/abc123", + evaultClient, +); +// => { uri, ename, metaEnvelopeId, publicUrl, filename, contentType, size } +``` + +## Error handling + +`parseFileUri` / `dereferenceFileUri` throw a descriptive `InvalidW3dsUriError` +or `Error` for: + +- Malformed URIs (not parseable, empty input). +- Wrong scheme (not `w3ds:`) or wrong host (not `file`). +- Missing `id` query parameter. +- `id` missing the `@` prefix or the `/` segment. +- Empty `ename` or `meta-envelope-id`. +- A non-existent `ename` (eVault cannot be resolved). +- A non-existent or non-file Meta Envelope. + +## Mapper integration + +The [Web3 Adapter](/docs/Infrastructure/Web3-Adapter) exposes a `__file()` +mapping directive that automatically references files on `toGlobal` and +dereferences them on `fromGlobal`. See +[Mapping Rules → File Referencing](/docs/Post%20Platform%20Guide/mapping-rules). diff --git a/infrastructure/web3-adapter/MAPPING_RULES.md b/infrastructure/web3-adapter/MAPPING_RULES.md index 977d60d2e..4799649bd 100644 --- a/infrastructure/web3-adapter/MAPPING_RULES.md +++ b/infrastructure/web3-adapter/MAPPING_RULES.md @@ -117,7 +117,7 @@ Different global field name (via a `,alias` suffix): object-storage URL; other values pass through unchanged. Requires an `EVaultClient` to be supplied to the mapper — the `Web3Adapter` -wires this automatically. See [`W3DS_URI.md`](./W3DS_URI.md) for the URI scheme. +wires this automatically. See the **File URIs** doc (`docs/docs/W3DS Protocol/File-URIs.md`) for the `w3ds://file` URI scheme. ## Owner Path diff --git a/infrastructure/web3-adapter/W3DS_URI.md b/infrastructure/web3-adapter/W3DS_URI.md deleted file mode 100644 index afecda530..000000000 --- a/infrastructure/web3-adapter/W3DS_URI.md +++ /dev/null @@ -1,94 +0,0 @@ -# The `w3ds://file` URI Scheme - -A standardised, human-readable URI scheme for referencing and dereferencing -files within the MetaState ecosystem. - -## Format - -```text -w3ds://file?id=@/ -``` - -| Component | Description | -| ------------------ | ------------------------------------------------------------------ | -| `w3ds://` | The scheme. Always lowercase. | -| `file` | The resource host. Identifies the URI as addressing a file. | -| `id` | Required query parameter carrying the file's address. | -| `@` | The owning user's entity name (`ename`), always `@`-prefixed. | -| `` | The ID of the Meta Envelope describing the file. | - -Example: - -```text -w3ds://file?id=@alice/envelope-abc123 -``` - -## How files are stored - -A file uploaded to eVault is: - -1. Streamed to object storage (DigitalOcean Spaces, S3-compatible) as a - `public-read` object. -2. Recorded as a **File Meta Envelope** (ontology `w3ds-file-v1`) with payload: - `{ filename, contentType, size, blobKey, publicUrl, uploadedAt }`. -3. Addressed by a `w3ds://file` URI built from the owner `ename` and the - Meta Envelope ID. - -## Constructing a URI - -The eVault `uploadFile` GraphQL mutation returns the `w3ds://file` URI directly. -In the web3-adapter: - -```ts -import { buildFileUri } from "./w3ds/uri"; - -buildFileUri({ ename: "@alice", metaEnvelopeId: "abc123" }); -// => "w3ds://file?id=@alice/abc123" -``` - -## Resolving (dereferencing) a URI - -There are two dereferencers: - -### HTTP — eVault core - -```http -GET /files/:metaEnvelopeId (header: X-ENAME: @) -``` - -Resolves the File Meta Envelope and responds with a **302 redirect** to the -file's public object-storage URL. - -- `400` — missing `X-ENAME` header or malformed ID. -- `404` — no File Meta Envelope for that ID, or it has no public URL. - -### Programmatic — web3-adapter - -```ts -import { dereferenceFileUri } from "./w3ds/resolver"; - -const file = await dereferenceFileUri( - "w3ds://file?id=@alice/abc123", - evaultClient, -); -// => { uri, ename, metaEnvelopeId, publicUrl, filename, contentType, size } -``` - -## Error handling - -`parseFileUri` / `dereferenceFileUri` throw a descriptive `InvalidW3dsUriError` -or `Error` for: - -- Malformed URIs (not parseable, empty input). -- Wrong scheme (not `w3ds:`) or wrong host (not `file`). -- Missing `id` query parameter. -- `id` missing the `@` prefix or the `/` segment. -- Empty `ename` or `meta-envelope-id`. -- A non-existent `ename` (eVault cannot be resolved). -- A non-existent or non-file Meta Envelope. - -## Mapper integration - -The web3-adapter mapper exposes a `__file()` directive that automatically -references files on `toGlobal` and dereferences them on `fromGlobal`. See -[`MAPPING_RULES.md`](./MAPPING_RULES.md#file-referencing-__file).