Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 14 additions & 0 deletions docs/developers-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,20 @@ make markdownlint 2>&1 | tee /tmp/markdownlint-digitalpuddle-${BRANCH}.out
make nixie 2>&1 | tee /tmp/nixie-digitalpuddle-${BRANCH}.out
```

Refresh the pinned DigitalOcean OpenAPI artefact when intentionally updating
the public API contract:

```bash
bun scripts/refresh-digitalocean-openapi.ts
```

The generated contract lives at `src/openapi/digitalocean.openapi.yaml`, and
its source, upstream commit, refresh command, response metadata, byte length,
and SHA-256 hash are recorded in
`src/openapi/digitalocean.openapi.provenance.json`. Updating the pin is a
compatibility event, so review operation-count and capability-classification
changes before committing it.

Build the package when changing the CommonJS CLI:

```bash
Expand Down
5 changes: 5 additions & 0 deletions docs/digitalpuddle-technical-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,11 @@ All public endpoints live under `/v2`. The implementation is contract-first:
route shapes and JSON schemas come from the pinned OpenAPI spec, while behaviour
is layered in selectively.

The pinned bundled contract lives at `src/openapi/digitalocean.openapi.yaml`.
Its source URL, upstream `digitalocean/openapi` commit, refresh command,
response metadata, byte length, and SHA-256 hash are recorded in
`src/openapi/digitalocean.openapi.provenance.json`.

### 8.2 v1 vertical slice

The first meaningful release targets the Nile Valley DOKS path, not the entire
Expand Down

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,12 +83,13 @@ This step answers whether public routing can be driven from a pinned
DigitalOcean OpenAPI contract while preserving Simulacrum as the HTTP backplane.
The outcome unlocks route-level implementation and capability reporting.

- [ ] 1.3.1. Add the pinned DigitalOcean OpenAPI artefact and a repeatable
- [x] 1.3.1. Add the pinned DigitalOcean OpenAPI artefact and a repeatable
refresh script with provenance.
- Requires 1.1.1.
- See `digitalpuddle-technical-design.md` §§4, 7.1, 8.1, and 20.
- Success: the repository records the upstream source, pin, refresh command,
and generated artefact hash.
- Pin: see `src/openapi/digitalocean.openapi.provenance.json`.
- [ ] 1.3.2. Build the operation registry from the pinned OpenAPI contract.
- Requires 1.1.3 and 1.3.1.
- See `digitalpuddle-technical-design.md` §§7.1, 8.1, 8.2, and 13.
Expand Down
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,8 @@
"typescript": "^6.0.2"
},
"overrides": {
"lodash": "4.18.1"
"lodash": "4.18.1",
"fast-uri": "3.1.2"
},
"inlinedDependencies": {
"starfx": "0.15.0"
Expand Down
136 changes: 136 additions & 0 deletions scripts/refresh-digitalocean-openapi.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
#!/usr/bin/env bun
/**
* @file Refreshes the pinned bundled DigitalOcean OpenAPI artefact.
*
* Downloads DigitalOcean's bundled public API v2 YAML, validates selected
* response headers for provenance, redacts webhook-shaped upstream examples,
* computes the SHA-256 hash of the checked-in artefact, and writes both
* `src/openapi/digitalocean.openapi.yaml` and
* `src/openapi/digitalocean.openapi.provenance.json`.
*
* Run `bun scripts/refresh-digitalocean-openapi.ts` when intentionally
* updating the pinned contract. The provenance records the source URL, upstream
* commit SHA, fetch timestamp, response headers, byte length, content hash, and
* redaction notes so contract updates are reproducible and reviewable.
*/

import {execFile} from 'node:child_process';
import {createHash} from 'node:crypto';
import {mkdir, writeFile} from 'node:fs/promises';
import {dirname} from 'node:path';
import {promisify} from 'node:util';
import {z} from 'zod';

const execFileAsync = promisify(execFile);

const artefactUrl = 'https://api-engineering.nyc3.digitaloceanspaces.com/spec-ci/DigitalOcean-public.v2.yaml';
const upstreamRepository = 'https://github.com/digitalocean/openapi.git';
const upstreamBranch = 'main';
const artefactPath = 'src/openapi/digitalocean.openapi.yaml';
const provenancePath = 'src/openapi/digitalocean.openapi.provenance.json';
const refreshCommand = 'bun scripts/refresh-digitalocean-openapi.ts';
const redactedSecretExamples = [
{
pattern: /https:\/\/hooks[.]slack[.]com\/services\/T00000000\/B00000000\/[A-Z]+/g,
description: 'Slack webhook placeholder with zeroed workspace and channel identifiers'
},
{
pattern: /https:\/\/hooks[.]slack[.]com\/services\/T1234567\/AAAAAAAA\/ZZZZZZ/g,
description: 'Slack webhook placeholder with sample workspace and channel identifiers'
}
] as const;
const redactedWebhookExample = 'https://example.invalid/redacted-slack-webhook';

const gitLsRemoteOutputSchema = z.string().regex(/^(?:[0-9a-f]{40}|[0-9a-f]{64})\s+refs\/heads\/main$/);
const fetchResponseHeadersSchema = z.object({
contentType: z.string().nullable(),
etag: z.string().nullable(),
lastModified: z.string().nullable()
});

type FetchArtefactResult = {
readonly response: Response;
readonly headers: z.infer<typeof fetchResponseHeadersSchema>;
};

const now = (): string => new Date().toISOString();

/**
* Resolves the current upstream branch commit with `git ls-remote`.
*
* @returns The upstream commit identifier as a SHA-1 or SHA-256 hex string.
* @throws Error when `git` fails or the output does not match the expected
* branch-ref shape.
*/
async function currentUpstreamCommit(): Promise<string> {
const {stdout} = await execFileAsync('git', ['ls-remote', upstreamRepository, `refs/heads/${upstreamBranch}`], {
encoding: 'utf8'
});
const output = gitLsRemoteOutputSchema.parse(stdout.trim());
const [sha] = output.split(/\s+/);
return sha;
}

/**
* Fetches the canonical bundled DigitalOcean OpenAPI artefact.
*
* Selected response headers are validated with `fetchResponseHeadersSchema` so
* Content-Type, ETag, and Last-Modified can be recorded in provenance.
*
* @returns The fetch response and validated provenance headers.
* @throws Error when the fetch fails, the response is not OK, or response
* header validation fails.
*/
async function fetchArtefact(signal?: AbortSignal): Promise<FetchArtefactResult> {
const response = await fetch(artefactUrl, {signal});
if (!response.ok) {
throw new Error(`Failed to fetch ${artefactUrl}: ${response.status} ${response.statusText}`);
}
const headers = fetchResponseHeadersSchema.parse({
contentType: response.headers.get('content-type'),
etag: response.headers.get('etag'),
lastModified: response.headers.get('last-modified')
});
return {response, headers};
}

const abortController = new AbortController();
const timeout = setTimeout(() => abortController.abort(), 30_000);
const {response, headers} = await fetchArtefact(abortController.signal);
clearTimeout(timeout);

const sourceBytes = new Uint8Array(await response.arrayBuffer());
const sourceText = new TextDecoder().decode(sourceBytes);
const artefactText = redactedSecretExamples.reduce(
(text, redaction) => text.replaceAll(redaction.pattern, redactedWebhookExample),
sourceText
);
const artefactBytes = new TextEncoder().encode(artefactText);
const sha256 = createHash('sha256').update(artefactBytes).digest('hex');
const upstreamCommit = await currentUpstreamCommit();

await mkdir(dirname(artefactPath), {recursive: true});
await writeFile(artefactPath, artefactBytes);

const provenance = {
artefact: artefactPath,
sourceUrl: artefactUrl,
upstreamRepository,
upstreamBranch,
upstreamCommit,
refreshCommand,
fetchedAt: now(),
contentLength: artefactBytes.byteLength,
sha256,
redactions: redactedSecretExamples.map(({description}) => ({
description,
replacement: redactedWebhookExample,
reason: 'Avoid committing webhook-shaped example secrets from the upstream specification.'
})),
responseHeaders: headers
} as const;

await writeFile(provenancePath, `${JSON.stringify(provenance, null, 2)}\n`);

console.log(`Wrote ${artefactPath} (${artefactBytes.byteLength} bytes, sha256:${sha256})`);
console.log(`Wrote ${provenancePath}`);
20 changes: 20 additions & 0 deletions src/openapi/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# DigitalOcean OpenAPI contract

This directory contains the pinned, bundled DigitalOcean public API v2
OpenAPI contract used by DigitalPuddle.

- `digitalocean.openapi.yaml` is the bundled OpenAPI artefact fetched from
DigitalOcean's generated specification endpoint.
- `digitalocean.openapi.provenance.json` records the source URL,
upstream `digitalocean/openapi` commit, refresh command, response
metadata, byte length, and SHA-256 hash.

Refresh the pin with:

```bash
bun scripts/refresh-digitalocean-openapi.ts
```

Updating this artefact is a compatibility event. Review the operation
count, hash, and downstream capability classification changes before
committing a refreshed pin.
28 changes: 28 additions & 0 deletions src/openapi/digitalocean.openapi.provenance.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"artefact": "src/openapi/digitalocean.openapi.yaml",
"sourceUrl": "https://api-engineering.nyc3.digitaloceanspaces.com/spec-ci/DigitalOcean-public.v2.yaml",
"upstreamRepository": "https://github.com/digitalocean/openapi.git",
"upstreamBranch": "main",
"upstreamCommit": "781eb485d1136605e02178f9c48a48fd679eb242",
"refreshCommand": "bun scripts/refresh-digitalocean-openapi.ts",
"fetchedAt": "2026-05-14T21:45:08.083Z",
"contentLength": 2927232,
"sha256": "604848f4ec226b1fea2be80552c3e501c5b3e695c3cc5dfc077450101f8bd1ba",
"redactions": [
{
"description": "Slack webhook placeholder with zeroed workspace and channel identifiers",
"replacement": "https://example.invalid/redacted-slack-webhook",
"reason": "Avoid committing webhook-shaped example secrets from the upstream specification."
},
{
"description": "Slack webhook placeholder with sample workspace and channel identifiers",
"replacement": "https://example.invalid/redacted-slack-webhook",
"reason": "Avoid committing webhook-shaped example secrets from the upstream specification."
}
],
"responseHeaders": {
"contentType": "application/yaml",
"etag": "\"bf6c729dd0cf8eea6a9c8a666ca92e41\"",
"lastModified": "Thu, 14 May 2026 16:03:32 GMT"
}
}
Loading
Loading