Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
49 changes: 47 additions & 2 deletions registry/CONTRACT.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Status: pre-production

Contract version: 0.1
Contract version: 0.2

## Purpose

Expand All @@ -29,6 +29,7 @@ Each fact has one authoring authority:
| Purpose, guidance, parameter meaning, and relationships | Registry Markdown |
| Curated examples and expected outcomes | Registry Markdown and fixtures |
| Lifecycle, visibility, access, and provenance | Registry Markdown |
| Recipe Writer eligibility and exclusion reason | Registry Markdown |
| Recipe JSON Schema | Generated output |
| Docusaurus pages | Generated output |
| Database/search records | Generated projection |
Expand Down Expand Up @@ -186,6 +187,8 @@ Every wrangle entry must declare:
public `slug`, and compatibility `aliases`
- `title` and `description`
- `status` and `visibility`
- an explicit `recipe_writer.eligible` decision; ineligible entries also
require a concise `recipe_writer.reason`
- `tags`
- a runtime package and Python symbol
- access flags
Expand Down Expand Up @@ -227,6 +230,23 @@ The shared `if`, `where`, and `where_params` definitions live in
`common/wrangle-controls.yaml`. Entries opt into them through `capabilities`.
They must not duplicate the common definitions in each wrangle file.

## Recipe Writer eligibility

Recipe Writer discovery is a consumer-specific, fail-closed Registry view.
Every entry must declare `recipe_writer.eligible`; omission is invalid rather
than an implicit opt-in. Eligible entries must be public, active, and reconciled
to a verified runtime contract. Ineligible entries must state a concise reason,
and that reason is emitted in the manifest and compiled contract so clients do
not need an Agent-owned exception list.

The initial view preserves the existing 88-key Recipe Writer baseline. The
compiler verifies both the count and a checksum of the sorted eligible keys, so
changing which wrangles are available requires an explicit reviewed update.
This initial schema is stock-only: it omits the full recipe schema's permissive
`custom.*` and `pandas.*` extension patterns. An extension wrangle therefore
cannot enter Recipe Writer merely because its name matches a broad pattern; it
needs a future explicit Registry-backed eligibility contract.

## Examples and verification

Each example has a stable ID, recipe YAML, input fixture, output fixture, and
Expand Down Expand Up @@ -256,9 +276,15 @@ The compiler produces:
- `manifest.json`, containing lightweight discovery metadata and links
- one comprehensive JSON contract per wrangle under
`wrangles-docs/static/registry/contracts/`, including all parameters
- a JSON Schema for those compiled contracts under
`wrangles-docs/static/registry/schema/wrangle-contract.schema.json`
- raw source Markdown and sanitized example fixtures
- a pre-production recipe JSON Schema under
`wrangles-docs/static/schemas/recipes/registry/schema.json`
- a Recipe Writer-specific schema containing only explicitly eligible entries
under `wrangles-docs/static/schemas/recipes/registry/recipe-writer.schema.json`
- a compact runtime-reconciliation summary under
`wrangles-docs/static/registry/runtime/reconciliation.json`
- deterministic JSON and Markdown reconciliation reports under
`registry/reports/`

Expand All @@ -270,9 +296,23 @@ Generated artifacts are deterministic: timestamps, local paths, credentials,
and environment-specific values are excluded. A clean compile followed by
`check:registry` must produce no Git diff.

The manifest records SHA-256 for every machine-facing bundle member and a
bundle checksum. Bundle members are ordered by their public path and framed as
the UTF-8 path, a NUL byte, the exact generated UTF-8 content, and a final NUL
byte. `manifest.json` is deliberately excluded from the bundle digest because
it contains the digest; a deployment can pin the manifest itself by source
commit or an external lock-file checksum without circular hashing.

The Recipe Writer eligible-key checksum uses the framing named by
`recipe_writer.eligible_keys_framing`: sorted keys encoded as UTF-8, each
terminated by one newline (`utf8-newline-separated-sorted-v1`).

## Versioning and lifecycle

The pre-production Registry version is `0.1.0`. A production release will
The pre-production Registry version is `0.2.0`. Version `0.2.0` makes the
fail-closed Recipe Writer eligibility decision required for every entry and
adds the compiled-contract and integrity metadata consumed by Registry clients.
A production release will
contain:

- an immutable Registry version
Expand All @@ -281,6 +321,11 @@ contain:
- immutable recipe-schema and Registry URLs
- a convenience `latest` alias that is not used for reproducible execution

The current runtime manifest records both the exact WranglesPY package version
and the pinned source revision used for reconciliation. The public manifest
exposes that evidence plus an exact compatible-version specifier so consumers
can fail closed before loading an incompatible runtime.

Wrangles progress through `draft`, `active`, `deprecated`, and `removed`.
Deprecated records stay discoverable and must identify their canonical
replacement with `replaced_by`. The compiler rejects a missing, self-referential,
Expand Down
9 changes: 8 additions & 1 deletion registry/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@ wrangles-docs/static/registry/ (machine-facing artifacts)
```

`runtime/wranglespy.json` is the pinned mechanical contract exported from the
WranglesPY recipe namespace. Its producer-owned schema is copied alongside the
WranglesPY recipe namespace. It records both the exact package version and
source revision. Its producer-owned schema is copied alongside the
Registry schema under `schema/`. The compiler reconciles the manifest with the
curated Markdown and writes the reviewable result to `reports/`. It also reads
all existing `wrangles-docs/wrangle-docs/**/_sources/*.md` quasi-registry
Expand All @@ -43,6 +44,12 @@ Runtime names, required status, defaults, symbols, and common controls must
agree; embedded Python schema and quasi-registry differences are retained as
migration evidence.

Each Registry entry also makes an explicit, fail-closed Recipe Writer
eligibility decision. The compiler publishes the eligible view, verifies it
against the pinned 88-key baseline, and emits checksums for every machine-facing
artifact plus a non-circular bundle checksum. The Recipe Writer schema is
stock-only and rejects broad `custom.*` and `pandas.*` extension names.

## Commands

Run these commands from `wrangles-docs`:
Expand Down
2 changes: 1 addition & 1 deletion registry/common/wrangle-controls.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
schema_version: "0.1"
schema_version: "0.2"
controls:
where:
description: Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.
Expand Down
3 changes: 2 additions & 1 deletion registry/reports/runtime-reconciliation.json
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
{
"format": "wrangles-registry-reconciliation",
"format_version": "0.2",
"registry_version": "0.1.0",
"registry_version": "0.2.0",
"runtime_source": {
"repository": "https://github.com/wrangleworks/WranglesPY",
"version": "1.20.2",
"revision": "7916bf158e8b7e561270a1bea7b808f88956edc4"
},
"summary": {
Expand Down
3 changes: 2 additions & 1 deletion registry/reports/runtime-reconciliation.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,9 @@

Generated file. Do not edit directly.

- Runtime version: `1.20.2`
- Runtime source: [`7916bf158e8b7e561270a1bea7b808f88956edc4`](https://github.com/wrangleworks/WranglesPY/commit/7916bf158e8b7e561270a1bea7b808f88956edc4)
- Registry version: `0.1.0`
- Registry version: `0.2.0`
- Runtime entries: 98
- Embedded Python schema docstrings: 96
- Existing quasi-registry records: 93
Expand Down
1 change: 1 addition & 0 deletions registry/runtime/wranglespy.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
"format_version": "0.1",
"source": {
"repository": "https://github.com/wrangleworks/WranglesPY",
"version": "1.20.2",
"revision": "7916bf158e8b7e561270a1bea7b808f88956edc4"
},
"entry_count": 98,
Expand Down
29 changes: 28 additions & 1 deletion registry/schema/wrangle-entry.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
"slug",
"status",
"visibility",
"recipe_writer",
"tags",
"runtime",
"access",
Expand All @@ -26,7 +27,7 @@
"sources"
],
"properties": {
"schema_version": {"const": "0.1"},
"schema_version": {"const": "0.2"},
"type": {"const": "wrangle"},
"id": {
"type": ["string", "null"],
Expand Down Expand Up @@ -71,6 +72,32 @@
"visibility": {
"enum": ["public", "internal"]
},
"recipe_writer": {
"type": "object",
"additionalProperties": false,
"required": ["eligible"],
"properties": {
"eligible": {
"type": "boolean",
"description": "Whether this wrangle is eligible for Recipe Writer discovery and authoring. Missing eligibility fails closed."
},
"reason": {
"type": "string",
"minLength": 1,
"description": "Required concise explanation when the wrangle is not eligible."
}
},
"allOf": [
{
"if": {
"properties": {"eligible": {"const": false}},
"required": ["eligible"]
},
"then": {"required": ["reason"]},
"else": {"not": {"required": ["reason"]}}
}
]
},
"tags": {
"type": "array",
"minItems": 1,
Expand Down
7 changes: 6 additions & 1 deletion registry/schema/wrangles-runtime-manifest.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,14 @@
"source": {
"type": "object",
"additionalProperties": false,
"required": ["repository", "revision"],
"required": ["repository", "version", "revision"],
"properties": {
"repository": {"type": "string", "format": "uri"},
"version": {
"type": "string",
"pattern": "^[0-9]+\\.[0-9]+\\.[0-9]+(?:[-+][0-9A-Za-z.-]+)?$",
"description": "Exact WranglesPY package version whose callable surface was exported."
},
"revision": {"type": "string", "minLength": 1}
}
},
Expand Down
5 changes: 4 additions & 1 deletion registry/wrangles/_root/accordion.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
schema_version: '0.1'
schema_version: '0.2'
type: wrangle
id: null
wrangle_name: accordion
Expand All @@ -13,6 +13,9 @@ aliases: []
slug: accordion
status: active
visibility: public
recipe_writer:
eligible: false
reason: Nested-list orchestration is not supported in the baseline.
tags:
- utility
- accordion
Expand Down
4 changes: 3 additions & 1 deletion registry/wrangles/_root/batch.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
schema_version: '0.1'
schema_version: '0.2'
type: wrangle
id: null
wrangle_name: batch
Expand All @@ -13,6 +13,8 @@ aliases: []
slug: batch
status: active
visibility: public
recipe_writer:
eligible: true
tags:
- utility
- batch
Expand Down
4 changes: 3 additions & 1 deletion registry/wrangles/_root/classify.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
schema_version: '0.1'
schema_version: '0.2'
type: wrangle
id: 06669ef8-cdd1-42f7-8078-98e0b7a42c30
wrangle_name: classify
Expand All @@ -11,6 +11,8 @@ aliases: []
slug: classify
status: active
visibility: public
recipe_writer:
eligible: true
tags:
- ai
- classify
Expand Down
4 changes: 3 additions & 1 deletion registry/wrangles/_root/clean-whitespaces.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
schema_version: '0.1'
schema_version: '0.2'
type: wrangle
id: e36e15c4-f0ad-43f8-8555-ef683a8ab892
wrangle_name: clean_whitespaces
Expand All @@ -13,6 +13,8 @@ aliases: []
slug: clean-whitespaces
status: active
visibility: public
recipe_writer:
eligible: true
tags:
- format
- clean-whitespaces
Expand Down
5 changes: 4 additions & 1 deletion registry/wrangles/_root/concurrent.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
schema_version: '0.1'
schema_version: '0.2'
type: wrangle
id: null
wrangle_name: concurrent
Expand All @@ -14,6 +14,9 @@ aliases: []
slug: concurrent
status: active
visibility: public
recipe_writer:
eligible: false
reason: Concurrent orchestration is not supported in the baseline.
tags:
- utility
- concurrent
Expand Down
4 changes: 3 additions & 1 deletion registry/wrangles/_root/copy.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
schema_version: '0.1'
schema_version: '0.2'
type: wrangle
id: b002fbdc-92c0-4347-889d-0f4bfeec99fa
wrangle_name: copy
Expand All @@ -11,6 +11,8 @@ aliases: []
slug: copy
status: active
visibility: public
recipe_writer:
eligible: true
tags:
- transform
- copy
Expand Down
4 changes: 3 additions & 1 deletion registry/wrangles/_root/date-calculator.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
schema_version: '0.1'
schema_version: '0.2'
type: wrangle
id: 19cfeb4f-02af-4ab2-895c-0ff2bb5cce19
wrangle_name: date_calculator
Expand All @@ -11,6 +11,8 @@ aliases: []
slug: date-calculator
status: active
visibility: public
recipe_writer:
eligible: true
tags:
- date
- date-calculator
Expand Down
4 changes: 3 additions & 1 deletion registry/wrangles/_root/drop.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
schema_version: '0.1'
schema_version: '0.2'
type: wrangle
id: 363bffbf-397e-4975-8382-e9efa5e9eed6
wrangle_name: drop
Expand All @@ -11,6 +11,8 @@ aliases: []
slug: drop
status: active
visibility: public
recipe_writer:
eligible: true
tags:
- select
- drop
Expand Down
4 changes: 3 additions & 1 deletion registry/wrangles/_root/explode.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
schema_version: '0.1'
schema_version: '0.2'
type: wrangle
id: 4e4b13ac-8d50-4b2c-85c8-2c31de1e817d
wrangle_name: explode
Expand All @@ -11,6 +11,8 @@ aliases: []
slug: explode
status: active
visibility: public
recipe_writer:
eligible: true
tags:
- split
- explode
Expand Down
4 changes: 3 additions & 1 deletion registry/wrangles/_root/filter.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
schema_version: '0.1'
schema_version: '0.2'
type: wrangle
id: e3242acf-d204-433f-8373-205b77481131
wrangle_name: filter
Expand All @@ -13,6 +13,8 @@ aliases: []
slug: filter
status: active
visibility: public
recipe_writer:
eligible: true
tags:
- select
- filter
Expand Down
Loading