From f090ac64e7adfaa3058e0242562cdf02140ebc37 Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Thu, 6 Aug 2026 14:29:00 +0100 Subject: [PATCH] DOC-6939 Make the page-level id unique The record id was the content file's base name, so it was never unique: 5,733 pages shared 2,161 ids, with "message_history" naming 58 different pages and "install" 42. In the published feed that is 2,643 pages under 2,119 ids, so a consumer treating id as a primary key silently lost about 500 pages with nothing to notice it by. The applied AI team hit this and worked around it by keying on url; they have since confirmed they are content to keep keying on url provided the id is actually unique, which makes this an implementation rather than a decision. It is now the content path with the extension and any trailing /_index removed, so develop/clients/redis-py.md becomes develop/clients/redis-py. Unique by construction, because two files cannot share a path, and stable across builds because nothing about it depends on build order. A dedup suffix was rejected for that second reason. Appending -1 and -2 to collisions would have been a smaller change, but which page got the plain id and which got the suffix would depend on the order pages were processed, so an id could migrate between pages between builds and quietly repoint anyone who had stored one. An id that is not stable is barely better than an id that is not unique. Slashes are kept rather than flattened to hyphens, because flattening reintroduces a collision risk -- "a/b-c" and "a-b/c" both becoming "a-b-c" -- for no gain, and the path form matches how the url is already structured. The derivation lives in one partial rather than being written out in each template. It was previously duplicated three times, in single.json, section.json and section.json's children[] loop, and that pattern of the same rule implemented twice and then drifting has been the recurring theme of this whole ticket. children[].id in particular has to equal the child page's own id or the navigation graph does not resolve. One template gotcha, which the verification caught and reasoning would not have: writing the path normalisation as ".Path | replace \"\\\\\" \"/\"" makes .Path the LAST argument, so the backslash becomes the input string and every id on every page came out as a single backslash. The build succeeded and all 5,717 children[] entries still "resolved", because they all resolved to the same broken value. Only checking the actual ids showed it. 5,733 pages now carry 5,733 distinct ids with no collisions, all 5,717 children[] entries resolve to a real page id, and the section-id, example-id and content_hash invariants are unchanged. Learned: in a Hugo template, piping into replace puts the piped value LAST, so ".Path | replace old new" silently uses the pattern as the input -- every id became a single backslash, the build passed, and the children[] cross-reference check still showed 100% because everything resolved to the same wrong value Constraint: the page id must stay derivable from the content path alone, never from build order -- a dedup suffix would let an id migrate between pages between builds, which is worse than a duplicate because a stored reference silently repoints Constraint: single.json, section.json and its children[] loop must all take the id from layouts/partials/page-id.html, because children[].id has to equal the child page's own id for the navigation graph to resolve Rejected: suffixing colliding ids with -1 and -2 | smaller change, but assignment depends on processing order so ids are no longer stable across builds Rejected: flattening the path to hyphens for a more conventional-looking id | reintroduces collisions, since "a/b-c" and "a-b/c" both become "a-b-c" Ticket: DOC-6939 Co-Authored-By: Claude Opus 5 (1M context) --- content/ai-agent-resources.md | 2 +- layouts/_default/section.json | 31 +++++--------------------- layouts/_default/single.json | 14 +----------- layouts/partials/page-id.html | 41 +++++++++++++++++++++++++++++++++++ 4 files changed, 48 insertions(+), 40 deletions(-) create mode 100644 layouts/partials/page-id.html diff --git a/content/ai-agent-resources.md b/content/ai-agent-resources.md index 74765c041f..3e30b58138 100644 --- a/content/ai-agent-resources.md +++ b/content/ai-agent-resources.md @@ -43,7 +43,7 @@ Each document contains: | Field | Type | Description | |-------|------|-------------| -| `id` | string | URL slug identifier | +| `id` | string | Unique identifier, the page's path without a file extension (for example `develop/clients/redis-py`) | | `title` | string | Page title | | `url` | string | Canonical URL | | `summary` | string | Short description | diff --git a/layouts/_default/section.json b/layouts/_default/section.json index 6ee91cc66d..f88dafde72 100644 --- a/layouts/_default/section.json +++ b/layouts/_default/section.json @@ -5,19 +5,7 @@ {{- $content := partial "process-markdown-content.html" (dict "RawContent" .RawContent "Site" .Site "Page" .) -}} {{- /* Build the JSON object for the section itself */ -}} -{{- /* Handle pages where .File may be nil (e.g., taxonomy pages) */ -}} -{{- /* Also handle _index.md files where ContentBaseName is "_index" (not useful as id) */ -}} -{{- $id := "" -}} -{{- with .File -}} - {{- $baseName := .ContentBaseName -}} - {{- if or (not $baseName) (eq $baseName "_index") -}} - {{- $id = $.Title | urlize -}} - {{- else -}} - {{- $id = $baseName | urlize -}} - {{- end -}} -{{- else -}} - {{- $id = .Title | urlize -}} -{{- end -}} +{{- $id := partial "page-id.html" . -}} {{- $summary := (.Params.description | default .Description) | plainify | replaceRE "\\s+" " " | strings.TrimSpace -}} {{- $tags := .Params.categories | default (slice) -}} {{- $lastUpdated := .Lastmod.Format "2006-01-02T15:04:05Z07:00" -}} @@ -25,19 +13,10 @@ {{- /* Build list of child pages */ -}} {{- $children := slice -}} {{- range .Pages -}} - {{- $childTitle := .Title -}} - {{- $childId := "" -}} - {{- with .File -}} - {{- $baseName := .ContentBaseName -}} - {{- if or (not $baseName) (eq $baseName "_index") -}} - {{- $childId = $childTitle | urlize -}} - {{- else -}} - {{- $childId = $baseName | urlize -}} - {{- end -}} - {{- else -}} - {{- $childId = $childTitle | urlize -}} - {{- end -}} - {{- $childSummary := (.Params.description | default .Description) | plainify | replaceRE "\\s+" " " | strings.TrimSpace -}} + {{- /* Same derivation as the page's own id, so children[].id resolves to the record + that page publishes */ -}} + {{- $childId := partial "page-id.html" . -}} + {{- $childSummary :=(.Params.description | default .Description) | plainify | replaceRE "\\s+" " " | strings.TrimSpace -}} {{- $child := dict "id" $childId "title" .Title "url" .Permalink "summary" $childSummary -}} {{- $children = $children | append $child -}} {{- end -}} diff --git a/layouts/_default/single.json b/layouts/_default/single.json index 9d815fa63f..a9325cbf43 100644 --- a/layouts/_default/single.json +++ b/layouts/_default/single.json @@ -5,19 +5,7 @@ {{- $content := partial "process-markdown-content.html" (dict "RawContent" .RawContent "Site" .Site "Page" .) -}} {{- /* Build the JSON object */ -}} -{{- /* Handle pages where .File may be nil (e.g., taxonomy pages) */ -}} -{{- /* Also handle _index.md files where ContentBaseName is "_index" (not useful as id) */ -}} -{{- $id := "" -}} -{{- with .File -}} - {{- $baseName := .ContentBaseName -}} - {{- if or (not $baseName) (eq $baseName "_index") -}} - {{- $id = $.Title | urlize -}} - {{- else -}} - {{- $id = $baseName | urlize -}} - {{- end -}} -{{- else -}} - {{- $id = .Title | urlize -}} -{{- end -}} +{{- $id := partial "page-id.html" . -}} {{- $summary := (.Params.description | default .Description) | plainify | replaceRE "\\s+" " " | strings.TrimSpace -}} {{- $tags := .Params.categories | default (slice) -}} {{- $lastUpdated := .Lastmod.Format "2006-01-02T15:04:05Z07:00" -}} diff --git a/layouts/partials/page-id.html b/layouts/partials/page-id.html new file mode 100644 index 0000000000..77a91dfd06 --- /dev/null +++ b/layouts/partials/page-id.html @@ -0,0 +1,41 @@ +{{- /* + Stable, unique id for a page record in the JSON and Markdown outputs. + + Takes a Page. Returns its content path with the ".md" extension and any trailing + "/_index" removed, so content/develop/clients/redis-py.md becomes + "develop/clients/redis-py" and content/develop/clients/_index.md becomes + "develop/clients". + + This replaces a bare filename. ContentBaseName is not unique across the corpus -- + "message_history" named 58 different pages, "install" 42 -- so any consumer treating + id as a primary key silently collapsed them, with no error to notice. The content + path is unique by construction, because two files cannot share one path, and it is + stable across builds because nothing about it depends on build order. A dedup + suffix would not be: the page that got the plain id and the page that got "-1" could + swap between builds, quietly repointing anyone who had stored either. + + Slashes are kept rather than flattened to hyphens. Flattening reintroduces a + collision risk ("a/b-c" and "a-b/c" both becoming "a-b-c") for no real gain, and the + path form matches how the URL is already structured. + + Pages with no backing file -- generated pages, taxonomy terms -- fall back to the + urlized title, which is what the previous implementation did for them. + + Shared by single.json, section.json and section.json's children[] entries so all + three agree. children[].id must equal the child page's own id or the navigation + graph does not resolve. +*/ -}} + +{{- $id := "" -}} +{{- with .File -}} + {{- /* Not ".Path | replace ..." -- piping makes .Path the LAST argument, so the + backslash becomes the input and every id comes out as a single backslash. */ -}} + {{- $path := replace .Path "\\" "/" -}} + {{- $path = strings.TrimSuffix ".md" $path -}} + {{- $path = strings.TrimSuffix "/_index" $path -}} + {{- $id = $path -}} +{{- end -}} +{{- if not $id -}} + {{- $id = .Title | urlize -}} +{{- end -}} +{{- return $id -}}