DOC-6909 Add link render hook to replace relref (investigation) - #3732
DOC-6909 Add link render hook to replace relref (investigation)#3732andy-stark-redis wants to merge 17 commits into
Conversation
…dings
Adds a prototype link render hook that resolves internal Markdown links to
their permalink, so [text](/path) can replace [text]({{< relref "/path" >}}),
plus the empirical findings in the dependency assessment. Internal-link parity
against a relref baseline is exact across the clients section.
The headline surprise overturned my first conclusion. relref and plain links
do NOT need an atomic migration to coexist. While a relref remains, its link
destination is still Hugo's shortcode placeholder at render-hook time (goldmark
runs before shortcode substitution), so the hook must pass placeholders through
untouched and let Hugo fill in the URL afterwards. The real atomic event is
installing the hook, not converting content, because the hook is global and
reprocesses every plain link already in the repo (the command reply-type links
are relative Markdown, not relref) with benign normalisation. Four defects only
showed up at corpus scale, each silently dropping pages or failing the build,
and are recorded in the assessment so the next attempt does not re-burn them.
Learned: coexistence via placeholder passthrough; hook-install is the atomic step; four corpus-scale hook bugs
Constraint: never dereference .Page.File in a render hook (nil on markdownify-generated and shortcode-inner pages); use .Page.Path
Rejected: urls.Parse for external-link detection | hard-errors on malformed destinations and fails the whole build, use a findRE scheme match
Directive: parity-test the hook against the whole site before migrating any content, then convert relref gradually; remove the HAHAHUGOSHORTCODE guard once no relref remains
Recheck: on Hugo upgrade, confirm the HAHAHUGOSHORTCODE placeholder marker still exists
Ticket: DOC-6909
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🧠 Redis MemoryFound 5 related items from repository history:
Memory updated at 61b6ee1 |
The first hook only tried .Page.GetPage, so links to companion source files in the use-case demos ([source](cache.rs) and friends) were reported as unresolved even though the files exist and the output was correct. These are page resources, not content pages. The hook now falls back to .Page.Resources.GetMatch before warning, which drops the false positives (build warnings 67 -> 40) and normalises the links to their permalink. The warnings that remain are genuine: real dead links, alias/redirect targets that GetPage cannot see, and static-directory files. Learned: internal links include page-bundle resources, not just pages; resolve them with .Page.Resources.GetMatch Ticket: DOC-6909 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🧠 Redis MemoryFound 5 related items from repository history:
Memory updated at 03de590 |
Links written with an explicit same-directory prefix, e.g. [bicycle dataset](./data/bicycles.txt), were reported as unresolved because neither GetPage nor Resources.GetMatch matches a `./`-prefixed path. Strip a single leading `./` before resolving; `../` is left untouched so GetPage can still resolve it relative to the current page. bicycles.txt now resolves to its permalink and stays published; build warnings drop 40 -> 39. The one that remains here is a genuine content bug the hook surfaces: ./data/products.txt linked from aggregations-syntax.md does not exist anywhere in content/ (a silent 404 before the hook). Ticket: DOC-6909 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🧠 Redis MemoryFound 5 related items from repository history:
Memory updated at 96cd672 |
|
Before adopting this solution we need to make sure that any tooling that uses the docs/build/version_archiver.py Lines 45 to 51 in 64af445 |
Applies two low-severity review points from Bugbot. Resolution now uses .PageInner rather than .Page so relative links resolve against the page whose Markdown contains them under transclusion, and fragment-only (#anchor) links get their own branch instead of being grouped with external links, matching the three-category model in the assessment. Both are behaviour-preserving: output is byte-identical to the previous build and the warning set is unchanged. Verified .PageInner changes nothing observable in this repo: embed-md transcludes via .Content (rendered in the target's own context), not .RenderShortcodes, so .PageInner equals .Page here, and the markdownify-context warnings still attribute to the calling page. Kept as canonical, future-proof practice for when .RenderShortcodes transclusion is introduced. Learned: .PageInner only diverges from .Page under .RenderShortcodes transclusion; this repo's embed-md uses .Content so PageInner is defensive here, not a live fix Ticket: DOC-6909 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Records the build and authoring tools that parse or generate relref as a literal string, so the migration plan accounts for them rather than silently breaking them: version_archiver.py (rewrites relrefs to version links), redisvl_docs_sync.py (emits relref in an importer), the check_shortcode_paths.py edit hook, and the Markdown/JSON output partial. Raised by paoloredis on the PR. Ticket: DOC-6909 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🧠 Redis MemoryFound 5 related items from repository history:
Memory updated at 7d340dc |
|
Edit: Claude wrote this, btw :-) Good catch, thanks @paoloredis — captured in the assessment (7d340dc). I audited
To be clear on scope: this PR is investigation-only — it adds the render hook and the assessment but converts no content, so none of these tools break yet. The migration plan now lists them explicitly so they're handled before any conversion. |
Documents a broader parity run: 825 files and ~5,000 relref calls converted across four structurally different sections at once (the Active-Active mount, Redis Cloud, Kubernetes, and integrate). No build errors, no dropped pages, no new warnings, and every rendered-link difference was benign normalisation. Notes the Active-Active mount resolves each relative link to the correct per-mount permalink via .PageInner, and flags that versioned trees were not yet covered. Ticket: DOC-6909 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🧠 Redis MemoryFound 5 related items from repository history:
Memory updated at c4e3953 |
Found by the versioned-tree stress test on a malformed link whose anchor embedded a full URL with its own `#`. Two defects: the resolved-link branches composed `RelPermalink + anchor` without safeURL, so Go's html/template autoescaper distrusted the embedded protocol and blanked the href to ZgotmplZ; and splitting the anchor on every `#` dropped everything after the second one. Now the resolved href is piped through safeURL, and the anchor is split on the first `#` only (via split/after/delimit) so embedded `#` survives intact. Verified: operate/rs/7.8 (326 files, all with url: overrides) reaches exact parity with the relref baseline bar the known cosmetic external-paren encoding, no dropped pages, normal anchors unchanged. A dead end en route: strings.Index does not exist in Hugo's template namespace (it silently resolves to nothing and the hook fails on every page), so the first-# split is done with split + after + delimit instead. Learned: Hugo has no strings.Index; resolved-link hrefs need safeURL or Go emits ZgotmplZ for anchors containing an embedded protocol Ticket: DOC-6909 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
operate/rs/7.8 (326 files, all with url: front-matter overrides) reaches exact parity with relref; GetPage honours url overrides identically. Notes the anchor-handling edge case the pass surfaced and how it was fixed. Ticket: DOC-6909 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🧠 Redis MemoryFound 5 related items from repository history:
Memory updated at f8bfc62 |
Three genuine link bugs found while auditing render-hook build warnings, fixed in the current relref/source form and independent of any migration (they also correct the live site): - references/rest-api/permissions.md (4 versions): the view_all_nodes_alerts entry was missing its `#`, so it linked to a non-existent page rather than the in-page anchor. - operate/rs/8.0/flex/_index.md: the Auto Tiering relref pointed at a malformed /operate/rs/8.0/7.22/... path; corrected to /operate/rs/7.22/... (the last version with the page - 8.0 replaces Auto Tiering with Flex). - clients/nodejs/amr.md: a doubled [Authority]([Authority](...)) link, unwrapped to a single link. Verified: the five corresponding warnings (2 REF_NOT_FOUND, 4 render-link) clear with no new warnings and no dropped pages. Left untouched: working alias links (not bugs), generated redisvl content (fix belongs in the sync script), and a missing dataset file. Ticket: DOC-6909 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🧠 Redis MemoryFound 5 related items from repository history:
Memory updated at e5665e0 |
…hook Clears all but one of the render-hook build warnings (34 -> 1) so merging the hook will not introduce a wave of warnings for the team. All fixed in current source form (no migration to plain links here): - Alias links canonicalised: /develop/interact/transactions -> /develop/using-commands/transactions (discard, exec, multi, unwatch, watch); /develop/use/keyspace-notifications -> /develop/pubsub/keyspace-notifications (expire). - setbit: data-types-intro#bitmaps -> data-types#bitmaps. - benchmarks: legacy /topics/pipelining -> /develop/using-commands/pipelining. - security: GPG-key link made bundle-relative so it resolves as a resource. - redisgraph release note: bare redisearch.io given an https scheme. - redisvl sql_to_redis_queries: reversed [url](docs) link corrected to [docs](url). - redisvl mcp_authentication (0.23.0 + latest): relative mcp.md link -> relref, which resolves in page context (these pages render via markdownify, where a relative link resolved against the wrong base). - redisvl 0.6.0/0.7.0 user guides: removed the dead "Release Guides" section (target never imported; the latest version already dropped it). Remaining and deliberately untouched: ./data/products.txt on aggregations-syntax - the dataset file does not exist anywhere in the repo, so it needs the asset added or the link removed, which is a team decision. Verified: 0 build errors, no dropped pages, no new warnings. Ticket: DOC-6909 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🧠 Redis MemoryFound 5 related items from repository history:
Memory updated at 49b53ca |
The note pointed readers to ./data/products.txt for download, but that dataset file has never existed in the repo, so the link was always broken. Removed the note (confirmed OK with the team). This clears the last outstanding render-hook warning: the build is now warning-free (0 render-link, 0 REF_NOT_FOUND, 0 errors, no dropped pages). Ticket: DOC-6909 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🧠 Redis MemoryFound 5 related items from repository history:
Memory updated at ce39f5b |
…Markdown
First real section migration piloting the render-hook approach: converts all
147 relref calls across the 14 redis-py pages from [text]({{< relref "/path" >}})
to ordinary [text](/path) Markdown links, resolved by the link render hook.
Rendered output is byte-identical to the relref baseline - 0 href differences
across the whole redis-py tree, 0 new build warnings, 0 errors, no dropped
pages. That includes the two bare-relative links
(develop/using-commands/transactions with no leading slash), which GetPage
resolves via the same site-wide fallback as relref.
Ticket: DOC-6909
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🧠 Redis MemoryFound 5 related items from repository history:
Memory updated at c7dd76f |
Adds layouts/_default/_markup/render-blockquote.html so GitHub-style alert
blockquotes (`> [!NOTE]`, `> [!WARNING]`, `> [!TIP]`, `> [!INFO]`, ...) render
with the same styling as the note/warning/tip/info/alert shortcodes, and
documents it in the assessment. Regular blockquotes keep Hugo's default
rendering (verified byte-identical across 21 sampled pages; site-wide count
unchanged at 70). Also fixes a pre-existing malformed alert in the RedisStack
7.2 release note whose body line lacked the `>` prefix, so it now renders inside
the Warning box that the hook activates.
The point is not cosmetic. A shortcode callout markdownifies its inner content
in a page-less context, so relative links inside it resolve against the site
root and break. A blockquote alert's body is native Markdown rendered in page
context, so its links resolve correctly - proven: an identical relative link
resolved inside `> [!NOTE]` but stayed raw inside `{{< note >}}`. This makes the
callout->blockquote migration a prerequisite for portable source-relative links
inside callouts, which is why it should land before the relative-link migration.
Verified: full build clean (0 errors, 0 warnings, 6484 pages).
Learned: blockquote-alert bodies render in page context (links resolve); shortcode callouts markdownify page-less (relative links break) - so blockquotes must precede relative-link migration
Ticket: DOC-6909
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🧠 Redis MemoryFound 5 related items from repository history:
Memory updated at f29556e |
… alerts
Pilots the callout->blockquote migration on develop/clients/redis-py: converts
all 8 `{{< note >}}` shortcodes across 5 files to portable `> [!NOTE]` Markdown
alerts, rendered by the new blockquote render hook.
Verified against the shortcode baseline: the 8 alert boxes render with identical
styling, identical body text, and 0 href differences across all 5 pages; full
build clean (0 errors, 0 warnings, 6484 pages). The only rendering delta is
benign `<p>`-wrapping of callout bodies (blockquotes always wrap paragraphs;
markdownify did not always) - no change to visible text, links, or the alert
box itself. redis-py had only note callouts, no custom titles, and no
block-level nested shortcodes, so it is the clean base case.
Converter (reusable, handles note/warning/tip/info): prefixes each inner line
with `> `, blank lines become `>` to preserve paragraph breaks. Links inside the
notes are currently absolute, so they already resolved; the portability payoff
(relative links working inside callouts) lands when the relative-link migration
is redone on this section next.
Ticket: DOC-6909
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🧠 Redis MemoryFound 5 related items from repository history:
Memory updated at 3139628 |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 3139628. Configure here.
Completes the redis-py portability pilot: converts all 147 internal links from root-relative absolute paths to source-relative Markdown links with the `.md` suffix (e.g. [x](../../reference/protocol-spec.md#resp-versions)), so they resolve in the repo (VS Code, GitHub) as well as via the render hook. Also enhances the link hook to strip a trailing `/_index.md` or `/index.md`, so links that point at a section or leaf-bundle file resolve. This depended on the callout->blockquote migration landed for redis-py: the six links that live inside callouts only resolve now because those callouts are blockquote alerts (rendered in page context) rather than markdownify shortcodes. With that in place the whole section is portable Markdown - relative links in both body and callouts - and the rendered site is byte-identical to the previous absolute-link version (0 href diffs across all 14 pages, 0 warnings, 0 errors). Every relative target was verified to point at a real repo file. Learned: source-relative .md links + blockquote callouts together give a fully repo-navigable section with identical published output; blockquotes had to land first or the in-callout links break Ticket: DOC-6909 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🧠 Redis MemoryFound 5 related items from repository history:
Memory updated at 2422cff |
…itten) Backs out the five RedisVL page edits from the broken-link cleanup (sql_to_redis_queries reversed-link, two mcp_authentication relative-link fixes, and the dead "Release Guides" section removal in 0.6.0/0.7.0). paoloredis clarified that the RedisVL docs under content/develop/ai/redisvl/ are pulled and converted from an external site on every sync - even old versions are refreshed - so hand-edits here are overwritten within a week or so. The real fix for those links belongs in the import/convert process (build/redisvl_docs_sync.py), not in the generated pages. Reverted to the merge-base version so this branch carries no RedisVL changes at all. This reintroduces the pre-existing RedisVL link warnings that the render hook surfaces (mcp.md relative link, the reversed sql_to_redis "docs" link, and the release_guide/ dead links) - deliberately left as out-of-scope, to be fixed in the sync tooling. aggregations-syntax.md is NOT RedisVL (hand-maintained search-and-query page) and keeps its fix. Directive: do not hand-edit content/develop/ai/redisvl/** - it is regenerated from an external source on every sync (even old versions); fix such issues in build/redisvl_docs_sync.py instead Ticket: DOC-6909 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🧠 Redis MemoryFound 5 related items from repository history:
Memory updated at c1ba272 |
Reinstates the five RedisVL page fixes reverted in c1ba272, per the decision to keep the broken-link cleanup (RedisVL included) in this PR. Restores the sql_to_redis reversed-link fix, the two mcp_authentication relative-link fixes, and the dead "Release Guides" section removal in 0.6.0/0.7.0 - clearing the 22 RedisVL render-link warnings the revert had reintroduced. Caveat retained for the record: RedisVL docs under content/develop/ai/redisvl/ are regenerated from an external source on every sync (even old versions), so these edits will be overwritten within ~a week; the durable fix belongs in build/redisvl_docs_sync.py. Keeping them here anyway keeps the build clean until that sync happens. Ticket: DOC-6909 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🧠 Redis MemoryFound 5 related items from repository history:
Memory updated at 07328c9 |

Note: implementing this revealed some existing problems that needed fixing before adding the new render hook stuff. The "interesting" stuff (if you can call it that) is in the
redis-pyclient-specific section. It should look and behave exactly as it does currently, but the links are relative to the current page and the notes/warnings are handled with a Github-flavour blockquote format.You can get a better impression of the benefit of this from the Github code view for the branch. Links in the
redis-pysection now work in the Github page when you click them and the notes/warnings are also formatted nicelyWhat this is
Investigation (DOC-6909) into replacing Hugo shortcodes with render hooks to make the Markdown sources more portable. This first step delivers a link render hook to replace the
relrefshortcode (~30k calls across ~3,500 files), plus the empirical findings that back the approach.This is a prototype + assessment, not a migration — no content is converted here. It adds:
layouts/_default/_markup/render-link.html— the hook.HUGO_DEPENDENCY_ASSESSMENT.md— the broad shortcode-portability assessment, with a new Prototype findings section under Links.How it was tested
Added the hook, converted every
relrefincontent/develop/clients/(1,072 calls / 110 files) to plain Markdown links, built the full site, and diffed rendered link targets against arelrefbaseline.Internal-link parity is exact across 125 client pages — anchors, mixed-case paths, bare page-relative paths, and
.mdsuffixes all resolve byte-for-byte identically torelref. The only residual differences are cosmetic percent-encoding of literal parentheses in external URLs ((→%28).Key findings
relrefand plain links can coexist — no atomic migration required. While arelrefremains, its destination is still Hugo's shortcode placeholder at hook time (goldmark runs before shortcode substitution), so the hook passes placeholders through untouched. A transition guard suppresses the ~26.8k spurious "unresolved" warnings this would otherwise produce; remove it oncerelrefis fully migrated..mdstripped, trailing slash). So the hook must be parity-tested site-wide when it lands; content can then be migrated gradually..Page.Filenil-panics on generated/markdownified pages;urls.Parsehard-errors on a pre-existing malformed link and fails the whole build; a fallback double-appended the anchor; andGetPagecan't resolve alias targets (false warnings). All are documented and handled/avoided in the committed hook.Suggested rollout
relrefbaseline.relref→ plain links gradually, section by section.relrefis gone, remove the placeholder guard and optionally make unresolved links a hard error.🤖 Generated with Claude Code
Note
Medium Risk
Global link rendering affects every internal Markdown link site-wide at build time; incorrect resolution could break navigation, though the pilot was parity-tested against relref.
Overview
Adds Hugo render hooks for links and GitHub-style alert blockquotes, plus
HUGO_DEPENDENCY_ASSESSMENT.mddocumenting how to reducerelrefand shortcode lock-in. The link hook resolves plain Markdown internal URLs likerelref, with a transition guard so unmigratedrelrefplaceholders still work during gradual migration.content/develop/clients/redis-py/is updated as the pilot:{{< relref >}}becomes source-relative.mdlinks, and many{{< note >}}callouts become> [!NOTE]blockquotes so links and callouts work in GitHub’s Markdown view.Smaller content fixes elsewhere: command pages point transaction/keyspace reference links at current paths, a few broken external/MCP links are repaired, RedisVL index TOC cleanup, and RS REST API permission tables fix a broken
view_all_nodes_alertsanchor.Reviewed by Cursor Bugbot for commit 07328c9. Bugbot is set up for automated code reviews on this repo. Configure here.