From 245c4fbf3a581926db9a2edef438ad355a327413 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 29 Mar 2026 11:37:45 +0000 Subject: [PATCH 01/30] Restyle docs to match landing page and restructure per Diataxis MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Replace Roboto with Inter + JetBrains Mono to match landing page typography - Port landing page design tokens (colors, shadows, radii) into docs CSS - Add subtle dot-grid background, amber glow, and terminal theme overrides - Style code blocks, admonitions, tables, sidebar, and links coherently - Add h1 accent underline and dashed hr matching landing page motifs - Reorganise index.rst toctrees into four Diataxis sections: Tutorials → installation, getting_started How-to Guides → troubleshooting, contributing Reference → manifest, manual, changelog, legal Explanation → vendoring, alternatives, internal - Add sphinx_design extension to conf.py (already used by landing page) https://claude.ai/code/session_0183Tzu6McYvSDwoXDoRjEyN --- doc/conf.py | 1 + doc/index.rst | 61 ++--- doc/static/css/custom.css | 456 +++++++++++++++++++++++++++++++++----- 3 files changed, 435 insertions(+), 83 deletions(-) diff --git a/doc/conf.py b/doc/conf.py index 62f56e426..6e7304e75 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -34,6 +34,7 @@ # ones. extensions = [ "sphinx_sitemap", + "sphinx_design", "plantweb.directive", "scenario_directive", "sphinx.ext.autodoc", diff --git a/doc/index.rst b/doc/index.rst index 7715f5f54..3cd888c0d 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -25,41 +25,48 @@ :width: 100% :align: center +Dfetch — *a source-only no-hassle project-dependency aggregator* +================================================================ + +We make products that can last 15+ years; because of this we want to be able to have all sources +available to build the entire project from source without depending on external resources. +For this, we needed a dependency manager flexible enough to retrieve dependencies as plain text +from various sources. `svn externals`, `git submodules` and `git subtrees` solve a similar +problem, but not in a VCS-agnostic or completely user-friendly way. +Dfetch must promote upstreaming changes, but allow for local customisations. +See :ref:`vendoring` for background on why this matters. + +.. asciinema:: asciicasts/basic.cast + +---- + .. toctree:: :maxdepth: 2 + :caption: Tutorials installation getting_started - manifest - manual - troubleshooting - contributing - changelog - alternatives - vendoring - legal - internal -Dfetch - *a source-only no-hassle project-dependency aggregator* -================================================================ +.. toctree:: + :maxdepth: 2 + :caption: How-to Guides -What is Dfetch? ---------------- + troubleshooting + contributing -We make products that can last 15+ years; because of this we want to be able to have all sources available -to build the entire project from source without depending on external resources. -For this, we needed a dependency manager that was flexible enough to retrieve dependencies as plain text -from various sources. `svn externals`, `git submodules` and `git subtrees` solve a similar -problem, but not in a VCS-agnostic way or completely user-friendly way. -We want self-contained code repositories without any hassle for end-users. -Dfetch must promote upstreaming changes, but allow for local customizations. -The problem is described thoroughly in `managing external dependencies `_ and sometimes -is also known as :ref:`vendoring`. +.. toctree:: + :maxdepth: 2 + :caption: Reference -Other tools that do similar things are ``Zephyr's West``, ``CMake ExternalProject`` and other meta tools. -See :ref:`alternatives` for a complete list. + manifest + manual + changelog + legal -Basic usage ------------ +.. toctree:: + :maxdepth: 2 + :caption: Explanation -.. asciinema:: asciicasts/basic.cast + vendoring + alternatives + internal diff --git a/doc/static/css/custom.css b/doc/static/css/custom.css index c502c39b1..da4012b25 100644 --- a/doc/static/css/custom.css +++ b/doc/static/css/custom.css @@ -1,44 +1,286 @@ -@import url("https://fonts.googleapis.com/css?family=Roboto:100,300,300i,400,500,700,900"); +@import url("https://fonts.googleapis.com/css2?family=Inter:wght@300;400;500;600;700;800&family=JetBrains+Mono:wght@400;500&display=swap"); -.sphinxsidebar .caption-text { - font-size: 130%; +/* ============================================================ + Design Tokens — mirrors the landing page palette exactly + ============================================================ */ +:root { + --primary: #c2620a; + --primary-dark: #a0510a; + --accent: #4e7fa0; + --accent-dark: #3a6682; + --text: #1c1917; + --text-2: #3d3530; + --text-muted: #78716c; + --bg-tint: #fef8f0; + --bg-mint: #eff6fa; + --border: #e7e0d8; + --grad: linear-gradient(135deg, #c2620a 0%, #7a3a0a 100%); + --shad-sm: 0 1px 3px rgba(0,0,0,.10), 0 1px 2px rgba(0,0,0,.06); + --shad-md: 0 4px 6px -1px rgba(0,0,0,.10), 0 2px 4px -2px rgba(0,0,0,.10); + --shad-lg: 0 10px 15px -3px rgba(0,0,0,.10), 0 4px 6px -4px rgba(0,0,0,.10); + --r: 12px; + --r-sm: 8px; + --r-lg: 20px; + --ease: 0.22s ease; } -.logo { - width: 100%; +/* ============================================================ + Keyframes + ============================================================ */ +@keyframes fade-slide-up { + from { opacity: 0; transform: translateY(18px); } + to { opacity: 1; transform: translateY(0); } } -.sphinxsidebarwrapper .internal, -.sphinxsidebarwrapper .external { - font-weight: 300; +/* ============================================================ + Base + ============================================================ */ +html, body { + overflow-x: clip; } body { - font-family: Roboto; + font-family: "Inter", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; + font-size: 16px; + -webkit-font-smoothing: antialiased; + -moz-osx-font-smoothing: grayscale; + color: var(--text); + background-color: #fff; + /* Subtle dot-grid background — same formula as landing page */ + background-image: + linear-gradient(to right, #fff 0%, transparent 10%, transparent 90%, #fff 100%), + radial-gradient(circle, rgba(194, 98, 10, 0.22) 1.5px, transparent 1.5px); + background-size: 100% 100%, 28px 28px; + background-repeat: no-repeat, repeat; } -h1, -h2, -h3, -h4, -h5, -h6 { - font-family: Roboto !important; - font-weight: 300 !important; +/* Keep content areas transparent so body dot-grid shows through */ +div.documentwrapper, +div.body { + background-color: transparent; + /* Soft amber glow at the top of each page */ + background-image: radial-gradient( + ellipse 120% 40% at 50% 0%, + rgba(194, 98, 10, 0.07) 0%, + transparent 65% + ); + background-size: 100% 100%; + background-repeat: no-repeat; } -.toctree-l1 { - padding-bottom: 0.5em; +/* ============================================================ + Typography + ============================================================ */ +h1, h2, h3, h4, h5, h6 { + font-family: "Inter", -apple-system, BlinkMacSystemFont, sans-serif !important; + font-weight: 700 !important; + letter-spacing: -0.02em; + line-height: 1.25 !important; + color: var(--text) !important; } +h1 { font-size: 2rem !important; } +h2 { font-size: 1.5rem !important; margin-top: 2.5rem !important; } +h3 { font-size: 1.2rem !important; margin-top: 2rem !important; } + .body p, .body dd, .body li { - font-family: Roboto; - font-size: 1em; - font-weight: 300; - line-height: 2; - text-align: justify; + font-family: "Inter", sans-serif; + font-size: 1.0625rem; + font-weight: 400; + line-height: 1.8; + color: var(--text-2); + text-align: left; +} + +strong { + font-weight: 600; + color: var(--text); +} + +/* Links */ +a { + color: var(--primary) !important; + text-decoration: none; + transition: color var(--ease), opacity var(--ease); +} +a:hover { + color: var(--primary-dark) !important; + opacity: 0.85; +} + +img { + height: auto; + max-width: 100%; + border-radius: var(--r-sm); +} + +/* ============================================================ + Code Blocks + ============================================================ */ +div.highlight { + border-radius: var(--r-sm) !important; + overflow: hidden !important; + box-shadow: var(--shad-sm) !important; + border: 1px solid var(--border) !important; +} + +div.highlight pre { + font-family: "JetBrains Mono", "Fira Code", "Cascadia Code", Consolas, monospace !important; + font-size: 0.875rem !important; + line-height: 1.7 !important; + padding: 1.25rem 1.5rem !important; +} + +/* Inline code */ +code, .pre, tt { + font-family: "JetBrains Mono", "Fira Code", Consolas, monospace !important; + font-size: 0.875em; + background: var(--bg-tint); + border: 1px solid var(--border); + border-radius: 4px; + padding: 0.1em 0.35em; + color: var(--primary-dark); +} + +/* ============================================================ + Admonitions + ============================================================ */ +div.admonition { + border: none !important; + border-radius: var(--r-sm) !important; + border-left: 4px solid var(--accent) !important; + box-shadow: var(--shad-sm) !important; + font-family: "Inter", sans-serif; + padding: 1.25rem 1.5rem !important; + background: var(--bg-mint) !important; + transition: box-shadow var(--ease) !important; + margin: 1.5rem 0 !important; +} + +div.admonition.warning, +div.admonition.caution { + border-left-color: var(--primary) !important; + background: var(--bg-tint) !important; +} + +div.admonition.danger, +div.admonition.error { + border-left-color: #c0544a !important; + background: #fff5f4 !important; +} + +div.admonition.tip, +div.admonition.hint { + border-left-color: #7aad6b !important; + background: #f3faf2 !important; +} + +div.admonition p.admonition-title { + font-family: "Inter", sans-serif; + font-weight: 700 !important; + color: var(--accent) !important; + margin: 0 0 0.5rem 0 !important; + font-size: 0.95rem !important; +} + +div.admonition.warning p.admonition-title, +div.admonition.caution p.admonition-title { color: var(--primary) !important; } +div.admonition.danger p.admonition-title, +div.admonition.error p.admonition-title { color: #c0544a !important; } +div.admonition.tip p.admonition-title, +div.admonition.hint p.admonition-title { color: #5a9a50 !important; } + +div.admonition p { + font-weight: 400 !important; + margin-top: 0.5rem !important; + color: var(--text-2); +} + +.admonition a { + font-weight: 500; +} + +/* ============================================================ + Tables + ============================================================ */ +table.docutils { + border-collapse: collapse; + width: 100%; + border-radius: var(--r-sm); + overflow: hidden; + box-shadow: var(--shad-sm); + border: 1px solid var(--border) !important; + margin: 1.5rem 0; +} + +table.docutils thead tr th { + background: var(--bg-tint); + font-family: "Inter", sans-serif; + font-weight: 600; + font-size: 0.875rem; + letter-spacing: 0.03em; + text-transform: uppercase; + color: var(--text) !important; + padding: 0.75rem 1rem; + border-bottom: 2px solid var(--border) !important; +} + +table.docutils tbody tr td { + padding: 0.625rem 1rem; + border-bottom: 1px solid var(--border) !important; + font-size: 0.9375rem; + color: var(--text-2); +} + +table.docutils tbody tr:last-child td { + border-bottom: none !important; +} + +table.docutils tbody tr:nth-child(even) td { + background: rgba(254, 248, 240, 0.4); +} + +/* ============================================================ + Sidebar + ============================================================ */ +.sphinxsidebar .caption-text { + font-family: "Inter", sans-serif; + font-size: 0.75rem; + font-weight: 700; + letter-spacing: 0.08em; + text-transform: uppercase; + color: var(--text-muted) !important; +} + +.sphinxsidebarwrapper .caption-text { + font-size: 0.8rem; +} + +.logo { + width: 100%; +} + +.sphinxsidebarwrapper .logo-name { + display: none; +} + +.sphinxsidebarwrapper .logo { + margin-bottom: 2em; +} + +.sphinxsidebarwrapper .internal, +.sphinxsidebarwrapper .external { + font-family: "Inter", sans-serif; + font-weight: 400; + font-size: 0.9rem; +} + +/* Sidebar active link */ +.sphinxsidebarwrapper a.current { + color: var(--primary) !important; + font-weight: 600; } .search { @@ -50,61 +292,163 @@ h6 { } .sphinxsidebar input[type="submit"] { - font-family: "Roboto", serif; + font-family: "Inter", sans-serif; + background: var(--primary); + color: #fff; + border: none; + border-radius: var(--r-sm); + padding: 0.4rem 0.8rem; + cursor: pointer; + font-weight: 600; + transition: background var(--ease); +} + +.sphinxsidebar input[type="submit"]:hover { + background: var(--primary-dark); +} + +/* ============================================================ + Toctree / Navigation + ============================================================ */ +.toctree-l1 { + padding-bottom: 0.5em; } .caption { - margin-top: 1em; + margin-top: 1.5em; } .caption-text { - font-weight: 500; + font-family: "Inter", sans-serif !important; + font-weight: 700 !important; + font-size: 0.75rem !important; + letter-spacing: 0.08em; + text-transform: uppercase; + color: var(--text-muted) !important; } -.sphinxsidebarwrapper .logo-name { - display: none; +/* Diataxis section colour-coding in sidebar captions */ +/* Tutorials → amber/primary */ +.toctree-wrapper p.caption:nth-of-type(1) .caption-text, +p.caption[id*="tutorial"] .caption-text { color: var(--primary) !important; } + +/* How-to → accent */ +.toctree-wrapper p.caption:nth-of-type(2) .caption-text, +p.caption[id*="how-to"] .caption-text { color: var(--accent) !important; } + +/* Reference → text */ +.toctree-wrapper p.caption:nth-of-type(3) .caption-text, +p.caption[id*="reference"] .caption-text { color: var(--text) !important; } + +/* Explanation → muted */ +.toctree-wrapper p.caption:nth-of-type(4) .caption-text, +p.caption[id*="explanation"] .caption-text { color: var(--text-muted) !important; } + +/* ============================================================ + Asciinema Player — match landing page terminal theme + ============================================================ */ +.asciinema-player-wrapper { + border-radius: var(--r) !important; + overflow: hidden !important; + box-shadow: var(--shad-lg) !important; + margin: 1.5rem 0 !important; } -.sphinxsidebarwrapper .logo { - margin-bottom: 2em; +.asciinema-player-theme-monokai { + --term-color-background: #1e1610; + --term-color-foreground: #f0e8dc; + --term-color-0: #2d2018; + --term-color-1: #c0544a; + --term-color-2: #7aad6b; + --term-color-3: #c2820a; + --term-color-4: #4e7fa0; + --term-color-5: #9b6eb0; + --term-color-6: #5da0b5; + --term-color-7: #e8ddd0; + --term-color-8: #6b5e54; + --term-color-9: #e07060; + --term-color-10: #c2620a; + --term-color-11: #d9a44a; + --term-color-12: #7aa8c5; + --term-color-13: #b08ac0; + --term-color-14: #7dc0d5; + --term-color-15: #f0e8dc; } -img { - height: auto; - max-width: 100%; +/* ============================================================ + Miscellaneous + ============================================================ */ + +/* Section permalink anchors — subtle */ +a.headerlink { + color: var(--border) !important; + font-size: 0.8em; + transition: color var(--ease) !important; +} +a.headerlink:hover { + color: var(--primary) !important; + opacity: 1 !important; } -div.admonition { +/* Horizontal rule */ +hr { border: none; - border-radius: 0; - box-shadow: 0 2px 2px 0 rgba(0, 0, 0, 0.14), 0 1px 5px 0 rgba(0, 0, 0, 0.12), - 0 3px 1px -2px rgba(0, 0, 0, 0.2); - font-family: Roboto; - padding: 2em; - position: relative; - transition: box-shadow 0.25s; + height: 2px; + background-image: repeating-linear-gradient( + 90deg, + rgba(194, 98, 10, 0.3) 0, + rgba(194, 98, 10, 0.3) 2px, + transparent 2px, + transparent 24px + ); + margin: 2.5rem 0; } -div.admonition p.admonition-title { - font-family: "Roboto", serif; - font-weight: 300; - margin: 0 0 5px 0; +/* Field lists (option tables) */ +table.field-list th { + font-family: "Inter", sans-serif; + font-weight: 600; + color: var(--text-muted); + font-size: 0.875rem; } -div.admonition p { - font-weight: 300; - margin-top: 1em; +/* Sphinx "rubric" headings */ +p.rubric { + font-family: "Inter", sans-serif !important; + font-weight: 700 !important; + font-size: 0.9rem !important; + letter-spacing: 0.06em; + text-transform: uppercase; + color: var(--text-muted) !important; + border-bottom: 1px solid var(--border); + padding-bottom: 0.4rem; + margin-bottom: 1rem; } -.admonition a { - font-weight: 300; +/* Page title (h1) accent underline */ +h1::after { + content: ""; + display: block; + width: 3rem; + height: 3px; + background: var(--grad); + border-radius: 2px; + margin-top: 0.5rem; } -strong { - font-weight: 400; +/* Sphinx tabs — match palette */ +.sphinx-tabs-tab { + font-family: "Inter", sans-serif !important; + font-weight: 500 !important; +} + +.sphinx-tabs-tab[aria-selected="true"] { + color: var(--primary) !important; + border-bottom-color: var(--primary) !important; } -.pre { - padding-left: 0.2em; - padding-right: 0.2em; +/* JSON schema tables */ +.json-schema-description p { + font-size: 0.9375rem; + line-height: 1.7; } From 72147e4bbe3418b3c45fd3e88df361411092bac0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 29 Mar 2026 11:42:23 +0000 Subject: [PATCH 02/30] Split add scenarios into separate feature files and wire up both casts - features/add-project-through-cli.feature now contains only the four non-interactive scenarios (plain add, auto-rename, dst guessing, overrides) - features/interactive-add.feature is a new file with all eight interactive scenarios (guided wizard, tag, src, ignore, update, abort, empty-src, pre-filled fields) - doc/manual.rst Add section now has two subsections: Non-interactive: add.cast + scenario-include for add-project-through-cli.feature Interactive: interactive-add.cast + scenario-include for interactive-add.feature https://claude.ai/code/session_0183Tzu6McYvSDwoXDoRjEyN --- doc/manual.rst | 23 ++- features/add-project-through-cli.feature | 213 +--------------------- features/interactive-add.feature | 218 +++++++++++++++++++++++ 3 files changed, 241 insertions(+), 213 deletions(-) create mode 100644 features/interactive-add.feature diff --git a/doc/manual.rst b/doc/manual.rst index 0f5b3c487..b1f980798 100644 --- a/doc/manual.rst +++ b/doc/manual.rst @@ -181,9 +181,30 @@ Add :prog: dfetch :path: add +.. automodule:: dfetch.commands.add + +Non-interactive +~~~~~~~~~~~~~~~ + +Pass a URL and *Dfetch* fills in sensible defaults (name, destination, default +branch) and appends the entry immediately — no prompts, no confirmation. +Use ``--name``, ``--dst``, ``--version``, ``--src``, and ``--ignore`` to +override individual fields. + .. asciinema:: asciicasts/add.cast -.. automodule:: dfetch.commands.add +.. scenario-include:: ../features/add-project-through-cli.feature + +Interactive +~~~~~~~~~~~ + +Pass ``--interactive`` / ``-i`` to be guided step-by-step through every +manifest field. Pre-fill fields with the flag options above and those prompts +are skipped. At the end you can optionally run ``dfetch update`` immediately. + +.. asciinema:: asciicasts/interactive-add.cast + +.. scenario-include:: ../features/interactive-add.feature CLI Cheatsheet diff --git a/features/add-project-through-cli.feature b/features/add-project-through-cli.feature index 8c4628520..4ef0d5180 100644 --- a/features/add-project-through-cli.feature +++ b/features/add-project-through-cli.feature @@ -5,12 +5,8 @@ Feature: Add a project to the manifest via the CLI fills in sensible defaults (name, destination, default branch), shows a preview, and appends the entry to ``dfetch.yaml`` after confirmation. - Pass ``--interactive`` / ``-i`` to be guided step-by-step through every - manifest field (name, destination, branch/tag/revision, optional src, - optional ignore list). - Use ``--name``, ``--dst``, ``--version``, ``--src``, ``--ignore`` to - pre-fill individual fields (works with and without ``-i``). + pre-fill individual fields. Background: Given a git repository "MyLib.git" @@ -70,189 +66,6 @@ Feature: Add a project to the manifest via the CLI dst: ext/MyLib """ - Scenario: Interactive add guides through each field - Given the manifest 'dfetch.yaml' - """ - manifest: - version: '0.0' - projects: - - name: ext/existing - url: some-remote-server/existing.git - """ - When I run "dfetch add -i some-remote-server/MyLib.git" with inputs - | Question | Answer | - | Project name | my-lib | - | Destination path | libs/my | - | Version | master | - | Source path | | - | Ignore paths | | - | Add project to manifest? | y | - | Run update | n | - Then the manifest 'dfetch.yaml' contains entry - """ - - name: my-lib - url: some-remote-server/MyLib.git - branch: master - dst: libs/my - """ - - Scenario: Interactive add with tag version - Given the manifest 'dfetch.yaml' - """ - manifest: - version: '0.0' - projects: - - name: existing - url: some-remote-server/existing.git - """ - When I run "dfetch add -i some-remote-server/MyLib.git" with inputs - | Question | Answer | - | Project name | my-lib | - | Destination path | my-lib | - | Version | v1 | - | Source path | | - | Ignore paths | | - | Add project to manifest? | y | - | Run update | n | - Then the manifest 'dfetch.yaml' contains entry - """ - - name: my-lib - url: some-remote-server/MyLib.git - tag: v1 - """ - - Scenario: Interactive add with src subpath - Given the manifest 'dfetch.yaml' - """ - manifest: - version: '0.0' - projects: - - name: existing - url: some-remote-server/existing.git - """ - When I run "dfetch add -i some-remote-server/MyLib.git" with inputs - | Question | Answer | - | Project name | my-lib | - | Destination path | my-lib | - | Version | master | - | Source path | docs/api | - | Ignore paths | | - | Add project to manifest? | y | - | Run update | n | - Then the manifest 'dfetch.yaml' contains entry - """ - - name: my-lib - url: some-remote-server/MyLib.git - branch: master - src: docs/api - """ - - Scenario: Interactive add with ignore list - Given the manifest 'dfetch.yaml' - """ - manifest: - version: '0.0' - projects: - - name: existing - url: some-remote-server/existing.git - """ - When I run "dfetch add -i some-remote-server/MyLib.git" with inputs - | Question | Answer | - | Project name | my-lib | - | Destination path | my-lib | - | Version | master | - | Source path | | - | Ignore paths | docs, tests | - | Add project to manifest? | y | - | Run update | n | - Then the manifest 'dfetch.yaml' contains entry - """ - - name: my-lib - url: some-remote-server/MyLib.git - branch: master - ignore: - - docs - - tests - """ - - Scenario: Interactive add triggers immediate fetch when update is accepted - Given the manifest 'dfetch.yaml' - """ - manifest: - version: '0.0' - projects: - - name: existing - url: some-remote-server/existing.git - """ - When I run "dfetch add -i some-remote-server/MyLib.git" with inputs - | Question | Answer | - | Project name | MyLib | - | Destination path | MyLib | - | Version | master | - | Source path | | - | Ignore paths | | - | Add project to manifest? | y | - | Run update | y | - Then the manifest 'dfetch.yaml' contains entry - """ - - name: MyLib - url: some-remote-server/MyLib.git - branch: master - """ - And 'MyLib/README.md' exists - - Scenario: Interactive add with abort does not modify manifest - Given the manifest 'dfetch.yaml' - """ - manifest: - version: '0.0' - projects: - - name: existing - url: some-remote-server/existing.git - """ - When I run "dfetch add -i some-remote-server/MyLib.git" with inputs - | Question | Answer | - | Project name | MyLib | - | Destination path | MyLib | - | Version | master | - | Source path | | - | Ignore paths | | - | Add project to manifest? | n | - Then the manifest 'dfetch.yaml' is replaced with - """ - manifest: - version: '0.0' - projects: - - name: existing - url: some-remote-server/existing.git - """ - - Scenario: Interactive add with empty src (repo root) does not add src field - Given the manifest 'dfetch.yaml' - """ - manifest: - version: '0.0' - projects: - - name: existing - url: some-remote-server/existing.git - """ - When I run "dfetch add -i some-remote-server/MyLib.git" with inputs - | Question | Answer | - | Project name | MyLib | - | Destination path | MyLib | - | Version | master | - | Source path | | - | Ignore paths | | - | Add project to manifest? | y | - | Run update | n | - Then the manifest 'dfetch.yaml' contains entry - """ - - name: MyLib - url: some-remote-server/MyLib.git - branch: master - """ - And the manifest 'dfetch.yaml' does not contain 'src:' - Scenario: Non-interactive add with field overrides Given the manifest 'dfetch.yaml' """ @@ -270,27 +83,3 @@ Feature: Add a project to the manifest via the CLI branch: master dst: libs/my-lib """ - - Scenario: Interactive add with pre-filled fields skips those prompts - Given the manifest 'dfetch.yaml' - """ - manifest: - version: '0.0' - projects: - - name: existing - url: some-remote-server/existing.git - """ - When I run "dfetch add -i some-remote-server/MyLib.git --name my-lib --dst libs/my" with inputs - | Question | Answer | - | Version | master | - | Source path | | - | Ignore paths | | - | Add project to manifest? | y | - | Run update | n | - Then the manifest 'dfetch.yaml' contains entry - """ - - name: my-lib - url: some-remote-server/MyLib.git - branch: master - dst: libs/my - """ diff --git a/features/interactive-add.feature b/features/interactive-add.feature new file mode 100644 index 000000000..4f8f5d180 --- /dev/null +++ b/features/interactive-add.feature @@ -0,0 +1,218 @@ +Feature: Add a project interactively via the CLI + + Pass ``--interactive`` / ``-i`` to ``dfetch add`` to be guided step-by-step + through every manifest field (name, destination, branch/tag/revision, + optional src, optional ignore list). + + Pre-fill individual fields with ``--name``, ``--dst``, ``--version``, + ``--src``, ``--ignore`` — those prompts are then skipped. + + Background: + Given a git repository "MyLib.git" + + Scenario: Interactive add guides through each field + Given the manifest 'dfetch.yaml' + """ + manifest: + version: '0.0' + projects: + - name: ext/existing + url: some-remote-server/existing.git + """ + When I run "dfetch add -i some-remote-server/MyLib.git" with inputs + | Question | Answer | + | Project name | my-lib | + | Destination path | libs/my | + | Version | master | + | Source path | | + | Ignore paths | | + | Add project to manifest? | y | + | Run update | n | + Then the manifest 'dfetch.yaml' contains entry + """ + - name: my-lib + url: some-remote-server/MyLib.git + branch: master + dst: libs/my + """ + + Scenario: Interactive add with tag version + Given the manifest 'dfetch.yaml' + """ + manifest: + version: '0.0' + projects: + - name: existing + url: some-remote-server/existing.git + """ + When I run "dfetch add -i some-remote-server/MyLib.git" with inputs + | Question | Answer | + | Project name | my-lib | + | Destination path | my-lib | + | Version | v1 | + | Source path | | + | Ignore paths | | + | Add project to manifest? | y | + | Run update | n | + Then the manifest 'dfetch.yaml' contains entry + """ + - name: my-lib + url: some-remote-server/MyLib.git + tag: v1 + """ + + Scenario: Interactive add with src subpath + Given the manifest 'dfetch.yaml' + """ + manifest: + version: '0.0' + projects: + - name: existing + url: some-remote-server/existing.git + """ + When I run "dfetch add -i some-remote-server/MyLib.git" with inputs + | Question | Answer | + | Project name | my-lib | + | Destination path | my-lib | + | Version | master | + | Source path | docs/api | + | Ignore paths | | + | Add project to manifest? | y | + | Run update | n | + Then the manifest 'dfetch.yaml' contains entry + """ + - name: my-lib + url: some-remote-server/MyLib.git + branch: master + src: docs/api + """ + + Scenario: Interactive add with ignore list + Given the manifest 'dfetch.yaml' + """ + manifest: + version: '0.0' + projects: + - name: existing + url: some-remote-server/existing.git + """ + When I run "dfetch add -i some-remote-server/MyLib.git" with inputs + | Question | Answer | + | Project name | my-lib | + | Destination path | my-lib | + | Version | master | + | Source path | | + | Ignore paths | docs, tests | + | Add project to manifest? | y | + | Run update | n | + Then the manifest 'dfetch.yaml' contains entry + """ + - name: my-lib + url: some-remote-server/MyLib.git + branch: master + ignore: + - docs + - tests + """ + + Scenario: Interactive add triggers immediate fetch when update is accepted + Given the manifest 'dfetch.yaml' + """ + manifest: + version: '0.0' + projects: + - name: existing + url: some-remote-server/existing.git + """ + When I run "dfetch add -i some-remote-server/MyLib.git" with inputs + | Question | Answer | + | Project name | MyLib | + | Destination path | MyLib | + | Version | master | + | Source path | | + | Ignore paths | | + | Add project to manifest? | y | + | Run update | y | + Then the manifest 'dfetch.yaml' contains entry + """ + - name: MyLib + url: some-remote-server/MyLib.git + branch: master + """ + And 'MyLib/README.md' exists + + Scenario: Interactive add with abort does not modify manifest + Given the manifest 'dfetch.yaml' + """ + manifest: + version: '0.0' + projects: + - name: existing + url: some-remote-server/existing.git + """ + When I run "dfetch add -i some-remote-server/MyLib.git" with inputs + | Question | Answer | + | Project name | MyLib | + | Destination path | MyLib | + | Version | master | + | Source path | | + | Ignore paths | | + | Add project to manifest? | n | + Then the manifest 'dfetch.yaml' is replaced with + """ + manifest: + version: '0.0' + projects: + - name: existing + url: some-remote-server/existing.git + """ + + Scenario: Interactive add with empty src (repo root) does not add src field + Given the manifest 'dfetch.yaml' + """ + manifest: + version: '0.0' + projects: + - name: existing + url: some-remote-server/existing.git + """ + When I run "dfetch add -i some-remote-server/MyLib.git" with inputs + | Question | Answer | + | Project name | MyLib | + | Destination path | MyLib | + | Version | master | + | Source path | | + | Ignore paths | | + | Add project to manifest? | y | + | Run update | n | + Then the manifest 'dfetch.yaml' contains entry + """ + - name: MyLib + url: some-remote-server/MyLib.git + branch: master + """ + And the manifest 'dfetch.yaml' does not contain 'src:' + + Scenario: Interactive add with pre-filled fields skips those prompts + Given the manifest 'dfetch.yaml' + """ + manifest: + version: '0.0' + projects: + - name: existing + url: some-remote-server/existing.git + """ + When I run "dfetch add -i some-remote-server/MyLib.git --name my-lib --dst libs/my" with inputs + | Question | Answer | + | Version | master | + | Source path | | + | Ignore paths | | + | Add project to manifest? | y | + | Run update | n | + Then the manifest 'dfetch.yaml' contains entry + """ + - name: my-lib + url: some-remote-server/MyLib.git + branch: master + dst: libs/my + """ From a265ea1264c35cb2d90c58783f98db2e10b44979 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 29 Mar 2026 11:48:21 +0000 Subject: [PATCH 03/30] Style vendoring page with landing-page icons, grids, and cards MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Replace plain prose sections with sphinx_design grids and cards: • Benefits → 2×2 icon-card grid in band-tint section • Costs → 2×2 icon-card grid in band-mint section • Languages → 3-column grid of language cards • Conclusion → tinted full-width card • Best practices → 9 full-width cards each with a Material icon title • Real-world projects → two band grids with linked cards • Further Reading → icon-prefixed bullet list - Add band-tint / band-mint CSS, sd-card / stat-card / card-tinted / sd-bg-dark rules, and sd-text-primary colour to docs custom.css (mirrors the landing page component library) https://claude.ai/code/session_0183Tzu6McYvSDwoXDoRjEyN --- doc/static/css/custom.css | 108 ++++++++ doc/vendoring.rst | 508 ++++++++++++++++++++++++-------------- 2 files changed, 425 insertions(+), 191 deletions(-) diff --git a/doc/static/css/custom.css b/doc/static/css/custom.css index da4012b25..36428a096 100644 --- a/doc/static/css/custom.css +++ b/doc/static/css/custom.css @@ -202,6 +202,114 @@ div.admonition p { font-weight: 500; } +/* ============================================================ + Full-bleed Color Bands (sphinx_design .. div:: band-tint / band-mint) + ============================================================ */ +.band-tint, +.band-mint { + position: relative; + padding: 3rem 0; + margin: 2.5rem 0; + z-index: 0; +} + +.band-tint::before, +.band-mint::before { + content: ""; + position: absolute; + top: 0; + bottom: 0; + width: 80vw; + z-index: -1; +} + +/* Bleeds to left viewport edge — rounded corner on the right */ +.band-tint::before { + left: calc(50% - 50vw); + border-radius: 0 var(--r-lg) var(--r-lg) 0; + background: rgba(254, 248, 240, 0.55); +} + +/* Bleeds to right viewport edge — rounded corner on the left */ +.band-mint::before { + left: calc(50% - 30vw); + border-radius: var(--r-lg) 0 0 var(--r-lg); + background: rgba(239, 246, 250, 0.55); +} + +/* Lead paragraph inside a band */ +.band-tint > p, +.band-mint > p { + font-size: 1.25rem !important; + font-weight: 700 !important; + color: var(--text) !important; + letter-spacing: -0.02em; + margin-bottom: 1.5rem !important; + line-height: 1.3 !important; +} + +/* ============================================================ + Cards — sphinx_design + ============================================================ */ +.sd-card { + border: 1px solid var(--border) !important; + border-top: 3px solid var(--primary) !important; + border-radius: var(--r) !important; + box-shadow: var(--shad-sm) !important; + transition: box-shadow var(--ease), transform var(--ease) !important; + overflow: hidden !important; + background: #fff !important; +} + +/* Extra spacing for standalone (non-grid) cards */ +:not(.sd-col) > .sd-card { + margin-top: 1.5rem !important; + margin-bottom: 1.5rem !important; +} + +.sd-card:hover { + box-shadow: var(--shad-lg) !important; + transform: translateY(-3px); +} + +.sd-card-body { + padding: 1.5rem 2rem !important; +} + +.sd-card-title { + font-family: "Inter", sans-serif !important; + font-size: 1.0625rem !important; + font-weight: 700 !important; + color: var(--text) !important; + margin-bottom: 0.75rem !important; + letter-spacing: -0.01em; +} + +/* Tinted variant */ +.card-tinted { + background: var(--bg-tint) !important; +} + +/* Dark variant */ +.sd-bg-dark { + background: linear-gradient(150deg, #1e1610 0%, #2d2018 100%) !important; + border-top-color: rgba(255, 255, 255, 0.12) !important; + border-color: rgba(255, 255, 255, 0.07) !important; +} +.sd-bg-dark .sd-card-title { color: #fff !important; } +.sd-bg-dark p { color: rgba(255,255,255,0.78) !important; } +.sd-bg-dark strong { color: #fff !important; font-weight: 600; } + +/* Stat cards (small centred grid items) */ +.stat-card .sd-card-title { + font-size: 1rem !important; +} + +/* sd-text-primary maps to accent blue on the landing page */ +.sd-text-primary { + color: var(--accent) !important; +} + /* ============================================================ Tables ============================================================ */ diff --git a/doc/vendoring.rst b/doc/vendoring.rst index bf42d9553..eff663557 100644 --- a/doc/vendoring.rst +++ b/doc/vendoring.rst @@ -1,5 +1,7 @@ +.. _vendoring: + Vendoring ========= @@ -13,8 +15,9 @@ engineers see it as a practical way to gain control and reliability, while other have historically described it as an anti-pattern. Both views exist for good reasons, and understanding the trade-offs matters more than choosing a side. + What People Mean by Vendoring ------------------------------ +------------------------------ At a basic level, vendoring means that a project can be built using only what is present in its repository. No network access is required, no external registries @@ -23,7 +26,7 @@ specific revision of the repository is sufficient to reproduce a build from that point in time. The term originally became common in communities that explicitly placed third- -party code into directories named `vendor`. Over time, the word broadened to +party code into directories named ``vendor``. Over time, the word broadened to describe any approach where dependency source code is copied into the project, regardless of directory layout or tooling. @@ -31,63 +34,93 @@ Vendoring does not necessarily imply permanent forks or heavy modification of third-party code. In many cases, vendored dependencies are kept pristine and updated mechanically from upstream sources. -Why Vendoring Can Be Helpful ----------------------------- +---- + +.. div:: band-tint + + :material-regular:`thumb_up;2em;sd-text-primary` **Why vendoring can be helpful** + + .. grid:: 1 1 2 2 + :gutter: 3 + + .. grid-item-card:: :material-regular:`replay;2em` Reproducibility + :text-align: center + :class-card: stat-card + + When dependencies are fetched dynamically, builds implicitly depend on the + availability of external services. Vendoring makes build inputs **explicit and + local** — no registry, no CDN, no network required. + + .. grid-item-card:: :material-regular:`visibility;2em` Visibility + :text-align: center + :class-card: stat-card + + Dependency code lives in the same source tree. + Easier to **inspect, debug, and understand** — no context switching, + no tooling required to fetch sources on demand. + + .. grid-item-card:: :material-regular:`security;2em` Supply-chain trust + :text-align: center + :class-card: stat-card + + Vendoring shifts trust from live infrastructure to explicit review. + The project decides **when** dependencies are updated and **exactly what + code** is being included. + + .. grid-item-card:: :material-regular:`tune;2em` Healthy friction + :text-align: center + :class-card: stat-card + + When adding a dependency requires consciously pulling in its source code, + teams become **more selective** and more aware of the long-term cost of + that decision. -The strongest argument for vendoring is reproducibility. +---- -When dependencies are fetched dynamically, builds implicitly depend on the -continued availability and behavior of external services. Registries can go -offline, packages can be removed or yanked, and transitive dependencies can change -in ways that are difficult to predict. Vendoring removes these uncertainties by -making the build inputs explicit and local. +.. div:: band-mint -Vendoring also improves visibility. When dependency code lives in the same source -tree, it is easier to inspect, debug, and understand. Developers can step into -library code without context switching or relying on tooling to fetch sources on -demand. This visibility can reveal how much code a project is actually relying on, -especially when compared to the apparent simplicity of a short dependency list in -a configuration file. + :material-regular:`warning;2em;sd-text-primary` **The costs and risks of vendoring** -Another benefit is reduced exposure to supply chain risk. Vendoring does not make -third-party code safe, but it shifts trust from live infrastructure to explicit -review. The project decides when dependencies are updated and exactly what code is -being included. + .. grid:: 1 1 2 2 + :gutter: 3 -Some proponents also argue that vendoring creates a healthy friction around -dependencies. When adding a dependency requires consciously pulling in its source -code, teams may be more selective and more aware of the long-term cost of that -decision. + .. grid-item-card:: :material-regular:`history_toggle_off;2em` History loss + :text-align: center + :class-card: stat-card -The Costs and Risks of Vendoring --------------------------------- + When code is copied into another repository, its original commit history, + tags, and branches are no longer directly visible. This can make updates + **harder**, especially if vendored code has been locally modified. -Vendoring introduces real downsides. + .. grid-item-card:: :material-regular:`call_split;2em` Divergence risk + :text-align: center + :class-card: stat-card -One common criticism is that vendoring discards upstream version control history. -When code is copied into another repository, its original commit history, tags, -and branches are no longer directly visible. This can make updates harder, -especially if vendored code has been locally modified. + When dependency code is nearby and easy to change, there is a temptation + to patch it locally rather than contribute fixes upstream. Over time this + can create **silent forks** that are difficult to reconcile. -Vendoring can also encourage divergence. When dependency code is nearby and easy -to change, there is a temptation to patch it locally rather than contribute fixes -upstream. Over time, this can result in silent forks that are difficult to -maintain or reconcile with new releases. + .. grid-item-card:: :material-regular:`storage;2em` Repository size + :text-align: center + :class-card: stat-card -Repository size and noise are practical concerns as well. Vendored dependencies -increase clone size and can dominate diffs during updates. Large dependency -refreshes can obscure meaningful changes to the project's own code, making review -and merging more difficult. + Vendored dependencies increase clone size and can **dominate diffs** + during updates. Large dependency refreshes can obscure meaningful changes + to the project's own code, making review and merging harder. + + .. grid-item-card:: :material-regular:`build;2em` Maintenance burden + :text-align: center + :class-card: stat-card + + Vendored dependencies do not update themselves. Security fixes, bug fixes, + and compatibility updates must be **pulled in manually**. Without a clear + policy and tooling support, vendored code can easily become outdated. -Maintenance responsibility is another cost. Vendored dependencies do not update -themselves. Security fixes, bug fixes, and compatibility updates must be pulled in -manually. Without a clear policy and tooling support, vendored code can easily -become outdated. Transitive Dependencies ----------------------- -Vendoring becomes significantly more complex once transitive dependencies are +:material-regular:`account_tree;1.2em;sd-text-primary` Vendoring becomes significantly more complex once transitive dependencies are considered. Vendoring a single library often requires vendoring everything that library @@ -100,10 +133,11 @@ graphs, handling version conflicts, and sharing common dependencies across projects. Vendoring replaces that automation with explicit ownership, which is sometimes desirable and sometimes overwhelming. + A Brief History --------------- -Vendoring predates modern package managers. +:material-regular:`history;1.2em;sd-text-primary` Vendoring predates modern package managers. Early C and C++ projects routinely shipped third-party libraries inline because there was no reliable way to depend on system-installed packages. Many Unix @@ -121,230 +155,322 @@ costs of relying entirely on external infrastructure. Today, vendoring is best understood as a trade-off rather than a relic. + Vendoring Across Languages -------------------------- -Different language ecosystems have adopted vendoring to very different degrees. +:material-regular:`code;1.2em;sd-text-primary` Different language ecosystems have adopted vendoring to very different degrees. + +.. grid:: 1 1 3 3 + :gutter: 2 + + .. grid-item-card:: Go + :text-align: center + :class-card: stat-card + + :material-regular:`check_circle;1.5em;sd-text-primary` + + Strongly associated with vendoring. + First-class workflow via ``go mod vendor``. + *"A little copying is better than a little dependency."* + + .. grid-item-card:: Rust + :text-align: center + :class-card: stat-card + + :material-regular:`check_circle;1.5em;sd-text-primary` + + Supported but not the default. Common in **embedded, regulated, and + long-term-support** projects where reproducibility is paramount. + + .. grid-item-card:: C / C++ + :text-align: center + :class-card: stat-card + + :material-regular:`check_circle;1.5em;sd-text-primary` -Go is strongly associated with vendoring. Early versions of Go lacked a central -dependency manager, which made vendoring a practical solution. Even after the -introduction of module support, vendoring remains a first-class workflow, with -tooling that understands and prioritizes vendored dependencies. One of Go's proverbs -is "A little copying is better than a little dependency." + **Frequent** vendoring due to the lack of a universal package manager, + ABI compatibility concerns, and platform differences. -Rust supports vendoring but does not encourage it by default. The Rust ecosystem -places a strong emphasis on reproducibility through its package registry and lock -files, reducing the need for vendoring in everyday development. Vendoring is still -common in embedded systems, regulated environments, and long-term-support -projects. + .. grid-item-card:: JavaScript + :text-align: center + :class-card: stat-card -JavaScript occupies an unusual position. Dependency source code is typically -present locally in directories such as `node_modules`, but it is rarely checked -into version control due to size and churn. Fully vendoring dependencies is -possible but uncommon. + :material-regular:`warning;1.5em` -Python has a mixed history. Vendoring was common in earlier projects and remains -common in small tools, embedded Python environments, and source-distributed -applications. Modern Python development more often relies on virtual environments -and lock files, but vendoring has never disappeared entirely. + ``node_modules`` is local, but **rarely committed** due to size and churn. + Fully vendoring is possible but uncommon. + + .. grid-item-card:: Python + :text-align: center + :class-card: stat-card + + :material-regular:`warning;1.5em` + + Mixed history. Common in **small tools and embedded environments**. + Modern development more often relies on virtual environments and lock files. + + .. grid-item-card:: Java / JVM + :text-align: center + :class-card: stat-card + + :material-regular:`remove_circle;1.5em` + + Rarely vendored. Maven and Gradle ecosystems rely heavily on remote + repositories and dependency resolution. -C and C++ continue to vendor dependencies frequently. The lack of a universal -package manager, combined with ABI compatibility concerns and platform -differences, makes vendoring a practical and sometimes unavoidable choice. Conclusion ---------- -Vendoring is neither a best practice nor an anti-pattern. +.. card:: + :class-card: card-tinted + + :material-regular:`balance;2em;sd-text-primary` **Vendoring is neither a best practice nor an anti-pattern.** -It is a deliberate trade-off that exchanges convenience and automatic updates for -control, predictability, and independence from external systems. In some contexts, -that trade is clearly worthwhile. In others, it introduces more cost than benefit. + It is a deliberate trade-off that exchanges convenience and automatic updates for + control, predictability, and independence from external systems. In some contexts, + that trade is clearly worthwhile. In others, it introduces more cost than benefit. -Used intentionally and with an understanding of its limitations, vendoring is -simply one tool among many for managing dependencies. + Used intentionally and with an understanding of its limitations, vendoring is + simply one tool among many for managing dependencies. + +---- Best Practices -------------- -The following practices are drawn from our own usage of *Dfetch* and real-world policies and respected guidelines. -They *mitigate* vendoring risks; they do not eliminate them. +The following practices are drawn from our own usage of *Dfetch* and real-world +policies and respected guidelines. They *mitigate* vendoring risks; they do not +eliminate them. + +.. card:: :material-regular:`push_pin;1.5em;sd-text-primary` Explicit Version Pinning and Provenance + + Every vendored dependency must be pinned to an explicit version, tag, or commit, + and its source must be documented. + + **Rationale** Vendored code is often added once and then forgotten. Without + automation, vulnerabilities, license issues, and inconsistencies can persist + unnoticed long after initial inclusion. + + * pip: ``vendor.txt`` tracks versions and sources + * Go/Kubernetes: ``go.mod``, ``go.sum``, ``vendor/modules.txt`` + * Cargo: ``Cargo.lock`` + vendored sources + * Guidelines: `OWASP SCVS `_, OpenSSF, NIST SP 800-161, SLSA + + *Dfetch* addresses this by having a declarative :ref:`manifest` and the option to :ref:`freeze` + dependencies to make each revision explicit. -.. admonition :: Explicit Version Pinning and Provenance +.. card:: :material-regular:`offline_bolt;1.5em;sd-text-primary` Reproducible and Offline Builds - Every vendored dependency must be pinned to an explicit version, tag, or commit, and its source must be documented. + Vendoring must enable fully reproducible and offline builds. - **Rationale** Vendored code is often added once and then forgotten. Without automation, vulnerabilities, license issues, - and inconsistencies can persist unnoticed long after initial inclusion. + **Rationale** The point of vendoring code is to remove external dependencies. + By making sure an offline build succeeds, it is proven that builds are not + dependent on external sources. - * pip: `vendor.txt` tracks versions and sources - * Go/Kubernetes: `go.mod`, `go.sum`, `vendor/modules.txt` - * Cargo: `Cargo.lock` + vendored sources - * Guidelines: `OWASP SCVS `_, OpenSSF, NIST SP 800-161, SLSA + * Go: committed ``vendor/`` + * Cargo: ``.cargo/config.toml`` + ``cargo vendor`` + * pip: vendored wheels and pure-Python dependencies + * Guidelines: SLSA, OpenSSF - *Dfetch* addresses this by having a declarative :ref:`manifest` and the option to :ref:`freeze` dependences to make each - revision explicit. + *Dfetch* doesn't directly address this — this is a policy to follow. -.. admonition :: Reproducible and Offline Builds +.. card:: :material-regular:`verified;1.5em;sd-text-primary` Trust Upstream After Due Diligence - Vendoring must enable fully reproducible and offline builds. + Vendored dependencies are not reviewed line-by-line; upstream unit tests are not + run. Do not auto-format vendored code. - **Rationale** The point of vendoring code is to remove external dependencies, by making sure an offline build succeeds it is - proven that builds are not dependent on external sources. + **Rationale** Reviewing every line of external code is costly and rarely effective. + By performing due diligence on upstream sources, you can trust their correctness + and security while minimising maintenance burden. - * Go: committed `vendor/` - * Cargo: `.cargo/config.toml` + `cargo vendor` - * pip: vendored wheels and pure-Python dependencies - * Guidelines: SLSA, OpenSSF + Reviewers should verify: - *Dfetch* doesn't directly address this, this is a policy to follow. + * Version changes and pinning + * Changelogs and release notes for regressions or security issues + * License compatibility + * CI status and test results of the upstream project + * Evidence of active maintenance and community support -.. admonition :: Trust Upstream After Due Diligence + Do not run upstream unit tests locally, apply formatting or style changes, or + make cosmetic changes to vendored code. - Vendored dependencies are not reviewed line-by-line, upstream (unit) tests are not run. - Do not auto-format vendored code. + Aligned with: OWASP SCVS · Google Open Source Security Guidelines · OpenSSF Best Practices. - **Rationale** Reviewing every line of external code is costly and rarely effective. By performing due diligence - on upstream sources, you can trust their correctness and security while minimizing maintenance burden. + *Dfetch* supports this approach via its manifest and metadata file + (``.dfetch_data.yaml``), which can be reviewed independently of the vendored code itself. - Reviewers should verify: +.. card:: :material-regular:`compare_arrows;1.5em;sd-text-primary` Separation of Vendor Updates from Product Changes - * Version changes and pinning - * Changelogs and release notes for regressions or security issues - * License compatibility - * CI status and test results of the upstream project - * Evidence of active maintenance and community support + Vendored dependency updates must be isolated from functional code changes. - Do not run upstream unit tests locally, apply formatting or style changes, or make cosmetic changes to vendored code. + * Separate PRs or commits + * Clear commit messages documenting versions and rationale - This principle is aligned with: + **Rationale** Separation reduces review noise, letting maintainers focus on + meaningful changes. - * OWASP Software Component Verification Standard - * Google Open Source Security Guidelines - * OpenSSF Best Practices + *Dfetch* doesn't address this directly. - *Dfetch* supports this approach via its manifest and metadata file (``.dfetch_data.yaml``), which can be reviewed independently - of the vendored code itself. +.. card:: :material-regular:`edit_off;1.5em;sd-text-primary` Minimize Local Modifications + Vendored code must not be modified directly unless unavoidable. -.. admonition :: Separation of Vendor Updates from Product Changes + If modifications are required: - Vendored dependency updates must be isolated from functional code changes. + * Document patches explicitly + * Prefer patch/overlay mechanisms + * Upstream changes whenever possible + * Cargo: ``[patch]`` mechanism + * Guidelines: Google OSS, OWASP, OpenSSF - * Separate PRs or commits - * Clear commit messages documenting versions and rationale + **Rationale** Keeping the vendored dependency identical to upstream makes it easy + to follow upstream updates. Upstreaming any changes lets others in the wider + community benefit from your fixes. - **Rationale** By separation the review noise will be reduced, letting maintainers focus on important changes. + *Dfetch* addresses this by providing a ``dfetch diff`` (:ref:`Diff`) command and a + ``patch`` (:ref:`Patch`) attribute in the manifest. It also has a CI check to detect + local changes using :ref:`Check`. - *Dfetch* doesn't address this directly. +.. card:: :material-regular:`verified_user;1.5em;sd-text-primary` Continuous Automation and Security Scanning -.. admonition :: Minimize Local Modifications + Vendored dependencies must be continuously verified through automation. - Vendored code must not be modified directly unless unavoidable. + * CI verifies vendor consistency + * Dependency and CVE scanning + * SBOM generation - If modifications are required: + **Rationale** By copy-pasting a dependency, there may be silent security + degradation since there are no automatic updates. - * Document patches explicitly - * Prefer patch/overlay mechanisms - * Upstream changes whenever possible - * pip: documented adaptations - * Cargo: `[patch]` mechanism - * Guidelines: Google OSS, OWASP, OpenSSF + *Dfetch* addresses this by providing a ``dfetch check`` (:ref:`Check`) command to + see if vendored dependencies are out-of-date and various report formats (including + SBoM) to check vulnerabilities. - **Rationale** The vendored dependency may diverge, keeping it the same as upstream makes it easy to keep following - upstream updates. Also by upstreaming any changes, more people outside the project can profit from any fixes. +.. card:: :material-regular:`gavel;1.5em;sd-text-primary` Track License and Legal Information - *Dfetch* addresses this by providing a ``dfetch diff`` (:ref:`Diff`) command and a ``patch`` (:ref:`Patch`) attribute in the manifest. - This attribute can point to one or more patches that should be applied to the vendored dependency. - Next to this there is a CI system to detect local changes using :ref:`Check`. + All vendored dependencies must have their license and legal information explicitly + recorded. -.. admonition :: Continuous Automation and Security Scanning + **Rationale** Ensuring license compliance prevents legal issues and maintains + compatibility with your project's license. - Vendored dependencies must be continuously verified through automation. + * Track license type for each vendored dependency + * Use machine-readable formats where possible (e.g. SPDX identifiers) + * Include license documentation alongside vendored code + * Guidelines: OWASP, OpenSSF, Google OSS best practices - * CI verifies vendor consistency - * Dependency and CVE scanning - * SBOM generation + *Dfetch* addresses this by retaining the license file even when only a subfolder + is fetched. - **Rationale** By copy-pasting a dependency, there maybe silent security degradation since there is no automatic updates. +.. card:: :material-regular:`filter_alt;1.5em;sd-text-primary` Vendor Only What You Need - *Dfetch* addresses this by providing a ``dfetch check`` (:ref:`Check`) command to see if vendored dependencies are out-of-date and - various report formats (including SBoM) to check vulnerabilities. + Minimise vendored code to what is strictly necessary for your project. -.. admonition :: Track License and Legal Information + **Rationale** Vendoring unnecessary code increases maintenance burden, security + risk, and potential for patch rot. Include only the specific modules, packages, + subfolder, or components your project depends on. - All vendored dependencies must have their license and legal information explicitly recorded. + *Dfetch* enables this by allowing you to fetch only a subfolder using the ``src:`` + attribute. - **Rationale** Ensuring license compliance prevents legal issues and maintains compatibility with your project's license. +.. card:: :material-regular:`workspaces;1.5em;sd-text-primary` Isolate Vendored Dependencies - * Track license type for each vendored dependency. - * Use machine-readable formats where possible (e.g., SPDX identifiers). - * Include license documentation alongside vendored code. - * Guidelines: OWASP, OpenSSF, Google OSS best practices. + Vendored dependencies must be clearly isolated from first-party code. - *Dfetch* addresses this by even when only a subfolder is fetched, retaining the license file. + **Rationale** Isolation prevents accidental coupling, avoids namespace conflicts, + and makes audits, updates, and removals easier. Vendored code should be + unmistakably identifiable as third-party code. -.. admonition :: Vendor Only What You Need + * Place vendored dependencies in a clearly named directory (e.g. ``vendor/``, ``_vendor/``) + * Avoid mixing vendored code with product or library sources + * Use language-supported namespace or module isolation where available + * Keep vendored code mechanically separable to enable future un-vendoring - Minimize vendored code to what is strictly necessary for your project. + Follows established practices in: Go (``vendor/``) · pip (``pip/_vendor``) · Cargo (``vendor/``) · Google OSS / OpenSSF. - **Rationale** Vendoring unnecessary code increases maintenance burden, security risk, and potential for patch rot. - Include only the specific modules, packages, subfolder or components your project depends on. + *Dfetch* enables this by allowing you to store the vendored dependency in a folder + using the ``dst:`` attribute. - *Dfetch* enables this by allowing to fetch only a subfolder using the ``src:`` attribute. +---- -.. admonition :: Isolate Vendored Dependencies +.. div:: band-tint - Vendored dependencies must be clearly isolated from first-party code. + :material-regular:`public;2em;sd-text-primary` **Real-world projects using vendoring** - **Rationale** Isolation prevents accidental coupling, avoids namespace conflicts, and makes audits, updates, and - removals easier. Vendored code should be unmistakably identifiable as third-party code. + .. grid:: 1 1 2 2 + :gutter: 2 - * Place vendored dependencies in a clearly named and well-known directory (e.g. ``vendor/``, ``_vendor/``). - * Avoid mixing vendored code with product or library sources. - * Use language-supported namespace or module isolation where available. - * Prevent accidental imports of vendored internals by first-party code. - * Keep vendored code mechanically separable to enable future un-vendoring. + .. grid-item-card:: Dynaconf (Python) + :link: https://github.com/dynaconf/dynaconf/tree/master/dynaconf/vendor + :link-type: url - This principle follows established practices in: + :material-regular:`code;1.2em` Python configuration management library. - * Go (``vendor/`` directory) - * pip (``pip/_vendor``) - * Cargo (``vendor/`` layout) - * Google OSS and OpenSSF guidelines + .. grid-item-card:: pip (Python) + :link: https://github.com/pypa/pip/tree/main/src/pip/_vendor + :link-type: url - *Dfetch* enables this by allowing to store the vendored dependency in a folder using the ``dst:`` attribute. + :material-regular:`code;1.2em` Python's own package installer vendors its dependencies. -Real-world projects using vendoring ------------------------------------ + .. grid-item-card:: Kubernetes (Go) + :link: https://github.com/kubernetes/kubernetes/tree/master/vendor + :link-type: url -- `Dynaconf - (Python) `_ -- `PIP - (Python) `_ -- `Kubernetes - (Go) `_ -- `Cargo - (Rust) `_ + :material-regular:`code;1.2em` The industry-standard container orchestrator. -Real world projects using Dfetch --------------------------------- + .. grid-item-card:: Cargo (Rust) + :link: https://doc.rust-lang.org/cargo/commands/cargo-vendor.html + :link-type: url -Here are some links to example projects using *Dfetch*. + :material-regular:`code;1.2em` Rust's package manager supports vendoring natively. -- `Dfetch`: https://github.com/dfetch-org/dfetch -- `ModbusScope`: https://github.com/ModbusScope/ModbusScope -- `Red Jackets Jazzband`: https://github.com/red-jackets-jazzband/website -- `Example Yocto`: https://github.com/dfetch-org/example-yocto -- `Example Zephyr`: https://github.com/dfetch-org/example-zephyr -Internally we use *Dfetch* for various projects and uses. +.. div:: band-mint + + :material-regular:`bolt;2em;sd-text-primary` **Real-world projects using Dfetch** + + .. grid:: 1 1 2 2 + :gutter: 2 + + .. grid-item-card:: Dfetch + :link: https://github.com/dfetch-org/dfetch + :link-type: url + + :material-regular:`hub;1.2em` Dfetch uses itself to vendor its own test fixtures. + + .. grid-item-card:: ModbusScope + :link: https://github.com/ModbusScope/ModbusScope + :link-type: url + + :material-regular:`hub;1.2em` Industrial Modbus visualisation tool. + + .. grid-item-card:: Example Yocto + :link: https://github.com/dfetch-org/example-yocto + :link-type: url + + :material-regular:`hub;1.2em` Dfetch in a Yocto/embedded Linux build. + + .. grid-item-card:: Example Zephyr + :link: https://github.com/dfetch-org/example-zephyr + :link-type: url + + :material-regular:`hub;1.2em` Dfetch in a Zephyr RTOS project. + + Internally we use *Dfetch* for various projects and uses. Further Reading --------------- -- `Vendoring is a vile anti pattern - Michael F. Lamb `_ -- `SO: What is "vendoring"? - Niels Bom `_ -- `Why we stopped vendoring our dependencies - Carlos Perez `_ -- `Vendoring - Carson Gross `_ -- `Our Software Dependency Problem - Russ Cox `_ -- `On Managing External Dependencies - Phillip Johnston `_ -- `PIP's vendoring policy `_ -- `SubPatch benefits `_ +:material-regular:`menu_book;1.2em;sd-text-primary` + +* `Vendoring is a vile anti pattern — Michael F. Lamb `_ +* `SO: What is "vendoring"? — Niels Bom `_ +* `Why we stopped vendoring our dependencies — Carlos Perez `_ +* `Vendoring — Carson Gross `_ +* `Our Software Dependency Problem — Russ Cox `_ +* `On Managing External Dependencies — Phillip Johnston `_ +* `PIP's vendoring policy `_ +* `SubPatch benefits `_ From d4130937e5eabe6af3eddd66eb6912eb9b947ccc Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 29 Mar 2026 11:52:21 +0000 Subject: [PATCH 04/30] Align README and docs index messaging with landing page MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit README.md: - Tagline: "DFetch can manage dependencies" → "Vendor dependencies without the pain." - Opening paragraph: internal rationale → user-facing pitch matching landing page ("copies source code directly...no Git submodules, no lock-in") - "What Dfetch Does" bullets: add supply-chain, SBOM, patch stack, dfetch import; drop vague items; lead with the strongest differentiators - "DFetch" → "Dfetch" throughout - "Github" → "GitHub" doc/index.rst: - OG/Twitter title: "Dfetch - a source-only no-hassle project-dependency aggregator" → "Dfetch — Vendor dependencies without the pain" (em-dash, landing page tagline) - OG/Twitter description: aligned to landing page wording ("No submodules, no lock-in, supply-chain ready.") - Sphinx meta description: aligned to landing page wording - Keywords: add "supply chain, sbom, license compliance"; drop "fetch tool, package manager, multi-project, monorepo" - H1 subtitle: aligned to landing page tagline - Intro paragraph: replaced internal backstory with user-facing pitch + supply-chain and patch-stack highlights https://claude.ai/code/session_0183Tzu6McYvSDwoXDoRjEyN --- README.md | 41 +++++++++++++++++++---------------------- doc/index.rst | 33 ++++++++++++++++----------------- 2 files changed, 35 insertions(+), 39 deletions(-) diff --git a/README.md b/README.md index ea1827510..dc64c2d8e 100644 --- a/README.md +++ b/README.md @@ -17,37 +17,34 @@ [![OpenSSF Best Practices](https://www.bestpractices.dev/projects/11245/badge)](https://www.bestpractices.dev/projects/11245) -**DFetch can manage dependencies** +**Vendor dependencies without the pain.** -We make products that can last 15+ years; because of this we want to be able to have all sources available -to build the entire project from source without depending on external resources. -For this, we needed a dependency manager that was flexible enough to retrieve dependencies as plain text -from various sources. `svn externals`, `git submodules` and `git subtrees` solve a similar -problem, but not in a VCS-agnostic way or completely user-friendly way. -We want self-contained code repositories without any hassle for end-users. -Dfetch must promote upstreaming changes, but allow for local customizations. -The problem is described thoroughly in [managing external dependencies](https://embeddedartistry.com/blog/2020/06/22/qa-on-managing-external-dependencies/) and sometimes -is also known as [*vendoring*](https://dfetch.readthedocs.io/en/latest/vendoring.html). +**Dfetch** copies source code directly into your project — no Git submodules, no SVN externals, +no hidden external links. Fetch from Git, SVN, or plain archive URLs. Dependencies live as plain, +readable files inside your own repository. You stay in full control of every line. -Dfetch supports three source types: **Git**, **SVN**, and **archive files** (`.tar.gz`, `.tgz`, `.tar.bz2`, `.tar.xz`, `.zip`). Archives can be verified with a cryptographic hash (`sha256`, `sha384`, or `sha512`) to guarantee integrity on every fetch. +Dfetch supports **Git**, **SVN**, and **archive files** (`.tar.gz`, `.tgz`, `.tar.bz2`, `.tar.xz`, `.zip`). +Archives can be verified with a cryptographic hash (`sha256`, `sha384`, or `sha512`) to guarantee +integrity on every fetch. No proprietary formats, no lock-in — switch tools any time. -Other tools that do similar things are ``Zephyr's West``, ``CMake ExternalProject`` and other meta tools. +Other tools that do similar things are Zephyr's West, CMake ExternalProject, and other meta tools. See [alternatives](https://dfetch.readthedocs.io/en/latest/alternatives.html) for a complete list. +The broader concept is known as [*vendoring*](https://dfetch.readthedocs.io/en/latest/vendoring.html). [**Getting started**](https://dfetch.readthedocs.io/en/latest/getting_started.html) | [**Manual**](https://dfetch.readthedocs.io/en/latest/manual.html) | [**Troubleshooting**](https://dfetch.readthedocs.io/en/latest/troubleshooting.html) | [**Contributing**](https://dfetch.readthedocs.io/en/latest/contributing.html) -## Problems DFetch Solves +## What Dfetch Does -* Declarative code reuse across projects ([inner sourcing](https://about.gitlab.com/topics/version-control/what-is-innersource/)) -* Compose multi-repo code bases into a single working tree -* Vendoring dependencies for reproducible builds -* Apply local patches while keeping upstream syncable -* VCS-agnostic dependency management (Git, SVN, and plain archives) +* Vendor source-only dependencies — fully self-contained, no external links at build time +* VCS-agnostic: mix Git, SVN, and plain archive URLs freely in one manifest * Fetch and verify archives with cryptographic integrity checks -* Self-contained exports for releases or audits +* Apply local patches while keeping upstream syncable (`dfetch diff` / `dfetch format-patch`) +* Supply-chain ready: SBOM generation, license detection, multi-format CI reports +* Migrate from Git submodules or SVN externals in seconds (`dfetch import`) +* Declarative code reuse across projects ([inner sourcing](https://about.gitlab.com/topics/version-control/what-is-innersource/)) ## Install @@ -122,10 +119,10 @@ manifest: hash: sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 ``` -## Github Action +## GitHub Action -You can use DFetch in your Github Actions workflow to check your dependencies. -The results will be uploaded to Github. Add the following to your workflow file: +You can use Dfetch in your GitHub Actions workflow to check your dependencies. +The results will be uploaded to GitHub. Add the following to your workflow file: ```yaml jobs: diff --git a/doc/index.rst b/doc/index.rst index 3cd888c0d..616d26131 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -1,23 +1,22 @@ .. meta:: - :description: Dfetch is a VCS-agnostic tool that simplifies dependency management by retrieving - source-only dependencies from various repositories, promoting upstream changes and - allowing local customizations. - :keywords: dfetch, dependency management, embedded development, fetch tool, vendoring, multi-repo, dependencies, git, svn, package manager, multi-project, monorepo + :description: Dfetch vendors source code from Git or SVN repositories or plain archives directly + into your project. No submodules, no lock-in, fully self-contained. Supply-chain ready. + :keywords: dfetch, dependency management, vendoring, git, svn, archive, embedded development, source-only dependencies, multi-repo, supply chain, sbom, license compliance :author: Dfetch Contributors :google-site-verification: yCnoTogJMh7Nm5gxlREDuONIXT4ijHcj972Y5k9p-sU .. raw:: html - - + + - - + + @@ -25,16 +24,16 @@ :width: 100% :align: center -Dfetch — *a source-only no-hassle project-dependency aggregator* -================================================================ +Dfetch — *vendor dependencies without the pain* +================================================ -We make products that can last 15+ years; because of this we want to be able to have all sources -available to build the entire project from source without depending on external resources. -For this, we needed a dependency manager flexible enough to retrieve dependencies as plain text -from various sources. `svn externals`, `git submodules` and `git subtrees` solve a similar -problem, but not in a VCS-agnostic or completely user-friendly way. -Dfetch must promote upstreaming changes, but allow for local customisations. -See :ref:`vendoring` for background on why this matters. +**Dfetch** copies source code directly into your project — no Git submodules, no SVN externals, +no hidden external links. Fetch from Git, SVN, or plain archive URLs. Dependencies live as plain, +readable files inside your own repository. You stay in full control of every line. + +Dfetch is supply-chain ready out of the box: generate SBOMs, detect licenses, and export +reports for Jenkins, SARIF, and Code Climate. Apply local patches and keep them syncable with +upstream. See :ref:`vendoring` for background on the problem this solves. .. asciinema:: asciicasts/basic.cast From af78f3e941fc36e8a6c3b3c9650366b4c8a9fe9e Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 29 Mar 2026 11:55:22 +0000 Subject: [PATCH 05/30] Style flow diagrams to match the docs design system MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit check.puml / update.puml: - Remove monochrome true and Frutiger font - Activity nodes: white fill, amber (#c2620a) border, 1.5px, rounded corners - Decision diamonds: mint tint (#eff6fa) fill, accent blue (#4e7fa0) border, accent-dark text (#3a6682) - Arrows / labels: muted grey (#78716c) - Start circle: amber; end circle: charcoal - Partitions (update.puml): warm tint fill (#fef8f0), muted border (#e7e0d8), small bold uppercase label in muted grey - Font: Arial (closest available to Inter in PlantUML) - Background: transparent so body dot-grid shows through custom.css: - img[src*="plantweb"] rule: white card with border, rounded corners, box-shadow, and generous padding — matches the sd-card / admonition style https://claude.ai/code/session_0183Tzu6McYvSDwoXDoRjEyN --- doc/static/css/custom.css | 18 ++++++++++++++++++ doc/static/uml/check.puml | 26 +++++++++++++++++++++++--- doc/static/uml/update.puml | 38 +++++++++++++++++++++++++++++--------- 3 files changed, 70 insertions(+), 12 deletions(-) diff --git a/doc/static/css/custom.css b/doc/static/css/custom.css index 36428a096..ab1ce112c 100644 --- a/doc/static/css/custom.css +++ b/doc/static/css/custom.css @@ -452,6 +452,24 @@ p.caption[id*="reference"] .caption-text { color: var(--text) !important; } .toctree-wrapper p.caption:nth-of-type(4) .caption-text, p.caption[id*="explanation"] .caption-text { color: var(--text-muted) !important; } +/* ============================================================ + PlantUML / plantweb flow diagrams + plantweb stores rendered images under _images/plantweb/ so + img[src*="plantweb"] selects exactly these and nothing else. + ============================================================ */ +img[src*="plantweb"] { + display: block; + margin: 2rem auto !important; + max-width: 620px; + width: 100%; + background: #fff; + border: 1px solid var(--border); + border-radius: var(--r); + box-shadow: var(--shad-md); + padding: 1.75rem 2rem; + box-sizing: border-box; +} + /* ============================================================ Asciinema Player — match landing page terminal theme ============================================================ */ diff --git a/doc/static/uml/check.puml b/doc/static/uml/check.puml index ba9fce82c..84c6ea238 100644 --- a/doc/static/uml/check.puml +++ b/doc/static/uml/check.puml @@ -1,8 +1,28 @@ @startuml -start +skinparam defaultFontName Arial +skinparam defaultFontSize 12 +skinparam defaultFontColor #1c1917 +skinparam shadowing false +skinparam BackgroundColor transparent +skinparam RoundCorner 8 + +skinparam ActivityBackgroundColor #ffffff +skinparam ActivityBorderColor #c2620a +skinparam ActivityBorderThickness 1.5 +skinparam ActivityFontColor #1c1917 +skinparam ActivityFontSize 12 +skinparam ActivityFontName Arial +skinparam ActivityArrowColor #78716c +skinparam ActivityArrowFontColor #78716c +skinparam ActivityArrowFontSize 11 +skinparam ActivityStartColor #c2620a +skinparam ActivityEndColor #1c1917 +skinparam ActivityBarColor #c2620a +skinparam ActivityDiamondBackgroundColor #eff6fa +skinparam ActivityDiamondBorderColor #4e7fa0 +skinparam ActivityDiamondFontColor #3a6682 -skinparam monochrome true -skinparam defaultFontName Frutiger +start :Get version on disk; diff --git a/doc/static/uml/update.puml b/doc/static/uml/update.puml index 66394eed8..e8f2a8a11 100644 --- a/doc/static/uml/update.puml +++ b/doc/static/uml/update.puml @@ -1,8 +1,34 @@ @startuml -start +skinparam defaultFontName Arial +skinparam defaultFontSize 12 +skinparam defaultFontColor #1c1917 +skinparam shadowing false +skinparam BackgroundColor transparent +skinparam RoundCorner 8 + +skinparam ActivityBackgroundColor #ffffff +skinparam ActivityBorderColor #c2620a +skinparam ActivityBorderThickness 1.5 +skinparam ActivityFontColor #1c1917 +skinparam ActivityFontSize 12 +skinparam ActivityFontName Arial +skinparam ActivityArrowColor #78716c +skinparam ActivityArrowFontColor #78716c +skinparam ActivityArrowFontSize 11 +skinparam ActivityStartColor #c2620a +skinparam ActivityEndColor #1c1917 +skinparam ActivityBarColor #c2620a +skinparam ActivityDiamondBackgroundColor #eff6fa +skinparam ActivityDiamondBorderColor #4e7fa0 +skinparam ActivityDiamondFontColor #3a6682 -skinparam monochrome true -skinparam defaultFontName Frutiger +skinparam PartitionBackgroundColor #fef8f0 +skinparam PartitionBorderColor #e7e0d8 +skinparam PartitionFontColor #78716c +skinparam PartitionFontStyle bold +skinparam PartitionFontSize 10 + +start partition "Update required" { @@ -14,12 +40,6 @@ partition "Update required" { else (no) stop endif - - ' if (Hash on disk same\nas in metadata?) then (yes) - ' elseif (Force flag given?) then (yes) - ' else - ' stop - ' endif } partition "Prepare for update" { From 9151aa4b30a9208567d9f2373d86f38b7234b5a8 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 29 Mar 2026 12:01:55 +0000 Subject: [PATCH 06/30] Restructure manual into four logical groups MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Commands are now organised from basic to advanced: Foundational — Init, Import, Add, Check, Update The complete everyday workflow: create/migrate a manifest, register dependencies, check versions, fetch them. Patching — Diff, Update patch, Format patch First-class patch workflow: capture, reapply, and export local changes while staying syncable with upstream. CI/CD Integration — Reporting (Jenkins/SARIF/Code Climate), Report/SBOM Machine-readable output for automated pipelines, security toolchains, and compliance audits. The Check Reporting sub-sections are moved out of Check and into this group so they sit next to Report/SBOM where readers looking for integration topics expect them. Utilities — Freeze, Environment, Validate Supporting commands: pin versions, verify setup, validate manifest. Other changes: - Remove the "Introduction" heading; intro prose stays under the page title - Separate groups with horizontal rules for visual breathing room - Add a one-paragraph section description for each group - Rewrite CLI Cheatsheet using the same four-group structure with labelled bold headings instead of a flat list - Update cheatsheet copy to match landing-page messaging https://claude.ai/code/session_0183Tzu6McYvSDwoXDoRjEyN --- doc/manual.rst | 257 ++++++++++++++++++++++++++++++++----------------- 1 file changed, 166 insertions(+), 91 deletions(-) diff --git a/doc/manual.rst b/doc/manual.rst index b1f980798..5735b1d62 100644 --- a/doc/manual.rst +++ b/doc/manual.rst @@ -3,18 +3,25 @@ Manual ====== -Introduction ------------- -*Dfetch* can perform various actions based on the projects listed in the `manifest `_. -Each of these actions are a separate command. Below an overview of all available commands and -their usage. For detailed information on each command, please refer to the respective sections below. -For a step-by-step guide see the `Getting Started `_. +*Dfetch* acts on the projects listed in the :ref:`Manifest`. +Each action is a separate sub-command. Commands are grouped here from the +core day-to-day workflow through patch management and CI/CD integration. +For a step-by-step walkthrough see :doc:`getting_started`. .. program-output:: dfetch --help :shell: +---- + +Foundational +------------ + +These five commands cover the complete everyday workflow: create or migrate a +manifest, register new dependencies, check for newer versions upstream, and +fetch them into your repository. + Init ------ +~~~~ .. argparse:: :module: dfetch.__main__ :func: create_parser @@ -26,7 +33,7 @@ Init .. automodule:: dfetch.commands.init Import ------- +~~~~~~ .. argparse:: :module: dfetch.__main__ :func: create_parser @@ -37,72 +44,75 @@ Import .. automodule:: dfetch.commands.import_ -Check ------ +Add +~~~ .. argparse:: :module: dfetch.__main__ :func: create_parser :prog: dfetch - :path: check + :path: add -.. asciinema:: asciicasts/check.cast +.. automodule:: dfetch.commands.add -.. automodule:: dfetch.commands.check +Non-interactive +``````````````` -Reporting -````````` -.. automodule:: dfetch.reporting.check.reporter +Pass a URL and *Dfetch* fills in sensible defaults (name, destination, default +branch) and appends the entry immediately — no prompts, no confirmation. +Use ``--name``, ``--dst``, ``--version``, ``--src``, and ``--ignore`` to +override individual fields. -Jenkins reporter -'''''''''''''''' -.. automodule:: dfetch.reporting.check.jenkins_reporter +.. asciinema:: asciicasts/add.cast -.. asciinema:: asciicasts/check-ci.cast +.. scenario-include:: ../features/add-project-through-cli.feature -Sarif reporter -'''''''''''''' -.. automodule:: dfetch.reporting.check.sarif_reporter +Interactive +``````````` -Code-climate reporter -''''''''''''''''''''' -.. automodule:: dfetch.reporting.check.code_climate_reporter +Pass ``--interactive`` / ``-i`` to be guided step-by-step through every +manifest field. Pre-fill fields with the flag options above and those prompts +are skipped. At the end you can optionally run ``dfetch update`` immediately. -Update ------- +.. asciinema:: asciicasts/interactive-add.cast + +.. scenario-include:: ../features/interactive-add.feature + +Check +~~~~~ .. argparse:: :module: dfetch.__main__ :func: create_parser :prog: dfetch - :path: update + :path: check -.. asciinema:: asciicasts/update.cast +.. asciinema:: asciicasts/check.cast -.. automodule:: dfetch.commands.update +.. automodule:: dfetch.commands.check -Report ------- +Update +~~~~~~ .. argparse:: :module: dfetch.__main__ :func: create_parser :prog: dfetch - :path: report + :path: update -.. asciinema:: asciicasts/report.cast +.. asciinema:: asciicasts/update.cast -.. automodule:: dfetch.commands.report +.. automodule:: dfetch.commands.update -List (default) -`````````````` -.. automodule:: dfetch.reporting.stdout_reporter +---- -Software Bill-of-Materials -`````````````````````````` -.. automodule:: dfetch.reporting.sbom_reporter +Patching +-------- -.. asciinema:: asciicasts/sbom.cast +*Dfetch* has a first-class patch workflow. ``dfetch diff`` captures local +changes as numbered patch files that are re-applied automatically on every +``dfetch update``. When a fix is ready to share upstream, ``dfetch +format-patch`` produces a contributor-ready unified diff. Diff ----- +~~~~ .. argparse:: :module: dfetch.__main__ :func: create_parser @@ -114,7 +124,7 @@ Diff .. automodule:: dfetch.commands.diff Update patch ------------- +~~~~~~~~~~~~ .. argparse:: :module: dfetch.__main__ :func: create_parser @@ -126,7 +136,7 @@ Update patch .. automodule:: dfetch.commands.update_patch Format patch ------------- +~~~~~~~~~~~~ .. argparse:: :module: dfetch.__main__ :func: create_parser @@ -137,8 +147,67 @@ Format patch .. automodule:: dfetch.commands.format_patch +---- + +CI/CD Integration +----------------- + +These commands are designed to plug into automated pipelines. Use ``dfetch +check`` report formats to surface stale or vulnerable dependencies in your +existing security toolchain. Use ``dfetch report`` to generate SBOMs and +inventory lists for compliance audits. + +Reporting +~~~~~~~~~ +.. automodule:: dfetch.reporting.check.reporter + +Jenkins reporter +'''''''''''''''' +.. automodule:: dfetch.reporting.check.jenkins_reporter + +.. asciinema:: asciicasts/check-ci.cast + +Sarif reporter +'''''''''''''' +.. automodule:: dfetch.reporting.check.sarif_reporter + +Code-climate reporter +''''''''''''''''''''' +.. automodule:: dfetch.reporting.check.code_climate_reporter + +Report +~~~~~~ +.. argparse:: + :module: dfetch.__main__ + :func: create_parser + :prog: dfetch + :path: report + +.. asciinema:: asciicasts/report.cast + +.. automodule:: dfetch.commands.report + +List (default) +'''''''''''''' +.. automodule:: dfetch.reporting.stdout_reporter + +Software Bill-of-Materials +'''''''''''''''''''''''''' +.. automodule:: dfetch.reporting.sbom_reporter + +.. asciinema:: asciicasts/sbom.cast + +---- + +Utilities +--------- + +Supporting commands for day-to-day maintenance: pin all versions to their +current state, verify your environment is correctly set up, and validate a +manifest without running a fetch. + Freeze ------- +~~~~~~ .. argparse:: :module: dfetch.__main__ :func: create_parser @@ -150,7 +219,7 @@ Freeze .. automodule:: dfetch.commands.freeze Environment ------------ +~~~~~~~~~~~ .. argparse:: :module: dfetch.__main__ :func: create_parser @@ -162,7 +231,7 @@ Environment .. automodule:: dfetch.commands.environment Validate --------- +~~~~~~~~ .. argparse:: :module: dfetch.__main__ :func: create_parser @@ -173,91 +242,97 @@ Validate .. automodule:: dfetch.commands.validate -Add ---- -.. argparse:: - :module: dfetch.__main__ - :func: create_parser - :prog: dfetch - :path: add +---- -.. automodule:: dfetch.commands.add +CLI Cheatsheet +-------------- -Non-interactive -~~~~~~~~~~~~~~~ +A quick-reference card for the most common *Dfetch* operations. All commands +discover ``dfetch.yaml`` automatically by searching up from the current directory. -Pass a URL and *Dfetch* fills in sensible defaults (name, destination, default -branch) and appends the entry immediately — no prompts, no confirmation. -Use ``--name``, ``--dst``, ``--version``, ``--src``, and ``--ignore`` to -override individual fields. +**Foundational** -.. asciinema:: asciicasts/add.cast +- Initialise a new manifest: -.. scenario-include:: ../features/add-project-through-cli.feature + .. code-block:: console -Interactive -~~~~~~~~~~~ + dfetch init -Pass ``--interactive`` / ``-i`` to be guided step-by-step through every -manifest field. Pre-fill fields with the flag options above and those prompts -are skipped. At the end you can optionally run ``dfetch update`` immediately. +- Add a dependency interactively or non-interactively: -.. asciinema:: asciicasts/interactive-add.cast + .. code-block:: console -.. scenario-include:: ../features/interactive-add.feature + dfetch add -i + dfetch add +- Migrate from git submodules or SVN externals: -CLI Cheatsheet --------------- + .. code-block:: console -A source-only, no-hassle project-dependency aggregator. -It uses a **manifest file** to describe your project's dependencies and fetches them into your codebase. -Also called vendoring. More info: ``_. + dfetch import -- Start a new manifest (`dfetch.yaml`) with placeholder content: +- Check which dependencies have newer versions available: .. code-block:: console - dfetch init + dfetch check [project] -- Add a new project to the manifest (interactive step-by-step wizard): +- Fetch / update one or all dependencies: .. code-block:: console - dfetch add -i + dfetch update [-f] [project] + +**Patching** - or non-interactively (auto-accept defaults, skip confirmation): +- Capture local changes to a vendored dependency as a patch file: .. code-block:: console - dfetch add + dfetch diff [project] -- Generate a manifest from existing git submodules or svn externals: +- Re-apply updated patches after an upstream bump: .. code-block:: console - dfetch import + dfetch update-patch [project] + +- Export a patch as a contributor-ready unified diff: -- Check for newer versions of dependencies and create a machine parseable report for your CI: + .. code-block:: console + + dfetch format-patch [project] + +**CI/CD Integration** + +- Check and emit a machine-readable report for your CI: .. code-block:: console dfetch check [--jenkins-json] [--sarif] [--code-climate] [project] -- Download one or all projects from the manifest: +- Generate an inventory list or SBOM: .. code-block:: console - dfetch update [-f] [project] + dfetch report [-o ] [-t {sbom,list}] [project] -- Freeze all projects to their current version: +**Utilities** + +- Pin all dependencies to their currently fetched version: .. code-block:: console dfetch freeze -- Report about the current state of the project(s): +- Verify the environment (VCS tools, versions): + + .. code-block:: console + + dfetch environment + +- Validate a manifest without fetching: .. code-block:: console - dfetch report [-o ] [-t {sbom,list}] [project] + dfetch validate From c3f8b6eec8b6a17269c3c0e9e2ea45f22c3069e4 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 29 Mar 2026 13:02:49 +0000 Subject: [PATCH 07/30] =?UTF-8?q?Restyle=20CLI=20Cheatsheet=20as=20a=20pri?= =?UTF-8?q?ntable=202=C3=972=20card=20grid?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit manual.rst — CLI Cheatsheet section: - Replace flat bulleted list with a raw HTML 2×2 grid - Four cards matching the manual's four groups: Foundational → amber header (--primary) Patching → slate-blue header (--accent) CI/CD → warm-charcoal header (#1e1610, dark card style) Utilities → warm-tint header (--bg-tint, softer) - Each card has a title + tagline in the header - Commands rendered as a two-column table: syntax (JetBrains Mono, amber) | description (Inter, muted); optional args shown lighter - Alternating row tint on even rows custom.css — new .cheatsheet / .cs-* rules: - CSS grid, 2-col on ≥700 px, 1-col on narrow viewports - .cs-cmd code resets the inline-code badge style (no bg, no border) - .cs-cmd em for optional args shown in --text-muted - .cs-table overrides general table.docutils rules (no shadow, no radius) - @media print: force background colours (print-color-adjust: exact), drop shadows, keep 2-col layout, suppress link-URL decoration https://claude.ai/code/session_0183Tzu6McYvSDwoXDoRjEyN --- doc/manual.rst | 205 +++++++++++++++++++++----------------- doc/static/css/custom.css | 176 ++++++++++++++++++++++++++++++++ 2 files changed, 292 insertions(+), 89 deletions(-) diff --git a/doc/manual.rst b/doc/manual.rst index 5735b1d62..6351e7623 100644 --- a/doc/manual.rst +++ b/doc/manual.rst @@ -247,92 +247,119 @@ Validate CLI Cheatsheet -------------- -A quick-reference card for the most common *Dfetch* operations. All commands -discover ``dfetch.yaml`` automatically by searching up from the current directory. - -**Foundational** - -- Initialise a new manifest: - - .. code-block:: console - - dfetch init - -- Add a dependency interactively or non-interactively: - - .. code-block:: console - - dfetch add -i - dfetch add - -- Migrate from git submodules or SVN externals: - - .. code-block:: console - - dfetch import - -- Check which dependencies have newer versions available: - - .. code-block:: console - - dfetch check [project] - -- Fetch / update one or all dependencies: - - .. code-block:: console - - dfetch update [-f] [project] - -**Patching** - -- Capture local changes to a vendored dependency as a patch file: - - .. code-block:: console - - dfetch diff [project] - -- Re-apply updated patches after an upstream bump: - - .. code-block:: console - - dfetch update-patch [project] - -- Export a patch as a contributor-ready unified diff: - - .. code-block:: console - - dfetch format-patch [project] - -**CI/CD Integration** - -- Check and emit a machine-readable report for your CI: - - .. code-block:: console - - dfetch check [--jenkins-json] [--sarif] [--code-climate] [project] - -- Generate an inventory list or SBOM: - - .. code-block:: console - - dfetch report [-o ] [-t {sbom,list}] [project] - -**Utilities** - -- Pin all dependencies to their currently fetched version: - - .. code-block:: console - - dfetch freeze - -- Verify the environment (VCS tools, versions): - - .. code-block:: console - - dfetch environment - -- Validate a manifest without fetching: - - .. code-block:: console - - dfetch validate +All commands discover ``dfetch.yaml`` automatically by searching up from the +current working directory. *Italicised* arguments are optional. + +.. raw:: html + +
+ + +
+
+ Foundational + Core workflow · daily use +
+ + + + + + + + + + + + + + + + + + + + + + + + + +
dfetch initCreate a new dfetch.yaml manifest
dfetch add <url>Add a dependency, auto-fill defaults
dfetch add -i <url>Add interactively, step-by-step wizard
dfetch importMigrate from git submodules / SVN externals
dfetch check [project]Show dependencies with newer versions available
dfetch update [-f] [project]Fetch / update one or all dependencies
+
+ + +
+
+ Patching + Local changes · upstream sync +
+ + + + + + + + + + + + + +
dfetch diff [project]Capture local changes as a patch file
dfetch update-patch [project]Re-apply patches after upstream version bump
dfetch format-patch [project]Export contributor-ready unified diff for upstream PR
+
+ + +
+
+ CI/CD Integration + Reports · SBOM · security +
+ + + + + + + + + + + + + + + + + + + + + +
dfetch check --jenkins-jsonJenkins-compatible JSON report
dfetch check --sarifSARIF report (GitHub Advanced Security etc.)
dfetch check --code-climateCode Climate / GitLab report
dfetch reportPrint a dependency inventory list
dfetch report -t sbomGenerate a Software Bill of Materials
+
+ + +
+
+ Utilities + Maintenance · setup · validation +
+ + + + + + + + + + + + + +
dfetch freezePin all dependencies to currently fetched version
dfetch environmentVerify VCS tools and environment setup
dfetch validateValidate the manifest without fetching
+
+ +
diff --git a/doc/static/css/custom.css b/doc/static/css/custom.css index ab1ce112c..ab38f27ac 100644 --- a/doc/static/css/custom.css +++ b/doc/static/css/custom.css @@ -578,3 +578,179 @@ h1::after { font-size: 0.9375rem; line-height: 1.7; } + +/* ============================================================ + CLI Cheatsheet + ============================================================ */ +.cheatsheet { + display: grid; + grid-template-columns: 1fr 1fr; + gap: 1.25rem; + margin: 1.5rem 0 2.5rem; +} + +.cs-group { + border: 1px solid var(--border); + border-radius: var(--r); + overflow: hidden; + box-shadow: var(--shad-sm); + background: #fff; + break-inside: avoid; + page-break-inside: avoid; +} + +/* -- group header -- */ +.cs-header { + padding: 0.55rem 1rem; + display: flex; + flex-direction: column; + gap: 0.1rem; +} + +.cs-title { + font-family: "Inter", sans-serif; + font-size: 0.875rem; + font-weight: 700; + letter-spacing: -0.01em; + line-height: 1.3; +} + +.cs-sub { + font-family: "Inter", sans-serif; + font-size: 0.6875rem; + font-weight: 400; + opacity: 0.72; +} + +.cs-primary { background: var(--primary); color: #fff; } +.cs-accent { background: var(--accent); color: #fff; } +.cs-dark { background: #1e1610; color: #f0e8dc; } +.cs-muted { + background: var(--bg-tint); + color: var(--text); + border-bottom: 2px solid var(--border); +} +.cs-muted .cs-sub { opacity: 0.5; } + +/* -- command rows -- */ +.cs-table { + width: 100%; + border-collapse: collapse; + /* override the general table.docutils rules */ + margin: 0 !important; + box-shadow: none !important; + border: none !important; + border-radius: 0 !important; +} + +.cs-table tr { + border-bottom: 1px solid var(--border); +} + +.cs-table tr:last-child { + border-bottom: none; +} + +.cs-table tr:nth-child(even) td { + background: rgba(254, 248, 240, 0.4) !important; +} + +.cs-cmd { + padding: 0.3rem 0.5rem 0.3rem 1rem; + vertical-align: middle; + width: 50%; + white-space: nowrap; +} + +/* the inside .cs-cmd should look like plain mono, not a styled badge */ +.cs-cmd code { + font-family: "JetBrains Mono", "Fira Code", Consolas, monospace !important; + font-size: 0.775rem !important; + font-weight: 500; + color: var(--primary-dark) !important; + background: none !important; + border: none !important; + padding: 0 !important; + line-height: 1.5; +} + +/* optional/variable parts shown in muted style */ +.cs-cmd em { + font-style: normal; + color: var(--text-muted); + font-weight: 400; +} + +.cs-desc { + padding: 0.3rem 1rem 0.3rem 0.5rem; + font-family: "Inter", sans-serif; + font-size: 0.775rem; + color: var(--text-muted); + line-height: 1.5; + vertical-align: middle; +} + +/* collapse to single column on narrow viewports */ +@media (max-width: 700px) { + .cheatsheet { + grid-template-columns: 1fr; + } +} + +/* ============================================================ + Print — cheatsheet + Forces background colours, removes shadows, 2-col on paper. + ============================================================ */ +@media print { + body { + background-image: none !important; + } + + div.documentwrapper, + div.body { + background-image: none !important; + } + + .cheatsheet { + grid-template-columns: 1fr 1fr; + gap: 0.75rem; + } + + .cs-group { + box-shadow: none; + border: 0.5pt solid #ccc; + } + + /* Force header background colours to print */ + .cs-primary { + background: #c2620a !important; + -webkit-print-color-adjust: exact; + print-color-adjust: exact; + } + .cs-accent { + background: #4e7fa0 !important; + -webkit-print-color-adjust: exact; + print-color-adjust: exact; + } + .cs-dark { + background: #1e1610 !important; + -webkit-print-color-adjust: exact; + print-color-adjust: exact; + } + .cs-muted { + background: #fef8f0 !important; + -webkit-print-color-adjust: exact; + print-color-adjust: exact; + } + + .cs-table tr:nth-child(even) td { + background: rgba(254, 248, 240, 0.5) !important; + -webkit-print-color-adjust: exact; + print-color-adjust: exact; + } + + /* suppress link-URL decoration that browsers add on print */ + a::after { + content: none !important; + } +} From aa3ae0dde77718cc317c3cb4c0c7daea68309f1a Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 29 Mar 2026 13:11:33 +0000 Subject: [PATCH 08/30] Redesign CLI Cheatsheet as a beautiful dark terminal card MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit HTML (manual.rst): - Replace 4-separate-white-cards with a single dark terminal card - Masthead: "df" amber badge logo + title + tagline + syntax-legend pills (subcommand / --flag / ) - Two-column body (col 1: Foundational + Utilities, col 2: Patching + CI/CD) split by a hairline - Section labels: coloured dot + ALL-CAPS name + muted tagline, underlined in the section colour (amber / blue / purple / green for the four groups) - Every command hand-tagged with token spans: .cs-kw dfetch → dim cream (de-emphasised) .cs-sc subcommand → bright amber (hero colour) .cs-fl --flag / -f → accent blue .cs-ag / [opt] → muted grey - Footer bar: docs + GitHub URL in small muted text CSS (custom.css): - Dark gradient background matching the landing-page dark card - Masthead with amber-tinted background band - Syntax legend token pills - Section label colour variants: cs-l-primary / cs-l-accent / cs-l-ci / cs-l-utility - Row separators: 4% white hairlines - Responsive: single-column below 720 px - Print: force colour-adjust, drop shadow, suppress link URLs https://claude.ai/code/session_0183Tzu6McYvSDwoXDoRjEyN --- doc/manual.rst | 234 ++++++++++++++------------ doc/static/css/custom.css | 339 +++++++++++++++++++++++++------------- 2 files changed, 349 insertions(+), 224 deletions(-) diff --git a/doc/manual.rst b/doc/manual.rst index 6351e7623..5f7119201 100644 --- a/doc/manual.rst +++ b/doc/manual.rst @@ -247,119 +247,143 @@ Validate CLI Cheatsheet -------------- -All commands discover ``dfetch.yaml`` automatically by searching up from the -current working directory. *Italicised* arguments are optional. - .. raw:: html
- -
-
- Foundational - Core workflow · daily use + +
+
+ +
+
Dfetch CLI Cheatsheet
+
Vendor dependencies without the pain  ·  dfetch.yaml found automatically
+
- - - - - - - - - - - - - - - - - - - - - - - - - -
dfetch initCreate a new dfetch.yaml manifest
dfetch add <url>Add a dependency, auto-fill defaults
dfetch add -i <url>Add interactively, step-by-step wizard
dfetch importMigrate from git submodules / SVN externals
dfetch check [project]Show dependencies with newer versions available
dfetch update [-f] [project]Fetch / update one or all dependencies
-
- - -
-
- Patching - Local changes · upstream sync +
+ subcommand + --flag + <arg>
- - - - - - - - - - - - - -
dfetch diff [project]Capture local changes as a patch file
dfetch update-patch [project]Re-apply patches after upstream version bump
dfetch format-patch [project]Export contributor-ready unified diff for upstream PR
- -
-
- CI/CD Integration - Reports · SBOM · security -
- - - - - - - - - - - - - - - - - - - - - -
dfetch check --jenkins-jsonJenkins-compatible JSON report
dfetch check --sarifSARIF report (GitHub Advanced Security etc.)
dfetch check --code-climateCode Climate / GitLab report
dfetch reportPrint a dependency inventory list
dfetch report -t sbomGenerate a Software Bill of Materials
-
- - -
-
- Utilities - Maintenance · setup · validation -
- - - - - - - - - - - - - -
dfetch freezePin all dependencies to currently fetched version
dfetch environmentVerify VCS tools and environment setup
dfetch validateValidate the manifest without fetching
+ +
+ + +
+ +
+
+ Foundational + core workflow · daily use +
+ +
+
dfetch init
+
Create a new dfetch.yaml manifest
+
+
+
dfetch add <url>
+
Add a dependency, auto-fill defaults
+
+
+
dfetch add -i <url>
+
Add interactively, step-by-step wizard
+
+
+
dfetch import
+
Migrate from git submodules / SVN externals
+
+
+
dfetch check [project]
+
Show dependencies with newer versions available
+
+
+
dfetch update [-f] [project]
+
Fetch / update one or all dependencies
+
+
+ +
+
+ Utilities + maintenance · setup · validation +
+ +
+
dfetch freeze
+
Pin all dependencies to currently fetched version
+
+
+
dfetch environment
+
Verify VCS tools and environment setup
+
+
+
dfetch validate
+
Validate the manifest without fetching
+
+
+ +
+ + +
+ +
+
+ Patching + local changes · upstream sync +
+ +
+
dfetch diff [project]
+
Capture local changes as a patch file
+
+
+
dfetch update-patch [project]
+
Re-apply patches after upstream version bump
+
+
+
dfetch format-patch [project]
+
Export contributor-ready unified diff
+
+
+ +
+
+ CI / CD Integration + reports · sbom · security +
+ +
+
dfetch check --jenkins-json
+
Jenkins-compatible JSON report
+
+
+
dfetch check --sarif
+
SARIF (GitHub Advanced Security etc.)
+
+
+
dfetch check --code-climate
+
Code Climate / GitLab report
+
+
+
dfetch report
+
Print a dependency inventory list
+
+
+
dfetch report -t sbom
+
Generate a Software Bill of Materials
+
+
+ +
+ +
+ +
+ dfetch.readthedocs.io  ·  github.com/dfetch-org/dfetch
diff --git a/doc/static/css/custom.css b/doc/static/css/custom.css index ab38f27ac..9b784b82f 100644 --- a/doc/static/css/custom.css +++ b/doc/static/css/custom.css @@ -580,177 +580,278 @@ h1::after { } /* ============================================================ - CLI Cheatsheet + CLI Cheatsheet — dark terminal card with syntax colouring ============================================================ */ + +/* Outer card */ .cheatsheet { - display: grid; - grid-template-columns: 1fr 1fr; - gap: 1.25rem; - margin: 1.5rem 0 2.5rem; + background: linear-gradient(150deg, #1c1410 0%, #281a0f 100%); + border-radius: var(--r-lg); + box-shadow: var(--shad-xl); + overflow: hidden; + margin: 2rem 0 3rem; + font-family: inherit; + -webkit-print-color-adjust: exact; + print-color-adjust: exact; } -.cs-group { - border: 1px solid var(--border); - border-radius: var(--r); - overflow: hidden; - box-shadow: var(--shad-sm); - background: #fff; - break-inside: avoid; - page-break-inside: avoid; +/* ── Masthead ─────────────────────────────────────────── */ +.cs-masthead { + display: flex; + align-items: center; + justify-content: space-between; + gap: 1rem; + padding: 1rem 1.5rem 0.9rem; + background: rgba(194, 98, 10, 0.12); + border-bottom: 1px solid rgba(194, 98, 10, 0.25); } -/* -- group header -- */ -.cs-header { - padding: 0.55rem 1rem; +.cs-masthead-brand { display: flex; - flex-direction: column; - gap: 0.1rem; + align-items: center; + gap: 0.875rem; +} + +/* "df" logo badge */ +.cs-logo { + display: inline-flex; + align-items: center; + justify-content: center; + width: 2.25rem; + height: 2.25rem; + border-radius: 8px; + background: var(--primary); + color: #fff; + font-family: "JetBrains Mono", monospace; + font-size: 0.875rem; + font-weight: 700; + letter-spacing: -0.04em; + flex-shrink: 0; + box-shadow: 0 2px 8px rgba(194, 98, 10, 0.5); } -.cs-title { +.cs-masthead-title { font-family: "Inter", sans-serif; - font-size: 0.875rem; + font-size: 0.9375rem; font-weight: 700; + color: #f0e8dc; letter-spacing: -0.01em; line-height: 1.3; } -.cs-sub { +.cs-masthead-tagline { font-family: "Inter", sans-serif; - font-size: 0.6875rem; + font-size: 0.7rem; + color: rgba(240, 232, 220, 0.45); + margin-top: 0.1rem; +} + +.cs-masthead-tagline code { + font-family: "JetBrains Mono", monospace !important; + font-size: 0.7rem !important; + background: none !important; + border: none !important; + padding: 0 !important; + color: rgba(194, 98, 10, 0.8) !important; +} + +/* Syntax legend in the top-right */ +.cs-masthead-legend { + display: flex; + gap: 0.625rem; + flex-shrink: 0; +} + +/* ── Token sample pills in the legend ── */ +.cs-tok { + font-family: "JetBrains Mono", monospace; + font-size: 0.65rem; + font-weight: 500; + padding: 0.2rem 0.5rem; + border-radius: 4px; + white-space: nowrap; +} + +.cs-tok-sc { background: rgba(232, 144, 74, 0.18); color: #e8904a; } +.cs-tok-fl { background: rgba(122, 168, 197, 0.18); color: #7aa8c5; } +.cs-tok-ag { background: rgba(160, 144, 128, 0.14); color: #9b8e85; } + +/* ── Two-column body ──────────────────────────────────── */ +.cs-body { + display: grid; + grid-template-columns: 1fr 1fr; + gap: 0; + padding: 1.25rem 1.5rem 1.25rem; + column-gap: 0; +} + +.cs-col { + /* hairline divider between columns */ + padding: 0 1.25rem; +} + +.cs-col:first-child { + padding-left: 0; + border-right: 1px solid rgba(255, 255, 255, 0.07); +} + +.cs-col:last-child { + padding-right: 0; +} + +/* ── Section labels ───────────────────────────────────── */ +.cs-section { + margin-bottom: 1.25rem; +} + +.cs-section:last-child { + margin-bottom: 0; +} + +.cs-label { + display: flex; + align-items: center; + gap: 0.5rem; + padding: 0.35rem 0; + margin-bottom: 0.2rem; + font-family: "Inter", sans-serif; + font-size: 0.65rem; + font-weight: 700; + letter-spacing: 0.1em; + text-transform: uppercase; + border-bottom: 1px solid; +} + +.cs-label-pip { + width: 6px; + height: 6px; + border-radius: 50%; + flex-shrink: 0; +} + +.cs-label-sub { font-weight: 400; - opacity: 0.72; + letter-spacing: 0; + text-transform: none; + opacity: 0.65; + margin-left: auto; + font-size: 0.6rem; } -.cs-primary { background: var(--primary); color: #fff; } -.cs-accent { background: var(--accent); color: #fff; } -.cs-dark { background: #1e1610; color: #f0e8dc; } -.cs-muted { - background: var(--bg-tint); - color: var(--text); - border-bottom: 2px solid var(--border); +/* Colour variants */ +.cs-l-primary { + color: #e8904a; + border-color: rgba(232, 144, 74, 0.3); } -.cs-muted .cs-sub { opacity: 0.5; } +.cs-l-primary .cs-label-pip { background: #e8904a; } -/* -- command rows -- */ -.cs-table { - width: 100%; - border-collapse: collapse; - /* override the general table.docutils rules */ - margin: 0 !important; - box-shadow: none !important; - border: none !important; - border-radius: 0 !important; +.cs-l-accent { + color: #7aa8c5; + border-color: rgba(122, 168, 197, 0.3); } +.cs-l-accent .cs-label-pip { background: #7aa8c5; } -.cs-table tr { - border-bottom: 1px solid var(--border); +.cs-l-ci { + color: #b08ac0; + border-color: rgba(176, 138, 192, 0.3); } +.cs-l-ci .cs-label-pip { background: #b08ac0; } -.cs-table tr:last-child { - border-bottom: none; +.cs-l-utility { + color: #8ba88b; + border-color: rgba(139, 168, 139, 0.3); } +.cs-l-utility .cs-label-pip { background: #8ba88b; } -.cs-table tr:nth-child(even) td { - background: rgba(254, 248, 240, 0.4) !important; +/* ── Command rows ─────────────────────────────────────── */ +.cs-row { + display: flex; + align-items: baseline; + gap: 0.75rem; + padding: 0.28rem 0; + border-bottom: 1px solid rgba(255, 255, 255, 0.04); } -.cs-cmd { - padding: 0.3rem 0.5rem 0.3rem 1rem; - vertical-align: middle; - width: 50%; +.cs-row:last-child { + border-bottom: none; +} + +.cs-syn { + font-family: "JetBrains Mono", "Fira Code", Consolas, monospace; + font-size: 0.775rem; white-space: nowrap; + flex-shrink: 0; + min-width: 0; } -/* the inside .cs-cmd should look like plain mono, not a styled badge */ -.cs-cmd code { - font-family: "JetBrains Mono", "Fira Code", Consolas, monospace !important; - font-size: 0.775rem !important; - font-weight: 500; - color: var(--primary-dark) !important; - background: none !important; - border: none !important; - padding: 0 !important; - line-height: 1.5; +/* Token colours */ +.cs-kw { color: rgba(240, 232, 220, 0.38); } /* dfetch — dim, always same */ +.cs-sc { color: #e8904a; font-weight: 600; } /* subcommand — amber hero */ +.cs-fl { color: #7aa8c5; } /* --flag / -f — accent blue */ +.cs-ag { color: #9b8e85; } /* / [opt] — muted */ + +.cs-dsc { + font-family: "Inter", sans-serif; + font-size: 0.75rem; + color: rgba(240, 232, 220, 0.52); + line-height: 1.45; + min-width: 0; } -/* optional/variable parts shown in muted style */ -.cs-cmd em { - font-style: normal; - color: var(--text-muted); - font-weight: 400; +/* inline code inside descriptions */ +.cs-dsc code { + font-family: "JetBrains Mono", monospace !important; + font-size: 0.7rem !important; + background: rgba(194, 98, 10, 0.15) !important; + border: none !important; + padding: 0.05em 0.3em !important; + border-radius: 3px !important; + color: #d97a3a !important; } -.cs-desc { - padding: 0.3rem 1rem 0.3rem 0.5rem; +/* ── Footer ───────────────────────────────────────────── */ +.cs-footer { + padding: 0.6rem 1.5rem; + background: rgba(0, 0, 0, 0.25); + border-top: 1px solid rgba(255, 255, 255, 0.06); font-family: "Inter", sans-serif; - font-size: 0.775rem; - color: var(--text-muted); - line-height: 1.5; - vertical-align: middle; + font-size: 0.6875rem; + color: rgba(240, 232, 220, 0.3); + text-align: center; + letter-spacing: 0.02em; } -/* collapse to single column on narrow viewports */ -@media (max-width: 700px) { - .cheatsheet { +/* ── Responsive ───────────────────────────────────────── */ +@media (max-width: 720px) { + .cs-body { grid-template-columns: 1fr; } + .cs-col:first-child { + border-right: none; + border-bottom: 1px solid rgba(255, 255, 255, 0.07); + padding-left: 0; + padding-bottom: 1.25rem; + margin-bottom: 1.25rem; + } + .cs-masthead { + flex-direction: column; + align-items: flex-start; + } } -/* ============================================================ - Print — cheatsheet - Forces background colours, removes shadows, 2-col on paper. - ============================================================ */ +/* ── Print ────────────────────────────────────────────── */ @media print { - body { - background-image: none !important; - } - - div.documentwrapper, - div.body { + body, div.documentwrapper, div.body { background-image: none !important; } .cheatsheet { - grid-template-columns: 1fr 1fr; - gap: 0.75rem; - } - - .cs-group { box-shadow: none; - border: 0.5pt solid #ccc; - } - - /* Force header background colours to print */ - .cs-primary { - background: #c2620a !important; - -webkit-print-color-adjust: exact; - print-color-adjust: exact; - } - .cs-accent { - background: #4e7fa0 !important; - -webkit-print-color-adjust: exact; - print-color-adjust: exact; - } - .cs-dark { - background: #1e1610 !important; - -webkit-print-color-adjust: exact; - print-color-adjust: exact; - } - .cs-muted { - background: #fef8f0 !important; - -webkit-print-color-adjust: exact; - print-color-adjust: exact; - } - - .cs-table tr:nth-child(even) td { - background: rgba(254, 248, 240, 0.5) !important; + border: 0.5pt solid #3a2a1e; -webkit-print-color-adjust: exact; print-color-adjust: exact; } /* suppress link-URL decoration that browsers add on print */ - a::after { - content: none !important; - } + a::after { content: none !important; } } From 16e6b3623792c3c0bb29a0c47fea6ed283e492d7 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 29 Mar 2026 13:17:31 +0000 Subject: [PATCH 09/30] Add command mindmap and wire up flow diagrams in manual - New commands.puml: mindmap with 4 colour-coded groups (Foundational amber, Patching blue, CI/CD purple, Utilities green) matching design tokens, placed right after the --help output in manual.rst - Wire check.puml and update.puml into their respective sections - Widen plantweb card max-width to 780px to give the mindmap room https://claude.ai/code/session_0183Tzu6McYvSDwoXDoRjEyN --- doc/manual.rst | 6 +++ doc/static/css/custom.css | 2 +- doc/static/uml/commands.puml | 91 ++++++++++++++++++++++++++++++++++++ 3 files changed, 98 insertions(+), 1 deletion(-) create mode 100644 doc/static/uml/commands.puml diff --git a/doc/manual.rst b/doc/manual.rst index 5f7119201..ea61a544b 100644 --- a/doc/manual.rst +++ b/doc/manual.rst @@ -11,6 +11,8 @@ For a step-by-step walkthrough see :doc:`getting_started`. .. program-output:: dfetch --help :shell: +.. uml:: /static/uml/commands.puml + ---- Foundational @@ -87,6 +89,8 @@ Check .. asciinema:: asciicasts/check.cast +.. uml:: /static/uml/check.puml + .. automodule:: dfetch.commands.check Update @@ -99,6 +103,8 @@ Update .. asciinema:: asciicasts/update.cast +.. uml:: /static/uml/update.puml + .. automodule:: dfetch.commands.update ---- diff --git a/doc/static/css/custom.css b/doc/static/css/custom.css index 9b784b82f..ad53a5efa 100644 --- a/doc/static/css/custom.css +++ b/doc/static/css/custom.css @@ -460,7 +460,7 @@ p.caption[id*="explanation"] .caption-text { color: var(--text-muted) !important img[src*="plantweb"] { display: block; margin: 2rem auto !important; - max-width: 620px; + max-width: 780px; width: 100%; background: #fff; border: 1px solid var(--border); diff --git a/doc/static/uml/commands.puml b/doc/static/uml/commands.puml new file mode 100644 index 000000000..da5fceab1 --- /dev/null +++ b/doc/static/uml/commands.puml @@ -0,0 +1,91 @@ +@startmindmap + + +* dfetch +** Foundational <> +*** init +*** import +*** add +*** check +*** update + +** Utilities <> +*** freeze +*** environment +*** validate + +left side + +** Patching <> +*** diff +*** update-patch +*** format-patch + +** CI / CD <> +*** check\n--jenkins-json +*** check\n--sarif +*** check\n--code-climate +*** report + +@endmindmap From 7608a25870638c43b5b61cf0aadeba6763b11c2e Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 29 Mar 2026 13:23:50 +0000 Subject: [PATCH 10/30] Improve command documentation clarity and completeness MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Remove duplicate diagram directives from manual.rst (check.puml and update.puml are already embedded in their module docstrings) - Expand thin module docstrings: init, report, environment, validate now explain what the command does, its output, and next steps - Fix validate.py circular self-reference ("Note that you can validate using validate") — rewrite as a proper command description - Fix typo in update.py: "don't what recommendations" → "want" - Improve --no-recommendations help on check and update: explain sub- manifest recursion instead of vague "ignore recommendations" - Clarify --force on update: warn that local changes will be overwritten - Fix reporter --help strings: consistent capitalisation (SARIF, GitHub, GitLab), clearer verb ("Write a … report to ") - Expand diff --revs and project help: show format, mention output file - Expand format-patch -o help: document default and output filename - Add SVN externals migration steps to import_.py matching the depth of the Git submodules section; include nested-externals caveat https://claude.ai/code/session_0183Tzu6McYvSDwoXDoRjEyN --- dfetch/commands/check.py | 14 +++++++++----- dfetch/commands/diff.py | 13 +++++++++++-- dfetch/commands/environment.py | 13 ++++++++++++- dfetch/commands/format_patch.py | 6 +++++- dfetch/commands/import_.py | 21 ++++++++++++++++++--- dfetch/commands/init.py | 9 ++++++++- dfetch/commands/report.py | 28 +++++++++++++++++++++++++--- dfetch/commands/update.py | 16 ++++++++++++---- dfetch/commands/validate.py | 16 ++++++++++++++-- doc/manual.rst | 4 ---- 10 files changed, 114 insertions(+), 26 deletions(-) diff --git a/dfetch/commands/check.py b/dfetch/commands/check.py index b42c59a08..a03f5123f 100644 --- a/dfetch/commands/check.py +++ b/dfetch/commands/check.py @@ -64,32 +64,36 @@ def create_menu(subparsers: dfetch.commands.command.SubparserActionType) -> None "--no-recommendations", "-N", action="store_true", - help="Ignore recommendations from fetched projects.", + help=( + "Do not check sub-manifests found inside fetched projects. " + "By default, dfetch.yaml files discovered in fetched dependencies " + "are also checked for outdated entries." + ), ) parser.add_argument( "projects", metavar="", type=str, nargs="*", - help="Specific project(s) to check", + help="Specific project(s) to check (default: all projects in manifest)", ) parser.add_argument( "--jenkins-json", metavar="outfile", type=str, - help="Generate a JSON that can be parsed by Jenkins.", + help="Write a Jenkins warnings-ng JSON report to .", ) parser.add_argument( "--sarif", metavar="outfile", type=str, - help="Generate a Sarif JSON that can be parsed by Github.", + help="Write a SARIF 2.1.0 report to (GitHub Advanced Security).", ) parser.add_argument( "--code-climate", metavar="outfile", type=str, - help="Generate a code-climate JSON that can be parsed by Gitlab.", + help="Write a Code Climate JSON report to (GitLab pipelines).", ) def __call__(self, args: argparse.Namespace) -> None: diff --git a/dfetch/commands/diff.py b/dfetch/commands/diff.py index 1cfa9ccd2..3f226ac4f 100644 --- a/dfetch/commands/diff.py +++ b/dfetch/commands/diff.py @@ -84,7 +84,13 @@ def create_menu(subparsers: dfetch.commands.command.SubparserActionType) -> None metavar="[:]", type=str, default="", - help="Revision(s) to generate diff from", + help=( + "Revision(s) to generate diff from. " + "Omit to diff from the last fetched revision to the working copy. " + "Supply one revision to use it as the start point. " + "Supply two revisions separated by ':' (e.g. abc123:def456) for an " + "explicit range." + ), ) parser.add_argument( @@ -92,7 +98,10 @@ def create_menu(subparsers: dfetch.commands.command.SubparserActionType) -> None metavar="", type=str, nargs=1, - help="Project to generate diff from", + help=( + "Project to generate diff from. " + "Output is written to .patch in the superproject root." + ), ) def __call__(self, args: argparse.Namespace) -> None: diff --git a/dfetch/commands/environment.py b/dfetch/commands/environment.py index 57cb655bb..b4cace015 100644 --- a/dfetch/commands/environment.py +++ b/dfetch/commands/environment.py @@ -1,4 +1,15 @@ -"""*Dfetch* can generate output about its working environment.""" +"""*Dfetch* can report information about its working environment. + +``dfetch environment`` prints: + +* **Platform** — operating system name and kernel version. +* **VCS tool versions** — the version of each supported VCS client + (Git, SVN) found on ``PATH``. + +Run this command when setting up a new machine to confirm that the required +VCS tools are installed and discoverable, or include the output when filing +a bug report so that developers can reproduce your environment exactly. +""" import argparse import platform diff --git a/dfetch/commands/format_patch.py b/dfetch/commands/format_patch.py index ad0926242..253eeb0a0 100644 --- a/dfetch/commands/format_patch.py +++ b/dfetch/commands/format_patch.py @@ -73,7 +73,11 @@ def create_menu(subparsers: dfetch.commands.command.SubparserActionType) -> None metavar="", type=str, default=".", - help="Output directory for formatted patches", + help=( + "Directory to write formatted patches to " + "(default: current working directory). " + "The output file is named formatted-.patch." + ), ) def __call__(self, args: argparse.Namespace) -> None: diff --git a/dfetch/commands/import_.py b/dfetch/commands/import_.py index 39ff43b8b..c8e8a6c40 100644 --- a/dfetch/commands/import_.py +++ b/dfetch/commands/import_.py @@ -70,11 +70,26 @@ Migrating from SVN externals ============================ -* Make sure your repository is up-to-date. +* Make sure your working copy is up-to-date (``svn update``). * Generate a manifest using :ref:`dfetch import`. -* Remove all svn externals (see `How do I remove svn::externals `_ ). +* Remove all SVN externals. Externals are stored as ``svn:externals`` + properties on directories. To remove them:: + + svn propdel svn:externals + svn commit -m "Remove SVN externals (switching to Dfetch)" + + Repeat for every directory that had an ``svn:externals`` property set. + See `How do I remove SVN externals `_ + for more details. * Download all your projects using :ref:`dfetch update`. -* Commit your projects as part of your project. +* Commit the fetched files as part of your project. + +.. note:: + + If your SVN externals themselves contain further externals (nested + externals), you will need to run ``dfetch import`` recursively and add + those entries to your manifest by hand, as *Dfetch* only inspects the + top-level ``svn:externals`` property. .. scenario-include:: ../features/import-from-svn.feature diff --git a/dfetch/commands/init.py b/dfetch/commands/init.py index f6d35fdec..4f8a13477 100644 --- a/dfetch/commands/init.py +++ b/dfetch/commands/init.py @@ -1,6 +1,13 @@ """*Dfetch* can generate a starting manifest. -It will be created in the current folder. +Running ``dfetch init`` creates a ``dfetch.yaml`` file in the current +directory. The file contains a minimal template that you can open and edit +directly, or populate incrementally using :ref:`dfetch add `. + +Once you have listed your dependencies, fetch them with :ref:`dfetch update `. + +If a ``dfetch.yaml`` already exists in the current directory, *Dfetch* +prints a warning and exits without overwriting it. """ import argparse diff --git a/dfetch/commands/report.py b/dfetch/commands/report.py index 18d637634..ee3610a1d 100644 --- a/dfetch/commands/report.py +++ b/dfetch/commands/report.py @@ -1,6 +1,28 @@ -"""*Dfetch* can generate multiple reports. - -There are several report types that *DFetch* can generate. +"""*Dfetch* can generate reports about the projects in your manifest. + +Two report types are available via the ``-t`` / ``--type`` flag: + +``stdout`` (default) + Prints a dependency inventory to the terminal. For each project it shows + the remote URL, branch/tag/revision, last-fetch timestamp, applied patches, + and any licences detected in the fetched source tree. + +``sbom`` + Generates a `CycloneDX 1.6 `_ Software Bill of + Materials (SBOM) as a JSON file (``report.json`` by default, override with + ``-o``). The SBOM includes package URLs (PURLs), VCS references, licence + evidence, and — for archive projects — an optional SHA-256 integrity hash. + + This can be uploaded to GitHub as a supply-chain asset or attached to a + GitLab pipeline as a ``cyclonedx`` artefact. + +Licence detection +~~~~~~~~~~~~~~~~~ +*Dfetch* scans each fetched project for common licence files (``LICENSE``, +``COPYING``, etc.) and uses a best-effort heuristic to identify the licence +type. Only matches with a confidence of 80 % or higher are reported; ambiguous +files are silently skipped. If no licence is detected, the field is left +empty rather than guessing. """ import argparse diff --git a/dfetch/commands/update.py b/dfetch/commands/update.py index 6b6ec505b..8965b6384 100644 --- a/dfetch/commands/update.py +++ b/dfetch/commands/update.py @@ -24,7 +24,7 @@ It is possible that fetched projects have manifests of their own. When these projects are fetched (with ``dfetch update``), the manifests are read as well -and will be checked to look for further dependencies. If you don't what recommendations, you can prevent *Dfetch* +and will be checked to look for further dependencies. If you don't want recommendations, you can prevent *Dfetch* checking sub-manifests with ``--no-recommendations``. .. scenario-include:: ../features/updated-project-has-dependencies.feature @@ -84,20 +84,28 @@ def create_menu(subparsers: dfetch.commands.command.SubparserActionType) -> None "-f", "--force", action="store_true", - help="Always perform update, ignoring version check or local changes.", + help=( + "Re-fetch and overwrite even if the project is already at the " + "requested version or has local modifications. " + "Any unsaved local changes in the destination directory will be lost." + ), ) parser.add_argument( "-N", "--no-recommendations", action="store_true", - help="Ignore recommendations from fetched projects.", + help=( + "Do not check sub-manifests found inside fetched projects. " + "By default, dfetch.yaml files discovered in fetched dependencies " + "are also checked for outdated entries." + ), ) parser.add_argument( "projects", metavar="", type=str, nargs="*", - help="Specific project(s) to update", + help="Specific project(s) to update (default: all projects in manifest)", ) def __call__(self, args: argparse.Namespace) -> None: diff --git a/dfetch/commands/validate.py b/dfetch/commands/validate.py index 4845f23a3..f31f49be5 100644 --- a/dfetch/commands/validate.py +++ b/dfetch/commands/validate.py @@ -1,6 +1,18 @@ -"""Note that you can validate your manifest using :ref:`validate`. +"""*Dfetch* can validate a manifest without fetching anything. -This will parse your :ref:`Manifest` and check if all fields can be parsed. +``dfetch validate`` parses ``dfetch.yaml`` and checks every field against the +manifest schema — project names, URLs, version strings, source paths, and +integrity hashes are all verified. Any structural or type error is reported +immediately with a clear message pointing at the offending field. + +This is useful in CI to catch manifest mistakes before a full ``dfetch update`` +run, or as a quick sanity-check after hand-editing the file. + +.. note:: + + Validation also runs automatically at the start of every ``dfetch update`` + and ``dfetch check`` — a separate ``dfetch validate`` step is only needed + when you want to check the manifest without triggering a fetch. .. scenario-include:: ../features/validate-manifest.feature diff --git a/doc/manual.rst b/doc/manual.rst index ea61a544b..1bbf47a83 100644 --- a/doc/manual.rst +++ b/doc/manual.rst @@ -89,8 +89,6 @@ Check .. asciinema:: asciicasts/check.cast -.. uml:: /static/uml/check.puml - .. automodule:: dfetch.commands.check Update @@ -103,8 +101,6 @@ Update .. asciinema:: asciicasts/update.cast -.. uml:: /static/uml/update.puml - .. automodule:: dfetch.commands.update ---- From ea104bda7d1ce7850cb424d7d9b10ddd33d08459 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 29 Mar 2026 13:30:15 +0000 Subject: [PATCH 11/30] Add visual Design Guide to contributing docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New "Design Guide" section under "Creating documentation" covers: - Colour palette: 8 swatches with token name, hex, role, and usage (primary amber, accent blue, near-black text, warm-gray muted, primary-dark, accent-dark, bg-tint cream, bg-mint cool) - Typography: two side-by-side specimens — Inter at all 6 weights and JetBrains Mono at 400/500 with usage guidance - CSS token reference table: all --primary/--accent/--text/--bg/ --border/--r/--shad tokens with colour dots and use descriptions - PlantUML skinparam guide: list-table mapping each diagram element (activity, diamond, arrow, partition, start node) to the correct token value so new diagrams stay visually consistent New CSS classes: .dg-palette, .dg-swatch*, .dg-type-grid/card/header/ body/specimen/weights, .dg-tokens (table overrides), .dg-dot https://claude.ai/code/session_0183Tzu6McYvSDwoXDoRjEyN --- doc/contributing.rst | 281 ++++++++++++++++++++++++++++++++++++++ doc/static/css/custom.css | 204 +++++++++++++++++++++++++++ 2 files changed, 485 insertions(+) diff --git a/doc/contributing.rst b/doc/contributing.rst index 23967f3c6..155e60432 100644 --- a/doc/contributing.rst +++ b/doc/contributing.rst @@ -99,6 +99,287 @@ Run ``script/create_docs.bat`` and open ``index.html`` in ``doc/_build/html`` to See `This example `_ for documenting the code. Alternatively in VSCode run the ``Create Docs`` task from the command palette. +Design Guide +~~~~~~~~~~~~ + +The docs and landing page share a single design system. Follow these tokens +whenever you add new pages, diagrams, or HTML blocks to keep the visual +language consistent. + +Colour palette +'''''''''''''' + +.. raw:: html + +
+ +
+
+
+
--primary
+
#c2620a
+
Amber Orange
+
CTAs, active borders, diagram start nodes, section pips
+
+
+ +
+
+
+
--primary-dark
+
#a0510a
+
Amber Dark
+
Hover states, gradient end, button borders
+
+
+ +
+
+
+
--accent
+
#4e7fa0
+
Slate Blue
+
Decision diamonds, links, secondary badges, note borders
+
+
+ +
+
+
+
--accent-dark
+
#3a6682
+
Slate Dark
+
Hover states on accent elements, diagram labels
+
+
+ +
+
+
+
--text
+
#1c1917
+
Near Black
+
Body copy, headings, diagram text
+
+
+ +
+
+
+
--text-muted
+
#78716c
+
Warm Gray
+
Captions, sub-labels, placeholder text, diagram arrow labels
+
+
+ +
+
+
+
--bg-tint
+
#fef8f0
+
Warm Cream
+
Alternate section bands, card backgrounds, table headers
+
+
+ +
+
+
+
--bg-mint
+
#eff6fa
+
Cool Mint
+
Tip/note backgrounds, accent section bands
+
+
+ +
+ +Typography +'''''''''' + +.. raw:: html + +
+ +
+
+ Inter + body · UI · headings +
+
+
+
+ 300 + Vendor dependencies without the pain +
+
+ 400 + Vendor dependencies without the pain +
+
+ 500 + Vendor dependencies without the pain +
+
+ 600 + Vendor dependencies without the pain +
+
+ 700 + Vendor dependencies without the pain +
+
+ 800 + Vendor dependencies without the pain +
+
+
+
+ +
+
+ JetBrains Mono + code · tokens · CLI +
+
+
+
+ 400 + dfetch update --force +
+
+ 500 + dfetch update --force +
+
+

+ Use for: inline code, CLI examples, hex values, token names, + cheatsheet syntax, diagram labels where monospace is needed. +

+
+
+ +
+ +CSS tokens reference +'''''''''''''''''''' + +Use CSS custom properties rather than raw hex values so changes propagate +everywhere automatically. + +.. raw:: html + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TokenValueUse for
--primary#c2620aPrimary action colour
--primary-dark#a0510aHover / pressed state
--accent#4e7fa0Secondary / info colour
--accent-dark#3a6682Hover on accent elements
--text#1c1917Body text, headings
--text-2#3d3530Secondary text
--text-muted#78716cCaptions, labels, placeholders
--bg-tint#fef8f0Warm section backgrounds
--bg-mint#eff6faCool section backgrounds
--border#e7e0d8Card borders, dividers
--r / --r-sm / --r-lg12px / 8px / 20pxBorder radius (card / badge / hero)
--shad-sm / --shad-md / --shad-lg—Box shadow tiers
+ +PlantUML diagrams +''''''''''''''''' + +All flow and mindmap diagrams must use the shared skinparam block from +``doc/static/uml/check.puml`` as a template. Key values: + +.. list-table:: + :header-rows: 1 + :widths: 35 30 35 + + * - Element + - Value + - Notes + * - Activity background + - ``#ffffff`` + - White nodes on transparent canvas + * - Activity border + - ``#c2620a`` (``--primary``) + - 1.5 pt thickness + * - Diamond background + - ``#eff6fa`` (``--bg-mint``) + - Decision nodes use the cool tint + * - Diamond border + - ``#4e7fa0`` (``--accent``) + - Consistent with accent colour + * - Arrow colour + - ``#78716c`` (``--text-muted``) + - Keeps arrows from competing with nodes + * - Start node + - ``#c2620a`` (``--primary``) + - Filled amber disc + * - Partition background + - ``#fef8f0`` (``--bg-tint``) + - Warm cream section bands + * - Default font + - Arial, 12 pt, ``#1c1917`` + - Matches ``--text`` + Releasing --------- diff --git a/doc/static/css/custom.css b/doc/static/css/custom.css index ad53a5efa..5a260ffef 100644 --- a/doc/static/css/custom.css +++ b/doc/static/css/custom.css @@ -855,3 +855,207 @@ h1::after { /* suppress link-URL decoration that browsers add on print */ a::after { content: none !important; } } + +/* ============================================================ + Design Guide — colour swatches, type specimens, tokens + ============================================================ */ + +/* ── Colour palette grid ─────────────────────────────────── */ +.dg-palette { + display: grid; + grid-template-columns: repeat(auto-fill, minmax(160px, 1fr)); + gap: 1rem; + margin: 1.5rem 0 2rem; +} + +.dg-swatch { + border-radius: var(--r); + border: 1px solid var(--border); + overflow: hidden; + box-shadow: var(--shad-sm); +} + +.dg-swatch-color { + height: 72px; + width: 100%; +} + +.dg-swatch-body { + padding: 0.6rem 0.75rem 0.7rem; + background: #fff; +} + +.dg-swatch-token { + font-family: "JetBrains Mono", monospace; + font-size: 0.7rem; + color: var(--primary); + letter-spacing: -0.01em; + margin-bottom: 0.1rem; +} + +.dg-swatch-hex { + font-family: "JetBrains Mono", monospace; + font-size: 0.8rem; + font-weight: 500; + color: var(--text); + margin-bottom: 0.25rem; +} + +.dg-swatch-label { + font-family: "Inter", sans-serif; + font-size: 0.7rem; + font-weight: 600; + color: var(--text); + margin-bottom: 0.1rem; +} + +.dg-swatch-usage { + font-family: "Inter", sans-serif; + font-size: 0.675rem; + color: var(--text-muted); + line-height: 1.4; +} + +/* ── Typography specimens ────────────────────────────────── */ +.dg-type-grid { + display: grid; + grid-template-columns: 1fr 1fr; + gap: 1.25rem; + margin: 1.5rem 0 2rem; +} + +@media (max-width: 640px) { + .dg-type-grid { grid-template-columns: 1fr; } +} + +.dg-type-card { + border: 1px solid var(--border); + border-radius: var(--r); + overflow: hidden; + box-shadow: var(--shad-sm); +} + +.dg-type-header { + padding: 0.6rem 0.875rem; + background: var(--bg-tint); + border-bottom: 1px solid var(--border); + display: flex; + align-items: baseline; + gap: 0.75rem; +} + +.dg-type-name { + font-family: "Inter", sans-serif; + font-size: 0.7rem; + font-weight: 700; + text-transform: uppercase; + letter-spacing: 0.08em; + color: var(--primary); +} + +.dg-type-meta { + font-family: "JetBrains Mono", monospace; + font-size: 0.65rem; + color: var(--text-muted); +} + +.dg-type-body { + padding: 1rem 0.875rem 0.875rem; + background: #fff; +} + +.dg-type-specimen { + font-family: "Inter", sans-serif; + margin: 0; + line-height: 1.4; +} + +.dg-type-specimen-mono { + font-family: "JetBrains Mono", monospace; + font-size: 0.9rem; + line-height: 1.6; + margin: 0; +} + +.dg-type-weights { + display: flex; + flex-direction: column; + gap: 0.35rem; + margin-top: 0.5rem; +} + +.dg-type-weight-row { + display: flex; + align-items: baseline; + gap: 0.75rem; +} + +.dg-weight-label { + font-family: "JetBrains Mono", monospace; + font-size: 0.6rem; + color: var(--text-muted); + width: 2.5rem; + flex-shrink: 0; +} + +.dg-weight-sample { + font-family: "Inter", sans-serif; + font-size: 0.9rem; + color: var(--text); + line-height: 1; +} + +/* ── Token reference table ───────────────────────────────── */ +.dg-tokens { + width: 100%; + border-collapse: collapse; + margin: 1.25rem 0 2rem; + font-size: 0.8rem; + border-radius: var(--r); + overflow: hidden; + box-shadow: var(--shad-sm) !important; + border: none !important; +} + +.dg-tokens th { + background: var(--bg-tint) !important; + color: var(--text) !important; + font-family: "Inter", sans-serif !important; + font-size: 0.7rem !important; + font-weight: 700 !important; + text-transform: uppercase !important; + letter-spacing: 0.06em !important; + padding: 0.55rem 0.875rem !important; + border-bottom: 2px solid var(--border) !important; + text-align: left !important; +} + +.dg-tokens td { + padding: 0.5rem 0.875rem !important; + border-bottom: 1px solid var(--border) !important; + vertical-align: middle !important; + background: #fff !important; + color: var(--text) !important; +} + +.dg-tokens tr:last-child td { border-bottom: none !important; } + +.dg-tokens code { + font-family: "JetBrains Mono", monospace !important; + font-size: 0.75rem !important; + background: var(--bg-tint) !important; + border: none !important; + padding: 0.1em 0.35em !important; + border-radius: 4px !important; + color: var(--primary) !important; +} + +.dg-dot { + display: inline-block; + width: 14px; + height: 14px; + border-radius: 50%; + vertical-align: middle; + margin-right: 0.4rem; + border: 1px solid rgba(0,0,0,.08); +} From 2e0ec55b6b2f97939e87d629b16aa3c36206e23e Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 29 Mar 2026 13:37:55 +0000 Subject: [PATCH 12/30] Add Diataxis section colouring to sidebar and page headers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New file: doc/static/js/diataxis.js - Classifies each page into Tutorial / How-to / Reference / Explanation via a PAGE_SECTIONS map (keyed by filename without .html) - Adds dxt-
class to → CSS paints coloured top strip - Injects a floating kind badge ("Tutorial", "How-to Guide", etc.) at the top of div.body - Finds sidebar p.caption elements by caption text, adds dxt-caption-
and dxt-section-
classes CSS additions (custom.css): - Four new tokens: --dxt-tutorial (#c2620a), --dxt-howto (#4e7fa0), --dxt-reference (#4a7a62 sage), --dxt-explanation (#7a5a9a purple) - 4px coloured top border on div.body per section - .dxt-badge pills: subtle tinted background + border in section colour - Sidebar caption: coloured left border + tinted background + dot pip + caption text colour per section - Sidebar links: section-coloured hover and bold active link Design Guide updated with a 4-swatch row documenting each section colour and a note on how to register new pages. https://claude.ai/code/session_0183Tzu6McYvSDwoXDoRjEyN --- doc/conf.py | 4 ++ doc/contributing.rst | 57 +++++++++++++++++ doc/static/css/custom.css | 127 ++++++++++++++++++++++++++++++++++++++ doc/static/js/diataxis.js | 84 +++++++++++++++++++++++++ 4 files changed, 272 insertions(+) create mode 100644 doc/static/js/diataxis.js diff --git a/doc/conf.py b/doc/conf.py index 6e7304e75..2f1c39b86 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -123,6 +123,10 @@ "css/custom.css", ] +html_js_files = [ + "js/diataxis.js", +] + html_logo = "images/dfetch_logo.png" # Theme options are theme-specific and customize the look and feel of a theme diff --git a/doc/contributing.rst b/doc/contributing.rst index 155e60432..189cd9612 100644 --- a/doc/contributing.rst +++ b/doc/contributing.rst @@ -380,6 +380,63 @@ All flow and mindmap diagrams must use the shared skinparam block from - Arial, 12 pt, ``#1c1917`` - Matches ``--text`` +Diataxis section colours +'''''''''''''''''''''''' + +Each of the four Diataxis sections has its own accent colour. These are +applied automatically by ``doc/static/js/diataxis.js``: it adds a CSS class +to ```` and to the sidebar caption elements so both the top page strip +and the sidebar navigation header reflect the section. + +.. raw:: html + +
+ +
+
+
+
--dxt-tutorial
+
#c2620a
+
Tutorials
+
Same as --primary; warm amber for learning-oriented pages
+
+
+ +
+
+
+
--dxt-howto
+
#4e7fa0
+
How-to Guides
+
Same as --accent; slate blue for task-oriented pages
+
+
+ +
+
+
+
--dxt-reference
+
#4a7a62
+
Reference
+
Sage green; neutral, precise tone for information pages
+
+
+ +
+
+
+
--dxt-explanation
+
#7a5a9a
+
Explanation
+
Soft purple; contemplative tone for conceptual pages
+
+
+ +
+ +To add a new page to a section, add its filename (without ``.rst``) to the +``PAGE_SECTIONS`` map in ``doc/static/js/diataxis.js``. + Releasing --------- diff --git a/doc/static/css/custom.css b/doc/static/css/custom.css index 5a260ffef..59d9fd65b 100644 --- a/doc/static/css/custom.css +++ b/doc/static/css/custom.css @@ -13,6 +13,12 @@ --text-muted: #78716c; --bg-tint: #fef8f0; --bg-mint: #eff6fa; + + /* Diataxis quadrant colours */ + --dxt-tutorial: #c2620a; /* amber — same as --primary */ + --dxt-howto: #4e7fa0; /* blue — same as --accent */ + --dxt-reference: #4a7a62; /* sage green */ + --dxt-explanation: #7a5a9a; /* soft purple */ --border: #e7e0d8; --grad: linear-gradient(135deg, #c2620a 0%, #7a3a0a 100%); --shad-sm: 0 1px 3px rgba(0,0,0,.10), 0 1px 2px rgba(0,0,0,.06); @@ -1059,3 +1065,124 @@ h1::after { margin-right: 0.4rem; border: 1px solid rgba(0,0,0,.08); } + +/* ============================================================ + Diataxis section colouring + Four quadrants: Tutorial · How-to · Reference · Explanation + Driven by diataxis.js which adds classes to and the + sidebar caption

/

+ +
@@ -142,3 +152,34 @@ CLI Cheatsheet
+ + diff --git a/doc/static/css/custom.css b/doc/static/css/custom.css index 58914429c..37e95c3b8 100644 --- a/doc/static/css/custom.css +++ b/doc/static/css/custom.css @@ -70,6 +70,11 @@ body { background-repeat: no-repeat, repeat; } +/* Clip band pseudo-elements so they don't bleed over the sidebar */ +div.bodywrapper { + overflow: hidden; +} + /* Keep content areas transparent so body dot-grid shows through */ div.documentwrapper, div.body { @@ -415,6 +420,22 @@ table.docutils tbody tr:nth-child(even) td { margin-bottom: 1em; } +.sphinxsidebar input[type="text"] { + font-family: "Inter", sans-serif; + border: 1px solid var(--border); + border-radius: var(--r-sm); + padding: 0.35rem 0.6rem; + width: 100%; + box-sizing: border-box; + font-size: 0.9rem; + transition: border-color var(--ease); +} + +.sphinxsidebar input[type="text"]:focus { + outline: none; + border-color: var(--primary); +} + .sphinxsidebar input[type="submit"] { font-family: "Inter", sans-serif; background: var(--primary); @@ -866,20 +887,136 @@ body.dxt-explanation h1::after { background: var(--dxt-explanation); } } } -/* ── Print ────────────────────────────────────────────── */ +/* ── Fullscreen button ───────────────────────────────── */ +.cs-fullscreen-btn { + display: inline-flex; + align-items: center; + justify-content: center; + width: 1.75rem; + height: 1.75rem; + padding: 0; + background: rgba(255, 255, 255, 0.06); + border: 1px solid rgba(255, 255, 255, 0.12); + border-radius: 5px; + color: rgba(240, 232, 220, 0.5); + cursor: pointer; + flex-shrink: 0; + transition: background 0.15s, color 0.15s; +} + +.cs-fullscreen-btn:hover { + background: rgba(255, 255, 255, 0.12); + color: rgba(240, 232, 220, 0.9); +} + +/* ── Fullscreen state ────────────────────────────────── */ +.cheatsheet:fullscreen, +.cheatsheet:-webkit-full-screen { + border-radius: 0; + overflow-y: auto; + padding: 2rem; + box-sizing: border-box; +} + +/* ── Print — light / black-and-white ─────────────────── */ @media print { + @page { size: A4 landscape; margin: 0.6cm; } + body, div.documentwrapper, div.body { background-image: none !important; + background: #fff !important; + } + + /* Hide everything on the page except the cheatsheet */ + .sphinxsidebar, + .related, + .footer, + div.bodywrapper > *:not(div.body), + div.body > *:not(section), + section > *:not(.cheatsheet):not(h1) { + display: none !important; } + h1 { display: none !important; } + .cheatsheet { - box-shadow: none; - border: 0.5pt solid #3a2a1e; + background: #fff !important; + box-shadow: none !important; + border: 0.5pt solid #ccc !important; + border-radius: 4px !important; + margin: 0 !important; + width: 100% !important; + box-sizing: border-box !important; + font-size: 8.5pt !important; -webkit-print-color-adjust: exact; print-color-adjust: exact; } - /* suppress link-URL decoration that browsers add on print */ + .cs-masthead { + background: #f5f5f5 !important; + border-bottom: 1pt solid #ccc !important; + padding: 0.4rem 0.8rem !important; + } + + .cs-body { + padding: 0.6rem 0.8rem !important; + } + + .cs-section { margin-bottom: 0.6rem !important; } + + .cs-row { padding: 0.15rem 0 !important; } + + .cs-label { padding: 0.2rem 0 !important; margin-bottom: 0.1rem !important; } + + .cs-footer { padding: 0.3rem 0.8rem !important; } + + .cs-masthead-title { color: #111 !important; } + .cs-masthead-tagline { color: #555 !important; } + .cs-masthead-tagline code { color: #555 !important; } + + .cs-logo { + background: #333 !important; + color: #fff !important; + box-shadow: none !important; + } + + .cs-tok-sc { background: #eee !important; color: #333 !important; } + .cs-tok-fl { background: #eee !important; color: #333 !important; } + .cs-tok-ag { background: #eee !important; color: #333 !important; } + + .cs-col:first-child { border-right-color: #ddd !important; } + + .cs-l-primary { color: #333 !important; border-color: #bbb !important; } + .cs-l-accent { color: #333 !important; border-color: #bbb !important; } + .cs-l-ci { color: #333 !important; border-color: #bbb !important; } + .cs-l-utility { color: #333 !important; border-color: #bbb !important; } + + .cs-l-primary .cs-label-pip { background: #555 !important; } + .cs-l-accent .cs-label-pip { background: #555 !important; } + .cs-l-ci .cs-label-pip { background: #555 !important; } + .cs-l-utility .cs-label-pip { background: #555 !important; } + + .cs-row { border-bottom-color: #eee !important; } + .cs-kw { color: #888 !important; } + .cs-sc { color: #111 !important; font-weight: 700 !important; } + .cs-fl { color: #444 !important; } + .cs-ag { color: #666 !important; } + + .cs-dsc { color: #444 !important; } + .cs-dsc code { + background: #f0f0f0 !important; + color: #333 !important; + border: none !important; + } + + .cs-footer { + background: #f5f5f5 !important; + border-top-color: #ccc !important; + color: #777 !important; + } + + .cs-fullscreen-btn { display: none !important; } + a::after { content: none !important; } } From baa843a1c5a965133a8c873c88bdd7950a296491 Mon Sep 17 00:00:00 2001 From: Ben Date: Sun, 29 Mar 2026 19:57:22 +0000 Subject: [PATCH 26/30] Fix pre format --- doc/static/css/custom.css | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/doc/static/css/custom.css b/doc/static/css/custom.css index 37e95c3b8..33f134f38 100644 --- a/doc/static/css/custom.css +++ b/doc/static/css/custom.css @@ -155,7 +155,7 @@ div.highlight pre { } /* Inline code */ -code, .pre, tt { +code, tt { font-family: "JetBrains Mono", "Fira Code", Consolas, monospace !important; font-size: 0.875em; background: var(--bg-tint); @@ -165,6 +165,18 @@ code, .pre, tt { color: var(--primary-dark); } +/* Sphinx wraps each word in inside inline code — strip the + per-word styling so only the outer element gets the box. */ +code .pre, tt .pre { + font-family: inherit; + font-size: inherit; + background: none; + border: none; + border-radius: 0; + padding: 0; + color: inherit; +} + /* ============================================================ Admonitions ============================================================ */ From fc694a0adea68506d1efa8de7c349bbad782b75f Mon Sep 17 00:00:00 2001 From: Ben Date: Sun, 29 Mar 2026 20:02:22 +0000 Subject: [PATCH 27/30] Reword commands --- doc/commands.rst | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/doc/commands.rst b/doc/commands.rst index ed2e3b0d0..dc7f9542f 100644 --- a/doc/commands.rst +++ b/doc/commands.rst @@ -3,9 +3,15 @@ Commands ======== -*Dfetch* acts on the projects listed in the :ref:`Manifest`. -Each action is a separate sub-command. -For step-by-step guides see :doc:`getting_started`, :doc:`patching`, :doc:`check-ci`, and :doc:`sbom`. +*Dfetch* is driven entirely from the command line. Each subcommand +operates on the projects listed in the :ref:`Manifest`, which +*Dfetch* searches for automatically from the current directory recursively +downward. + +This page is the complete CLI reference — flags, arguments, and +behaviour for every subcommand. If you are new to *Dfetch*, start with +:doc:`getting_started` instead. For specific tasks the How-to Guides +in the sidebar go further. .. program-output:: dfetch --help :shell: @@ -34,6 +40,8 @@ Import .. automodule:: dfetch.commands.import_ +.. seealso:: :doc:`migration` — step-by-step guide for switching from Git submodules or SVN externals. + Add --- .. argparse:: @@ -44,6 +52,8 @@ Add .. automodule:: dfetch.commands.add +.. seealso:: :doc:`adding-a-project` — walks through adding a new dependency from start to finish. + Check ----- .. argparse:: @@ -56,6 +66,8 @@ Check .. automodule:: dfetch.commands.check +.. seealso:: :doc:`check-ci` — how to run dependency checks in CI pipelines and interpret the output formats. + Update ------ .. argparse:: @@ -68,6 +80,8 @@ Update .. automodule:: dfetch.commands.update +.. seealso:: :doc:`updating-projects` — covers the update workflow, pinning versions, and force-fetching. + Diff ---- .. argparse:: @@ -78,6 +92,8 @@ Diff .. asciinema:: asciicasts/diff.cast +.. seealso:: :doc:`patching` — creating, applying, and maintaining patches across upstream version bumps. + Update patch ------------ .. argparse:: @@ -110,6 +126,8 @@ Report .. automodule:: dfetch.reporting.stdout_reporter +.. seealso:: :doc:`sbom` — generating a Software Bill of Materials with ``dfetch report``. + Freeze ------ .. argparse:: From d474874a7f3991db8d90d12474c0ee59c85db484 Mon Sep 17 00:00:00 2001 From: Ben Date: Sun, 29 Mar 2026 20:06:59 +0000 Subject: [PATCH 28/30] Restructure docs --- doc/_ext/scenario_directive.py | 11 ++++++- doc/conf.py | 5 +++- doc/{ => explanation}/alternatives.rst | 0 doc/{ => explanation}/internal.rst | 0 doc/{ => explanation}/vendoring.rst | 2 -- doc/{ => howto}/adding-a-project.rst | 4 +-- doc/{ => howto}/check-ci.rst | 18 ++++++------ doc/{ => howto}/contributing.rst | 0 doc/{ => howto}/migration.rst | 0 doc/{ => howto}/patching.rst | 6 ++-- doc/{ => howto}/sbom.rst | 2 +- doc/{ => howto}/troubleshooting.rst | 2 +- doc/{ => howto}/updating-projects.rst | 2 +- doc/index.rst | 36 +++++++++++------------ doc/{ => reference}/changelog.rst | 0 doc/{ => reference}/cli_cheatsheet.rst | 0 doc/{ => reference}/commands.rst | 38 +++++++++++++------------ doc/{ => reference}/legal.rst | 0 doc/{ => reference}/manifest.rst | 0 doc/{ => tutorials}/getting_started.rst | 0 doc/{ => tutorials}/installation.rst | 2 +- 21 files changed, 70 insertions(+), 58 deletions(-) rename doc/{ => explanation}/alternatives.rst (100%) rename doc/{ => explanation}/internal.rst (100%) rename doc/{ => explanation}/vendoring.rst (99%) rename doc/{ => howto}/adding-a-project.rst (97%) rename doc/{ => howto}/check-ci.rst (94%) rename doc/{ => howto}/contributing.rst (100%) rename doc/{ => howto}/migration.rst (100%) rename doc/{ => howto}/patching.rst (96%) rename doc/{ => howto}/sbom.rst (98%) rename doc/{ => howto}/troubleshooting.rst (98%) rename doc/{ => howto}/updating-projects.rst (98%) rename doc/{ => reference}/changelog.rst (100%) rename doc/{ => reference}/cli_cheatsheet.rst (100%) rename doc/{ => reference}/commands.rst (64%) rename doc/{ => reference}/legal.rst (100%) rename doc/{ => reference}/manifest.rst (100%) rename doc/{ => tutorials}/getting_started.rst (100%) rename doc/{ => tutorials}/installation.rst (97%) diff --git a/doc/_ext/scenario_directive.py b/doc/_ext/scenario_directive.py index d1ff07ccd..6361654c4 100644 --- a/doc/_ext/scenario_directive.py +++ b/doc/_ext/scenario_directive.py @@ -59,10 +59,19 @@ def list_of_scenarios(self, feature_file_path: str) -> Tuple[str]: def run(self): """Generate the same literalinclude block for every scenario.""" + env = self.state.document.settings.env feature_file = self.arguments[0].strip() scenarios_available = self.list_of_scenarios(feature_file) + # Compute a path for literalinclude that is relative to the current + # RST document's directory. literalinclude resolves relative to the + # source file, while list_of_scenarios uses srcdir as the base — these + # differ once RST files are in sub-directories. + feature_abs = os.path.abspath(os.path.join(env.app.srcdir, feature_file)) + current_doc_dir = os.path.dirname(os.path.join(env.app.srcdir, env.docname)) + include_path = os.path.relpath(feature_abs, current_doc_dir) + scenario_titles = [ title.strip() for title in self.options.get("scenario", "").splitlines() @@ -84,7 +93,7 @@ def run(self):
Example: {html.escape(scenario_title)} -.. literalinclude:: {feature_file} +.. literalinclude:: {include_path} :language: gherkin :caption: {feature_file} :force: diff --git a/doc/conf.py b/doc/conf.py index f3cfb68e7..8e52d03b1 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -97,7 +97,10 @@ autosectionlabel_maxdepth = 3 # Suppress warnings about duplicate labels from argparse directive -suppress_warnings = ["autosectionlabel.commands"] +suppress_warnings = [ + "autosectionlabel.reference/commands", + "autosectionlabel.howto/updating-projects", +] # Options for sphinx-autoissues issuetracker = "github" diff --git a/doc/alternatives.rst b/doc/explanation/alternatives.rst similarity index 100% rename from doc/alternatives.rst rename to doc/explanation/alternatives.rst diff --git a/doc/internal.rst b/doc/explanation/internal.rst similarity index 100% rename from doc/internal.rst rename to doc/explanation/internal.rst diff --git a/doc/vendoring.rst b/doc/explanation/vendoring.rst similarity index 99% rename from doc/vendoring.rst rename to doc/explanation/vendoring.rst index eff663557..ffb84b0ec 100644 --- a/doc/vendoring.rst +++ b/doc/explanation/vendoring.rst @@ -1,7 +1,5 @@ -.. _vendoring: - Vendoring ========= diff --git a/doc/adding-a-project.rst b/doc/howto/adding-a-project.rst similarity index 97% rename from doc/adding-a-project.rst rename to doc/howto/adding-a-project.rst index 4efb8d34b..d956ae884 100644 --- a/doc/adding-a-project.rst +++ b/doc/howto/adding-a-project.rst @@ -61,7 +61,7 @@ path based on your existing projects. dfetch add https://github.com/some-org/some-repo.git -.. asciinema:: asciicasts/add.cast +.. asciinema:: ../asciicasts/add.cast Override individual fields with flags: @@ -90,7 +90,7 @@ inside the remote repository, or configure which paths to ignore. dfetch add -i https://github.com/some-org/some-repo.git -.. asciinema:: asciicasts/interactive-add.cast +.. asciinema:: ../asciicasts/interactive-add.cast The wizard walks through each field in turn: diff --git a/doc/check-ci.rst b/doc/howto/check-ci.rst similarity index 94% rename from doc/check-ci.rst rename to doc/howto/check-ci.rst index 32bc1a618..c3ba3884b 100644 --- a/doc/check-ci.rst +++ b/doc/howto/check-ci.rst @@ -22,7 +22,7 @@ Running dfetch check in CI .. automodule:: dfetch.reporting.check.reporter -.. asciinema:: asciicasts/check-ci.cast +.. asciinema:: ../asciicasts/check-ci.cast Without extra flags the results are printed to stdout and the build fails if any issue is found: @@ -45,7 +45,7 @@ Jenkins (warnings-ng) *Dfetch* writes a report in the `warnings-ng native JSON format`_ that the `warnings-ng plugin`_ can ingest directly. -.. asciinema:: asciicasts/check.cast +.. asciinema:: ../asciicasts/check.cast **Severity mapping** @@ -61,12 +61,12 @@ Jenkins (warnings-ng) Jenkins will show an overview of all issues: -.. image:: images/out-of-date-jenkins2.png +.. image:: ../images/out-of-date-jenkins2.png :alt: Cpputest is out-of-date and requires updating. Clicking an issue navigates to the exact line in the manifest: -.. image:: images/out-of-date-jenkins.png +.. image:: ../images/out-of-date-jenkins.png :alt: Cpputest is out-of-date and requires updating. **Pipeline snippet** @@ -114,17 +114,17 @@ findings appear inline in pull requests. Results appear on the Actions summary: -.. image:: images/github-actions-result.png +.. image:: ../images/github-actions-result.png :alt: Github action has run during a pull request. A locally changed project surfaces like this: -.. image:: images/local-change-github.png +.. image:: ../images/local-change-github.png :alt: A project was locally changed. Clicking *details* brings you to the manifest entry: -.. image:: images/local-change-github-details.png +.. image:: ../images/local-change-github-details.png :alt: A project was locally changed. **GitHub Actions workflow** @@ -195,12 +195,12 @@ branch. GitLab shows the results on the pipeline page: -.. image:: images/gitlab-check-pipeline-result.png +.. image:: ../images/gitlab-check-pipeline-result.png :alt: Gitlab detected issues. Clicking an issue navigates to the manifest: -.. image:: images/gitlab-highlighted-manifest.png +.. image:: ../images/gitlab-highlighted-manifest.png :alt: Gitlab highlights the project in the manifest with the issue. **``.gitlab-ci.yml`` snippet** diff --git a/doc/contributing.rst b/doc/howto/contributing.rst similarity index 100% rename from doc/contributing.rst rename to doc/howto/contributing.rst diff --git a/doc/migration.rst b/doc/howto/migration.rst similarity index 100% rename from doc/migration.rst rename to doc/howto/migration.rst diff --git a/doc/patching.rst b/doc/howto/patching.rst similarity index 96% rename from doc/patching.rst rename to doc/howto/patching.rst index 0fad90384..4acfbbe29 100644 --- a/doc/patching.rst +++ b/doc/howto/patching.rst @@ -37,7 +37,7 @@ run: metadata file and writes a patch file named ``some-project.patch`` (or ``some-project-N.patch`` if multiple patches already exist). -.. asciinema:: asciicasts/diff.cast +.. asciinema:: ../asciicasts/diff.cast **Controlling which revisions are compared** @@ -100,7 +100,7 @@ This regenerates the last patch for ``some-project`` from the current working tree, keeping the upstream revision unchanged. It is safe to run repeatedly as you iterate on the fix. -.. asciinema:: asciicasts/update-patch.cast +.. asciinema:: ../asciicasts/update-patch.cast .. tabs:: @@ -148,7 +148,7 @@ You can verify the formatted patch applies cleanly before submitting: svn patch formatted-some-project.patch -.. asciinema:: asciicasts/format-patch.cast +.. asciinema:: ../asciicasts/format-patch.cast .. tabs:: diff --git a/doc/sbom.rst b/doc/howto/sbom.rst similarity index 98% rename from doc/sbom.rst rename to doc/howto/sbom.rst index c96d788dd..d6726b1a6 100644 --- a/doc/sbom.rst +++ b/doc/howto/sbom.rst @@ -9,7 +9,7 @@ dependency with its URL, revision, and auto-detected license. Downstream tools can use the SBOM to monitor for known vulnerabilities or enforce a license policy across an organisation. -.. asciinema:: asciicasts/sbom.cast +.. asciinema:: ../asciicasts/sbom.cast .. code-block:: sh diff --git a/doc/troubleshooting.rst b/doc/howto/troubleshooting.rst similarity index 98% rename from doc/troubleshooting.rst rename to doc/howto/troubleshooting.rst index cc70bdecd..cc7488103 100644 --- a/doc/troubleshooting.rst +++ b/doc/howto/troubleshooting.rst @@ -19,7 +19,7 @@ This shows missing or incompatible dependencies. Run: Compare the output to the expected tools your commands require. -.. asciinema:: asciicasts/environment.cast +.. asciinema:: ../asciicasts/environment.cast Step 2: Use verbose mode ------------------------ diff --git a/doc/updating-projects.rst b/doc/howto/updating-projects.rst similarity index 98% rename from doc/updating-projects.rst rename to doc/howto/updating-projects.rst index b54a9e89c..ba319cfc2 100644 --- a/doc/updating-projects.rst +++ b/doc/howto/updating-projects.rst @@ -25,7 +25,7 @@ Run without arguments to fetch every project in the manifest: dfetch update -.. asciinema:: asciicasts/update.cast +.. asciinema:: ../asciicasts/update.cast *Dfetch* reads ``dfetch.yaml``, resolves each project's VCS type, and writes the requested revision into the destination folder. A ``.dfetch_data.yaml`` diff --git a/doc/index.rst b/doc/index.rst index ec96ca4d8..c2b9ad0f5 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -43,36 +43,36 @@ upstream. See :ref:`vendoring` for background on the problem this solves. :maxdepth: 2 :caption: Tutorials - installation - getting_started + tutorials/installation + tutorials/getting_started .. toctree:: :maxdepth: 2 :caption: How-to Guides - migration - adding-a-project - updating-projects - patching - check-ci - sbom - troubleshooting - contributing + howto/migration + howto/adding-a-project + howto/updating-projects + howto/patching + howto/check-ci + howto/sbom + howto/troubleshooting + howto/contributing .. toctree:: :maxdepth: 2 :caption: Reference - manifest - commands - cli_cheatsheet - changelog - legal + reference/manifest + reference/commands + reference/cli_cheatsheet + reference/changelog + reference/legal .. toctree:: :maxdepth: 2 :caption: Explanation - vendoring - alternatives - internal + explanation/vendoring + explanation/alternatives + explanation/internal diff --git a/doc/changelog.rst b/doc/reference/changelog.rst similarity index 100% rename from doc/changelog.rst rename to doc/reference/changelog.rst diff --git a/doc/cli_cheatsheet.rst b/doc/reference/cli_cheatsheet.rst similarity index 100% rename from doc/cli_cheatsheet.rst rename to doc/reference/cli_cheatsheet.rst diff --git a/doc/commands.rst b/doc/reference/commands.rst similarity index 64% rename from doc/commands.rst rename to doc/reference/commands.rst index dc7f9542f..274a8c9da 100644 --- a/doc/commands.rst +++ b/doc/reference/commands.rst @@ -10,7 +10,7 @@ downward. This page is the complete CLI reference — flags, arguments, and behaviour for every subcommand. If you are new to *Dfetch*, start with -:doc:`getting_started` instead. For specific tasks the How-to Guides +:doc:`../tutorials/getting_started` instead. For specific tasks the How-to Guides in the sidebar go further. .. program-output:: dfetch --help @@ -24,7 +24,7 @@ Init :prog: dfetch :path: init -.. asciinema:: asciicasts/init.cast +.. asciinema:: ../asciicasts/init.cast .. automodule:: dfetch.commands.init @@ -36,11 +36,11 @@ Import :prog: dfetch :path: import -.. asciinema:: asciicasts/import.cast +.. asciinema:: ../asciicasts/import.cast .. automodule:: dfetch.commands.import_ -.. seealso:: :doc:`migration` — step-by-step guide for switching from Git submodules or SVN externals. +.. seealso:: :doc:`../howto/migration` — step-by-step guide for switching from Git submodules or SVN externals. Add --- @@ -52,7 +52,7 @@ Add .. automodule:: dfetch.commands.add -.. seealso:: :doc:`adding-a-project` — walks through adding a new dependency from start to finish. +.. seealso:: :doc:`../howto/adding-a-project` — walks through adding a new dependency from start to finish. Check ----- @@ -62,11 +62,11 @@ Check :prog: dfetch :path: check -.. asciinema:: asciicasts/check.cast +.. asciinema:: ../asciicasts/check.cast .. automodule:: dfetch.commands.check -.. seealso:: :doc:`check-ci` — how to run dependency checks in CI pipelines and interpret the output formats. +.. seealso:: :doc:`../howto/check-ci` — how to run dependency checks in CI pipelines and interpret the output formats. Update ------ @@ -76,11 +76,11 @@ Update :prog: dfetch :path: update -.. asciinema:: asciicasts/update.cast +.. asciinema:: ../asciicasts/update.cast .. automodule:: dfetch.commands.update -.. seealso:: :doc:`updating-projects` — covers the update workflow, pinning versions, and force-fetching. +.. seealso:: :doc:`../howto/updating-projects` — covers the update workflow, pinning versions, and force-fetching. Diff ---- @@ -90,9 +90,9 @@ Diff :prog: dfetch :path: diff -.. asciinema:: asciicasts/diff.cast +.. asciinema:: ../asciicasts/diff.cast -.. seealso:: :doc:`patching` — creating, applying, and maintaining patches across upstream version bumps. +.. seealso:: :doc:`../howto/patching` — creating, applying, and maintaining patches across upstream version bumps. Update patch ------------ @@ -102,7 +102,9 @@ Update patch :prog: dfetch :path: update-patch -.. asciinema:: asciicasts/update-patch.cast +.. asciinema:: ../asciicasts/update-patch.cast + +.. _format-patch: Format patch ------------ @@ -112,7 +114,7 @@ Format patch :prog: dfetch :path: format-patch -.. asciinema:: asciicasts/format-patch.cast +.. asciinema:: ../asciicasts/format-patch.cast Report ------ @@ -122,11 +124,11 @@ Report :prog: dfetch :path: report -.. asciinema:: asciicasts/report.cast +.. asciinema:: ../asciicasts/report.cast .. automodule:: dfetch.reporting.stdout_reporter -.. seealso:: :doc:`sbom` — generating a Software Bill of Materials with ``dfetch report``. +.. seealso:: :doc:`../howto/sbom` — generating a Software Bill of Materials with ``dfetch report``. Freeze ------ @@ -136,7 +138,7 @@ Freeze :prog: dfetch :path: freeze -.. asciinema:: asciicasts/freeze.cast +.. asciinema:: ../asciicasts/freeze.cast .. automodule:: dfetch.commands.freeze @@ -148,7 +150,7 @@ Environment :prog: dfetch :path: environment -.. asciinema:: asciicasts/environment.cast +.. asciinema:: ../asciicasts/environment.cast .. automodule:: dfetch.commands.environment @@ -160,6 +162,6 @@ Validate :prog: dfetch :path: validate -.. asciinema:: asciicasts/validate.cast +.. asciinema:: ../asciicasts/validate.cast .. automodule:: dfetch.commands.validate diff --git a/doc/legal.rst b/doc/reference/legal.rst similarity index 100% rename from doc/legal.rst rename to doc/reference/legal.rst diff --git a/doc/manifest.rst b/doc/reference/manifest.rst similarity index 100% rename from doc/manifest.rst rename to doc/reference/manifest.rst diff --git a/doc/getting_started.rst b/doc/tutorials/getting_started.rst similarity index 100% rename from doc/getting_started.rst rename to doc/tutorials/getting_started.rst diff --git a/doc/installation.rst b/doc/tutorials/installation.rst similarity index 97% rename from doc/installation.rst rename to doc/tutorials/installation.rst index 5efd777e6..5e01cac17 100644 --- a/doc/installation.rst +++ b/doc/tutorials/installation.rst @@ -80,4 +80,4 @@ Run the following command to verify the installation dfetch environment -.. asciinema:: asciicasts/environment.cast +.. asciinema:: ../asciicasts/environment.cast From 63bbfe5aab41388e050946465d6c8ffe9af10b62 Mon Sep 17 00:00:00 2001 From: Ben Date: Sun, 29 Mar 2026 20:34:45 +0000 Subject: [PATCH 29/30] shorter titles --- doc/howto/patching.rst | 4 ++-- doc/howto/sbom.rst | 4 ++-- doc/howto/troubleshooting.rst | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/doc/howto/patching.rst b/doc/howto/patching.rst index 4acfbbe29..c2d2bc463 100644 --- a/doc/howto/patching.rst +++ b/doc/howto/patching.rst @@ -1,8 +1,8 @@ .. _patching: -Patch a vendored dependency -=========================== +Patch a dependency +================== *Dfetch* has a first-class patch workflow. When you need to fix a bug or apply a customisation to a vendored dependency, you can track that change as a diff --git a/doc/howto/sbom.rst b/doc/howto/sbom.rst index d6726b1a6..57632da8c 100644 --- a/doc/howto/sbom.rst +++ b/doc/howto/sbom.rst @@ -1,8 +1,8 @@ .. _sbom: -Generate a Software Bill-of-Materials -====================================== +Generate a SBoM +=============== ``dfetch report`` generates a `CycloneDX`_ SBOM listing every vendored dependency with its URL, revision, and auto-detected license. Downstream diff --git a/doc/howto/troubleshooting.rst b/doc/howto/troubleshooting.rst index cc7488103..bf5df5bbd 100644 --- a/doc/howto/troubleshooting.rst +++ b/doc/howto/troubleshooting.rst @@ -1,7 +1,7 @@ -Troubleshoot common problems -============================ +Troubleshoot +============ Sometimes *Dfetch* may not behave as expected. This is could be because it relies on standard command-line tools such as ``git`` to be available on your system. This section will help you From 3632eb144cdfe3ffcf4b59018d44ce51c4f7aafd Mon Sep 17 00:00:00 2001 From: Ben Date: Sun, 29 Mar 2026 22:51:09 +0200 Subject: [PATCH 30/30] Update doc/howto/sbom.rst Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> --- doc/howto/sbom.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/howto/sbom.rst b/doc/howto/sbom.rst index 57632da8c..0d5ae13ec 100644 --- a/doc/howto/sbom.rst +++ b/doc/howto/sbom.rst @@ -1,7 +1,7 @@ .. _sbom: -Generate a SBoM +Generate an SBOM =============== ``dfetch report`` generates a `CycloneDX`_ SBOM listing every vendored