Skip to content

feat(ls): add 'dt list' and a tracked-object tree with md/html export (0.26.0) - #189

Merged
johnyaku merged 2 commits into
mainfrom
feat/ls-tree
Aug 31, 2026
Merged

johnyaku merged 2 commits into
mainfrom
feat/ls-tree

Conversation

@johnyaku

@johnyaku johnyaku commented Aug 31, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds dt list (an alias for dt ls) and a --tree mode that renders the repository as a nested tree in three formats — plain text, Markdown, and a self-contained interactive HTML page.

Tree scope — the tracked-object view

By default the tree shows the git-tracked + DVC-tracked objects and nothing else:

  • untracked files are excluded (add them with --all);
  • git-ignored and dvc-ignored files are never shown (.snakemake/, __pycache__/, …);
  • DVC / VCS bookkeeping is hidden — *.dvc pointers, dvc.lock, .gitignore, .dvcignore, and the .dvc/ and .dt/ directories. The tracked object is shown, not the file that points at or configures it.

DVC-tracked data is always shown even though DVC keeps it under .gitignore (matched via isout). dvc list already honours .dvcignore, so dvc-ignored paths never appear.

Flag Effect
--tree Render as a tree (plain text)
-o {text,md,html} Render format (md/html imply --tree)
--all Also include untracked files (still never ignored ones)
--dvc-only Restrict to DVC outputs
-L, --level N Cap depth; deeper contents collapse to … (N files)

A remote URL or a committed --rev is treated as an already-tracked tree (only bookkeeping is stripped).

Rendering

  • Text / Markdown print the repository name and the revision — short SHA, any tags on that commit, and its date.
  • HTML titles the repository (linked to its web page), shows the revision subtitle, and reuses the collapsible fold-up from dt diff -o html (per-directory <details>, Expand/Collapse all).
    • Git-tracked files link straight to GitHub — each git-tracked file (e.g. a README) is a direct link to its blob on the repo's web page, pinned to the listed revision's SHA. Browse docs straight from the tree.
    • DVC objects get a command popup — their data isn't on the git host, so the dvc button opens copy-pastable dvc get / dvc import commands with HTTPS / SSH tabs (default HTTPS, remembered across popups). Both clone URLs are derived from the repo, so SSH works for private repos. (Directories offer the same popup.)
myrepo
b802a00 · v1.2.0 · 2026-08-31
├── data/
│   └── samples.h5ad        ← DVC object: dvc get/import popup
└── README.md               ← git-tracked: links to GitHub

Performance

dt ls / dt list no longer ask dvc list for --size / --show-hash unless a flag or filter needs them, keeping dvc off its heavy, mmap-intensive resolution path on large (100k+ file) trees. Subtree counts are memoized so HTML rendering is linear in the number of nodes.

Docs

  • docs/ls.md — new Tree view section (scope, options, formats, HTML features incl. GitHub links) + alias note.
  • docs/commands.md — updated dt ls entry.

Tests

  • Unit (tests/unit/test_ls.py): tree build/render (text/md/html), depth collapse, the tracked/--all/--dvc-only filtering model, bookkeeping/.dvc/.dt suppression, repo-URL parsing (HTTPS/SSH), popup contents, the revision subtitle, and git-file-to-GitHub linking (+ no-web-URL fallback).
  • Integration (tests/integration/test_ls_cmd.py): dt list alias parity, tree formats against a real DVC repo, default excludes untracked/ignored/bookkeeping, --all re-includes untracked, HTML repo-link + popup, git-tracked files link to GitHub while DVC objects keep the popup, and the revision/tag subtitle.

All unit tests pass locally (90) and the full ls integration suite passes on an NCI compute node (25).

🤖 Generated with Claude Code

johnyaku and others added 2 commits August 31, 2026 14:04
… (0.26.0)

Adds `dt list` as an alias for `dt ls`, and a `--tree` mode that renders the
repository as a nested tree in three formats: plain text, Markdown, and a
self-contained interactive HTML page.

Tree scope (the tracked-object view):
- Default shows git-tracked + DVC-tracked objects only.
- `--all` also includes untracked files (never git-/dvc-ignored ones).
- `--dvc-only` restricts to DVC outputs; `-L/--level N` caps depth.
- Excludes untracked, git-ignored, and dvc-ignored paths, plus DVC/VCS
  bookkeeping (`*.dvc`, `dvc.lock`, `.gitignore`, `.dvcignore`, `.dvc/`,
  `.dt/`) -- the tracked object is shown, not the pointer/config. DVC data is
  always shown even though it is gitignored (matched via `isout`).
- A remote URL or a committed `--rev` is treated as an already-tracked tree.

Rendering:
- Text/Markdown print the repo name and revision (short SHA, tags, date).
- HTML titles the repo linked to its web page, shows the revision subtitle,
  and reuses the collapsible fold-up from `dt diff -o html`. Each node has a
  popup of copy-pastable `dvc get`/`dvc import` commands with HTTPS/SSH tabs
  (default HTTPS, remembered); both clone URLs are derived from the repo so
  SSH works for private repos.

Performance: `dt ls`/`dt list` no longer request `--size`/`--show-hash` from
`dvc list` unless a flag or filter needs them, keeping dvc off its heavy,
mmap-intensive resolution path on large (100k+ file) trees; subtree counts are
memoized so HTML rendering is linear.

Docs updated (docs/ls.md, docs/commands.md). Unit + integration tests added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In the HTML tree, git-tracked files (isout=False) now render as direct links
to their GitHub blob, pinned to the listed revision's SHA -- handy for opening
a README straight from the tree. DVC-tracked objects are unchanged: their data
isn't on the git host, so they keep the dvc get/import popup. Directories keep
the popup too.

File entries in the tree now carry the per-file blob URL (build_tree stores
{name, gh}); _annotate_github computes it for HTML output only, skipping
untracked files (reachable only via --all) which aren't on the host. Falls
back to the popup when the repo has no browsable web URL.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@johnyaku
johnyaku merged commit 58acca2 into main Aug 31, 2026
1 check passed
@johnyaku
johnyaku deleted the feat/ls-tree branch August 31, 2026 06:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant