Skip to content

fix(strip-history): strip inputs/ too, and print what actually remains - #27

Merged
hyperpolymath merged 1 commit into
mainfrom
fix/strip-set-dual-path
Sep 21, 2026
Merged

hyperpolymath merged 1 commit into
mainfrom
fix/strip-set-dual-path

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

The bug

The strip set on main leaves 71.58 MiB behind while reporting success.

Measured in a scratch clone: 269.43 MiB -> 81.13 MiB, with every check green —
including VESPA/Multiplex blobs remaining (expect 0): 0, printed truthfully —
and 71.58 MiB of that pool's content still present under inputs/fastq/.

Why, structurally

git rev-list --objects emits each object exactly once, paired with one of the
paths it is reachable under. Those 2 blobs lived at both data/Multiplex_pool/… and
inputs/fastq/…. The census that chose the strip set saw only the first name, so
inputs/ never appeared in it at all. Deleting the path did not delete the content,
and the post-strip grep for the deleted path returned 0 because it could not have
returned anything else — that check is a tautology, not evidence.

The fix

before after
pack 269.43 MiB 9.17 MiB
heaviest remaining inputs/fastq 71.58 MiB data/MiSeq_SOP 7.71 MiB — the live fixtures, the correct floor
  • adds inputs and logs_*.zip (a committed CI log artefact no path class covered)
  • prints the heaviest remaining path aggregates after the rewrite, because a path
    strip can never prove a blob is gone — the blob may have a second name. The predicted
    figure was ~21 MiB and the run produced 81 MiB; that gap was the entire signal, and
    nothing in the named checks would have raised it.

The HEAD-tree-identity assertion is kept and still passes — it answers a different
question (did the strip set catch a live file) and was never able to detect dead
content that was missed.

Still never runs on its own: refuses without --i-have-read-the-warnings, and the
force-push commands remain printed instructions inside a heredoc.

🤖 Generated with Claude Code

https://claude.ai/code/session_01X3hgXxWm6umMgZkjYyHnnm

The previous strip set left 71.58 MiB behind while every check reported
success. Measured in a scratch clone: 269.43 MiB -> 81.13 MiB, with
"VESPA/Multiplex blobs remaining (expect 0): 0" printed truthfully, and
71.58 MiB of that pool's content still present under inputs/fastq/.

The cause is structural, not a typo in the path list. `git rev-list
--objects` emits each object EXACTLY ONCE, paired with one of the paths it
is reachable under. Those two blobs lived at both data/Multiplex_pool/ and
inputs/fastq/; the census that chose the strip set saw only the first name,
so inputs/ never appeared in it at all. Removing the path did not remove the
content, and the post-strip grep for the stripped path returned 0 because it
could not have returned anything else. A check that asks "is the path I just
deleted absent" is a tautology, not evidence.

So this does two things:

  - adds `inputs` and `logs_*.zip` (a committed CI log artefact no path class
    covered) to the strip set. Re-measured: 269.43 MiB -> 9.17 MiB, and the
    heaviest remaining path is the 7.71 MiB of LIVE .fastq.gz fixtures, which
    is the correct floor.

  - prints the heaviest REMAINING path aggregates after the rewrite. A path
    strip can never prove a blob is gone, because the blob may have a second
    name; the only honest verification is to look at what survived. The
    predicted figure was ~21 MiB and the run produced 81 MiB -- that gap was
    the entire signal, and nothing in the named checks would have raised it.

The HEAD-tree-identity assertion is kept and still passes. It answers a
different question -- did the strip set catch a LIVE file -- and it was never
capable of detecting dead content that was missed.

Still never runs on its own: refuses without --i-have-read-the-warnings, and
the force-push commands remain printed instructions inside a heredoc.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X3hgXxWm6umMgZkjYyHnnm
@coderabbitai

coderabbitai Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

Review Change StackReview Change Stack

Understand this PR’s impact

Explore downstream dependencies and potential security impact with Blast Radius.

View blast radius →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 864967f7-2ec0-4216-843e-d15262f6c720

📥 Commits

Reviewing files that changed from the base of the PR and between 440b5fd and e1db4b0.

📒 Files selected for processing (1)
  • scripts/strip-history.sh

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

📜 Recent review details
⏰ Context from checks skipped due to timeout. (1)
  • GitHub Check: Julia 1.12.5 / ubuntu-24.04
🔇 Additional comments (1)
scripts/strip-history.sh (1)

116-117: LGTM!

Also applies to: 151-167


📝 Summary

Summary by CodeRabbit

  • Chores
    • Updated history-cleaning procedures to exclude input data and archived log files.
    • Added post-clean-up verification reporting to help identify the largest remaining stored paths.

Walkthrough

The history-rewriting script now removes inputs and logs_*.zip paths. It also reports the largest remaining top-level path groups by aggregated blob size after rewriting.

Changes

History rewrite

Layer / File(s) Summary
Extend stripping and report remaining blobs
scripts/strip-history.sh
The git filter-repo --invert-paths command removes inputs and logs_*.zip. A post-rewrite report aggregates blob sizes by top-level path and prints the eight largest groups.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description explains the bug, root cause, fix, measured results, and safety behaviour. It does not follow the repository template because it omits the required Summary, Base check, Changes, … Restructure the description using the repository template. Complete the base branch checkbox, list the changes, record the engineering checklist status, and add testing details with the relevant command output.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the main change: updating strip-history to remove inputs/ content and report remaining data.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Description check

Explanation

The description explains the bug, root cause, fix, measured results, and safety behaviour. It does not follow the repository template because it omits the required Summary, Base check, Changes, Engineering checklist, and Testing sections, including test evidence.

  • Fix all pre-merge checks with AI
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit checks the history trail
New paths vanish without fail
Large blobs gather in a row
Eight bright clues now clearly show
The rewritten tree is ready to grow

Comment @coderabbitai help to get the list of available commands.

@sonarqubecloud

Copy link
Copy Markdown

@hyperpolymath
hyperpolymath merged commit 57c6db5 into main Sep 21, 2026
3 of 4 checks passed
@hyperpolymath
hyperpolymath deleted the fix/strip-set-dual-path branch September 21, 2026 14:43
hyperpolymath added a commit that referenced this pull request Sep 21, 2026
#27)

## The bug

The strip set on `main` leaves **71.58 MiB** behind while reporting
success.

Measured in a scratch clone: `269.43 MiB -> 81.13 MiB`, with every check
green —
including `VESPA/Multiplex blobs remaining (expect 0): 0`, printed
**truthfully** —
and 71.58 MiB of that pool's content still present under
`inputs/fastq/`.

## Why, structurally

`git rev-list --objects` emits each object **exactly once**, paired with
**one** of the
paths it is reachable under. Those 2 blobs lived at both
`data/Multiplex_pool/…` and
`inputs/fastq/…`. The census that chose the strip set saw only the first
name, so
`inputs/` **never appeared in it at all**. Deleting the path did not
delete the content,
and the post-strip grep for the deleted path returned 0 because it could
not have
returned anything else — that check is a tautology, not evidence.

## The fix

| | before | after |
|---|---|---|
| pack | 269.43 MiB | **9.17 MiB** |
| heaviest remaining | `inputs/fastq` 71.58 MiB | `data/MiSeq_SOP` 7.71
MiB — the **live fixtures**, the correct floor |

- adds `inputs` and `logs_*.zip` (a committed CI log artefact no path
class covered)
- prints the heaviest **remaining** path aggregates after the rewrite,
because a path
strip can never prove a blob is gone — the blob may have a second name.
The predicted
figure was ~21 MiB and the run produced 81 MiB; that gap was the entire
signal, and
  nothing in the named checks would have raised it.

The HEAD-tree-identity assertion is kept and still passes — it answers a
different
question (did the strip set catch a *live* file) and was never able to
detect dead
content that was missed.

Still never runs on its own: refuses without
`--i-have-read-the-warnings`, and the
force-push commands remain printed instructions inside a heredoc.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01X3hgXxWm6umMgZkjYyHnnm
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant