repo refactor: Repair mainline benchmark hygiene and make benchmark/report artifacts reproducible from a fresh clone#11
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
Refactors benchmark/reproducibility documentation and ignore rules so benchmarking and report-related artifacts are reproducible from a clean clone without committing generated outputs.
Changes:
- Update
report/benchmark_report.mdto be a hand-maintained reproducibility note (no checked-in generated benchmark outputs). - Update
README.mdto document a fresh-clone workflow using an out-of-tree temp build dir plus CTest + pytest verification. - Expand
.gitignoreto cover common CMake/build outputs and generated benchmark/report artifacts.
Reviewed changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
report/benchmark_report.md |
Reframes the report as a reproducibility note; documents fixture scope and the fresh-clone run sequence/commands. |
README.md |
Aligns top-level docs with the “fresh clone” reproducible build/test/benchmark workflow and clarifies fixture vs proprietary dataset scope. |
.gitignore |
Adds comprehensive ignore rules for build trees, binaries, and generated benchmark/report byproducts. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Repo job repo_refactor for objective #38 is ready for review and merge.
Objective: Build project: Real-Time Limit Order Book Engine in C++
Job instructions:
Starting from main, remove any git-tracked generated build artifacts and ensure build outputs are ignored. Reconcile the previously dirty benchmark/report state so the repository contains only source datasets, scripts, reports, and code needed for reproducible benchmarking. Verify the benchmark binaries, correctness verifier, and report-generation commands configure/build/run successfully from a fresh clone using documented commands. If benchmark datasets for AAPL plus the additional tickers are synthetic or reduced, document that clearly in repo-visible artifacts rather than implying full LOBSTER distribution data. Update README/benchmark docs only as needed to reflect the exact runnable command set on main.
Why this PR looks credible:
Tests passed on the current branch. The change is coherent and appropriately bounded for the benchmark-hygiene task. It hardens ignore rules for generated CMake/build/test/benchmark byproducts, updates repo-visible docs to describe an actually reproducible fresh-clone workflow, and clearly corrects the dataset framing so the checked-in CSVs are presented as reduced/synthetic fixtures rather than full LOBSTER distributions. The reported validation covers clean configure/build, ctest including the benchmark smoke path, the required pytest command, and all four documented benchmark invocations from an isolated copy, which is sufficient for this cleanup-focused refactor. Audit cleared the diff, with 1 remaining validation gap(s) to watch. Backend verification returned insufficient evidence.
Audit summary:
The change is coherent and appropriately bounded for the benchmark-hygiene task. It hardens ignore rules for generated CMake/build/test/benchmark byproducts, updates repo-visible docs to describe an actually reproducible fresh-clone workflow, and clearly corrects the dataset framing so the checked-in CSVs are presented as reduced/synthetic fixtures rather than full LOBSTER distributions. The reported validation covers clean configure/build, ctest including the benchmark smoke path, the required pytest command, and all four documented benchmark invocations from an isolated copy, which is sufficient for this cleanup-focused refactor.
Audit must-fix items:
Validation gaps:
git ls-filesfor old build outputs before merging.