Conversation
Add an isolated alex-256slot-lowlf competitor copied from alex-256slot, preserving the original variants while lowering copied data-node density thresholds for space/performance experiments. Agent-Task: Tune ALEX-256slot occupancy/load-factor constants toward ART-relative memory and measure performance. Agent-Decision: Keep the change as a separate benchmark index so baseline alex and alex-256slot remain comparable; only adjust copied density constants to kMax=0.6, kInit=0.5, kMin=0.4 after rejecting a lower 0.4/0.35/0.3 probe that crashed full-size bulk load. Agent-Limitation: Single global density tuning did not meet the ART-relative memory target across datasets; benchmark artifacts document the miss.
Replace the single low-load-factor probe with three explicit ALEX-256slot variants tuned for covid, fb-1, and osm memory matching against ART. Keep the original alex and alex-256slot baselines unchanged. Agent-Task: Find dataset-specific ALEX-256slot load-factor settings whose memory is within 10% of ART on covid, fb-1, and osm, then measure performance with three repeats. Agent-Decision: Encode kMinDensity in separate index names alex-256slot-lf024, alex-256slot-lf020, and alex-256slot-lf019; retain kMaxDensity=0.6 and kInitDensity=0.5 across variants; remove the earlier alex-256slot-lowlf negative-result artifact. Agent-Limitation: These variants are dataset-specific experimental controls for memory-matched comparisons, not throughput improvements over ART.
l05k
changed the title
Add a seaborn notebook under result/motivation that plots space versus insert/read throughput for ART, ALEX, LIPP, LIPP-256slot, and the dataset-specific memory-matched ALEX-256slot variants across covid, fb-1, and osm. Agent-Task: Provide a Jupyter notebook plotting space vs throughput using seaborn for the relevant benchmark result directory. Agent-Decision: Keep the notebook output-free so users run it with the repository uv notebook extra to render figures locally. Agent-Limitation: The notebook reads existing checked-in summary CSVs and does not regenerate benchmark data.
Owner
Author
|
Added |
Update the seaborn notebook to include the linear dataset subplot using alex-256slot as the memory-close variant, and move the shared legend from the right side to the top of each figure. Agent-Task: Add linear to the plotting notebook and place legends at the top. Agent-Decision: Use the existing motivation summary for linear because no dataset-specific load-factor variant was needed there; keep tuned dataset rows from the load-factor summary. Agent-Limitation: The notebook remains output-free and must be run by the user to render figures.
Owner
Author
|
Updated |
…-throughput plot The notebook filtered motivation data to BASELINE_INDEXES (which excludes alex-256slot) before looking up the linear dataset's alex-256slot row. Split into motivation_full (unfiltered, used for linear lookup) and motivation (filtered to baselines, used for plotting). Agent-Task: fix ValueError for linear dataset in alex_256slot_space_throughput.ipynb Agent-Decision: keep original alex-256slot for linear (not tuned variant) since space is already within 10% of ART Agent-Limitation: none
…hroughput plot Replace the Jupyter notebook with a pure Python script that: 1. Prints the combined plot_data table (dataset x index x throughput x memory). 2. Prints the melted throughput_data table. 3. Saves insert and read scatter plots as PNG files. This avoids the notebook runtime dependency and makes the analysis directly reproducible with a single command: uv run --extra notebook python3 result/motivation/alex_256slot_space_throughput.py Agent-Task: convert alex_256slot_space_throughput.ipynb to standalone script Agent-Decision: savefig to PNG since headless environment has no display Agent-Limitation: none
- Merge all alex-256slot(-lf*) variants into a single 'alex-256slot' legend label to avoid overcrowded legend/item overlap. Printed tables retain original index names. - Switch to Carlito (Calibri metric-compatible) as the default sans-serif font with consistent font sizes across all elements. Agent-Task: unify alex-256slot legend, switch to Calibri/Carlito font Agent-Decision: normalize labels after printing tables but before plotting Agent-Limitation: none
…y note Two minor documentation fixes from PR review: - Research record: note benchmarked commit 36f63c9 in Validation Method - space_throughput.py: document dependency on motivation summary CSV Agent-Task: Fix minor issues found during PR #4 review Agent-Model: claude-opus-4.5 Agent-Decision: Follow-up commit per review workflow; do not amend Agent-Limitation: none
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Task Description
Find ALEX-256slot load-factor settings for
covid,fb-1, andosmsuch that each dataset has a parameter version whose memory is within 10% of ART, then compare performance with three benchmark repeats.What Changed
Added three separate ALEX-256slot variants, each copied from
alex-256slotand registered as an explicit benchmark index:alex-256slot-lf024covidalex-256slot-lf020fb-1alex-256slot-lf019osmThe original
alexandalex-256slotimplementations are unchanged. The earlier singlealex-256slot-lowlfprobe was replaced because the intended experiment is dataset-specific matching, not one global setting.Benchmark artifacts added:
result/motivation/alex_256slot_dataset_load_factor_raw.csvresult/motivation/alex_256slot_dataset_load_factor_summary.csvresult/motivation/alex_256slot_dataset_load_factor.mdreports/benchmark/20260602-alex-256slot-dataset-load-factor.mdKey Design Decisions
kMaxDensity_=0.6andkInitDensity_=0.5fixed after quick probes; tunekMinDensity_per dataset.linearper the clarified scope.Alternatives Considered
kInitDensity_=0.45,kMinDensity_=0.35: rejected because full-size bulk-load probes crashed.Test Coverage
microbenchalex-256slot-lf019,alex-256slot-lf020, andalex-256slot-lf024:success_insert: 250,success_read: 500.test_suite=22, 100M bulkload / 100M shuffled inserts, datasets from/root/workspace/datasets/{covid,fb-1,osm}.Known Limitations / Follow-up Tasks
These are dataset-specific experimental controls for memory-matched comparisons. They are not throughput improvements over ART.
Acceptance summary:
Review Guidance
Focus on whether the dataset-specific variants and naming are acceptable for controlled memory-matched benchmark comparisons, and whether the benchmark artifacts/report provide enough evidence for the next analysis step.