"Don't Merge": Code Review and Sanity Check

.gitignore

@@ -0,0 +1,22 @@
+*.o
+# *example*
+*.tmp
+
+# .vscode
+.vscode/*
+
+*callgrind*
+
+# build
+**/build/**
+
+# log
+**/*.log
+
+# text
+*.txt
+secret.txt
+
+tracing_strategy.md
+
+workload_0p*

.vscode/launch.json

@@ -0,0 +1,29 @@
+{
+    // Use IntelliSense to learn about possible attributes.
+    // Hover to view descriptions of existing attributes.
+    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
+    "version": "0.2.0",
+    "configurations": [
+
+      // {
+      //   "type": "node",
+      //   "request": "launch",
+      //   "name": "launch program",
+      //   "skipFiles": [ "<node_internals>/**"],
+      //   "preLaunchTask": "simple_example",
+      //   "program": "${workspaceFolder}/simple_example",
+      // }
+      {
+        "name": "simple_example",
+        "type": "cppdbg",
+        "request": "launch",
+        "program": "${fileDirname}/../development/rocksdb/self_RD/simple_example",
+        "args": [">", "log"],
+        "stopAtEntry": false,
+        "cwd": "${fileDirname}",
+        "environment": [],
+        "externalConsole": false,
+        "MIMode": "gdb",
+      }
+    ]
+}

.vscode/log

@@ -0,0 +1 @@
+Clearing system cache ...

.vscode/tasks.json

@@ -0,0 +1,29 @@
+{
+    "tasks": [
+
+        // {
+            // "type": "cppbuild",
+            // "label": "C/C++: g++ build active file",
+            // "command": "/usr/bin/g++",
+            // "args": [
+            //     "-fdiagnostics-color=always",
+            //     "-g",
+            //     "${file}",
+            //     "-o",
+            //     "${fileDirname}/${fileBasenameNoExtension}"
+            // ],
+            // "options": {
+            //     "cwd": "${fileDirname}"
+            // },
+            // "problemMatcher": [
+            //     "$gcc"
+            // ],
+            // "group": {
+            //     "kind": "build",
+            //     "isDefault": true
+            // },
+            // "detail": "Task generated by Debugger."
+        // }
+    ],
+    "version": "2.0.0"
+}

README.md

@@ -1,2 +1,59 @@
-# LSMRangeDeletes
-Making range deletes better in LSMs
+# LSMRangeDeletes
+Making range deletes better in LSMs.
+
+## Building the Project
+1. **Build RocksDB Core**:
+   Go to `./development/rocksdb/` and run:
+   ```bash
+   make clean             # Run if DB file relation is broken
+   cp ../make_config ./
+   make static_lib -j8
+   ```
+
+2. **Build Examples**:
+   Go to experiment directory `./development/rocksdb/self_rd_e[number]_xxx` and run:
+   ```bash
+   make simple_example -j8
+   ```
+
+## Running Experiments
+Run the `simple_example` with specific parameters:
+```bash
+python run_task_256B_string_key.py
+```
+
+## Progress
+The project has implemented several advanced range delete filter mechanisms in RocksDB to optimize lookup performance in LSM-trees under frequent range deletions.
+
+### Implemented Filters
+1.  **Spry-Vec (Range Delete Filter with Vector / PLRDF_STRINGKEY)**:
+    - **Concept**: A per-level filter that tracks range tombstones at each level of the LSM-tree.
+    - **Mechanism**: Manages a set of non-overlapping range tombstones `[start, end)` per level. It uses a specialized `isEntryAlive` check to determine if a key is covered by any range tombstone at a given level.
+    - **Optimization**: Features automatic **Adjacent Range Merging** (e.g., merging `[10, 20)` and `[20, 30)` into `[10, 30)`) to minimize filter size and lookup overhead.
+
+2.  **Spry-Vec-Frag (Range Delete Filter with Vector and Fragmentation operation / SPLIT_PLRDF_STRINGKEY)**:
+    - An extension of PLRDF that supports key-space splitting (by keeping all the newly arrived point queries that fall within older range tombstones) to further refine range tracking within a level, improving filtering efficiency on range deleted keys.
+
+3. **Top-Level-Spry (Top Level Range Delete Filter / TOP_LEVEL_RDF)**:
+    - **Concept**: A variant of Spry-Vec-Frag that keeps all the rangetombstones on level 1 in the buffer. Theoretically, the fastest filter for point queries on range deleted keys.
+
+4. **Spry-Trie (Range Delete Filter with Trie / SuRF_RDF)**:
+    - **Concept**: A trie-based range delete filter which utilized LOUDS representation to compress the trie.
+    - **Mechanism**: Uses a compressed trie to store range tombstones inside the buffer for each file. It allows for efficient "is deleted" checks by traversing the trie. 
+    - **Optimization**: Compresses the trie to store range tombstones.
+
+5. **Spry-Trie-Frag (Range Delete Filter with Trie and Fragmentation operation / SPLIT_SuRF_RDF)**:
+    - An extension of Spry-Trie that supports key-space splitting (by keeping all the newly arrived point queries that fall within older range tombstones) to further refine range tracking within a level, improving filtering efficiency on range deleted keys.It reaches the balance between memory footprint and speed.
+
+6.  **SkyLineRDF (Skyline Range Delete Filter)**:
+    - **Concept**: A more sophisticated filter that accounts for the sequence numbers (timestamps) of range deletions.
+    - **Mechanism**: Uses a "skyline" approach to track the maximum sequence number for any given key range. It allows for precise "is alive" checks by comparing a point key's sequence number against the skyline's maximum at that position.
+    - **Optimization**: Efficiently aggregates overlapping range tombstones into a minimal set of non-overlapping "skyline" segments.
+
+
+### Core Integration Refinements
+- **SST Boundary Tracking**: Modified `FileMetaData` and the MANIFEST to track precise point key and range tombstone boundaries separately, enabling accurate clipping of filters at the SST level.
+- **Flush-time Aggregation**: Range tombstones from multiple memtables are consolidated during the flush process to ensure consistent RDF state in Level 0.
+- **Compaction Management**: Implemented a "Gather & Shift/Clip" strategy to efficiently update RDF metadata during compaction, ensuring that range deletes are correctly propagated or clipped across SST boundaries.
+
+