Skip to content

fix #10: adapt to d-engine main data_dir API, fix docker-release CI + local CVE gate - #11

Merged
JoshuaChi merged 3 commits into
mainfrom
feature/10-main-breaking-changes
Aug 8, 2026
Merged

JoshuaChi merged 3 commits into
mainfrom
feature/10-main-breaking-changes

Conversation

@JoshuaChi

@JoshuaChi JoshuaChi commented Aug 7, 2026 •

Copy link
Copy Markdown
Contributor

What Does This PR Do?

Adapts to d-engine main's breaking data_dir API change; fixes 2 broken CI/CVE gates.

Type:

  • Bug Fix (with test)

Why Is This Needed?

  • d-engine main now requires explicit data_dir arg on start_custom/start_node — d-lmdb didn't compile against it.
  • docker-release.yml: push step tagged with wrong var, invalid Docker ref on every release; arm64 smoke test flaky under QEMU — switched to native runner.
  • docker-cve-check: hard-failed on unfixable upstream Debian CVEs — now only gates on CVEs with an available fix.

Checklist

  • make check passes
  • make test passes
  • Added tests for new code — n/a, signature-only change
  • Commits squashed

Testing

  • cargo nextest run --all-features: 91/91 passed
  • make test (build + smoke test + CVE gate): clean, multiple runs

Does This Fit d-lmdb's Scope?

  • Fault-tolerance only, no scaling
  • Simple
  • No public API bloat

Reviewer Notes

Also added docker-preflight: checks buildx/scout are reachable before build — this repo's .envrc sets a project-scoped DOCKER_CONFIG missing cli-plugins/, which otherwise fails several steps later with a confusing error.

Estimated review complexity:

  • Quick (< 100 lines)

Summary by CodeRabbit

  • New Features

    • Added Docker environment checks for required build and security tooling.
    • Docker builds now support dedicated AMD64 and ARM64 runners with clearer platform image tags.
    • Container vulnerability checks now separate unfixed issues and fail on fixable Medium-or-higher vulnerabilities.
  • Bug Fixes

    • Prevented multiple processes from simultaneously using the same database directory.
    • Improved database startup and directory handling across deployment examples.
  • Chores

    • Expanded ignore rules for local database and log data directories.

@JoshuaChi

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 7, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai

coderabbitai Bot commented Aug 7, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: ffc58eca-2087-4678-a336-331da4830c5b

📥 Commits

Reviewing files that changed from the base of the PR and between 2b1601e and e8f4faf.

📒 Files selected for processing (3)
  • Makefile
  • d-lmdb/src/lock.rs
  • d-lmdb/src/lock_test.rs
🚧 Files skipped from review as they are similar to previous changes (3)
  • d-lmdb/src/lock_test.rs
  • d-lmdb/src/lock.rs
  • Makefile

📝 Walkthrough

Walkthrough

The pull request updates Docker release execution and vulnerability checks. It adds exclusive LMDB data-directory locking, updates three-node storage paths, removes legacy directory settings, and broadens ignored directory patterns.

Changes

Docker validation and release workflow

Layer / File(s) Summary
Platform-specific release builds
.github/workflows/docker-release.yml
The workflow assigns AMD64 and ARM64 builds to explicit runners, removes QEMU setup, and uses normalized platform values in image targets.
Docker preflight and vulnerability checks
Makefile
The Makefile validates Docker and its required plugins before verification. Docker Scout reports unfixed vulnerabilities separately and gates only on fixable Medium-or-higher CVEs.

LMDB data-directory locking

Layer / File(s) Summary
Data-directory lock implementation
d-lmdb/src/lib.rs, d-lmdb/src/lock.rs, d-lmdb/src/lock_test.rs
DataDirLock creates and exclusively locks .d-lmdb.lock. Tests cover directory creation, contention, and reacquisition after release.
LMDB startup integration
d-lmdb/src/db.rs
Both startup paths acquire and retain DataDirLock, pass the configured data directory to node startup, and stop overriding raft_config.cluster.db_root_dir.
Example storage and ignore paths
examples/three-node/config/*.toml, .gitignore
Three-node configurations use ./data/* and omit legacy directory settings. Ignore patterns now match data, database, and log directories at any repository depth.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

  • deventlab/d-lmdb#6: It introduced overlapping Docker verification and CVE-check targets that this pull request extends.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main changes to the d-engine data_dir API, Docker release workflow, and local CVE gate.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/10-main-breaking-changes

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/docker-release.yml:
- Line 66: Update the Docker release step that assigns TARGET to avoid
interpolating `${{ env.IMAGE }}` inside the Bash script; pass the value through
the step environment and reference `$IMAGE` instead. Validate IMAGE before
constructing the target if it is mutable, while preserving the existing Docker
image tag format and workflow behavior.

In `@d-lmdb/src/db.rs`:
- Line 62: Acquire the d-lmdb node/data-directory lock in LmdbStorageEngine::new
and LmdbStateMachine::new’s setup path before either custom LMDB environment is
opened or initialized, and hold it across both constructors until startup
proceeds to start_custom or start_node. Reuse the existing lock mechanism or
d-engine entry point that supports a pre-acquired lock.

In `@Makefile`:
- Around line 323-336: Update the retry loop around docker scout cves to capture
each command’s exit status separately from OUT, and after retries fail when the
final status is nonzero by printing OUT and exiting before parsing LINE or
vulnerability counts. Preserve the existing FAILSIG retry behavior and only
continue to the vulnerability-count parsing when the command completed
successfully.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7b7368a1-2244-4b5d-9c4d-fe2af0e90127

📥 Commits

Reviewing files that changed from the base of the PR and between 9da47d6 and 2761cd9.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (8)
  • .github/workflows/docker-release.yml
  • .gitignore
  • Makefile
  • d-lmdb/src/db.rs
  • examples/single-node/config.toml
  • examples/three-node/config/n1.toml
  • examples/three-node/config/n2.toml
  • examples/three-node/config/n3.toml
💤 Files with no reviewable changes (1)
  • examples/single-node/config.toml

Comment thread .github/workflows/docker-release.yml Outdated
Comment thread d-lmdb/src/db.rs
Comment thread Makefile Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
Makefile (1)

256-276: 🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift

Gate each platform image before publishing it.

The release workflow bypasses docker-preflight and docker-cve-check. Add an equivalent check after the smoke test and before Push tested image by digest. Scan d-lmdb-server:compose with ${{ matrix.platform }}; the existing docker-cve-check target rebuilds and scans d-lmdb-server:local instead.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Makefile` around lines 256 - 276, Add a release-workflow check after the
smoke test and before “Push tested image by digest” that scans
d-lmdb-server:compose for the current matrix platform using docker scout, rather
than invoking docker-cve-check, which rebuilds and scans d-lmdb-server:local.
Ensure each platform image is gated before its digest is published.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@d-lmdb/src/lock.rs`:
- Around line 25-30: Update the lock acquisition error mapping around
file.try_lock so only TryLockError::WouldBlock produces the existing Error::Path
contention message; map TryLockError::Error(err) to Error::Io while preserving
the original I/O error.

In `@Makefile`:
- Around line 318-322: Replace the human-readable output parsing after the
docker scout cves invocation with a status-based gate using --only-fixed
--only-severity critical,high,medium --exit-code. Treat exit status 2 as a
vulnerability finding, while treating every other non-zero status as a scanner
failure; remove the LINE summary extraction and related validation.

---

Outside diff comments:
In `@Makefile`:
- Around line 256-276: Add a release-workflow check after the smoke test and
before “Push tested image by digest” that scans d-lmdb-server:compose for the
current matrix platform using docker scout, rather than invoking
docker-cve-check, which rebuilds and scans d-lmdb-server:local. Ensure each
platform image is gated before its digest is published.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 83c857d3-ac69-4b02-8ba1-66fbbf0fa8a1

📥 Commits

Reviewing files that changed from the base of the PR and between 2761cd9 and 2b1601e.

📒 Files selected for processing (7)
  • .github/workflows/docker-release.yml
  • .gitignore
  • Makefile
  • d-lmdb/src/db.rs
  • d-lmdb/src/lib.rs
  • d-lmdb/src/lock.rs
  • d-lmdb/src/lock_test.rs
🚧 Files skipped from review as they are similar to previous changes (1)
  • .gitignore

Comment thread d-lmdb/src/lock.rs Outdated
Comment thread Makefile Outdated
… scout --exit-code, document lock test intent
@JoshuaChi
JoshuaChi merged commit 6d3671f into main Aug 8, 2026
6 checks passed
@JoshuaChi
JoshuaChi deleted the feature/10-main-breaking-changes branch August 8, 2026 08:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant