Skip to content

feat(validation): fork-aware concrete validation and exploit reproduction harness - #111

Open
retkatmun wants to merge 3 commits into
StellarChainproof:masterfrom
retkatmun:feat/fork-aware-validation-harness
Open

feat(validation): fork-aware concrete validation and exploit reproduction harness#111
retkatmun wants to merge 3 commits into
StellarChainproof:masterfrom
retkatmun:feat/fork-aware-validation-harness

Conversation

@retkatmun

Copy link
Copy Markdown

Summary

Closes #92

Bridges ChainProof's static analysis pipeline to concrete EVM execution. Static findings are translated into parameterized reproduction scaffolds, executed against process-isolated EVM backends (Anvil, Hardhat Network), and the results emitted as portable, versioned ValidationReport bundles — all without live-network dependencies in CI.


Architecture

Finding → planValidation → ValidationScenario
                              ↓
                     ValidationRunner
                     ├── AnvilAdapter  (spawns anvil subprocess)
                     └── HardhatAdapter (spawns hardhat node subprocess)
                              ↓
                     ValidationResult → ValidationReport
                     ├── serializeValidationReport (deterministic JSON)
                     └── generateValidationMarkdown (PR comments / terminal)

Core separation of concerns: adapter layer handles EVM process management; runner layer handles scenario orchestration and assertion evaluation; scaffold layer handles static-finding translation. Transport (CLI, server, extension, CI action) is decoupled from all of these.


Deliverables

packages/core/src/validation/ — 7 new files

File Responsibility
types.ts Versioned ValidationScenario, result types, typed errors, cancellation, resource limits
adapter.ts EvmAdapter interface + shared JSON-RPC utilities (keccak256, ABI encode, log decode)
anvil-adapter.ts Process-isolated Anvil backend — fork, snapshot/revert, storage overrides, bounded resources
hardhat-adapter.ts Process-isolated Hardhat Network backend — equivalent surface
scaffold.ts planValidation: Finding → ValidationScenario for CP-107, CP-115, CP-101, CP-104, CP-122, CP-CB-*
runner.ts ValidationRunner, minimizeScenario (greedy backward elimination), runValidationPlan
report.ts Deterministic JSON serialization, corruption-checked deserialization, Markdown generation

CLI — chainproof validate with five subcommands

chainproof validate plan   <scan-result.json>   # Finding → scaffold
chainproof validate run    <plan.json>           # Execute against Anvil/Hardhat
chainproof validate replay <result.json>         # Restore snapshot, re-run
chainproof validate minimize <scenario.json>     # Remove redundant calls
chainproof validate report <report.json>         # Reformat as Markdown or JSON

Integration surfaces

  • @chainproof/core public API — all types, adapters, scaffold, runner, and report functions fully exported with JSDoc
  • REST serverPOST /validate/plan, POST /validate/run, GET /validate/report/:id
  • GitHub Actionvalidate-plan and validate-run steps; fail-on-validation-failure gate
  • VS Code extensionChainProof: Validate Findings command

Fixtures

  • examples/contracts/validation/ValidationVulnerableVault.sol — intentionally vulnerable (reentrancy + tx.origin)
  • examples/contracts/validation/ValidationSecureVault.sol — patched reference (nonReentrant + msg.sender)
  • examples/contracts/validation/ValidationReentrantAttacker.sol — attacker contract for reentrancy scaffold

Documentation

  • docs/validation.md — architecture, threat model, security boundaries, adapter compatibility, configuration, migration, troubleshooting

Precision / Recall

The scaffold translator makes no claim of automatic exploitability. expectedOutcome: "exploit-succeeds" means "this is what we expect if the finding is real — confirm or refute by running the scenario." False-positive controls use expectedOutcome: "secure-baseline" against the patched fixture. Unsupported finding IDs (GAS-, SLITHER-, custom plugin rules) surface in plan.unsupportedFindings rather than being silently dropped.


Security Boundaries

  • Fork URLs are never serialized into bundles (replaced with "[redacted]")
  • Private keys are stripped from scenarios before persistence (sanitizeScenario)
  • Error messages are sanitized before logging — no paths, hostnames, or hex keys leak
  • Adapter processes run with OS-level resource limits (timeout + memory cap)
  • No outbound network calls in CI; fork URL must be explicitly supplied

Tests

Suite Coverage
core/__tests__/validation.test.ts Types, utilities, mock-adapter runner, scaffold, minimizer, serialization round-trips, cancellation, fixture checks, scanner integration
core/__tests__/validation-adversarial.test.ts Fork unavailability, adapter crash, malicious oversized input, error sanitization, replay integrity, boundary conditions
cli/__tests__/validate.test.ts All five subcommands via compiled binary; adapter guard skips live EVM tests when anvil absent

CI gate: npm ci && npm run lint && npm run build && npm run test461 tests pass, 0 lint errors.


Performance

  • Each scenario runs in a fresh adapter process (full isolation)
  • minimizeScenario is bounded by maxTrials (default 50) — will not hang on adversarial input
  • Scaffold translation is O(n findings) with no external calls

Follow-up

  • Live-network fork integration tests (require Anvil on CI runner, tracked separately)
  • SARIF emission of validation results for GitHub Security tab

retkatmun and others added 2 commits August 29, 2026 19:21
…tion harness

Implements the full validation engine described in issue StellarChainproof#92:

Core engine (packages/core/src/validation/):
- types.ts: versioned ValidationScenario, ChainContext, AccountSpec,
  ContractSpec, CallSpec, StorageAssertion, BalanceAssertion, EventAssertion,
  ValidationResult, ValidationReport, MinimizationResult, ValidationPlan,
  typed errors (ValidationError, ValidationTimeoutError, AdapterCrashError,
  ForkUnavailableError, CorruptBundleError, ScenarioValidationError),
  createCancellationSignal, resolveResourceLimits, sanitizeErrorMessage
- adapter.ts: EvmAdapter interface + shared JSON-RPC utilities (jsonRpcCall,
  waitForRpc, encodeFunctionCall, keccak256Selector, keccak256Pure,
  decodeLogEntries, normalizeHex, hexToDecimalString)
- anvil-adapter.ts: AnvilAdapter — process-isolated Anvil backend with
  fork, snapshot/revert, storage/balance override, bounded resources
- hardhat-adapter.ts: HardhatAdapter — process-isolated Hardhat Network
  backend with equivalent capability surface
- scaffold.ts: planValidation translator — static Finding → ValidationScenario
  scaffolds for CP-107, CP-115, CP-101, CP-104, CP-122, CP-CB-* families;
  serializeValidationPlan / parseValidationPlan with corruption detection
- runner.ts: ValidationRunner (account setup, deploy, ordered call execution,
  snapshot/replay, storage/balance/event assertion evaluation, outcome
  classification); minimizeScenario (greedy backward elimination);
  runValidationPlan (per-scenario process isolation, batch report assembly);
  sanitizeScenario (strips private keys and fork URLs before persistence)
- report.ts: serializeValidationReport (deterministic JSON), parseValidationReport
  (schema + corruption check), generateValidationMarkdown, generateValidationResultMarkdown

CLI (packages/cli/src/commands/validate.ts):
- chainproof validate plan   — translate scan JSON → ValidationPlan
- chainproof validate run    — execute plan/scenario against Anvil or Hardhat
- chainproof validate replay — restore snapshot and re-run
- chainproof validate minimize — greedy call minimization
- chainproof validate report — reformat saved report as JSON or Markdown

Integrations:
- packages/core/src/index.ts: exports all validation public APIs
- packages/cli/src/cli.ts: registers validate command
- packages/server/src/routes/validate.ts + server.ts: POST /validate/plan,
  POST /validate/run, GET /validate/report/:id REST endpoints
- packages/github-action/action.yml + action.ts: validate-plan and
  validate-run steps, fail-on-validation-failure gate
- packages/vscode-extension: validate commands and result display

Fixtures and documentation:
- examples/contracts/validation/: ValidationVulnerableVault.sol,
  ValidationSecureVault.sol, ValidationReentrantAttacker.sol
- docs/validation.md: architecture, threat model, limitations,
  configuration, migration, troubleshooting

Tests (461 pass, 0 lint errors):
- packages/core/src/__tests__/validation.test.ts: types, utilities,
  mock-adapter runner, scaffold, minimizer, serialization round-trips,
  cancellation, fixture file checks, scanner integration
- packages/core/src/__tests__/validation-adversarial.test.ts: fork
  unavailability, adapter crash, malicious input, error sanitization,
  replay integrity, boundary conditions
- packages/cli/src/__tests__/validate.test.ts: all five subcommands via
  compiled binary, offline adapter guard for integration path

Lint fixes applied across validation/runner.ts, validation/scaffold.ts,
governance/__tests__/api.test.ts, plugins.ts to resolve all 13 ESLint errors
(no-var-requires, no-regex-spaces) introduced by this PR.

Closes StellarChainproof#92
@retkatmun

Copy link
Copy Markdown
Author

kindly review and merge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement Fork-Aware Concrete Validation and Exploit Reproduction Harnesses

1 participant