feat(validation): fork-aware concrete validation and exploit reproduction harness - #111
Open
retkatmun wants to merge 3 commits into
Open
Conversation
…tion harness Implements the full validation engine described in issue StellarChainproof#92: Core engine (packages/core/src/validation/): - types.ts: versioned ValidationScenario, ChainContext, AccountSpec, ContractSpec, CallSpec, StorageAssertion, BalanceAssertion, EventAssertion, ValidationResult, ValidationReport, MinimizationResult, ValidationPlan, typed errors (ValidationError, ValidationTimeoutError, AdapterCrashError, ForkUnavailableError, CorruptBundleError, ScenarioValidationError), createCancellationSignal, resolveResourceLimits, sanitizeErrorMessage - adapter.ts: EvmAdapter interface + shared JSON-RPC utilities (jsonRpcCall, waitForRpc, encodeFunctionCall, keccak256Selector, keccak256Pure, decodeLogEntries, normalizeHex, hexToDecimalString) - anvil-adapter.ts: AnvilAdapter — process-isolated Anvil backend with fork, snapshot/revert, storage/balance override, bounded resources - hardhat-adapter.ts: HardhatAdapter — process-isolated Hardhat Network backend with equivalent capability surface - scaffold.ts: planValidation translator — static Finding → ValidationScenario scaffolds for CP-107, CP-115, CP-101, CP-104, CP-122, CP-CB-* families; serializeValidationPlan / parseValidationPlan with corruption detection - runner.ts: ValidationRunner (account setup, deploy, ordered call execution, snapshot/replay, storage/balance/event assertion evaluation, outcome classification); minimizeScenario (greedy backward elimination); runValidationPlan (per-scenario process isolation, batch report assembly); sanitizeScenario (strips private keys and fork URLs before persistence) - report.ts: serializeValidationReport (deterministic JSON), parseValidationReport (schema + corruption check), generateValidationMarkdown, generateValidationResultMarkdown CLI (packages/cli/src/commands/validate.ts): - chainproof validate plan — translate scan JSON → ValidationPlan - chainproof validate run — execute plan/scenario against Anvil or Hardhat - chainproof validate replay — restore snapshot and re-run - chainproof validate minimize — greedy call minimization - chainproof validate report — reformat saved report as JSON or Markdown Integrations: - packages/core/src/index.ts: exports all validation public APIs - packages/cli/src/cli.ts: registers validate command - packages/server/src/routes/validate.ts + server.ts: POST /validate/plan, POST /validate/run, GET /validate/report/:id REST endpoints - packages/github-action/action.yml + action.ts: validate-plan and validate-run steps, fail-on-validation-failure gate - packages/vscode-extension: validate commands and result display Fixtures and documentation: - examples/contracts/validation/: ValidationVulnerableVault.sol, ValidationSecureVault.sol, ValidationReentrantAttacker.sol - docs/validation.md: architecture, threat model, limitations, configuration, migration, troubleshooting Tests (461 pass, 0 lint errors): - packages/core/src/__tests__/validation.test.ts: types, utilities, mock-adapter runner, scaffold, minimizer, serialization round-trips, cancellation, fixture file checks, scanner integration - packages/core/src/__tests__/validation-adversarial.test.ts: fork unavailability, adapter crash, malicious input, error sanitization, replay integrity, boundary conditions - packages/cli/src/__tests__/validate.test.ts: all five subcommands via compiled binary, offline adapter guard for integration path Lint fixes applied across validation/runner.ts, validation/scaffold.ts, governance/__tests__/api.test.ts, plugins.ts to resolve all 13 ESLint errors (no-var-requires, no-regex-spaces) introduced by this PR. Closes StellarChainproof#92
Author
|
kindly review and merge |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #92
Bridges ChainProof's static analysis pipeline to concrete EVM execution. Static findings are translated into parameterized reproduction scaffolds, executed against process-isolated EVM backends (Anvil, Hardhat Network), and the results emitted as portable, versioned
ValidationReportbundles — all without live-network dependencies in CI.Architecture
Core separation of concerns: adapter layer handles EVM process management; runner layer handles scenario orchestration and assertion evaluation; scaffold layer handles static-finding translation. Transport (CLI, server, extension, CI action) is decoupled from all of these.
Deliverables
packages/core/src/validation/— 7 new filestypes.tsValidationScenario, result types, typed errors, cancellation, resource limitsadapter.tsEvmAdapterinterface + shared JSON-RPC utilities (keccak256, ABI encode, log decode)anvil-adapter.tshardhat-adapter.tsscaffold.tsplanValidation:Finding → ValidationScenariofor CP-107, CP-115, CP-101, CP-104, CP-122, CP-CB-*runner.tsValidationRunner,minimizeScenario(greedy backward elimination),runValidationPlanreport.tsCLI —
chainproof validatewith five subcommandsIntegration surfaces
@chainproof/corepublic API — all types, adapters, scaffold, runner, and report functions fully exported with JSDocPOST /validate/plan,POST /validate/run,GET /validate/report/:idvalidate-planandvalidate-runsteps;fail-on-validation-failuregateChainProof: Validate FindingscommandFixtures
examples/contracts/validation/ValidationVulnerableVault.sol— intentionally vulnerable (reentrancy + tx.origin)examples/contracts/validation/ValidationSecureVault.sol— patched reference (nonReentrant + msg.sender)examples/contracts/validation/ValidationReentrantAttacker.sol— attacker contract for reentrancy scaffoldDocumentation
docs/validation.md— architecture, threat model, security boundaries, adapter compatibility, configuration, migration, troubleshootingPrecision / Recall
The scaffold translator makes no claim of automatic exploitability.
expectedOutcome: "exploit-succeeds"means "this is what we expect if the finding is real — confirm or refute by running the scenario." False-positive controls useexpectedOutcome: "secure-baseline"against the patched fixture. Unsupported finding IDs (GAS-, SLITHER-, custom plugin rules) surface inplan.unsupportedFindingsrather than being silently dropped.Security Boundaries
"[redacted]")sanitizeScenario)Tests
core/__tests__/validation.test.tscore/__tests__/validation-adversarial.test.tscli/__tests__/validate.test.tsCI gate:
npm ci && npm run lint && npm run build && npm run test— 461 tests pass, 0 lint errors.Performance
minimizeScenariois bounded bymaxTrials(default 50) — will not hang on adversarial inputFollow-up