v3.0.0 PR - #3
Merged
Merged
Conversation
- Updated DecentDB dependency reference in pubspec.yaml to v2.14.0. - Adjusted engine version in logging tests and metadata contracts to reflect the new version. - Enhanced schema models to support new metadata fields introduced in DecentDB v2.14.0. - Implemented Parquet export infrastructure with cursor-based streaming to handle large datasets. - Added Parquet export dialog for user interaction during export operations. - Introduced structured error diagnostics in the bridge failure mapping to improve error handling. - Updated release plan and ADRs to reflect changes and new features for version 2.1.0.
- Introduced `DatabaseOpenSettings` to manage database connection profiles and plan cache settings. - Implemented `DecentDbDoctorService` for running diagnostics on databases using CLI and sys.* views as fallback. - Created `DecentDbDoctorDialog` for displaying diagnostic results in the UI. - Added tests for typed batch classification and SQL vocabulary. - Updated ADRs for DecentDB v2.17.0 upgrade and diagnostics boundary. - Introduced constants for expected DecentDB version in tests.
- Introduced `processCoordinationTimeoutMs` and `openBridgeTimeoutMs` in `DatabaseOpenSettings` to allow configuration of database open timeouts. - Updated `AppConfig` to support serialization and deserialization of new timeout settings in TOML format. - Enhanced `DecentDbBridge` to resolve effective timeouts based on configuration and environment variables. - Implemented error handling for `DDB_ERR_TIMEOUT` in `DecentDbMigrationService` with user-friendly explanations and suggestions. - Added tests for new timeout features and validation of engine version mismatches. - Updated README with troubleshooting steps for database opening issues related to timeouts and sidecar files.
… short-circuiting
There was a problem hiding this comment.
Pull request overview
This PR upgrades Decent Bench to DecentDB v2.17.0 and ships a breaking, mandatory migration path (format 13 → 14), while also adopting a broad set of v2.15–v2.17 engine capabilities (doctor/diagnostics, new sys.* metrics, open-option profiles/plan-cache settings, import batching, schema/explain enhancements) and updating supporting docs/tests.
Changes:
- Bump pinned DecentDB engine/tag to v2.17.0 and app version to 3.0.0+1, with updated changelog/docs/ADRs for the breaking migration.
- Add guided in-place migration infrastructure + coordination-timeout guidance + new Doctor panel (CLI-first with sys.* fallback).
- Adopt engine capability additions across imports (typed batch), schema models/UI, SQL vocabulary/formatter, EXPLAIN visualization, and operational metrics.
Reviewed changes
Copilot reviewed 66 out of 67 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| THIRD_PARTY_NOTICES.md | Updates dependency attributions for engine bump |
| README.md | Updates engine version references + adds troubleshooting docs |
| design/SPEC.md | Updates export deferment history and future backlog |
| design/RELEASE_PLAN_2.1.0.md | Adds historical phased release plan doc |
| design/adr/0063-worker-isolate-restart-on-timeout.md | Documents worker restart/busy short-circuit decision |
| design/adr/0062-database-performance-profile-open-options.md | Documents new open-options surfaced in config/UI |
| design/adr/0061-decentdb-doctor-diagnostics-boundary.md | Documents CLI doctor primary + sys.* fallback boundary |
| design/adr/0060-decentdb-2-17-format-14-guided-migration.md | Documents mandatory format-14 migration contract |
| design/adr/0059-structured-decentdb-error-diagnostics.md | Documents structured diagnostics mapping decision |
| design/adr/0058-schema-snapshot-metadata-parity-decentdb-2_14.md | Documents schema snapshot parity expansion |
| design/adr/0057-column-reordering-ux-contract.md | Documents column reordering UX/persistence contract |
| design/adr/0056-parquet-export-implementation-strategy.md | Documents Parquet export strategy/options |
| design/adr/0025-decentdb-git-dependency-rationale.md | Updates pinned git ref discussion to v2.17.0 |
| design/adr/0003-pinned-decentdb-sql-capability-baseline.md | Updates pinned capability baseline to v2.17.0 |
| CHANGELOG.md | Adds 3.0.0 breaking release notes + features |
| apps/decent-bench/test/widget_test.dart | Centralizes engine version assertions + updates grid state expectations |
| apps/decent-bench/test/support/fakes.dart | Extends fake gateway/models for new engine features |
| apps/decent-bench/test/support/decentdb_test_constants.dart | Adds shared expected engine version constant |
| apps/decent-bench/test/features/workspace/presentation/shell/menu_command_contract.dart | Adds Tools → Database Doctor to menu contract |
| apps/decent-bench/test/features/workspace/infrastructure/decentdb_open_timeout_test.dart | Tests new open timeout + TOML round-trips |
| apps/decent-bench/test/features/workspace/infrastructure/decentdb_migration_service_test.dart | Adds migrate-in-place + timeout explanation tests |
| apps/decent-bench/test/features/workspace/infrastructure/decentdb_engine_version_guard_test.dart | Adds engine/native mismatch warning tests |
| apps/decent-bench/test/features/workspace/infrastructure/decentdb_doctor_service_test.dart | Adds doctor CLI parsing + sys.* fallback tests |
| apps/decent-bench/test/features/workspace/infrastructure/decentdb_bridge_worker_recovery_test.dart | Adds worker-busy + restart-on-timeout orchestration tests |
| apps/decent-bench/test/features/workspace/infrastructure/decentdb_bridge_smoke_test.dart | Extends sys.* view coverage + versioned test names |
| apps/decent-bench/test/features/workspace/infrastructure/app_config_store_test.dart | Tests new database_open config round-trips |
| apps/decent-bench/test/features/workspace/domain/workspace_metadata_contract_test.dart | Updates tooling metadata engine version expectation |
| apps/decent-bench/test/features/workspace/domain/sql_vocabulary_test.dart | Tests fulltext/ALTER INDEX vocabulary + autocomplete |
| apps/decent-bench/test/features/workspace/domain/sdk_generation_test.dart | Updates generated SDK engine version constant test |
| apps/decent-bench/test/features/workspace/domain/schema_models_test.dart | Adds tests for new schema snapshot model fields |
| apps/decent-bench/test/features/workspace/domain/query_phase_models_test.dart | Updates bridge failure test labeling/version |
| apps/decent-bench/test/features/workspace/domain/explain_plan_visualization_test.dart | Adds tests for new EXPLAIN operators + cost parsing |
| apps/decent-bench/test/features/workspace/application/workspace_controller_test.dart | Tests describeQueryContract timeout/busy fallback behavior |
| apps/decent-bench/test/features/import/infrastructure/typed_batch_classification_test.dart | Adds tests for typed-batch classification helpers |
| apps/decent-bench/test/app/logging/app_logger_test.dart | Updates logged engine_version assertions |
| apps/decent-bench/README.md | Updates app-specific engine/version docs |
| apps/decent-bench/pubspec.yaml | Bumps app version + engine ref + dependency updates |
| apps/decent-bench/pubspec.lock | Updates resolved dependency graph for new versions |
| apps/decent-bench/lib/main.dart | Installs engine version mismatch guard at startup |
| apps/decent-bench/lib/features/workspace/presentation/workspace_screen.dart | Adds doctor UI, schema metadata surfacing, timeout help, in-place migration flow |
| apps/decent-bench/lib/features/workspace/presentation/shell/schema_explorer_pane.dart | Adds index-kind icons + AUTOINCREMENT badge |
| apps/decent-bench/lib/features/workspace/presentation/shell/results_pane.dart | Renders estimated cost chips + wider plan badges |
| apps/decent-bench/lib/features/workspace/presentation/shell/app_menu_bar.dart | Registers Tools → Database Doctor menu command |
| apps/decent-bench/lib/features/workspace/presentation/export_results_parquet_dialog.dart | Adds Parquet export dialog UI |
| apps/decent-bench/lib/features/workspace/presentation/decentdb_migration_dialog.dart | Adds in-place migration confirmation dialog |
| apps/decent-bench/lib/features/workspace/presentation/decentdb_doctor_dialog.dart | Adds doctor report dialog UI |
| apps/decent-bench/lib/features/workspace/infrastructure/sqlite_import_support.dart | Adds executeBatchTyped fast path for SQLite import |
| apps/decent-bench/lib/features/workspace/infrastructure/sql_dump_import_support.dart | Adds executeBatchTyped fast path for SQL dump import |
| apps/decent-bench/lib/features/workspace/infrastructure/excel_import_support.dart | Adds executeBatchTyped fast path for Excel import |
| apps/decent-bench/lib/features/workspace/infrastructure/decentdb_migration_service.dart | Adds coordination-timeout helpers + migrateInPlace implementation |
| apps/decent-bench/lib/features/workspace/infrastructure/decentdb_doctor_service.dart | Implements doctor CLI runner + sys.* fallback |
| apps/decent-bench/lib/features/workspace/domain/sql_vocabulary.dart | Adds fulltext/ALTER INDEX keywords + functions |
| apps/decent-bench/lib/features/workspace/domain/sql_formatter.dart | Adds new newline clauses for formatter |
| apps/decent-bench/lib/features/workspace/domain/schema_models.dart | Extends schema snapshot domain models (rowCount/FKs/view deps/index includes/fresh/autoinc) |
| apps/decent-bench/lib/features/workspace/domain/query_result_models.dart | Adds ParquetExportResult model |
| apps/decent-bench/lib/features/workspace/domain/query_phase_models.dart | Adds engineVersionWarning to DatabaseSession |
| apps/decent-bench/lib/features/workspace/domain/explain_plan_visualization.dart | Adds multi-word operator parsing + estimated cost extraction |
| apps/decent-bench/lib/features/workspace/domain/database_open_settings_model.dart | Adds DatabaseOpenSettings model + open options fragment builder |
| apps/decent-bench/lib/features/workspace/domain/app_config.dart | Adds [database_open] TOML serialization + parsing |
| apps/decent-bench/lib/features/workspace/application/workspace_controller.dart | Threads database_open into open, adds describe-contract timeout fallback, adds maintenance actions + doctor/sys view query helper |
| apps/decent-bench/lib/features/import/infrastructure/typed_batch_classification.dart | Adds shared typed-batch type/signature normalization helpers |
| apps/decent-bench/lib/features/import/infrastructure/import_execution_service.dart | Uses executeBatchTyped for generic import when possible |
| apps/decent-bench/lib/features/export/infrastructure/parquet_exporter.dart | Adds Parquet export infrastructure placeholder |
| apps/decent-bench/lib/app/headless_quality_runner.dart | Adds legacy-format actionable hint for headless quality |
| apps/decent-bench/lib/app/headless_import_runner.dart | Adds legacy-format actionable hint for headless import |
| .kilo/plans/1785879082050-decentdb-2-17-upgrade.md | Adds detailed v2.17 upgrade plan and validation checklist |
Comment on lines
+114
to
+121
| static bool isCoordinationTimeoutMessage(String? message) { | ||
| final normalized = message?.toLowerCase() ?? ''; | ||
| return normalized.contains('ddb_err_timeout') || | ||
| normalized.contains('err_timeout') || | ||
| normalized.contains('writer lock') || | ||
| normalized.contains('timed out') || | ||
| normalized.contains('timeout'); | ||
| } |
Comment on lines
+134
to
+150
| final coordNote = databasePath == null | ||
| ? 'A stale <database>.ddb.coord file from a previous run can also ' | ||
| 'cause this. Closing other DecentDB-backed processes and ' | ||
| 'removing the .coord sidecar (it is rebuildable) usually ' | ||
| 'clears it. To raise the engine wait, set ' | ||
| 'process_coordination_timeout_ms in [database_open] of ' | ||
| 'config.toml. If the bridge wrapper times out first, also ' | ||
| 'raise open_bridge_timeout_ms (or set the ' | ||
| 'DECENT_BENCH_OPEN_TIMEOUT_MS environment variable).' | ||
| : 'A stale "$databasePath.ddb.coord" sidecar file from a previous ' | ||
| 'run can also cause this. Closing other DecentDB-backed ' | ||
| 'processes and removing the .coord sidecar (it is rebuildable) ' | ||
| 'usually clears it. You can also raise the wait by setting ' | ||
| 'process_coordination_timeout_ms in the [database_open] ' | ||
| 'section of your config.toml (engine side). If the bridge ' | ||
| 'wrapper times out before the engine replies, also raise ' | ||
| 'open_bridge_timeout_ms (or DECENT_BENCH_OPEN_TIMEOUT_MS).'; |
Comment on lines
+297
to
+305
| static String _randomToken() { | ||
| final mix = DateTime.now().microsecondsSinceEpoch ^ | ||
| DateTime.now().microsecondsSinceEpoch; | ||
| final hex = mix.toRadixString(16); | ||
| if (hex.length >= 10) { | ||
| return hex.substring(0, 10); | ||
| } | ||
| return hex.padLeft(10, '0'); | ||
| } |
Comment on lines
+206
to
+214
| final cliPath = | ||
| await (_cliPathResolver ?? DecentDbCliResolver().resolve)(); | ||
| final args = buildDoctorArguments( | ||
| databasePath: normalizedPath, | ||
| checks: checks, | ||
| verifyAllIndexes: verifyAllIndexes, | ||
| verifyIndexes: verifyIndexes, | ||
| maxIndexVerify: maxIndexVerify, | ||
| ); |
Comment on lines
+10
to
+16
| class ParquetExportResult { | ||
| const ParquetExportResult({ | ||
| required this.rowCount, | ||
| required this.path, | ||
| this.schemaFingerprint, | ||
| this.warnings = const <String>[], | ||
| this.duration, |
Comment on lines
+501
to
+503
| /// True when every column in [columns] can be expressed in the typed-batch | ||
| /// signature (i/b/f/t). UUID columns are coerced to the `t` signature so | ||
| /// they can ride the typed path, but require text form in the row values. |
Comment on lines
+4
to
+19
| void main() { | ||
| test('INTEGER / DOUBLE / TEXT map to i/f/t; BOOLEAN is excluded in v2.17', | ||
| () { | ||
| expect(typedBatchSignatureChar('INTEGER'), 'i'); | ||
| expect(typedBatchSignatureChar('BIGINT'), 'i'); | ||
| expect( | ||
| typedBatchSignatureChar('BOOLEAN'), | ||
| isNull, | ||
| reason: | ||
| 'The Dart binding for v2.17 only accepts i/t/f; BOOLEAN rides the ' | ||
| 'bindAll path.', | ||
| ); | ||
| expect(typedBatchSignatureChar('DOUBLE PRECISION'), 'f'); | ||
| expect(typedBatchSignatureChar('TEXT'), 't'); | ||
| expect(typedBatchSignatureChar('VARCHAR(64)'), 't'); | ||
| }); |
…CLI resolution fallback, duplicate class, doc comment, sqlite3 version, test formatting Co-authored-by: sphildreth <193334+sphildreth@users.noreply.github.com>
Co-authored-by: sphildreth <193334+sphildreth@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces a major upgrade to Decent Bench, centering on the DecentDB v2.17.0 engine update and a mandatory one-way migration of all user databases to the new on-disk format (format 14). The release is versioned as 3.0.0 to reflect this breaking change. In addition to the migration path, the update adds several Tier 1 engine features, a new Doctor panel for diagnostics, improvements to import/export, and enhanced schema and workspace features. The changelog is updated to document all major changes and their rationale.
Breaking changes and migration:
New and enhanced engine features:
profile), plan cache management, and 8 newsys.*operational metric views. [1] [2]Import, export, and schema improvements:
executeBatchTypedAPI when possible, improving throughput and reliability; adds a shared column classification module. [1] [2]Changelog and documentation:
CHANGELOG.mdfor 3.0.0, including all breaking changes, new features, and rationale for the version bump. Updates release links and references. [1] [2]For full details, see the migration plan and ADR references in the updated documentation.