v2.0.0 work - #2
Merged
Merged
Conversation
…ucture Refactor the `DecentDbBridge` worker isolate into a `_BridgeWorkerState` class to improve encapsulation and maintainability. This change replaces procedural dispatch with dedicated handler methods for database operations and encapsulates database/cursor state. Additionally, improve the test suite and CI pipeline by: - Grouping tests logically using `group()` blocks across unit, widget, and integration tests. - Introducing fixture helpers to reduce boilerplate in test setup. - Enabling coverage collection and Codecov integration in the GitHub Actions workflow. - Updating the CI cache key strategy for Flutter dependencies. Includes a new design document outlining future improvement goals.
Introduce a series of Architecture Decision Records (ADRs) to define the technical direction for upcoming features, including inline table editing, workspace management, and data visualization. Additionally, revise the `FUTURE_WINS.md` roadmap to establish a ranked backlog prioritized by user impact. This update clarifies the project's strategic direction and provides a structured plan for vNext development. ADRs added: - 0028: Inline table data editor - 0029: Workspace project file and query library - 0030: Charting library and visualization contract - 0031: Parquet and Excel export dependency strategy - 0032: Database snapshot and safe run - 0033: Computed column transforms during import
Introduce the Command Palette (`Ctrl+Shift+P`) to allow users to quickly find and execute any registered menu command. The palette features a fuzzy-search interface, keyboard navigation, and displays command metadata including icons and keyboard shortcuts. - Create `CommandPalette` widget and integrate into `WorkspaceScreen` - Add `view_command_palette` to default keybindings and command registry - Implement fuzzy search filtering over the `MenuCommandRegistry` - Include widget tests for command filtering and execution flow - Update changelog and project roadmap to reflect implementation status
Implement comprehensive error catching and logging for configuration loading, state persistence, and platform integrations. This ensures that failures in non-critical operations (like native menu detection or config parsing) are recorded without crashing the application, while providing better diagnostic context for state save failures. - Add logging for workspace state persistence failures - Inject logger into shell controller to monitor preference updates - Provide fallback to default config on parsing errors in AppConfigStore - Log platform-specific and archive extraction errors in the UI layer
…tracts Enable first-class support for the DecentDB v2.5.1 engine by surfacing native semantic types, tooling metadata, and query contracts. This update allows the workbench to provide type-aware cell rendering, schema-drift warnings via fingerprints, and structured parameter input. - Support ENUM, IPADDR, MACADDR, temporal, and spatial types - Add query contract integration for parameter hints and result metadata - Implement persistent query history with a new results subtab - Add streaming JSON and NDJSON export functionality - Introduce SQL risk assessment for destructive DML operations - Upgrade DecentDB dependency to v2.5.1 and update roadmap docs BREAKING CHANGE: Persisted workspace state bumped to schema version 3.
Update all application manifests, metadata, and runner configurations to reflect the version 2.0.0 milestone. This release finalizes the integration of DecentDB v2.5.1 features and stabilizes the integration testing infrastructure to ensure reliable CI runs. - Bump version to 2.0.0 in pubspec.yaml, app_metadata.dart, and native runners - Update theme compatibility constraints to support the 2.x.x range - Refactor integration tests to centralize setup and improve teardown reliability - Synchronize CHANGELOG, README, and design docs with the 2.0.0 shipping state
Synchronize documentation with the latest application capabilities, covering new import formats, workbench enhancements, and CLI flags. - Add NDJSON and archive-wrapped sources to supported formats - Detail the command palette and per-tab history in SQL Workbench - Add section for SQL risk classification and query contracts - Update CLI usage examples with --help and --version - Remove obsolete status section from table of contents
…tructure Introduce the foundation for DecentDB native branching and snapshot management, including UI integration for the branch workbench and state tracking in the controller. - Add WorkspaceBranchInfo, WorkspaceSnapshotInfo, and WorkspaceBranchDiff domain models - Extend WorkspaceDatabaseGateway with branch and snapshot operations - Integrate branch workbench controls into the app menu and toolbar - Add branch label to the status bar for active context visibility - Enhance spatial data handling with WKT and GeoJSON copy options in the results grid - Improve SQL parameter input with type-aware validation based on native type descriptors - Update design documents and ADRs to reflect revised feature priorities This implementation provides the UI and controller logic for the branching workflow while the bridge layer handles cases where the underlying Dart bindings for native operations are pending.
Introduce a comprehensive suite of features for data introspection, transformation, and export to finalize the v2.0.0 workbench foundation. - Add result charts (bar, line, pie, scatter) and EXPLAIN plan visualization - Implement row-level import transforms for filters and computed columns - Add Excel (.xlsx) export support using a native XML-based writer - Introduce column and database statistics for deeper data analysis - Implement a persistent saved-query library and project manifest system - Add schema-first SDK generation prototype for TypeScript output - Support import/export profiles for repeatable headless workflows - Enable type-aware inline table editing for single-table results - Update design documentation and ADRs to reflect implemented v2.0.0 logic
Introduce design documentation and architectural decision record for a read-only ERD viewer. This feature facilitates schema discovery and navigation without enabling DDL mutations. - Add ADR-0035 for read-only ERD viewer and image export - Update PRD and SPEC to include ERD viewer requirements - Create ERD UI implementation plan - Reprioritize ERD viewer in future roadmap documentation
Expand the ERD technical specification to address edge cases in foreign key visualization and UI scalability. - Specify synthetic grouping logic for multi-column foreign keys - Define column truncation and responsive display density rules - Add placeholder node behavior for missing schema references - Limit initial scope to table objects, excluding views - Detail search filtering and empty state UI requirements
…ints Revise ERD implementation guidance to prioritize shipping a usable first slice over pursuing production-grade graph layout. Introduce conservative raster export size limits to prevent out-of-memory crashes on large diagrams. - Replace aspirational Sugiyama layout with simple deterministic layered-grid approach scoped to initial slice - Add timebox boundary for layout complexity; defer to dependency ADR if custom layout becomes untenable - Enforce 8192 px and 64 megapixel limits on image export canvas allocation - Clarify fallback strategies for export scale violations and future tiled export - Update ADR and spec to reflect pragmatic first-implementation constraints
… patterns Establish comprehensive implementation guidance for ERD viewer covering canvas rendering strategies, node/edge interaction models, and performance optimization techniques. Document best practices for handling large diagrams and user feedback mechanisms to support the initial production release.
…h image export Add complete ERD viewer implementation including deterministic schema-relationship graph generation, layered-grid layout algorithm, navigation-pane UI with search and neighborhood mode, workspace menu/command integration, and PNG/JPG raster export with safe pixel limits. Update dependencies to include image package v4.3.0 for raster encoding, integrate with existing table-preview navigation, and mark all six implementation phases as complete in the design guide.
…ormat-version upgrade Implement safe copy-based migration for legacy DecentDB files that fail to open due to unsupported format versions. The workflow detects format-version errors, runs the official `decentdb-migrate` tool into a new destination, and opens the migrated database after success. Add migration service, dialog UI, native tool resolution from PATH/environment/bundle/release asset, and staging helper support for packaging. Update workspace controller to gracefully handle missing tooling metadata. Extend native library resolver and release asset logic to support migration tool discovery and caching alongside the native library.
…RD layout - Update `decentdb_bridge` to resolve foreign keys defined at the table level by scanning the table's foreign key constraints when serializing columns. - Increase ERD node heights for wide and medium densities to improve visibility of column lists. - Add smoke tests for table-level foreign key resolution. - Add widget tests for ERD table card column overflow behavior.
Update the DecentDB Dart binding and runtime dependency from v2.5.1 to v2.6.0. This alignment includes updating internal documentation, ADRs, and version tracking across the repository. A new enhancement plan is added to track the adoption of v2.6.0 surfaces such as queued writes, operational metrics, and SQL/PRAGMA parity. Test fixtures and expectations are also updated to reflect the new engine version.
Introduce ADR-0038 through ADR-0042 to define the technical approach for integrating new DecentDB v2.6.0 capabilities, including queued writes, the local web console companion process, reactive stream management, sync diagnostics, and the Lua extension trust model. The enhancement plan is updated with a phase map that links these architectural decisions to the project's implementation workstreams, formalizing the roadmap for feature adoption following the initial dependency alignment.
Introduce support for DecentDB v2.6.0 operational metrics, optional queued write execution for table edits, and a managed Web Console companion process with CLI resolution. Update the SQL editor with v2.6 parity for PRAGMA metadata, system schema views, and schema qualifiers. Mark ADR-0038 through ADR-0042 as accepted and transition the enhancement plan to implemented.
Apply project-specific branding to the Linux, macOS, and Windows desktop runners. This includes implementing runtime asset resolution for the Linux GTK window icon and updating the native asset collections for macOS and Windows. Also, remove the implemented DECENTDB_2_6_ENHANCEMENT_PLAN.md and update the changelog to reflect the branding changes.
Improve the handling of DecentDB v2.6 operational metrics by detecting Dart binding limitations regarding 'sys.*' inspection views. Instead of reporting individual access errors for each metric, the bridge now collapses these into a single informative note explaining the current paging path restriction. This prevents the diagnostics view from being cluttered with redundant error messages when running against environments where the Dart binding cannot yet page results from the system schema. - Add detection for schema-qualified boundary errors in the bridge. - Implement a consolidated summary view for unavailable system metrics. - Add smoke test to verify collapsing behavior. - Update CHANGELOG.md to reflect improved diagnostics handling.
…ment Enable comprehensive database branching and snapshot workflows via the DecentDB bridge. This includes support for creating, listing, deleting, and diffing branches, as well as executing queries against specific branches. Additionally, implement standard file lifecycle operations (Save, Save As, Close) to manage workspace state, query libraries, and configurations with explicit durability. - Connect UI commands to native branch and snapshot APIs. - Implement workspace state persistence and file duplication logic. - Resolve DecentDB system metrics using paged inspection views. - Switch DecentDB dependency to a local path for integrated builds. - Add architectural documentation and menu command contract tests.
Implement comprehensive testing for application-menu keybindings to ensure every default shortcut targets a valid command and respects activation state. - Verify shortcut-to-command mappings in the menu contract audit. - Add UI smoke tests for global shortcut dispatch (Ctrl+Shift+P, F1, Ctrl+Q) within the production shell. - Enforce constraints against duplicate accelerator assignments and deferred command bindings. - Remove obsolete ERD UI implementation and planning documents to clean up the design directory.
Introduce a comprehensive Help Center accessible via Help > Documentation or the F1 key. This replaces the previous static documentation dialog with a dynamic, searchable interface powered by bundled Markdown articles. - Add `flutter_markdown_plus` dependency for rich content rendering. - Implement Help Center UI with support for local search and topic navigation. - Integrate bundled Markdown assets into the application. - Add domain, infrastructure, and presentation layers for help content management. - Update release guidelines to include documentation maintenance procedures. - Add widget tests to verify Help Center accessibility and navigation.
Reduces visual clutter by replacing the expansive command toolbar with a compact set of essential actions. Most commands are moved to the application menus or the command palette. - Consolidate import commands into a single `PopupMenuButton`. - Limit toolbar buttons to New, Open, Import, and Commands. - Update widget tests to reflect the new UI structure and use keys for more robust selection. - Update CHANGELOG to document the UI simplification.
…ation Refreshes the product and engineering priority index by consolidating suggestions from multiple coding agents. - Reorganizes the document structure to focus on agent-driven feedback. - Implements new consolidation rules to group overlapping ideas and workflow-based enhancements. - Updates the purpose and review basis to reflect the new roadmap methodology. - Removes deprecated v2.0.0 status in favor of an agent feedback consolidation refresh.
Introduce architectural decision records and a validation plan for data quality profiling. - Add WIN_DATA_QUALITY_PROFILING_VALIDATION_PLAN.md - Add ADR for data quality persistence and project contracts - Add ADR for data quality execution and paging contracts - Add ADR for data quality report privacy contracts
…hive wrappers Update READMEs, help assets, and design documents to reflect expanded import capabilities, including new archive wrapper support (tar/gzip/bzip2) and a more comprehensive list of supported delimited and structured formats. - Update project READMEs with expanded import format lists - Refine help center documentation for delimited, JSON, XML, and HTML imports - Update import support plan status to reflect completed features - Clarify archive wrapper routing logic in technical documentation
Enables the data quality suite to analyze the output of saved queries by materializing them into temporary tables. High-latency validation rules like regex and similarity checks are now executed in background isolates to prevent UI blocking. - Add "Run Quality Profile" button to successful import summaries - Persist preferred quality profiles and modes in workspace files - Implement temporary table lifecycle for query profiling - Offload non-SQL checks to multi-threaded background workers - Expand test coverage with dashboard integration and domain unit tests - Prune completed roadmap items from documentation
Adds regression tests for isolate-backed validation rules to ensure they correctly persist violation details for paged retrieval. Marks the Data Quality Profiling & Validation suite as 100% complete in the tracking documentation. - Add test case for successful validation runs with zero failures - Validate that regex and near-duplicate rules support paged detail loading - Update design document checklist to reflect full implementation status - Document final performance strategies for large-table validation flows
Refine DataQualityController state management to provide accurate lifecycle tracking and robust data synchronization. - introduce canCancelRun getter for precise UI control - track execution via cancellation tokens instead of persisted status - synchronize current run with repository state after completion - detect stale 'running' statuses in freshness computation - add error handling for result persistence failures
Reorganize import system documentation to align with the modular architecture and improve clarity for both users and developers. - delete design/IMPORT_FORMATS.md and design/IMPORT_SUPPORT_PLAN.md - migrate the supported format list to the bundled Help Center - unify the future format backlog in WIN_IMPORT_FORMAT_EXPANSION_PLAN.md - update ADRs, PRD, and project READMEs to reflect the new structure - refactor documentation validation tests to target help assets
…ation Add a "Testing Tooling Policy" to the import expansion plan to guide the creation of test fixtures using containerized tools. - mandate Docker for format-specific tools like PostgreSQL and DuckDB - preserve dependency-free execution for standard test runs - require pinned image tags and documented cleanup procedures - define preferred tooling for Parquet, ODS, and SQL dump formats
Update the import format implementation plan to clarify prioritization logic and document strategies for new target formats. - add implementation complexity to factors influencing wave movement - clarify Wave 1 and Wave 2 definitions to distinguish between bounded implementation paths and those requiring ADRs or worker decisions - expand the fixture generation table with specific tooling strategies for Microsoft Access, DBF/FoxPro, Markdown, YAML, and XZ/7-Zip - bump last-reviewed timestamp to 2026-05-23
Complete the implementation of several Wave 1 import formats, moving them from planned/investigate status to fully supported features. This includes new generic wizards, specialized infrastructure support, and UI integration for clipboard-based ingestion. - add support for clipboard tables (TSV, CSV, Markdown, HTML) - implement fixed-width text, ODS, and SpreadsheetML importers - add structured log support for JSON streams and common templates - support Markdown pipe tables and HAR browser archives - expand SQL dump support to include PostgreSQL plain dumps and COPY - implement XZ archive extraction and routing - update help documentation and internal module catalog - add ADR-0054 for Parquet runtime requirements
Introduce cross-platform window management to preserve application position, size, and display state (normal, maximized, or fullscreen) between sessions. This ensures the workspace environment is consistently restored on Linux, macOS, and Windows. - introduce native platform channels for window geometry synchronization - enable background persistence of window state to the TOML configuration - bump configuration schema to version 3 to accommodate window metadata - add ADR-0055 documenting the window placement persistence design - update changelog and project specifications for workspace persistence - ensure state restoration occurs during early application bootstrap - implement periodic and lifecycle-triggered state capture in Dart
Refactor the monolithic `WorkspaceController` and `WorkspaceDatabaseGateway` to improve maintainability and testability by introducing specialized controllers and interfaces. - Extract `BranchController` to manage branch and snapshot state and workflow logic. - Decompose `WorkspaceDatabaseGateway` into single-responsibility interfaces: `DatabaseLifecycleGateway`, `SchemaIntrospectionGateway`, `QueryExecutionGateway`, `ExportGateway`, `ImportGateway`, and `BranchWorkflowGateway`. - Update `WorkspaceController` to delegate branch operations to the new `BranchController`. - Upgrade `decentdb` dependency to v2.7.0 via git reference. - Implement a global error boundary in `main.dart` to capture unhandled Flutter and asynchronous errors. - Update documentation and changelog to reflect architectural changes and dependency upgrades.
Improve responsiveness and reliability of branch-related operations by introducing specific timeouts and non-blocking state updates. - Introduce `_branchRequestTimeout` (10s) to `DecentDbBridge` requests to prevent long-running branch operations from hanging indefinitely. - Use `unawaited` for `refreshBranchState` in `WorkspaceController` to prevent blocking the main workspace restoration flow. - Update `DecentDbBridge` methods to utilize the new branch-specific timeout.
Introduce a configurable timeout mechanism for all database operations to prevent indefinite hangs when worker isolates become unresponsive. - Add `query_timeout_seconds` to `AppConfig` (default: 60s) and update TOML schema to version 4. - Implement timeout support in `DecentDbBridge` and `WorkspaceDatabaseGateway` interfaces. - Update `WorkspaceController` to enforce timeouts on queries, pagination, and exports. - Add `updateQueryTimeout` method to allow runtime configuration updates. - Update integration and unit tests to support the new `timeout` parameter in gateway mocks.
…ogic Refactor the workspace and import features to improve maintainability and performance. - Refactor `DecentDbBridge` to use a generic `_ImportOperation` instead of multiple specialized operation classes, reducing code duplication in import handling. - Implement a size limit for ZIP archives in `ImportDetectionService` to prevent excessive memory usage during in-memory processing. - Optimize `WorkspaceController` by introducing a cache for `queryHistory` to avoid redundant computations. - Clean up `WorkspaceController` by removing redundant private logging methods. - Update several ADRs to reflect status changes (Accepted/Superseded). - Add new tests for import application logic and branch controller.
Add a collection of sample datasets for testing different text-based import formats, including: - CSV (Comma-Separated Values) - GZ (Compressed CSV) - Fixed-width text files - PSV (Pipe-Separated Values) - TSV (Tab-Separated Values)
Decompose the monolithic workspace domain models into granular, specialized files to improve maintainability and organization. Changes include: - Splitting `workspace_models.dart` and `app_config.dart` into individual model files (e.g., `appearance_settings_model.dart`, `query_phase_models.dart`, `schema_models.dart`). - Renaming `test-data/text_seperated_values` to `test-data/text_separated_values` to correct the spelling. - Updating test fixtures and documentation to reflect the new directory structure. - Updating `THIRD_PARTY_NOTICES.md` to bump `decentdb` dependency to `v2.7.0`.
Update the pinned DecentDB Dart binding and runtime dependency from v2.7.0 to v2.8.0 to leverage new engine capabilities and diagnostics. - Implement structured error diagnostic support by parsing rich JSON output from the engine, including subcodes, retryability, and documentation anchors. - Add support for new operational metrics including process coordination, process readers, and process lock metrics. - Improve import reliability by implementing better cleanup of temporary database files (WAL, SHM, journal, etc.) on completion or failure. - Enhance SQLite import with savepoint-based transaction management for better atomicity during multi-table imports. - Update documentation, changelogs, and test suites to reflect the v2.8.0 alignment and new engine features.
…e refresh Add calls to `refreshBranchState()` in workspace controller tests to ensure the branch state is correctly synchronized after database initialization or opening. This ensures that assertions regarding `canUseNativeBranchWorkflow` and `branchState` are tested against the most current state.
…logging Replace the DecentDB-based logging mechanism with a file-based `ClefAppLogger` that generates timestamped session logs. This change includes: - Updating `AppLogger` to use `logDirectoryPath` and `sessionLogFilePath` instead of a single database path. - Implementing `ClefAppLogger` for managing text-based log files. - Adding `log_directory` support to `AppConfig` and `LoggingSettings`. - Introducing a `LogViewerDialog` to allow users to view log files via the UI. - Integrating logging into the SQLite import worker for better traceability of background tasks. - Updating existing tests to reflect the new logging architecture.
…mport worker Update the SQLite import logic to use explicit transactions for each table instead of savepoints. This improves reliability and changes how transaction state is tracked during the import process. - Replace `savepoint` and `releaseSavepoint` with `begin` and `commit` for each table. - Update `transactionOpen` flag management to reflect the new transactional structure. - Refactor logging to provide granular feedback per table completion. - Adjust the progress update sequence to occur after the final checkpointing phase.
Improve the robustness of the SQLite import worker by adding specialized handling for malformed or empty timestamp values. - Return epoch zero for empty strings when target type is TIMESTAMP. - Add validation to treat strings consisting only of separators (e.g., "---", "::", "T") as null to prevent parsing errors.
Add validation to ensure that import operations (headless CLI, generic import dialog, and workspace controller imports) use the correct `.ddb` extension for target files. - Implement `validateDecentDbTargetPath` to check for empty paths and incorrect extensions. - Integrate validation into the headless import runner, import dialog, and Excel/SQL/SQLite import workflows. - Prevent users from attempting to import into standard `.db` files by providing descriptive error messages.
Improve the reliability and performance of the schema refresh process in the `WorkspaceController` and enhance metadata lookup logic. - Parallelize `loadSchema` and `getToolingMetadata` calls using `Future.wait` to reduce refresh latency. - Encapsulate tooling metadata fetching in a safe wrapper to prevent metadata failures from blocking schema loading. - Update `ToolingMetadata.getColumnTypeIndex` to fallback to iterating over `columnTypeMetadata` if the index map is empty, ensuring more robust type lookups.
Update documentation to reflect changes in how application logs are handled, moving from a DecentDB-backed log database to structured JSON.CLEF files stored in a per-session `logs/` directory.
Add a lint ignore comment for `use_null_aware_elements` when spreading the `actions` list to resolve static analysis warnings in the PanelCard widget.
Simplify the PanelCard widget implementation by using Material instead of DecoratedBox. This replaces manual BoxDecoration properties with Material's built-in shape and color handling, streamlining the widget structure.
Replace ColoredBox with Material in the _HelpNavigationPane to ensure proper elevation and ink effects within the help center dialog.
Update GitHub Actions workflows to download and configure both the Dart-native assets and the DecentDB runtime assets. This includes setting up environment variables for the migration tool and CLI to ensure they are available during build and release processes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Upgrades Decent Bench to DecentDB v2.8.0 and delivers the v2.0.0 feature set: modular import system, data quality suite, ERD viewer, structured logging, schema explorer optimization, and hardened SQLite/Excel/SQL-dump import paths.
Highlights
DecentDB v2.8.0 Upgrade
Import System Overhaul
Logging
Schema Explorer Performance
Other
Testing
Intentionally Excluded
Stats
440 files changed, ~68K insertions, ~7K deletions across 55 commits.