Skip to content

Add support for setting retention period using RETAIN syntax - #348

Merged
CGodiksen merged 27 commits into
mainfrom
dev/vacuum-retain
Sep 9, 2025
Merged

Add support for setting retention period using RETAIN syntax#348
CGodiksen merged 27 commits into
mainfrom
dev/vacuum-retain

Conversation

@CGodiksen

Copy link
Copy Markdown
Collaborator

Closes #343 by changing the parser to support the syntax VACUUM table_name RETAIN num_seconds. Note that RETAIN is optional.

The PR also removes the previous method for setting the retention period which was through the environment variable MODELARDBD_RETENTION_PERIOD_IN_SECONDS and the configuration retention_period_in_seconds. This method was removed to avoid confusion between the two ways to set the retention period and since the setting was also only supposed to be used in the server (modelardbd). It could have been kept in the server but this might cause confusion on how it is handled in the manager and Operations API.

@CGodiksen CGodiksen self-assigned this Sep 7, 2025
@CGodiksen
CGodiksen requested a review from Copilot September 7, 2025 07:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for setting retention periods in VACUUM commands using the RETAIN num_seconds syntax, while removing the previous environment variable-based configuration approach. The changes enable users to specify retention periods directly in SQL rather than through server configuration.

Key changes:

  • Extended the VACUUM SQL parser to support optional RETAIN num_seconds syntax
  • Updated all vacuum operations to accept an optional retention period parameter
  • Removed environment variable and configuration-based retention period settings

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
crates/modelardb_types/src/flight/protocol.proto Removed retention_period_in_seconds from Configuration protobuf
crates/modelardb_storage/src/parser.rs Extended VACUUM parser to support RETAIN syntax with validation
crates/modelardb_storage/src/delta_lake.rs Updated vacuum_table to accept optional retention period
crates/modelardb_server/tests/integration_test.rs Updated tests to use new VACUUM syntax instead of configuration
crates/modelardb_server/src/remote.rs Updated FlightService to pass retention period from parsed SQL
crates/modelardb_server/src/context.rs Updated vacuum_table method signature and removed configuration usage
crates/modelardb_server/src/configuration.rs Removed retention period fields and methods from ConfigurationManager
crates/modelardb_manager/src/remote.rs Updated manager to handle new vacuum syntax and parameter passing
crates/modelardb_embedded/src/operations/mod.rs Updated Operations trait vacuum method signature
crates/modelardb_embedded/src/operations/data_folder.rs Removed environment variable usage, updated vacuum implementation
crates/modelardb_embedded/src/operations/client.rs Updated client vacuum to construct SQL with RETAIN syntax
crates/modelardb_embedded/src/capi.rs Updated C API to accept retention period parameter
crates/modelardb_embedded/bindings/python/tests/test_operations.py Updated Python tests to use new vacuum API
crates/modelardb_embedded/bindings/python/modelardb/operations.py Updated Python bindings to support retention period parameter

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread crates/modelardb_storage/src/parser.rs
Comment thread crates/modelardb_storage/src/parser.rs Outdated
Comment thread crates/modelardb_storage/src/delta_lake.rs Outdated
Comment thread crates/modelardb_server/src/context.rs Outdated
@skejserjensen
skejserjensen self-requested a review September 7, 2025 07:42
Comment thread crates/modelardb_embedded/bindings/python/modelardb/operations.py Outdated
Comment thread crates/modelardb_embedded/src/capi.rs Outdated
Comment thread crates/modelardb_embedded/src/operations/client.rs Outdated
Comment thread crates/modelardb_embedded/src/operations/data_folder.rs Outdated
Comment thread crates/modelardb_embedded/src/operations/mod.rs Outdated
Comment thread crates/modelardb_types/src/types.rs
Comment thread crates/modelardb_storage/src/delta_lake.rs
Comment thread crates/modelardb_types/src/types.rs
Comment thread crates/modelardb_storage/src/delta_lake.rs Outdated
Comment thread crates/modelardb_storage/src/parser.rs Outdated
@CGodiksen
CGodiksen merged commit 5f89e65 into main Sep 9, 2025
4 checks passed
@CGodiksen
CGodiksen deleted the dev/vacuum-retain branch September 9, 2025 18:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for setting the retention period in VACUUM statements.

4 participants