Add support for vacuum to delete files no longer referenced by a table - #344
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for SQL VACUUM statements to delete files that are no longer referenced by a table. VACUUM can be used to vacuum all tables or specific tables by name, with retention period controlled via environment variable.
- Adds
VACUUMSQL statement parsing and execution support - Implements vacuum functionality across all components (server, manager, embedded)
- Adds configuration for retention period with default of 7 days
Reviewed Changes
Copilot reviewed 15 out of 16 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| protocol.proto | Adds retention_period_in_seconds field to Configuration message |
| parser.rs | Adds VACUUM statement parsing using ShowVariable as substitute |
| delta_lake.rs | Implements vacuum_table method using Delta Lake vacuum operations |
| Cargo.toml | Adds chrono dependency for time delta calculations |
| integration_test.rs | Adds comprehensive vacuum integration tests |
| remote.rs | Adds vacuum SQL statement support to Flight service |
| context.rs | Implements vacuum_table method with retention period configuration |
| configuration.rs | Adds retention period configuration management |
| operations/mod.rs | Adds vacuum method to Operations trait |
| operations/data_folder.rs | Implements vacuum for local data folder operations |
| operations/client.rs | Implements vacuum for remote client operations |
| capi.rs | Adds C API binding for vacuum operations |
| operations.py | Adds Python binding for vacuum functionality |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
skejserjensen
requested changes
Aug 20, 2025
skejserjensen
approved these changes
Aug 20, 2025
chrthomsen
approved these changes
Aug 21, 2025
Closed
9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds support for simple
VACUUMSQL statements to delete files that are no longer referenced by a table. The PR adds support forVACUUMwhich vacuums all tables as well asVACUUM table_nameandVACUUM table_name_1, table_name_2.Note that it is only possible to set the retention period using an environment variable for now. #343 has been created to make it possible to specify the retention period in the statement in the future.