Skip to content

feat: add rolling manifest writer - #443

Merged
wgtmac merged 3 commits into
apache:mainfrom
zhjwpku:add_rolling_manifest_writer
Dec 30, 2025
Merged

wgtmac merged 3 commits into
apache:mainfrom
zhjwpku:add_rolling_manifest_writer

Conversation

@zhjwpku

@zhjwpku zhjwpku commented Dec 27, 2025

Copy link
Copy Markdown
Collaborator

No description provided.

@zhjwpku
zhjwpku force-pushed the add_rolling_manifest_writer branch from a7bb821 to 24182ac Compare December 29, 2025 06:35
/// \brief Add an added entry for a file.
///
/// \param file a data file
/// \return Status::OK() if the entry was written successfully

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated to this PR: I think we don't have Status::OK() method. We may need to fix all these comments to not confuse readers.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to \return Status indicating success or failure

Comment thread src/iceberg/manifest/rolling_manifest_writer.h
Comment thread src/iceberg/manifest/rolling_manifest_writer.h Outdated
RollingManifestWriter::~RollingManifestWriter() {
// Ensure we close the current writer if not already closed
if (!closed_) {
(void)CloseCurrentWriter();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
(void)CloseCurrentWriter();
std::ignore = CloseCurrentWriter();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, why not directly calling Close() so you don't need to check closed_?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to Close()

Comment thread src/iceberg/manifest/rolling_manifest_writer.cc Outdated
if (length_result.has_value()) {
return length_result.value() >= target_file_size_in_bytes_;
}
// If we can't get the length, don't roll

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is a good idea. What if it is a fatal error returned by the underlying writer?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left this as a TODO, let's revisit this later.

@wgtmac
wgtmac merged commit bbdb227 into apache:main Dec 30, 2025
10 checks passed
@zhjwpku
zhjwpku deleted the add_rolling_manifest_writer branch December 30, 2025 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants