A zlib chunk with no trailer is read to its declared size - #35
Merged
Merged
Conversation
The V Maps pack was written by HPIZ Archiver, which leaves the four-byte adler32 trailer off every zlib chunk. The original reads its bytes and never looks for the trailer; RWE's decompressor demanded a clean end of stream and threw "ZLib decompress failed" on every one of the pack's 628 maps, from the lobby as well as the loader. A chunk is now also complete when all of its input has been consumed and exactly the declared number of bytes came out. The chunk's own checksum still covers the compressed data, so a genuinely short or corrupt chunk is still refused, and a test pins both sides. The "Expected property name" crash on [V] A Better Fate from the same pack does not reproduce: with the chunks readable, all 628 OTAs parse. The lobby no longer lets a parse failure escape to the fatal handler either -- a malformed map shows the parser's line and column in a message box and leaves the previous selection standing. Issue: #8 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PJCbwC9MGZnM6erMDKptun
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #8: the zlib failure on the large
.ufo(upstream MHeasell#93) and the "Expected property name" crash on [V] A Better Fate (upstream #38). With this, every item of #8 has a PR.The pack. "V maps 26.5.2020" from ModDB, 2.1 GB, one
V maps.ufoholding 628 maps. The Google Drive link in the upstream issue is dead.What was wrong. Dissecting the archive in Python: every chunk header is sound and every chunk checksum matches, but each zlib stream inflates to exactly its declared size and then stops without the four-byte adler32 trailer. That is the HPIZ Archiver bug the upstream thread mentions.
decompressZLibrequiredZ_STREAM_END, so every file in the pack threw, whether selected in the lobby or loaded from the command line.What changed.
decompressZLibalso accepts a stream whose input is fully consumed and whose output is exactly the declared size. A short chunk, an overlong one, and garbage are still refused. The chunk checksum over the compressed bytes is unchanged and still enforced before this runs.MainMenuScene::setCandidateSelectedMapcatches a read or parse failure, logs it, and shows it in the message box with the parser's line and column instead of letting it reachmain's fatal handler. The previous selection stays.On the second item. Once the chunks read, [V] A Better Fate loads through to the AI's map survey, and a sweep of all 628 extracted OTAs through
parseTdfFromStringandparseOtagives 628 parsed, 0 failed. Whatever the 2019 report hit has been fixed in the pack since; the lobby guard covers the class regardless.Verified.
io/hpi/hpi_util.test.cpp: a whole stream, a trailer-less stream, a truncated stream (refused), an overrun (refused), garbage (refused).hpi_test extractof every one of the 628 OTAs in the pack: 628 ok, 0 failed, where before the fix it was 0 and 628.rwe_test: all 553 test cases pass (71484 assertions), Linux gcc-14 Debug.🤖 Generated with Claude Code
https://claude.ai/code/session_01PJCbwC9MGZnM6erMDKptun