Zstd compression overrides, parallel decompression, and misc fixes - #128
Merged
Conversation
Log a throttled warning when a disk's high-usage warning fires, and apply the same package to other repeat-failure log sites (auto-save failures, snapshot pruning/blob GC, helper service reconciliation and pipe errors) so a persistent condition doesn't spam the log file.
… serialization Consolidates repeated path-collision checks, MessageBox boilerplate, compression-level mapping, and legacy-image-header parsing into shared helpers; fixes a UI-thread stall where editing a disk's password during a remount ran SetPassword synchronously outside the Task.Run that wraps the mount; and adds a round-trip test for the DiskOptions/DiskProfile mapping.
Consolidates the identical path/FileInfo/security-descriptor read-write logic shared by DiskImageSerializer and SnapshotStore into NodeMetadataIO, and moves MainViewModel's stateless helper methods into MainViewModelHelpers; documents the CLI adapter's exact contract surface with MainViewModel.
…hes for snapshot diffing Write previously took DateTimeOffset.UtcNow three times per call (metadata timestamps, MarkDirty, content-access tracking); it now captures one timestamp and reuses it. FileNode gains a ContentVersion counter bumped only when content bytes actually change (Write/Overwrite/SetFileSizeCore), letting SnapshotManager.ComputeHash cache a file's SHA-256 across repeated diff/GC passes instead of rehashing unchanged files on every dirty auto-save tick.
GetChildren (used by CanDelete on every directory-delete check) scanned the whole sorted namespace from the start to find where a path prefix's run began. FileNodeMap now keeps a plain Dictionary for O(1) node lookup alongside a SortedSet<string> of keys, and GetChildren/RenameDescendants seek directly into the prefix's range via GetViewBetween in O(log n) instead of a full-map scan.
Save/Load/PeekHeader open the .mdr file with the default 4KB buffer, causing many small syscalls when writing per-node metadata directly to the stream (most noticeable on uncompressed images, where there's no GZipStream buffering in front of it).
…ewModel The epsilon check meant to dedupe near-identical intermediate ticks could also swallow a final Report(1.0) that landed within epsilon of the last stored value, leaving the progress bar visibly stuck just short of 100%.
SaveToImageWithSnapshot always split progress into [0, 0.5] for the image save and [0.5, 1.0] for the snapshot write, even when snapshot retention isn't configured. TryWriteSnapshot then no-ops and jumps straight to 1.0, so the bar visibly stalled around 50% before snapping to done. Now the full [0, 1] range goes to the image save unless a snapshot write may actually happen.
Frame each compressed chunk as [Int32 length][bytes] (mirroring ChunkedGcm's framing) instead of relying on transparent concatenated Zstd frames, so load can dispatch each chunk's decompression to a worker pool the same way save already parallelizes compression. SnapshotStore's blob reader picks up the same ParallelZstd.ReadStream since it shares the writer.
Let a disk's save/snapshot compression optionally override the exact Zstd level (1-22) instead of the coarse Fastest/Optimal/SmallestSize mapping, exposed as an advanced option in CreateDiskDialog and persisted per-disk via DiskProfile/DiskOptions. Bumps SharpCompress to 0.50.3.
List every distributed NuGet dependency (ZstdSharp.Port, ThrottledLogging, System.CommandLine, Spectre.Console, YamlDotNet, Microsoft.Extensions.*, Serilog and its sinks) in THIRD-PARTY-NOTICES.md, and replace the per-package WinFsp/SharpCompress hyperlinks in AboutDialog with a single link to that file so future dependency additions only require a doc update.
Wires DiskOptions.CustomZstdLevel through the CLI mount path (CliMountOverrides, MountOptionsFactory.MountOverrides, and MainViewModel.MountImageAsync) so --custom-zstd-level (1-22) can override the preset level mapped from --compression, matching the GUI's existing custom-level support.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
mdrive mount.mdrnode region in parallel across Zstd chunks for faster loadsBusyOverlayViewModelThrottledLoggingTest plan
dotnet buildsucceeds with 0 warnings/errorsdotnet test tests/ManagedDrive.Tests(run in CI)