Create deterministic immutable compressed archives

[jpraynaud]

Why

IPFS content addressing (CID) requires byte identical archives across nodes, but non deterministic compression produces a different CID for the same immutable data.

What

Produce deterministic compressed archives of the immutable files so that identical input always yields an identical CID.

How

• Identify the sources of non determinism in the current archive creation (timestamps, entry ordering, metadata, compression level)
• Make the archive creation deterministic (https://docs.rs/tar/latest/tar/enum.HeaderMode.html)
• Verify that identical inputs produce identical CIDs across nodes
• Add tests covering the determinism of the archives
• Lock versions of crate creating tar and compressor crates?
• How to support breaking changes in the future:
  • Manual operation for aggregator operators?
  • Or automatic detection support?