Use of libdeflate to speed up compression

[rouault]

Potentially faster (35% - 50% improvement is typical) compression/decompression could be possible by using libdeflate, as an optional extra dependency : https://github.com/ebiggers/libdeflate (packaged in ubuntu since 20.04 or in recent Debian)

See https://github.com/OSGeo/gdal/pull/3072/files for an example (here using zlib encapsulation, for gzip use the libdeflate_gzip_XXXX() family).

I believe however that, given that libdeflate doesn't work in streaming mode, you could only use libdeflate for the compression side (by providing an output buffer with a size corresponding to the worse case which is given by a function of the libdeflate API), but that would be already a win since deflate compression is much slower than decompression. For optimal efficiency in case of very frequent calls to gzip(), I don't know if that's possible for postgres extensions, you'd probably want to re-use the output buffer and libdeflate compression object between calls

To be able to use libdeflate on decompression you'd need to know in advance the size of the uncompressed buffer, or bet on the compression ratio to provide a decompression buffer large enough, so this is likely not doable.

I've noted that at what is the zlib default compression level (6), libdeflate sometimes produces slightly larger output. Using 7 fixes that.

[patrickmichalina]

This going to happen?

[pramsey]

No, I don't think so. Adds complexity to what I'd like to keep as just a simple simple simple extension and build.