IronCompress

C++ compression methods joined together in one native library, cross-compiled for multiple architectures and exposed as a .NET library. .NET has built-in support for Gzip and Brotli (which is what this library is using) but other compression methods are either available only as native libraries hard (impossible for some) to consume, or ill-ported C# alternatives. In fact, I myself wrote one of them. Using native, original implementations is the way to go if you want to keep performance, security, and features up to date.

The library supports the following formats:

• Google Snappy [source].
• Facebook's Zstandard (zstd) [source].
• Gzip [source].
• Google Brotli [source] (.net standard 2.0 build uses native version instead).
• LZO [source].
• LZ4 [source].

And following architectures:

I periodically update to the latest binaries. All the binaries are compiled from C/C++ source with CMake for major systems i.e. Linux, Windows and Mac OSX. They are then wrapped safely with a .NET interface.

Logic

Although managed versions are available, they will be only used as a fallback when native library is not available, because native libraries are faster and more up to date with latest advancements.

You can check which version was used by reading IronCompressResult.NativeUsed flag.

Using

This library only compresses buffers. It may work with streams in the future, but I am currently only interested in buffers. Essentially, you pass ReadOnlySpan<byte> as an input, and receive Span<T> as an output in the most efficient way.

Here is an example of how to compress buffer with snappy codec:

using IronCompress;	// root namespace

// Construct library entry point and optionally pass an implementation of ArrayPool.
// I will pass default shared pool here.
var iron = new Iron();

byte[] input = ...;
using(IronCompressResult compressed = iron.Compress(Codec.Snappy, input.AsSpan())) {
    // ... use data
}

Compressand Decompress methods actually return a Result class which wraps byte array of compression or decompression operation. You can access the underlying result by calling to .AsSpan() method inside the result. On dispose, Result makes sure the underlying memory is freed up - if pooling was used, it will be returned back to the pool.

To decompress:

using (IronCompressResult uncompressed = iron.Decompress(Codec.Snappy, compressed, input.Length)) {
	// ... use data
}

As with compression, this returns Result with decompressed data. It's worth nothing one important difference - decompression needs the length of output buffer specified as third parameter (input.Length). Although some decompressors can guess uncompressed length more or less correct, the others won't know it beforehand. In reality this problem is solved by using a framing format that adds metadata about resulting length, however many compression formats do not define that and consider compressed packets to be implementation specific.

You will need more or less recent C++ compiler, CMake and .NET SDK 8 to build the code.

Building

See workflow file for building instructions.

Hint: To develop managed only code locally you can download the latest artifact from Actions output and put it into native/ubin so you have binaries for all platforms.