Awesome
Supersnappy
nimble install supersnappy
Supersnappy is a pure Nim implementation of Google's Snappy compression algorithm. The goal of this library is to be small, straightforward, dependency-free and highly performant.
Supersnappy can be used at compile time too. This is great for baking assets into executables in compressed form. Check out an example here.
To ensure Supersnappy is compatible with other Snappy implementations, tests/validate.nim can be run. This script verifies that data compressed by Supersnappy can be uncompressed by other implementations (and that other implementations can uncompress data compressed by Supersnappy).
Supersnappy works well using Nim's relatively new --gc:arc and --gc:orc as well as the default garbage collector. This library also works using both nim c and nim cpp, in addition to --cc:vcc on Windows.
I have also verified that Supersnappy builds with --experimental:strictFuncs on Nim 1.4.0.
Performance
Benchmarks can be run comparing different Snappy implementations. My benchmarking shows this library performs significantly better in all cases than alternatives. Check the performance yourself by running tests/benchmark.nim.
nim c --gc:arc -d:release -r .\tests\benchmark.nim (100 compress-uncompress cycles, lower time is better)
https://github.com/guzba/supersnappy
name ............................... min time avg time std dv runs
alice29.txt ........................ 0.504 ms 0.513 ms ±0.020 x1000
asyoulik.txt ....................... 0.447 ms 0.451 ms ±0.006 x1000
fireworks.jpg ...................... 0.013 ms 0.015 ms ±0.001 x1000
geo.protodata ...................... 0.100 ms 0.106 ms ±0.008 x1000
html ............................... 0.120 ms 0.128 ms ±0.013 x1000
html_x_4 ........................... 0.524 ms 0.534 ms ±0.014 x1000
kppkn.gtb .......................... 0.422 ms 0.435 ms ±0.028 x1000
lcet10.txt ......................... 1.362 ms 1.380 ms ±0.028 x1000
paper-100k.pdf ..................... 0.040 ms 0.041 ms ±0.002 x1000
plrabn12.txt ....................... 1.885 ms 1.898 ms ±0.020 x1000
urls.10K ........................... 1.565 ms 1.574 ms ±0.009 x1000
https://github.com/dfdeshom/nimsnappy
alice29.txt ........................ 0.590 ms 0.599 ms ±0.017 x1000
asyoulik.txt ....................... 0.525 ms 0.529 ms ±0.005 x1000
fireworks.jpg ...................... 0.011 ms 0.011 ms ±0.000 x1000
geo.protodata ...................... 0.100 ms 0.101 ms ±0.002 x1000
html ............................... 0.118 ms 0.127 ms ±0.017 x1000
html_x_4 ........................... 0.553 ms 0.559 ms ±0.006 x1000
kppkn.gtb .......................... 0.517 ms 0.523 ms ±0.009 x1000
lcet10.txt ......................... 1.585 ms 1.599 ms ±0.018 x1000
paper-100k.pdf ..................... 0.017 ms 0.017 ms ±0.000 x1000
plrabn12.txt ....................... 2.224 ms 2.237 ms ±0.017 x1000
urls.10K ........................... 1.826 ms 1.842 ms ±0.021 x1000
https://github.com/NimCompression/nimsnappyc
alice29.txt ........................ 0.605 ms 0.609 ms ±0.004 x1000
asyoulik.txt ....................... 0.513 ms 0.518 ms ±0.005 x1000
fireworks.jpg ...................... 0.014 ms 0.020 ms ±0.004 x1000
geo.protodata ...................... 0.144 ms 0.150 ms ±0.010 x1000
html ............................... 0.169 ms 0.172 ms ±0.004 x1000
html_x_4 ........................... 0.701 ms 0.712 ms ±0.009 x1000
kppkn.gtb .......................... 0.510 ms 0.515 ms ±0.005 x1000
lcet10.txt ......................... 1.653 ms 1.666 ms ±0.022 x1000
paper-100k.pdf ..................... 0.047 ms 0.049 ms ±0.003 x1000
plrabn12.txt ....................... 2.073 ms 2.092 ms ±0.024 x1000
urls.10K ........................... 1.873 ms 1.899 ms ±0.028 x1000
https://github.com/status-im/nim-snappy
alice29.txt ........................ 0.609 ms 0.615 ms ±0.005 x1000
asyoulik.txt ....................... 0.529 ms 0.536 ms ±0.015 x1000
fireworks.jpg ...................... 0.010 ms 0.011 ms ±0.001 x1000
geo.protodata ...................... 0.131 ms 0.135 ms ±0.004 x1000
html ............................... 0.151 ms 0.156 ms ±0.004 x1000
html_x_4 ........................... 0.647 ms 0.657 ms ±0.016 x1000
kppkn.gtb .......................... 0.519 ms 0.523 ms ±0.006 x1000
lcet10.txt ......................... 1.640 ms 1.650 ms ±0.010 x1000
paper-100k.pdf ..................... 0.019 ms 0.020 ms ±0.001 x1000
plrabn12.txt ....................... 2.232 ms 2.243 ms ±0.011 x1000
urls.10K ........................... 1.796 ms 1.810 ms ±0.023 x1000
Testing
nimble test
To prevent Supersnappy from causing a crash or otherwise misbehaving on bad input data, a fuzzer has been run against it. You can do run the fuzzer any time by running nim c -r tests/fuzz.nim
Credits
This implementation was heavily influenced by snappy-c.