Awesome
pithy
pithy is licensed under the terms of the BSD license.<br /> Copyright © 2011, John Engelhart.
Notes
pithys roots can be traced back to Googles snappy compression library, but has diverged quite a bit.
pithy can not read or write the snappy compression format, and vice versa.
TODO
Write the README.md
:).
Benchmarks
- Benchmarks were performed on an iPhone 4 (not S) and compiled with
clang
version 3.0 using-Os
. - Compression and decompression speed numbers are in megabytes per second.
- The files used are from the snappy distributions test suite.
zlib 1
andzlib 6
correspond to thezlib
tunable compression effort, with1
being fastest, and6
being the default.pithy 0
andpithy 2
correspond to thepithy
tunable compression effort.
Size
Name | Uncompressed | pithy 0 | pithy 2 | snappy | lzf | lzo | zlib 1 | zlib 6 |
---|---|---|---|---|---|---|---|---|
alice29.txt | 152089 | 93270 | 87767 | 90965 | 82985 | 83568 | 65148 | 54416 |
asyoulik.txt | 125179 | 83039 | 78170 | 80207 | 72081 | 73849 | 56809 | 48909 |
baddata1.snappy | 27512 | 27521 | 27521 | 26675 | 26228 | 26491 | 23151 | 22953 |
baddata2.snappy | 27483 | 27495 | 27495 | 26724 | 26272 | 26531 | 23214 | 23035 |
baddata3.snappy | 28384 | 28346 | 28346 | 27476 | 27105 | 27388 | 23927 | 23730 |
cp.html | 24603 | 12398 | 11990 | 11838 | 11869 | 11677 | 9046 | 7973 |
fields.c | 11150 | 4960 | 4790 | 4728 | 4667 | 4680 | 3665 | 3134 |
geo.protodata | 118588 | 19984 | 19524 | 27459 | 27748 | 20170 | 18845 | 15143 |
grammar.lsp | 3721 | 1859 | 1839 | 1800 | 1768 | 1783 | 1344 | 1234 |
house.jpg | 126958 | 126846 | 126843 | 126803 | 130003 | 127169 | 126488 | 126525 |
html | 102400 | 21725 | 21077 | 24140 | 22534 | 21059 | 17049 | 13711 |
html_x_4 | 409600 | 21753 | 21105 | 96472 | 89962 | 82730 | 67470 | 53379 |
kennedy.xls | 1029744 | 427736 | 425992 | 425735 | 402525 | 359590 | 242311 | 204004 |
kppkn.gtb | 184320 | 73768 | 68287 | 70535 | 75486 | 72623 | 49877 | 38763 |
lcet10.txt | 426754 | 247515 | 230250 | 243710 | 225007 | 223028 | 174142 | 144916 |
mapreduce-osdi-1.pdf | 94330 | 79281 | 79331 | 77477 | 79684 | 77241 | 76414 | 74940 |
plrabn12.txt | 481861 | 340325 | 317582 | 329339 | 290030 | 297444 | 228901 | 195273 |
ptt5 | 513216 | 87776 | 83676 | 93455 | 80756 | 86277 | 65571 | 56477 |
sum | 38240 | 19018 | 18728 | 19837 | 20274 | 17609 | 14130 | 13002 |
urls.10K | 702087 | 343976 | 316942 | 357267 | 350981 | 312748 | 253275 | 222625 |
xargs.1 | 4227 | 2603 | 2545 | 2509 | 2441 | 2451 | 1864 | 1748 |
Compression Speed in MB/s
Name | pithy 0 | pithy 2 | snappy | lzf | lzo | zlib 1 | zlib 6 |
---|---|---|---|---|---|---|---|
alice29.txt | 34.31 | 30.48 | 26.92 | 18.46 | 27.91 | 5.01 | 1.36 |
asyoulik.txt | 32.67 | 29.09 | 25.63 | 17.52 | 25.91 | 4.60 | 1.24 |
baddata1.snappy | 332.22 | 236.41 | 37.59 | 12.61 | 13.20 | 3.33 | 2.79 |
baddata2.snappy | 287.97 | 213.15 | 40.39 | 12.58 | 13.12 | 3.45 | 2.87 |
baddata3.snappy | 287.98 | 213.21 | 34.57 | 12.57 | 13.03 | 3.38 | 2.82 |
cp.html | 44.02 | 37.66 | 29.81 | 22.54 | 29.00 | 6.32 | 3.36 |
fields.c | 43.06 | 35.57 | 26.45 | 25.68 | 39.53 | 6.03 | 3.56 |
geo.protodata | 90.33 | 82.31 | 63.04 | 37.91 | 60.77 | 12.47 | 5.42 |
grammar.lsp | 35.15 | 33.17 | 23.19 | 24.99 | 36.98 | 3.73 | 2.93 |
house.jpg | 712.25 | 450.11 | 209.48 | 11.53 | 10.41 | 2.67 | 2.46 |
html | 77.14 | 69.61 | 58.09 | 41.57 | 57.68 | 11.02 | 4.65 |
html_x_4 | 171.03 | 131.75 | 56.00 | 37.63 | 51.18 | 11.18 | 4.36 |
kennedy.xls | 45.41 | 37.88 | 36.68 | 34.34 | 40.08 | 7.97 | 1.76 |
kppkn.gtb | 47.66 | 42.32 | 37.38 | 31.08 | 40.32 | 6.80 | 1.23 |
lcet10.txt | 33.98 | 29.69 | 27.15 | 17.84 | 23.87 | 4.92 | 1.37 |
mapreduce-osdi-1.pdf | 165.96 | 147.48 | 90.87 | 14.25 | 13.31 | 3.47 | 2.91 |
plrabn12.txt | 29.28 | 24.61 | 22.84 | 15.80 | 20.64 | 4.14 | 1.03 |
ptt5 | 80.67 | 70.76 | 68.08 | 41.08 | 49.54 | 11.46 | 3.53 |
sum | 44.91 | 38.59 | 30.31 | 22.71 | 32.02 | 5.93 | 1.97 |
urls.10K | 39.15 | 31.68 | 31.11 | 20.44 | 23.31 | 5.71 | 2.47 |
xargs.1 | 33.60 | 25.68 | 19.67 | 20.26 | 30.31 | 3.77 | 2.99 |
Decompression Speed in MB/s
Name | pithy 0 | pithy 2 | snappy | lzf | lzo | zlib 1 | zlib 6 |
---|---|---|---|---|---|---|---|
alice29.txt | 115.76 | 114.84 | 48.92 | 69.53 | 68.74 | 28.71 | 33.94 |
asyoulik.txt | 112.63 | 110.95 | 47.17 | 66.92 | 65.92 | 27.99 | 31.66 |
baddata1.snappy | 1544.53 | 1642.51 | 228.20 | 114.57 | 143.39 | 20.58 | 21.42 |
baddata2.snappy | 1093.85 | 1049.47 | 251.99 | 115.96 | 144.84 | 20.60 | 20.77 |
baddata3.snappy | 1355.65 | 1355.65 | 213.21 | 114.71 | 143.99 | 20.19 | 21.67 |
cp.html | 172.58 | 172.58 | 76.19 | 74.72 | 100.70 | 30.83 | 32.77 |
fields.c | 154.19 | 156.49 | 67.73 | 81.80 | 100.34 | 33.76 | 37.18 |
geo.protodata | 340.65 | 343.80 | 135.29 | 150.60 | 156.64 | 54.24 | 58.54 |
grammar.lsp | 154.64 | 154.24 | 72.43 | 78.96 | 104.45 | 22.32 | 23.19 |
house.jpg | 1985.66 | 897.23 | 1017.70 | 230.62 | 229.32 | 27.60 | 36.28 |
html | 279.83 | 280.60 | 115.30 | 130.03 | 136.39 | 50.97 | 55.11 |
html_x_4 | 832.94 | 847.38 | 108.72 | 122.60 | 129.39 | 46.41 | 50.31 |
kennedy.xls | 147.34 | 147.61 | 52.08 | 56.72 | 95.54 | 32.96 | 36.43 |
kppkn.gtb | 132.47 | 115.95 | 56.02 | 85.62 | 84.27 | 34.07 | 41.23 |
lcet10.txt | 109.02 | 107.61 | 45.58 | 60.34 | 61.26 | 27.58 | 30.86 |
mapreduce-osdi-1.pdf | 478.53 | 463.82 | 257.78 | 182.13 | 185.10 | 29.29 | 30.33 |
plrabn12.txt | 94.13 | 90.73 | 37.60 | 55.46 | 52.74 | 23.68 | 26.55 |
ptt5 | 241.46 | 182.63 | 116.23 | 131.32 | 133.36 | 49.16 | 52.61 |
sum | 162.12 | 147.64 | 67.91 | 74.89 | 98.04 | 29.18 | 33.77 |
urls.10K | 156.08 | 153.04 | 68.10 | 62.93 | 77.84 | 29.52 | 31.03 |
xargs.1 | 149.63 | 144.20 | 63.98 | 72.03 | 93.80 | 22.03 | 23.30 |