Home

Awesome

<a id="top"></a>

Better Faster Stronger Mixer

<!--ts--> <!--te-->

Testing framework for the quest to find a fast & strong mixer, e. g. for hashtables. Many modern hashtables like robin_hood::unordered_map or Abseil's hash tables require high quality hashing for their tables to work efficiently.

Mixing Speed

Generated with nanobench, g++ 9.2, -O3 -march=native, on an Intel i7-8700 CPU locked to 3.20GHz

ns/opop/serr%ins/opcyc/opIPCbra/opmiss%totalbenchmark
1.25799,436,335.340.0%3.004.000.7510.000.0%0.00FNV1A_Pippip
4.07245,934,865.720.0%4.0012.990.3080.000.0%0.00aes2
5.32188,075,153.370.0%5.0016.980.2950.000.0%0.00aes3
1.88532,956,443.530.0%5.005.990.8350.000.0%0.00crc_mul
4.14241,447,855.030.0%15.0013.231.1340.000.0%0.00ettinger_mixer1
5.17193,519,486.340.0%21.0016.501.2730.000.0%0.00ettinger_mixer2
10.6394,049,146.350.0%38.0133.951.1190.000.0%0.00fnv1a_64
3.15317,831,100.560.3%18.0010.061.7900.000.0%0.00lemire_stronglyuniversal
3.44290,636,264.570.0%9.0010.990.8190.000.0%0.00mum3_mixer
3.13319,642,931.980.0%9.009.990.9010.000.0%0.00mumxmumxx1
3.44290,583,035.870.0%11.0010.991.0010.000.0%0.00mumxmumxx2
3.13319,760,942.470.0%8.009.990.8010.000.0%0.00mumxmumxx3
3.76266,269,957.680.0%13.0011.991.0840.000.0%0.00murmurhash3_fmix64
1.56639,449,772.840.0%4.005.000.8010.000.0%0.00robin_hood_hash_int
4.16240,133,099.180.1%13.0013.300.9770.000.0%0.00rrmxmx
4.51221,797,240.960.1%15.0014.391.0420.000.0%0.00rrxmrrxmsx_0
3.76266,285,145.560.0%13.0012.001.0840.000.0%0.00staffort_mix13
5.32187,920,592.190.0%19.0017.001.1180.000.0%0.00twang_mix64
3.83260,771,165.130.0%14.0012.251.1430.000.0%0.00wyhash3_mix
5.95167,926,373.820.0%21.0019.021.1040.000.0%0.00xxh3_mixer

Mixing Quality

The testing protocol is taken from by Pelle Evensen's blog post Better, stronger mixer and a test procedure:

Successive increasing numbers, which are rotated right, then optionally bitorder is reversed, are fed into a mixer. The mixer's output is fed into PractRand, which analyzes the quality of the generated number. So basically, the algorithm is this:

// e.g. for rotation 37, and enabled bitreverse:
int rotation = 37;
uint64_t ctr = 0;
while (true) {
    feedPractRand(mixer(bitreverse64(rotr(ctr, rotation))));
    ++ctr;
}

In my tests, I run PractRand 0.95 with the arguments RNG_test stdin64 -tf 2 -tlmin 10 -tlmax 40. The test aborts when it detects a failure in the quality of the mixer results.

Ideally, a mixer doesn't fail the practrand test and is as fast as possible. In the following plot I show the results of the mixers that I have already evaluated. Pareto optimums are green (I consider 2^10 still a failure).

Results

practrand results

An ideal mixer should be in the upper right corner. The pareto front is colored green (except anything that already fails on tlmax=10. I consider these mixers as too bad for practical use).

FNV1A_Pippip

identity, rotation 0 - 63

0123456789101112131415
010101010101010101010101010101010
1610101010101010101010101010101010
3210101010101010101010101010101010
4810101010101010101010101010101010

min: 2^10, max: 2^10, mean: 2^10.0

identity, rotation 0 - 63

0123456789101112131415
010101010101010101010101010101010
1610101010101010101010101010101010
3210101010101010101010101010101010
4810101010101010101010101010101010

aes2

identity, rotation 0 - 63

0123456789101112131415
015151616161616161613131414141415
1615151514141415151514141414141515
3215161616161616161614141414141415
4815141414141415151514141414141515

min: 2^13, max: 2^16, mean: 2^14.7

identity, rotation 0 - 63

0123456789101112131415
015161616161616161614141314151515
1615141414141415151515131414141415
3215161616161616161614141314141515
4815141414141415151514131414141515

min: 2^13, max: 2^16, mean: 2^14.7

aes3

TODO only tested up to -tlmax 20

identity, rotation 0 - 63

0123456789101112131415
0>20>20>20>20>20>20>20>20>20>2019>20>20>20>20>20
16>20>20>20>20>20>20>20>20>20>20>20>20>20>20>20>20
32>20>20>20>20>20>20>20>20>20>2019>20>20>20>20>20
48>20>20>20>20>20>20>20>20>20>20>20>20>20>20>20>20

min: 2^19, max: 2^20, mean: 2^20.0

identity, rotation 0 - 63

0123456789101112131415
0>20>20>20>20>20>20>20>20>20>2019>20>20>20>20>20
16>20>20>20>20>20>20>20>20>20>20>20>20>20>20>20>20
32>20>20>20>20>20>20>20>20>20>2019>20>20>20>20>20
48>20>20>20>20>20>20>20>20>20>20>20>20>20>20>20>20

min: 2^19, max: 2^20, mean: 2^20.0

crc_mul

identity, rotation 0 - 63

0123456789101112131415
013131413131313131313131313131313
1613131413141313131313131313131313
3213131313131413141413131313131313
4813131313131314131313131313131313

min: 2^13, max: 2^14, mean: 2^13.1

identity, rotation 0 - 63

0123456789101112131415
013131313131314131313131313131313
1613131313131313131313141313131313
3213141313131313131313131414141414
4813131313131313131313131313131313

min: 2^13, max: 2^14, mean: 2^13.1

ettinger_mixer1

TODO

ettinger_mixer2

TODO

fnv1a_64

identity, rotation 0 - 63

0123456789101112131415
012121212101010101010101010101010
1610101010101110101010101011131310
3213131313131313131312131312131213
4813121312121212121212131213131313

min: 2^10, max: 2^13, mean: 2^11.6

identity, rotation 0 - 63

0123456789101112131415
010101010111212121212131314131313
1612121212121212131313141213131313
3213131312131313131013101010101010
4810101010101010101010101010101010

min: 2^10, max: 2^14, mean: 2^11.5

lemire_stronglyuniversal

identity, rotation 0 - 63

0123456789101112131415
012121212121212131312131310121213
1613121310121312121212131212131312
3212121212121313131212121312131313
4813121312121313121312131210121011

min: 2^10, max: 2^13, mean: 2^12.2

identity, rotation 0 - 63

0123456789101112131415
013141412131212121210121014131313
1613131313141313131313131313131313
3212131313131313141313121213131213
4813131313121213131313131313121313

min: 2^10, max: 2^14, mean: 2^12.8

mum3_mixer

identity, rotation 0 - 63

0123456789101112131415
016171717171717161717171717171717
1617171717161616171717161716171717
3217161717161717161717161616171717
4816161617171717171716171716161617

min: 2^16, max: 2^17, mean: 2^16.7

identity, rotation 0 - 63

0123456789101112131415
017171616171717171617161716161717
1617161717171716161617171616161616
3216161716171617171717171717171616
4816161717171717171717161717171717

min: 2^16, max: 2^17, mean: 2^16.6

mumxmumxx1

identity, rotation 0 - 63

0123456789101112131415
038>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40
16>40>40>40>40>40>40>40>40>40>40>40>40>40323331
3234>40>4038>40>40>40373436383636333930
4832303030333432313837363736323237

min: 2^30, max: 2^40, mean: 2^37.2

identity, rotation 0 - 63

0123456789101112131415
036>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40
16>40>40>40>40>40>40>40>40>40>40>40>40>40>40>4034
3235>403334>403838>4040393836>403828>40
4831313229303131323433333133343638

min: 2^28, max: 2^40, mean: 2^37.3

mumxmumxx2

identity, rotation 0 - 63

0123456789101112131415
0>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40
16>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40
32>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40
48>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40

min: 2^40, max: 2^40, mean: 2^40.0

identity, rotation 0 - 63

0123456789101112131415
0>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40
16>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40
32>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40
48>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40

min: 2^40, max: 2^40, mean: 2^40.0

mumxmumxx3

TODO

murmurhash3_fmix64

identity, rotation 0 - 63

0123456789101112131415
017181817171616161616161715151514
1614141414141515151615151616161515
3216171717161514141515141515151515
4814141414141415151516151617171717

min: 2^14, max: 2^18, mean: 2^15.4

identity, rotation 0 - 63

0123456789101112131415
015161718181819191817171717161717
1617161414141414161717171718181715
3215141416161718181818171717171717
4817161515141414151617171717181817

min: 2^14, max: 2^19, mean: 2^16.5

robin_hood_hash_int

identity, rotation 0 - 63

0123456789101112131415
012131313121313131212121212121213
1613121013121012121212121310121213
3212101313121313131213121212121213
4813121013121012121212121310121213

min: 2^10, max: 2^13, mean: 2^12.1

identity, rotation 0 - 63

0123456789101112131415
013131313131313141213131312121313
1613131313121313131313131313131414
3213131313131313131213131312121313
4813131313121313131313131313151313

min: 2^12, max: 2^15, mean: 2^13.0

rrmxmx

TODO

rrxmrrxmsx_0

TODO

staffort_mix13

identity, rotation 0 - 63

0123456789101112131415
019171818181718181818181919161717
1617171617161617171617161717171818
3219191919191919191919202019201919
4817171617171716171617171718181919

min: 2^16, max: 2^20, mean: 2^17.8

identity, rotation 0 - 63

0123456789101112131415
016181818181818181921202219211818
1621192020202019202019202019191818
3218192222202122212022192119182119
4820192020192020192020191918181817

min: 2^16, max: 2^22, mean: 2^19.4

twang_mix64

identity, rotation 0 - 63

0123456789101112131415
013151516161617151515141414131313
1615151516151615161718181618181919
3220191919191820202019191820202121
4821211617171615151515141414131313

min: 2^13, max: 2^21, mean: 2^16.6

identity, rotation 0 - 63

0123456789101112131415
014161616161616171516131313141414
1614141516161616171818192018181918
3219211820212122222221181918171917
4817161616151615151516161516151513

min: 2^13, max: 2^22, mean: 2^16.8

wyhash3_mix

identity, rotation 0 - 63

0123456789101112131415
0>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40
16>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40
32>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40
48>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40

min: 2^40, max: 2^40, mean: 2^40.0

identity, rotation 0 - 63

0123456789101112131415
0>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40
16>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40
32>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40
48>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40>40

min: 2^40, max: 2^40, mean: 2^40.0

xxh3_mixer

identity, rotation 0 - 63

0123456789101112131415
021191618182024252627262728282626
1623252625262929282725262422212223
3219192221212221181919201918181818
4818191615171918181918181819212120

min: 2^15, max: 2^29, mean: 2^21.6

identity, rotation 0 - 63

0123456789101112131415
018182224242625242220201817171718
1617181515181818181819181922211820
3223181822212020221819222320202222
4822212016182223202627262324242221

min: 2^15, max: 2^27, mean: 2^20.4