Home

Awesome

mmh3

Documentation Status GitHub Super-Linter Build PyPi Version Python Versions License: MIT Total Downloads Recent Downloads

mmh3 is a Python extension for MurmurHash (MurmurHash3), a set of fast and robust non-cryptographic hash functions invented by Austin Appleby.

By combining mmh3 with probabilistic techniques like Bloom filter, MinHash, and feature hashing, you can develop high-performance systems in fields such as data mining, machine learning, and natural language processing.

Another popular use of mmh3 is to calculate favicon hashes, which are utilized by Shodan, the world's first IoT search engine.

This page provides a quick start guide. For more comprehensive information, please refer to the documentation.

Installation

pip install mmh3

Usage

Basic usage

>>> import mmh3
>>> mmh3.hash(b"foo") # returns a 32-bit signed int
-156908512
>>> mmh3.hash("foo") # accepts str (UTF-8 encoded)
-156908512
>>> mmh3.hash(b"foo", 42) # uses 42 as the seed
-1322301282
>>> mmh3.hash(b"foo", 0, False) # returns a 32-bit unsigned int
4138058784

mmh3.mmh3_x64_128_digest(), introduced in version 5.0.0, efficienlty hashes buffer objects that implement the buffer protocol (PEP 688) without internal memory copying. The function returns a bytes object of 16 bytes (128 bits). It is particularly suited for hashing large memory views, such as bytearray, memoryview, and numpy.ndarray, and performs faster than the 32-bit variants like hash() on 64-bit machines.

>>> mmh3.mmh3_x64_128_digest(numpy.random.rand(100))
b'\x8c\xee\xc6z\xa9\xfeR\xe8o\x9a\x9b\x17u\xbe\xdc\xee'

Various alternatives are available, offering different return types (e.g., signed integers, tuples of unsigned integers) and optimized for different architectures. For a comprehensive list of functions, refer to the API Reference.

hashlib-style hashers

mmh3 implements hasher objects with interfaces similar to those in hashlib from the standard library, although they are still experimental. See Hasher Classes in the API Reference for more information.

Changelog

See Changelog for the complete changelog.

5.0.1 - 2024-09-22

Fixed

5.0.0 - 2024-09-18

Added

Changed

Deprecated

Fixed

4.1.0 - 2024-01-09

Added

Fixed

License

MIT, unless otherwise noted within a file.

Known Issues

Different results from other MurmurHash3-based libraries

By default, mmh3 returns signed values for the 32-bit and 64-bit versions and unsigned values for hash128 due to historical reasons. To get the desired result, use the signed keyword argument.

Starting from version 4.0.0, mmh3 is endian-neutral, meaning that its hash functions return the same values on big-endian platforms as they do on little-endian ones. In contrast, the original C++ library by Appleby is endian-sensitive. If you need results that comply with the original library on big-endian systems, please use version 3.*.

For compatibility with Google Guava (Java), see https://stackoverflow.com/questions/29932956/murmur3-hash-different-result-between-python-and-java-implementation.

For compatibility with murmur3 (Go), see https://github.com/hajimes/mmh3/issues/46.

Contributing Guidelines

See Contributing.

Authors

MurmurHash3 was originally developed by Austin Appleby and distributed under public domain https://github.com/aappleby/smhasher.

Ported and modified for Python by Hajime Senuma.

External Tutorials

High-performance computing

The following textbooks and tutorials are great resources for learning how to use mmh3 (and other hash algorithms in general) for high-performance computing.

Internet of things

Shodan, the world's first IoT search engine, uses MurmurHash3 hash values for favicons (icons associated with web pages). ZoomEye follows Shodan's convention. Calculating these values with mmh3 is useful for OSINT and cybersecurity activities.

Related Libraries