Home

Awesome

simdjson-java

Build Status

A Java version of simdjson - a JSON parser using SIMD instructions, based on the paper Parsing Gigabytes of JSON per Second by Geoff Langdale and Daniel Lemire.

Code Sample

DOM Parser

byte[] json = loadTwitterJson();

SimdJsonParser parser = new SimdJsonParser();
JsonValue jsonValue = parser.parse(json, json.length);
Iterator<JsonValue> tweets = jsonValue.get("statuses").arrayIterator();
while (tweets.hasNext()) {
    JsonValue tweet = tweets.next();
    JsonValue user = tweet.get("user");
    if (user.get("default_profile").asBoolean()) {
        System.out.println(user.get("screen_name").asString());
    }
}

Schema-Based Parser

byte[] json = loadTwitterJson();

SimdJsonParser parser = new SimdJsonParser();
SimdJsonTwitter twitter = simdJsonParser.parse(buffer, buffer.length, SimdJsonTwitter.class);
for (SimdJsonStatus status : twitter.statuses()) {
    SimdJsonUser user = status.user();
    if (user.default_profile()) {
        System.out.println(user.screen_name());
    }
}

record SimdJsonUser(boolean default_profile, String screen_name) {
}

record SimdJsonStatus(SimdJsonUser user) {
}

record SimdJsonTwitter(List<SimdJsonStatus> statuses) {
}

Installation

The library is available in the Maven Central Repository. To include it in your project, add the following dependency to the build.gradle file:

implementation("org.simdjson:simdjson-java:0.1.0")

or to the pom.xml file:

<dependency>
    <groupId>org.simdjson</groupId>
    <artifactId>simdjson-java</artifactId>
    <version>0.1.0</version>
</dependency>

Please remember about specifying the desired version.

Note that simdjson-java follows the SemVer specification, which means, for example, that a major version of zero indicates initial development, so the library's API should not be considered stable.

We require Java 18 or better.

Benchmarks

To run the JMH benchmarks, execute the following command:

./gradlew jmh

Tests

To run the tests, execute the following command:

./gradlew test

Performance

This section presents a performance comparison of different JSON parsers available as Java libraries. The benchmark used the twitter.json dataset, and its goal was to measure the throughput (ops/s) of parsing and finding all unique users with a default profile.

256-bit Vectors

Environment:

DOM parsers (ParseAndSelectBenchmark):

LibraryVersionThroughput (ops/s)
simdjson-java (padded)0.3.0783.878
simdjson-java0.3.0760.426
fastjson22.0.49308.660
jackson2.17.0259.536

Schema-based parsers (SchemaBasedParseAndSelectBenchmark):

LibraryVersionThroughput (ops/s)
simdjson-java (padded)0.3.01237.432
simdjson-java0.3.01216.891
jsoniter-scala2.28.4614.138
fastjson22.0.49494.362
jackson2.17.0339.904

512-bit Vectors

Environment:

DOM parsers (ParseAndSelectBenchmark):

LibraryVersionThroughput (ops/s)
simdjson-java (padded)0.3.01842.146
simdjson-java0.3.01765.592
fastjson22.0.49718.133
jackson2.17.0616.617

Schema-based parsers (SchemaBasedParseAndSelectBenchmark):

LibraryVersionThroughput (ops/s)
simdjson-java (padded)0.3.03164.274
simdjson-java0.3.02990.289
jsoniter-scala2.28.41826.229
fastjson22.0.491259.622
jackson2.17.0789.030

To reproduce the benchmark results, execute the following command:

./gradlew jmh -Pjmh.includes='.*ParseAndSelectBenchmark.*'

The benchmark may take several minutes. Remember that you need Java 18 or better.