Awesome

Benchmarks of Go serialization methods

This is a test suite for benchmarking various Go serialization methods.

Tested serialization methods

• encoding/gob
• encoding/json
• github.com/json-iterator/go
• github.com/alecthomas/binary
• github.com/davecgh/go-xdr/xdr
• github.com/Sereal/Sereal/Go/sereal
• github.com/ugorji/go/codec
• github.com/vmihailenco/msgpack/v4
• labix.org/v2/mgo/bson
• github.com/tinylib/msgp (code generator for msgpack)
• google.golang.org/protobuf (generated code)
• github.com/gogo/protobuf (generated code, optimized version of goprotobuf)
• go.dedis.ch/protobuf (reflection based)
• github.com/google/flatbuffers
• github.com/hprose/hprose-go/io
• github.com/glycerine/go-capnproto
• zombiezen.com/go/capnproto2
• github.com/andyleap/gencode
• github.com/pascaldekloe/colfer
• github.com/linkedin/goavro
• github.com/ikkerens/ikeapack
• github.com/raszia/gotiny
• github.com/prysmaticlabs/go-ssz
• github.com/200sc/bebop (generated code)
• github.com/shamaton/msgpackgen (generated code)
• github.com/ymz-ncnk/musgo (generated code)

Running the benchmarks

go get -u -t
go test -bench='.*' ./

To update the table in the README:

./stats.sh

Recommendation

If performance, correctness and interoperability are the most important factors, gogoprotobuf is currently the best choice. It does require a pre-processing step (eg. via Go 1.4's "go generate" command).

But as always, make your own choice based on your requirements.

Data

The data being serialized is the following structure with randomly generated values:

type A struct {
    Name     string
    BirthDay time.Time
    Phone    string
    Siblings int
    Spouse   bool
    Money    float64
}

Results

Results with Go 1.20 linux/amd64 on an Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz

Totals:

Issues

The benchmarks can also be run with validation enabled.

VALIDATE=1 go test -bench='.*' ./

Unfortunately, several of the serializers exhibit issues:

1. (minor) BSON drops sub-microsecond precision from time.Time.
2. (minor) Ugorji Binc Codec drops the timezone name (eg. "EST" -> "-0500") from time.Time.

--- FAIL: BenchmarkBsonUnmarshal-8
    serialization_benchmarks_test.go:115: unmarshaled object differed:
        &{20b999e3621bd773 2016-01-19 14:05:02.469416459 -0800 PST f017c8e9de 4 true 0.20887343719329818}
        &{20b999e3621bd773 2016-01-19 14:05:02.469 -0800 PST f017c8e9de 4 true 0.20887343719329818}
--- FAIL: BenchmarkUgorjiCodecBincUnmarshal-8
    serialization_benchmarks_test.go:115: unmarshaled object differed:
        &{20a1757ced6b488e 2016-01-19 14:05:15.69474534 -0800 PST 71f3bf4233 0 false 0.8712180830484527}
        &{20a1757ced6b488e 2016-01-19 14:05:15.69474534 -0800 -0800 71f3bf4233 0 false 0.8712180830484527}

All other fields are correct however.

Additionally, while not a correctness issue, FlatBuffers, ProtoBuffers, Cap'N'Proto and ikeapack do not support time types directly. In the benchmarks an int64 value is used to hold a UnixNano timestamp.

3. (major) Goprotobuf has been disabled in the above, as it is no longer maintained and is incompatible with the latest changes to Google's Protobuf package. See discussions: 1, 2.

panic: protobuf tag not enough fields in ProtoBufA.state:

goroutine 148 [running]:
github.com/gogo/protobuf/proto.(*unmarshalInfo).computeUnmarshalInfo(0xc0000d9c20)
 /home/ken/src/go/pkg/mod/github.com/gogo/protobuf@v1.3.2/proto/table_unmarshal.go:341 +0x135f
github.com/gogo/protobuf/proto.(*unmarshalInfo).unmarshal(0xc0000d9c20, {0xcff820?}, {0xc000122fc0, 0x35, 0x35})
 /home/ken/src/go/pkg/mod/github.com/gogo/protobuf@v1.3.2/proto/table_unmarshal.go:138 +0x67
github.com/gogo/protobuf/proto.(*InternalMessageInfo).Unmarshal(0xc00013a8c0?, {0xe6d5f0, 0xc000114060}, {0xc000122fc0?, 0x35?, 0x35?})
 /home/ken/src/go/pkg/mod/github.com/gogo/protobuf@v1.3.2/proto/table_unmarshal.go:63 +0xd0
github.com/gogo/protobuf/proto.(*Buffer).Unmarshal(0xc00009fe28, {0xe6d5f0, 0xc000114060})
 /home/ken/src/go/pkg/mod/github.com/gogo/protobuf@v1.3.2/proto/decode.go:424 +0x153
github.com/gogo/protobuf/proto.Unmarshal({0xc000122fc0, 0x35, 0x35}, {0xe6d5f0, 0xc000114060})
 /home/ken/src/go/pkg/mod/github.com/gogo/protobuf@v1.3.2/proto/decode.go:342 +0xe6
github.com/alecthomas/go_serialization_benchmarks.Benchmark_Goprotobuf_Unmarshal(0xc0000d5680)
 /home/ken/src/go/src/github.com/alecthomas/go_serialization_benchmarks/serialization_benchmarks_test.go:853 +0x26a
testing.(*B).runN(0xc0000d5680, 0x1)
 /usr/local/go/src/testing/benchmark.go:193 +0x102
testing.(*B).run1.func1()
 /usr/local/go/src/testing/benchmark.go:233 +0x59
created by testing.(*B).run1
 /usr/local/go/src/testing/benchmark.go:226 +0x9c
exit status 2
FAIL github.com/alecthomas/go_serialization_benchmarks 61.159s