Awesome
Triple Store
Triple Store is a library to manipulate RDF triples in a fast and fluent fashion.
RDF triples allow to represent any data and its relations to other data. It is a very versatile concept and is used in Linked Data, graphs traversal and storage, etc....
Here the RDF triples implementation follows along the W3C RDF concepts. (Note that reification is not implemented.). More digestible info on RDF Wikipedia
Features overview
- Create and manage triples through a convenient DSL
- Snapshot and query RDFGraphs
- Binary encoding/decoding
- Lenient NTriples encoding/decoding (see W3C Test suite in testdata/ntriples/w3c_suite/)
- DOT encoding
- Stream encoding/decoding (for binary & NTriples format) for memory conscious program
- CLI (Command line interface) utility to read and convert triples files.
Library
This library is written using the Golang language. You need to install Golang before using it.
Get it:
go get -u github.com/wallix/triplestore
Test it:
go test -v -cover -race github.com/wallix/triplestore
Bench it:
go test -run=none -bench=. -benchmem
Import it in your source code:
import (
"github.com/wallix/triplestore"
// tstore "github.com/wallix/triplestore" for less verbosity
)
Get the CLI with:
go get -u github.com/wallix/triplestore/cmd/triplestore
Concepts
A triple is made of 3 components:
subject -> predicate -> object
... or you can also view that as:
entity -> attribute -> value
So
- A triple consists of a subject, a predicate and a object.
- A subject is a unicode string.
- A predicate is a unicode string.
- An object is a resource (or IRI) or a literal (blank node are not supported).
- A literal is a unicode string associated with a datatype (ex: string, integer, ...).
- A resource, a.k.a IRI, is a unicode string which point to another resource.
And
- A source is a persistent yet mutable source or container of triples.
- A RDFGraph is an immutable set of triples. It is a snapshot of a source and queryable .
- A dataset is a basically a collection of RDFGraph.
You can also view the library through the godoc
Usage
Create triples
Although you can build triples the way you want to model any data, they are usually built from known RDF vocabularies & namespace. Ex: foaf, ...
triples = append(triples,
SubjPred("me", "name").StringLiteral("jsmith"),
SubjPred("me", "age").IntegerLiteral(26),
SubjPred("me", "male").BooleanLiteral(true),
SubjPred("me", "born").DateTimeLiteral(time.Now()),
SubjPred("me", "mother").Resource("mum#121287"),
)
or dynamically and even shorter with
triples = append(triples,
SubjPredLit("me", "age", "jsmith"), // String literal object
SubjPredLit("me", "age", 26), // Integer literal object
SubjPredLit("me", "male", true), // Boolean literal object
SubjPredLit("me", "born", time.now()) // Datetime literal object
SubjPredRes("me", "mother", "mum#121287"), // Resource object
)
or with blank nodes and language tag in literal
triples = append(triples,
SubjPred("me", "name").Bnode("jsmith"),
BnodePred("me", "name").StringLiteral("jsmith"),
SubjPred("me", "name").StringLiteralWithLang("jsmith", "en"),
)
Create triples from a struct
As a convenience you can create triples from a singular struct, where you control embedding through bnode.
Here is an example.
type Address struct {
Street string `predicate:"street"`
City string `predicate:"city"`
}
type Person struct {
Name string `predicate:"name"`
Age int `predicate:"age"`
Size int64 `predicate:"size"`
Male bool `predicate:"male"`
Birth time.Time `predicate:"birth"`
Surnames []string `predicate:"surnames"`
Addr Address `predicate:"address" bnode:"myaddress"` // empty bnode value will make bnode value random
}
addr := &Address{...}
person := &Person{Addr: addr, ....}
tris := TriplesFromStruct("jsmith", person)
src := NewSource()
src.Add(tris)
snap := src.Snapshot()
snap.Contains(SubjPredLit("jsmith", "name", "..."))
snap.Contains(SubjPredLit("jsmith", "size", 186))
snap.Contains(SubjPredLit("jsmith", "surnames", "..."))
snap.Contains(SubjPredLit("jsmith", "surnames", "..."))
snap.Contains(SubjPred("me", "address").Bnode("myaddress"))
snap.Contains(BnodePred("myaddress", "street").StringLiteral("5th avenue"))
snap.Contains(BnodePred("myaddress", "city").StringLiteral("New York"))
Equality
me := SubjPred("me", "name").StringLiteral("jsmith")
you := SubjPred("me", "name").StringLiteral("fdupond")
if me.Equal(you) {
...
}
)
Triple Source
A source is a persistent yet mutable source or container of triples
src := tstore.NewSource()
src.Add(
SubjPredLit("me", "age", "jsmith"),
SubjPredLit("me", "born", time.now()),
)
src.Remove(SubjPredLit("me", "age", "jsmith"))
RDFGraph
A RDFGraph is an immutable set of triples you can query. You get a RDFGraph by snapshotting a source:
graph := src.Snapshot()
tris := graph.WithSubject("me")
for _, tri := range tris {
...
}
Codec
Triples can be encoded & decoded using either a simple binary format or more standard text format like NTriples, ...
Triples can therefore be persisted to disk, serialized or sent over the network.
For example
enc := NewBinaryEncoder(myWriter)
err := enc.Encode(triples)
...
dec := NewBinaryDecoder(myReader)
triples, err := dec.Decode()
Create a file of triples under the lenient NTriples format:
f, err := os.Create("./triples.nt")
if err != nil {
return err
}
defer f.Close()
enc := NewLenientNTEncoder(f)
err := enc.Encode(triples)
Encode to a DOT graph
tris := []Triple{
SubjPredRes("me", "rel", "you"),
SubjPredRes("me", "rdf:type", "person"),
SubjPredRes("you", "rel", "other"),
SubjPredRes("you", "rdf:type", "child"),
SubjPredRes("other", "any", "john"),
}
err := NewDotGraphEncoder(file, "rel").Encode(tris...)
...
// output
// digraph "rel" {
// "me" -> "you";
// "me" [label="me<person>"];
// "you" -> "other";
// "you" [label="you<child>"];
//}
Load a binary dataset (i.e. multiple RDFGraph) concurrently from given files:
path := filepath.Join(fmt.Sprintf("*%s", fileExt))
files, _ := filepath.Glob(path)
var readers []io.Reader
for _, f := range files {
reader, err := os.Open(f)
if err != nil {
return g, fmt.Errorf("loading '%s': %s", f, err)
}
readers = append(readers, reader)
}
dec := tstore.NewDatasetDecoder(tstore.NewBinaryDecoder, readers...)
tris, err := dec.Decode()
if err != nil {
return err
}
...
triplestore CLI
This CLI is mainly ised for triples files conversion and inspection. Install it with go get github.com/wallix/triplestore/cmd/triplestore
. Then triplestore -h
for help.
Example of usage:
triplestore -in ntriples -out bin -files fuzz/ntriples/corpus/samples.nt
triplestore -in ntriples -out bin -files fuzz/ntriples/corpus/samples.nt
triplestore -in bin -files fuzz/binary/corpus/samples.bin
RDFGraph as a Tree
A tree is defined from a RDFGraph given:
- a specific predicate as an edge
- and considering triples pointing to RDF resource Object
You can then navigate the tree using the existing API calls
tree := tstore.NewTree(myGraph, myPredicate)
tree.TraverseDFS(...)
tree.TraverseAncestors(...)
tree.TraverseSiblings(...)
Have a look at the godoc fro more info
Note that at the moment, constructing a new tree from a graph does not verify if the tree is valid namely no cycle and each child at most one parent.