Home

Awesome

Quetzal (Query Tranzlation Libraries)

SPARQL to SQL translation engine for multiple backends, such as DB2, PostgreSQL and Apache Spark.

Philosophy

The goal of Quetzal is to provide researchers with a framework to experiment with various techniques to store and query graph data efficiently. To this end, we provide 3 modular components that:

Overview of Components

For more information on how to get started, click on the Wiki to this repository

Install and build issues

If you are building from source, get the following: git clone https://github.com/themadcreator/rabinfingerprint and build using maven.

Storage of graph data on cloud SQL backing stores such as Spanner and BigQuery

Since the time we worked on Quetzal, a number of cloud databases have emerged that support the complex SQL queries needed to access graph data. One question that we started to ask recently is whether storage of graph data is better suited for a column oriented, nested type data layout such as BigQuery, or whether a row store such as Spanner is better suited for storage of graph data. There are tradeoffs to each, and this is by no means an exhaustive comparison of the two different approaches, but we performed some very initial experiments on the following layout on BigQuery versus Spanner for a simple graph query which is not just a 1 hop neighborhood of a node, and we note the rather interesting results here.

<http\://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl\#undergraduateDegreeFrom>=col_8 <http\://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl\#emailAddress>=col_6 <http\://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl\#mastersDegreeFrom>=col_5 <http\://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl\#memberOf>=col_12 <http\://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl\#worksFor>=col_3 <http\://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl\#teachingAssistantOf>=col_15 <http\://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl\#subOrganizationOf>=col_16 <http\://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl\#researchInterest>=col_9 <http\://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl\#teacherOf>=col_7 <http\://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl\#name>=col_2 <http\://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl\#headOf>=col_11 <http\://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl\#telephone>=col_4 <http\://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl\#publicationAuthor>=col_0 <http\://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl\#takesCourse>=col_14 <http\://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl\#doctoralDegreeFrom>=col_10 <http\://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl\#advisor>=col_13 <http\://www.w3.org/1999/02/22-rdf-syntax-ns\#type>=col_1

{"schema":{"fields":[{"name":"subject","type":"string"},{"name":"col_8","type":"string"},{"name":"col_6","type":"string"},{"name":"col_5","type":"string"},{"name":"col_12","type":"string"},{"name":"col_3","type":"string"},{"name":"col_15","type":"string"},{"name":"col_9","type":"string"},{"name":"col_16","type":"string"},{"name":"col_7","type":"string","mode":"repeated"},{"name":"col_2","type":"string"},{"name":"col_11","type":"string"},{"name":"col_4","type":"string"},{"name":"col_0","type":"string","mode":"repeated"},{"name":"col_14","type":"string","mode":"repeated"},{"name":"col_10","type":"string"},{"name":"col_13","type":"string"},{"name":"col_1","type":"string","mode":"repeated"}]}}

static void createDatabase(DatabaseAdminClient dbAdminClient, DatabaseId id) { Operation<Database, CreateDatabaseMetadata> op = dbAdminClient.createDatabase( id.getInstanceId().getInstance(), id.getDatabase(), Arrays.asList("CREATE TABLE DPH (\n" + " subject STRING(MAX) NOT NULL,\n" + " col_0 ARRAY<STRING(MAX)>, \n" + " col_1 ARRAY<STRING(MAX)>, \n" + " col_2 STRING(MAX), \n" + " col_3 STRING(MAX), \n" + " col_4 STRING(MAX), \n" + " col_5 STRING(MAX), \n" + " col_6 STRING(MAX), \n" + " col_7 ARRAY<STRING(MAX)>, \n" + " col_8 STRING(MAX), \n" + " col_9 STRING(MAX), \n" + " col_10 STRING(MAX), \n" + " col_11 STRING(MAX), \n" + " col_12 STRING(MAX), \n" + " col_13 STRING(MAX), \n" + " col_14 ARRAY<STRING(MAX)>, \n" + " col_15 STRING(MAX), \n" + " col_16 STRING(MAX)) \n" + " PRIMARY KEY (subject)")); Database db = op.waitFor().getResult(); System.out.println("Created database [" + db.getId() + "]"); }

with t1 as (select subject as student, col_13 as advisor from lubm.DPH where col_13 is not null), t2 as (select subject as student, col_14 as course from lubm.DPH where col_14 is not null), t3 as (select subject as teacher, col_7 as course from lubm.DPH where col_7 is not null), t4 as (select teacher, course from t3 t, t.course course), t5 as (select student, course from t2 t, t.course course) select t5.student, t4.teacher, t4.course from t4, t5, t1 where t4.course = t5.course and t4.teacher = t1.advisor and t5.student = t1.student

select dph1.subject as student, dph1.col_13 as advisor, course, dph2.subject as teacher, c from DPH as dph1, DPH as dph2 cross join unnest(dph1.col_14) as course cross join unnest(dph2.col_7) as c where dph1.col_13 is not null and dph1.col_14 is not null and dph2.col_7 is not null and course = c and dph1.col_13 = dph2.subject