Home

Awesome

DBStage – Flexible, Staged Query Compilation Playground

stability-experimental

Introduction

This repository contains a proof of concept for a configurable language-integrated runtime query compiler based on staging. The implementation relies on the Squid type-safe metaprogramming framework for Scala, which makes its code manipulation and generation capabilities fairly robust.

The main features are:

Step by step

1. define the database relations

case object Person extends Relation {
  val Id = Column[Int]("Id", primary = true)
  val Name = Column[String]("Name")
  val Age = Column[Int]("Age")
  val Sex = Column[Sex]("Sex")
}

2.a. register queries to be executed later, using a SQL-like DSL

  import Person._
  val q0 = from(Person) where ir"$Age > 18" where ir"$Sex == Male" select (Name,Age)

(Of course, one can write where ir"$Age > 18 && $Sex == Male" equivalently.)

Note that column types are checked at compile-time, but column reference consistency and ambiguities are checked at query construction time (runtime). For example if I had written select (Name,Age,Salary) it would have complained at runtime that there are no such Salary column available. (It would be easy to have a compile-time linter written in Squid to catch these errors earlier.)

2.b. load the data from the file system

  Person.loadDataFromFile("data/persons.csv", compileCode = true)

This compiles a program on-the-fly to efficiently load the data given the relation schema.

2.c. on-the-fly compile and execute queries

  q0.plan.foreach { case (name, age) => assert(age > 18); println(s"$name $age") }

Notice that the types for name and age are correctly inferred as String and Int, respectively.

Importantly, steps 2.a, 2.b and 2.c can be done in any order and can be interleaved.

Another example: all pairs of people of the same age but opposite sex:

  val m = from(Person)
  val f = from(Person)
  val q = ((m where ir"$Sex == Male") join (f where ir"$Sex == Female"))(ir"${m.Age} == ${f.Age}")
    .select (m.Age, m.Name, f.Name, m.Id, f.Id)
  q.printLines

Which prints the following:

Age(0)Name(0)Name(1)Id(0)Id(1)
41bob parkerjulia kenn16
...............

The currently supported functionalities are:

What I'd like to have in the future: