One of the greatest benefits of Clojure is its ability to create simple, powerful abstractions that operate at the level of the problem while also operating at the level of the language.
This talk discusses a query processing engine built in Clojure that leverages this abstraction power to combine streams of data for efficient concurrent execution. The first part of the talk will discuss the representation and manipulation of query execution plans:
* Representing processing trees as s-expressions
* Streams as sequences of data
* Optimizing processing trees by manipulating s-expressions
The second part will discuss simple and concurrent execution of query execution plans:
* Direct execution of s-expression trees
* Compilation of s-expressions into nodes and pipes
* Concurrent processing nodes and pipes using a fork/join pool