© 2019 Strange Loop
Time is intrinsic to information and yet it is usually an afterthought in database designs. We present Crux, a general purpose open source document database with bitemporal graph queries.
This talk will explore the journey of how Crux was conceived from JUXT's consulting experiences of building global system integrations and providing temporal query capabilities for financial services projects.
Patrik and Jeremy from the core team will outline the overarching goals for Crux as an "unbundled", pluggable, and layered database with bitemporal ACID transactions. Bitemporality cleanly separates the time when the system learns about a fact (transaction time) from the times relating to the actual life-cycle of a fact (valid time). Modelling valid time is essential in a streaming architecture for maintaining a consistent temporal view of data. Bitemporality in Crux provides efficient and horizontally scalable point-in-time Datalog queries whilst enabling programmatic use of the database "as a value" with globally consistent queries.
The team will also discuss the design choices made to support a Datalog query engine on-top of Key/Value stores such as RocksDB and LMDB, the use of semi-immutable log storage to support eviction of sensitive data, clustering with Kafka, and the use of Clojure for constructing a layered and extensible system. Finally, they will share a view on future directions and how JUXT plans to collaborate with the open source community.
Jeremy Taylor is an Offering Manager at JUXT where he leads the development of Crux as an open source database product. Following 7 years of technical pre-sales for IBM's eCommerce systems business, Jeremy transformed his interest in Clojure and Datalog from a hobby into a career evolution by joining JUXT, the UK's foremost Clojure consultancy. As an Offering Manager, Jeremy is responsible for leading the growth of Crux towards long-term viability as a supported open source product in the enterprise database marketplace. Jeremy is passionate about database technologies and their use for knowledge representation, and he has contributed to the DataScript library through the course of his personal research.
Patrik Kårlin is one of JUXT's senior developers working on Crux. He has been developing with Clojure professionally for over 5 years and has engaged on several major consulting projects with JUXT. Prior to joining the Crux team, Patrik implemented a bitemporal data-store at an investment bank to provide queryability via GraphQL across large production data-sets for traders and developers to consume. On another project with JUXT he was instrumental in creating a secure and PCI DSS compliant cloud platform for handling personal data.