Strange Loop

Next: September 12-14 2019

/

Stifel Theatre

/

St. Louis, MO

Building Scalable Data Pipelines with PostgreSQL, GraphQL and React

We have a lot of great front-end technologies today, and many interesting ways to store and process data in the backend. But often the abstractions break down when we try to build a simple architecture that processes and visualizes data as its primary goal. We might be able to create a nice frontend, but the ingestion and storage is a mess. Or the other way around, depending on the individual's background.

Most recently, GraphQL has shown that this barrier exists - a lot of people would love to use it, but not that many do, since they are missing the capable backend that can run the queries.

In this workshop I'll introduce a clean architecture for ingesting, storing, processing and visualizing data - built on PostgreSQL, GraphQL, React and d3. I'll talk about the trade-offs, the weaknesses, and what I think is a good set of best practices when starting a new data-intensive, frontend-heavy project using these technologies.

This workshop builds on my experience scaling PostgreSQL for many different kinds of companies, as well as my personal interest and background in data analysis and visualization using React and d3.

Basic knowledge of the React-based front-end stack, as well as PostgreSQL is welcome, but not necessary - this workshop is intermediate level and hopes to give you a good starting point to dive deeper into the individual pieces, or see a good example case study for you to take inspiration from.

Lukas Fittl

Citus Data

Lukas is hacking on PostgreSQL-related projects full time at Citus Data. He's previously been part of the Founding Team at Product Hunt, as well as Co-Founder at Soup.io and Spark59.