Data systems, explained from first principles.

Deep dives into Spark, Postgres and the lakehouse, with benchmarks you can rerun. Written for the people who decide how data platforms get built.

Writing

2 items

Benchmarks & Reports

2 items
Infra - Ram per core analysis of AWS EC2 instances Ram per core statistics of AWS EC2 instances Python's for-loop vs list-comprehension performance A benchmark of the 2 ways to loop over arrays, over all major python versions

Microbooks

1 item
The Spark Field Manual An engineer-focused field manual on Spark internals for new data engineers and seasoned experts who need a refresher.

Tools

5 tools
PVLDB Explorer Search every paper in the Proceedings of the VLDB Endowment, read the abstract and preview the PDF. Spark Fast Docs Instant search across the PySpark SQL and Structured Streaming API: signatures, parameters and examples. Parquet Viewer Open a Parquet file and inspect its schema, row groups and values. Nothing leaves the browser. Databricks Instances Compare cloud instance families on cores, memory and cost per DBU. Spark Memory Calculator Work out the largest partition your executors can hold before they spill.

Topics

10 tags