Tag: data-engineering (2)

All tags: blog conventions data-engineering gen-ai git python spark

Spark Adaptive Query Engine (AQE) - all the details you need to know
Spark Adaptive Query Engine - what problems it solves and how it works in detail.
2024-02-20 15 min spark data-engineering

Spark join strategies
Spark join strategies
2024-01-22 16 min spark data-engineering

Want to build a websites imilar to this? The recipie is quite simple once you sit down to do it.
A daily cron job runs a python script that converts markdown files into a static website, and pushes it to git.
CI-CD is set up on the git branch, which deploys the website onto cloudflare pages. Take just a day to build up from scratch.