Niraj Zade
Senior Data Engineer | Software Engineer

I help companies build centralized data lakehouses - by gathering data from disparate sources (APIs, databases, tools etc) - to enable AI, Data Science & Data Driven Business Decisions.

All this while not breaking the bank.

Tech I most frequently work with: Spark, FastAPI, Python, SQL, Azure (and also AWS), Airflow, Docker

CONTACT

Email: niraj[at]theniraj[dot]com
LinkedIn - linkedin.com/in/nirajzade/

I simply enjoy computers. A LOT. Way too much actually.
If computing wasn't a career - I'd be doing some other job in the day, and be glued to computers at the night.
Haven't found any other line of work that's so (frustratingly) loveable. I'm glad that the world has a path to turn software engineering into a career.

Blog posts

Articles

I learn through writing. Most of these are ever evolving pieces.

python

python // 2023 Sep 17 | 5m | 1058 words

Python gotchas compilation

Variable scope leakage, global variable read-write issues

Contents:
  1. Variable scope "leakage"
  2. Can read a global without global keyword, but can't write to it
python // 2024 May 06 | 7m | 1303 words

Unicode string normalization schemes in Python

Universally applicable string comparison in python. For string comparisons, and for de-duplication in datasets.

Contents:
  1. The problem
  2. What makes unicode string comparison hard?
  3. Solution
  4. Normalization in depth
  5. When to and when not to use normalization
  6. Resources

resources

resources // 2025 Jun 15 | 3m | 626 words

Links collection

An ever evolving list of links. Will keep on getting cleaned, organized, appended, disorganized.

Contents:
  1. Blogs list
  2. Podcasts
  3. Computers
  4. Homelab
  5. Infra
  6. Human
  7. Math
resources // 2025 Jun 02 | 3m | 715 words

Papers, books, talks etc

A centralized collection of papers, talks, lectures around computer science, engineering, and the overall industry

Contents:
  1. Meta
  2. General
  3. Software engineering
  4. AI (not theory)
  5. Datastores
  6. Networking
  7. Web3
  8. Security

spark

spark // 2026 Jan 03 | 13m | 2496 words

Delta Lake - performance optimization and maintenance

Delta lake optmization and maintenance techniques

Contents:
  1. WORK IN PROGRESS ========
  2. Core factors affecting performance
  3. Gist
  4. Liquid clustering - Faster reads via data skipping
  5. Auto-compaction - Control file size
  6. Automating table maintenance (Databricks only)
  7. Manual intervention
  8. OPTIMIZE - data layout optimization
  9. Auto-optimization - wip - WHERE TO PUT THIS IN THIS NOTE?
  10. Z-ORDER - faster reads by data-skipping
  11. Partitioning - WIP
  12. VACUUM - Removing stale files by trimming history
spark // 2025 Oct 16 | 26m | 4773 words

Spark Microbook

A collection of spark (and pySpark) knowledge. Wrote this partly to help myself learn, and partly to have one dependable place where my colleagues and I can find the information we need.

Contents:
  1. Adaptive Query Engine (AQE)
  2. Join strategies
spark // 2024 Jan 22 | 13m | 2450 words

Spark join strategies

Spark join strategies

Contents:
  1. Types of joins
  2. Overview of available join strategies
  3. Hash Joins
  4. Nested Loop joins
  5. Sort Merge join
  6. How does spark choose a join strategy?
  7. Notes
spark // 2025 Oct 11 | 4m | 887 words

Spark performance optimization compendium

Speed up data reads from sql databases, joins, column generation

Contents:
  1. Speed up reads
  2. Slow transformations
  3. Random issues

sqlserver

sqlserver // 2025 Jul 10 | 2m | 519 words

SqlServer reference - All date & time formatters

SqlServer time formatters, to use in convert() function. Used to convert date time objects into string of appropriate formats.

Contents:
  1. Date Formatters
  2. Time Formatters
  3. Date and Time Formatters

webtech

webtech // 2024 Apr 02 | 12m | 2321 words

JWT - a quick overview of its working and encryption algorithms

A quick overview on the workings of JWT

Contents:
  1. Anatomy of a JWT
  2. JWT cryptography
  3. Vulnerabilities
  4. Performance
  5. Resources
  6. Ending notes

work

work // 2024 Oct 14 | 1hr18m | 14441 words

Lecture - You and your research by Dr. Richard Hamming

The lecture by Richard Hamming

Contents:
  1. Introduction of Dr. Richard W. Hamming
  2. THE TALK: You and Your Research by Dr. Richard W. Hamming
  3. Biographical sketch of Richard Hamming
  4. Acknowledgement
work // 2025 Oct 13 | 6m | 1253 words

Why charge more as an engineer

Why you should charge more as an engineer

Contents:
  1. Pricing signals quality
  2. The "too good to be true"
  3. The uphill battle against the first impression
  4. When charging low, make sure you're already known to charge high
  5. Positives

Tools

Engineering tools:

Other tools:

© Niraj Zade 2025 - Website, Linkedin
Website was autogenerated on 18 Jan 2026
Whoever owns storage, owns computing