Niraj Zade
Data Engineer
I work at the intersection of computing and data.
Typically help companies/people with data ingestion, processing, and storage on the cloud.
To unlock downstream data science and business intelligence.
At any scale.
Have spent a decade working on computers, and 2.5 years of it working at tech companies.
I seriously think about things like tech debt, maintainability/operational burden, on-boarding speed (documentation etc), testability, fault-tolerance, load bearing capacity etc.
---- ---- ---- ---- ---- ---- ---- ----
Have been a relatively early adopter of the cloud- since 2011 itself.
(Used to borrow a fleet of build servers. Later on, built a data & media sharing platform for my group).
I bring sharp understanding and intuitions of the overall field, developed over a decade and half of extremely heavy studies and hands-on experimentation - in my home lab, and on the cloud.
Past work includes:
Contact
Computers understanding humans makes codebases irrelevant
A consequence of good code generation models
2023 Apr 08
8 min
blog (4)
gen-ai (1)
Own your email's domain
Protect identity loss by owning the email's domain
2023 Feb 12
6 min
blog (4)
Why python needs virtual environments
An opinion on the python package and env management shitshow
2023 Feb 04
10 min
package-management (1)
python (2)
blog (4)
Why interpreted languages make superior scripting languages
Why {python, bash, ruby, ...} are preferred for scripting over {go, c, ...}
2023 Jan 03
1 min
blog (4)
Resources
These are constantly evolving notes.
aws (1) conventions (1) data-engineering (1) dbms (2) distributed-systems (1) git (1) infra (1) infrastructure (2) linux (1) postgres (2) python (2) software-engineering (2) sql (1) study (1)
Postgres - data types
pg data types, and when to use them
2024 Oct 13
4 min
postgres (2)
dbms (2)
Postgres - index types and usage scenarios
Postgres index types, their tradeoffs, and when to use them
2024 Oct 13
7 min
postgres (2)
dbms (2)
Data and Computing books list
An evolving list of books
2024 Oct 07
1 min
study (1)
SSH tunneling templates
SSH config templates for various scenarios
2024 Sep 22
1 min
infrastructure (2)
linux (1)
Software port list
An ever-expanding list of default ports of common services, for on-prem infra and home-labs.
2024 Sep 21
2 min
infra (1)
UUID - versions, details, use cases
There are 8 versions of uuid, each with its own specification. You must know how a version works, to know the use cases where it can be used.
2024 Sep 13
16 min
distributed-systems (1)
software-engineering (2)
Git based development - Standards and Etiquette
Standards and guidelines on writing commit messages, creating branches, creating pull requests.
2024 Aug 28
9 min
git (1)
conventions (1)
software-engineering (2)
Python unicode string comparison and normalization schemes
Universally applicable string comparison in python. For string comparisons, and for de-duplication in datasets.
2024 May 06
8 min
python (2)
Ram per core analysis of AWS EC2 instances
Ram per core statistics of AWS EC2 instances
2024 Mar 13
34 min
aws (1)
infrastructure (2)
SQL logical clause execution order
Like operator precedence, but for SQL clauses.
2023 Nov 15
2 min
sql (1)
data-engineering (1)
Tools