niraj zade :::: ~/articles ~/tools ~/hire

Recommended python packages

By Niraj Zade | 2024 Oct 14 | 0 min read

Infrastructure
- python-dotenv - read key-value pairs from .env file and set them as env variables
- nuikta - python compiler to package the application into a single binary. Useful for avoiding pip while building images, or shipping a single executable to users (eg - desktop apps, binaries to deploy via ansible etc)
Testing
- pytest - Easier to work with. Better testing than unittest (fixtures etc)
HTTP requests
- httpx (built on top of requests. Brings in async support, http2)
- requests - not recommended as httpx now exists.
HTML parsing
- selectolax - generally 20x faster than beautifulsoup4.
- beautifulsoup4 - simple to use. Has limited parallelism as its HTTP calls are blocking.

Table Of Contents

None

Other Articles

[2024 Oct 14] Lecture - You and your research by Dr. Richard Hamming

[2025 Jul 10] SqlServer - All date & time formatters

[2024 Jan 22] Performance Optimization - Spark join strategies

[2024 Feb 20] Performance Optimization - Spark AQE - all the details you need to know

[2024 Mar 13] Infra - Ram per core analysis of AWS EC2 instances

[2024 May 06] Unicode string normalization schemes in Python

[2024 Oct 14] Recommended python packages

[2024 Sep 22] Linux - SSH tunneling templates

[2024 Oct 13] Postgres - index types and usage scenarios

[2025 Aug 25] Catalyzing research within India

[2025 Jun 15] Links collection

[2025 Jun 02] Papers, books, talks etc

[2025 Jul 16] My subconscious doesn't like LLMs

[2023 Jan 03] Why interpreted languages make superior scripting languages

[2023 Feb 12] Own your email's domain

[2023 Apr 08] Computers understanding humans makes codebases irrelevant