Python — Quick pointers while kick starting new python project

Every time a new python project kickstarts, following list of pointers could serve as a checklist or reference. It helps to bring in the best practices. I will try to keep this list updated based on latest trends.

Python version setup and dependency management

pyenv lets you easily switch between multiple versions of Python. It’s simple, unobtrusive, and follows the UNIX tradition of single-purpose tools that do one thing well.

pipx — Install and Run Python Applications in Isolated Environments. With this, we will install poetry or other python applications so they will reside in their own virtual environments. It should in no case be installed in the environment of the project that is to be managed by Poetry. This ensures that Poetry’s own dependencies will not be accidentally upgraded or uninstalled. (Each of the following installation methods ensures that Poetry is installed into an isolated environment.) In addition, the isolated virtual environment in which poetry is installed should not be activated for running poetry commands.

Poetry is an alternative tool for pipenv. It can help manage python dependency and also help with packaging the python application. This is my default dependency manager now.

Pipenv is a tool that aims to bring the best of all packaging worlds (bundler, composer, npm, cargo, yarn, etc.) to the Python world. Windows is a first-class citizen, in our world.

Install pipx

# Example script for Mac
brew install pipx
pipx ensurepath
sudo pipx ensurepath --global # optional to allow pipx actions with --global argument

• More detailed installation steps are here

Using pyenv and poetry

If the application that we like to develop needs python packaging needs, we might prefer poetry.

pyenv versions       # to show already installed versions
pyenv install --list # list available versions that can be installed 
pyenv install 3.10.3 # only needed for installing this python
pyenv local 3.8.1    # Activate Python
eval "$(pyenv init --path)" # exports shims in path. https://github.com/pyenv/pyenv#understanding-shims

poetry init
poetry config --list # will show configuration. We can configure globally or locally
poetry config virtualenvs.in-project true # Globat set. I like to have virtualenv within project folder.
poetry config --local virtualenvs.in-project true # set only for local
# after setting in-project, while we add dependency, poetry will create venv and do everything.
poetry shell # to shell into virtual environment

• In poetry if we need to add packages extras like `uvicorn[standard]`

## to add extras like uvicorn[standard] use --extras
poetry add --extras standard uvicorn==0.17.6

Using pyenv and pipenv

# Install pyenv (Mac)
brew install pyenv
# Install python
pyenv install 3.8.1
# Set application python version
# this will save it in a .python-version file
pyenv local 3.8.1
# this will initialize python version
eval "$(pyenv init -)"
# this will initialize pipenv with our chosen python version
pipenv install --python 3.8.1
# If you prefer to have the virtual env dir in same directory
# set the PIPENV_VENV_IN_PROJECT environment variable to any value PIPENV_VENV_IN_PROJECT=1 pipenv install --python 3.8.1
# To activate this project's virtualenv
pipenv shell
# Alternatively, run a command inside the virtualenv 
pipenv run

• pyenv doesn’t have the feature to list python versions that can be installed. As of now, we need to find the version details from online — https://www.python.org/downloads/

$ pipenv shell and $ pipenv run will automatically load it .env file. This is very useful to initialize PYTHONPATH or other initializations.

#.env file content
PYTHONPATH=${PYTHONPATH}:${PWD}

To automatically create virtual env always in same directory., we can have following in .bashrc or zshrc :

# for pipenv to maintain the venv in same directory
export PIPENV_VENV_IN_PROJECT=1

Pipenv useful features

# this will open a dependency library, to have quick look
pipenv open flask

REST API projects

Earlier my goto framework would be flask or django. Recently, my goto framework is FastAPI. It brings best python skills + best REST API practices. FastAPI is a modern, fast (high-performance), web framework for building APIs with Python 3.6+ based on standard Python type hints

Pydantic

Pydantic settings is a simple handy feature to configure application settings. Must have while kickstarting a new project.

Building data models based on pydantic may feel like slowing us down but it actually pays lot of dividend when kick starting new project.

Pydantic helps data validation and type hinting. It also enriches IDE like VSCode with more contextual help.

Alembic for database migration ( in SQLAlchemy environment )

While using FastAPI for relational database, I always combine it with Alembic. Alembic is a lightweight database migration tool for usage with the SQLAlchemy Database Toolkit for Python.

In this article, I have tried to capture how to setup FastAPI and alembic in detail:

Configuring Python FastAPI with SqlAlchemy and Alembic

Don’t have module level singleton objects

from functools import lru_cache


class SomeThirdPartyConnection:
    pass

@lru_cache(1)
def get_instance()
    return SomeThirdPartyConnection()


# Simple singleton
connection = get_instance()
# Don't do the above style

Don’t do this style of singleton object creation. This style of singleton objects makes unit testing much harder. The code will also have instantiation behaviours during module import.

DBMate for database migration

If the project doesn’t use SQLAlchemy, then using Alembic is not feasible. In such cases, using programming language agnostic tools is very helpful.

I like dbmate. It’s open source and it has all required features. A lightweight, framework-independent database migration tool. It has all basic features needed in a db migration tool.

I didn’t like flyway, for one important reason — they have kept undoing a migration ( `migrate down` ) as a paid feature. In a db migration library, migrate up & down are basic features. It’s like giving a free car and keeping the reverse gear as a paid feature.