Python Virtual Environments: Manage Dependencies with virtualenv, Poetry, Pipenv, and UV

Introduction

Welcome to this lab on Python Virtual Environments. The use of a virtual environment is to test Python code in encapsulated environments and to avoid filling the base Python installation with libraries that might only be used for a single project. This practice is crucial for maintaining clean, reproducible, and conflict-free development setups.

In this lab, you will get hands-on experience with five popular tools for managing Python virtual environments and dependencies:

virtualenv: The foundational tool for creating isolated Python environments.
Poetry: An all-in-one tool for dependency management, packaging, and publishing.
Pipenv: A tool that aims to bring the best of all packaging worlds to Python.
Anaconda: A popular platform for data science that includes powerful environment management capabilities.
UV: An extremely fast, next-generation Python package installer and resolver.

By the end of this lab, you will be able to choose and use the right tool for your project's needs.

virtualenv

In this step, you will learn to use virtualenv to create and manage isolated Python environments. virtualenv is a tool to create isolated Python environments, allowing you to test Python code in encapsulated environments and avoid filling the base Python installation with libraries that might only be used for a single project. The tool has been pre-installed for you.

First, let's create a directory to store your virtual environments and then create a new virtual environment named HelloWorld. Open a terminal in the WebIDE and run the following commands:

mkdir -p ~/.virtualenvs
virtualenv ~/.virtualenvs/HelloWorld

You will see some output as the environment is created. The virtual environment will be created in the ~/.virtualenvs/HelloWorld directory.

created virtual environment CPython3.10.12.final.0-64 in 257ms
  creator CPython3Posix(dest=/home/labex/.virtualenvs/HelloWorld, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/labex/.local/share/virtualenv)
    added seed packages: pip==24.0, setuptools==69.5.1, wheel==0.43.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

Now, let's activate the virtual environment. This will change your terminal prompt to include (HelloWorld), indicating that the virtual environment is now active. Anything you install using pip will now be specific to this environment and will not affect your global Python installation.

source ~/.virtualenvs/HelloWorld/bin/activate

After activation, your prompt will change to show (HelloWorld):

(HelloWorld) labex:project/ $

To exit the virtual environment, simply use the deactivate command:

deactivate

Notice how the (HelloWorld) prefix disappears from your terminal prompt.

To reactivate the environment later, you can use the source command again from anywhere:

source ~/.virtualenvs/HelloWorld/bin/activate

Your prompt will change back, and you'll be in the HelloWorld environment again, ready to work on your project.

Poetry

In this step, you will explore Poetry, a modern tool for dependency management and packaging in Python. It allows you to declare the libraries your project depends on, and it will manage (install/update) them for you. Poetry is already installed in this environment.

Make sure you have deactivated your virtual environment.

deactivate

First, let's create a new project using Poetry. This command will set up a standard directory structure for your project.

poetry new my-poetry-project

This command creates a new directory named my-poetry-project with the following structure:

my-poetry-project
├── pyproject.toml
├── README.md
├── my_poetry_project
│   └── __init__.py
└── tests
    ├── __init__.py
    └── test_my_poetry_project.py

The most important file here is pyproject.toml, which orchestrates your project and its dependencies. Let's view its initial content.

cat my-poetry-project/pyproject.toml

The output will look similar to this:

[tool.poetry]
name = "my-poetry-project"
version = "0.1.0"
description = ""
authors = ["Your Name <you@example.com>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.10"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Now, navigate into your new project directory:

cd my-poetry-project

To add a new dependency, use the poetry add command. Poetry will automatically find a suitable version and add it to pyproject.toml. Let's add the pendulum library.

poetry add pendulum

To install all dependencies listed in pyproject.toml (for example, when cloning a project), you would run:

poetry install

Finally, to remove a dependency, use the poetry remove command:

poetry remove pendulum

This removes the package and updates the pyproject.toml file accordingly.

Pipenv

In this step, you'll learn to use Pipenv, a tool that aims to bring the best of all packaging worlds to Python. It automatically creates and manages a virtualenv for your projects, as well as adding/removing packages from your Pipfile as you install/uninstall packages.

First, create a directory for your new project and navigate into it.

cd ~/project
mkdir my_pipenv_project

cd my_pipenv_project

Now, let's install a package. Pipenv will automatically create a new virtual environment and a Pipfile to track your dependencies. Let's install the requests library.

pipenv install requests

After the command finishes, you will see two new files in your directory: Pipfile and Pipfile.lock. The Pipfile is used to track which dependencies your project needs. Let's inspect its contents:

cat Pipfile

The output will show the requests package listed under [packages].

[[source]]
url = "https://pypi.org/simple"
verify_ssl = false
name = "pypi"

[packages]
requests = "*"

[dev-packages]

[requires]
python_version = "3.10"

To activate the virtual environment associated with your project, run:

pipenv shell

Your terminal prompt will change, indicating you are now inside the project's isolated environment.

To uninstall a package, you can run the uninstall command from within the activated shell:

pipenv uninstall requests

To exit the virtual environment shell, simply type exit:

exit

Anaconda

In this step, you will get started with Anaconda. Anaconda is a distribution of Python and R for scientific computing, that simplifies package management and deployment. Its package manager, conda, is excellent for creating and managing environments. The conda command is available in your terminal.

First, let's create a new environment named my_conda_env. We will also specify the Python version to use. The -y flag automatically confirms any prompts.

cd ~/project
conda create -n my_conda_env python=3.10 -y

Once the environment is created, you need to activate it to start using it.

conda activate my_conda_env

Your terminal prompt will now be prefixed with (my_conda_env), indicating that the environment is active. Any packages you install now will be specific to this environment. Let's install numpy as an example.

conda install numpy -y

When you are finished working in the environment, you can deactivate it.

conda deactivate

The prefix will disappear from your prompt, and you will return to the base environment.

UV

In this step, you will discover UV, an extremely fast Python package installer and resolver from Astral, the makers of Ruff. It's designed as a drop-in replacement for pip and pip-tools workflows and can be 10-100x faster. UV has been pre-installed for you.

Let's start by initializing a new project. The uv init command will create a pyproject.toml file and a virtual environment in a .venv directory.

cd ~/project
uv init my-uv-project

Now, change into the newly created project directory.

cd my-uv-project

To add a dependency, use the uv add command. This is similar to poetry add or npm install. Let's add the requests library.

uv add requests

UV automatically detects and uses the virtual environment in the current directory. You can run commands within this managed environment using uv run. Let's check the Python version inside the environment.

uv run python --version

You should see the Python version of the virtual environment, not the global one.

If you prefer to activate the shell manually, you can do so just like with a standard venv:

source .venv/bin/activate

Your prompt will change to show (.venv), indicating the environment is active. To leave the activated shell, use the deactivate command.

deactivate

UV simplifies workflows by providing a single, incredibly fast binary for package and environment management.

Summary

In this lab, you gained hands-on experience with several essential tools for managing Python virtual environments and dependencies. Properly managing your project's environment is a fundamental skill for any Python developer, ensuring reproducibility and avoiding conflicts.

You learned to:

Use virtualenv for creating, activating, and managing basic, isolated environments.
Leverage Poetry for a unified workflow that handles project creation, dependency management, and packaging through the pyproject.toml file.
Utilize Pipenv to manage application dependencies declaratively with a Pipfile, simplifying the process of setting up a development environment.
Work with Anaconda (conda) to create and manage environments, a tool especially popular in the data science community for its robust handling of complex packages.
Discover UV, the new, extremely fast tool that unifies package installation, resolution, and environment management into a single binary.

Each tool has its strengths, and the best choice often depends on your project's specific needs and your personal workflow preferences. You are now equipped to make an informed decision and effectively manage your Python projects.