They’re good for allowing some projects to simulate having their own dedicated setup (computer). So that:
- You don’t need to give the program admin permissions.
- It’s especially good if you don’t think you’ll be using the program anywhere else.
- Prevents conflicts
- say you like python3, but need python2 for a program, just install python2 in the environment
- You can always close and open past environments.
Python Virtual Environments vs. Docker
A Python environment (like a virtual environment) and Docker both help manage dependencies, but they work at different levels:
Python Virtual Environment (venv, conda, etc.)
- 🏠 Manages Python packages for a specific project.
- 🐍 Isolates dependencies inside Python (e.g., different versions of
numpy
,pandas
). - 🚫 Doesn’t handle system dependencies (e.g., database servers, different OS versions).
- 💻 Runs on the host machine.
Docker
- 📦 Encapsulates everything (OS, libraries, code, and dependencies).
- 🌍 Works across different machines without compatibility issues.
- 🏗️ Can run different versions of Python, databases, or even entire applications.
- 🚀 More powerful for deploying apps on servers.
Key Difference
👉 Python environments isolate dependencies at the Python level, while Docker isolates the entire system environment.
Pip & Venv vs. Conda Package Management
pip
: Installs Python packages from PyPI. It resolves dependencies but doesn’t handle system libraries.conda
: Installs packages from Anaconda repositories. It manages both Python and system dependencies, making it more reliable for scientific computing.
Use pip
for general Python packages and conda
for scientific/ML packages with complex dependencies.
venv
: Only manages Python environments; you install packages withpip
. Doesn’t handle system dependencies.conda
: Manages both Python environments and system dependencies (e.g., CUDA, MKL). Works withconda
packages instead of justpip
.
Use venv
for lightweight projects and conda
for ML, data science, or complex dependencies.
Handling Virtual Environments
Create
conda create --name env_name python=3.10
Activate
conda activate env_name
Deactivate
conda deactivate
Terminology
- Global Environment is called base