Installation

This guide covers all the ways you can install and set up Axolotl for your environment.

1 Requirements

  • NVIDIA GPU (Ampere architecture or newer for bf16 and Flash Attention) or AMD GPU
  • Python ≥3.11
  • PyTorch ≥2.9.0

2 Installation

Important

For Blackwell GPUs, please use Pytorch 2.9.1 and CUDA 12.8.

2.1 Quick Install

Axolotl uses uv as its package manager. uv is a fast, reliable Python package installer and resolver built in Rust.

Install uv if not already installed:

curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env

Choose your CUDA version (e.g. cu128, cu130), create a venv, and install:

export UV_TORCH_BACKEND=cu128  # or cu130
uv venv --no-project --relocatable
source .venv/bin/activate
uv pip install --no-build-isolation axolotl[flash-attn,deepspeed]

2.2 Edge/Development Build

For the latest features between releases:

git clone https://github.com/axolotl-ai-cloud/axolotl.git
cd axolotl
export UV_TORCH_BACKEND=cu128  # or cu130
uv sync --extra flash-attn --extra deepspeed
source .venv/bin/activate

uv sync creates a .venv, installs exact pinned versions from uv.lock, and sets up an editable install automatically.

2.3 Docker

docker run --gpus '"all"' --rm -it --ipc=host axolotlai/axolotl-uv:main-latest

For development with Docker:

docker compose up -d
TipAdvanced Docker Configuration 
docker run --privileged --gpus '"all"' --shm-size 10g --rm -it \
  --name axolotl --ipc=host \
  --ulimit memlock=-1 --ulimit stack=67108864 \
  --mount type=bind,src="${PWD}",target=/workspace/axolotl \
  -v ${HOME}/.cache/huggingface:/root/.cache/huggingface \
  axolotlai/axolotl-uv:main-latest
Important

For Blackwell GPUs, please use axolotlai/axolotl-uv:main-py3.11-cu128-2.9.1 or the cloud variant axolotlai/axolotl-cloud-uv:main-py3.11-cu128-2.9.1.

Please refer to the Docker documentation for more information on the different Docker images that are available.

3 Cloud Environments

3.1 Cloud GPU Providers

For providers supporting Docker:

3.2 Google Colab

4 Platform-Specific Instructions

4.1 macOS

uv pip install --no-build-isolation -e '.'

See Section 7 for Mac-specific issues.

4.2 Windows

Important

We recommend using WSL2 (Windows Subsystem for Linux) or Docker.

5 Migrating from pip to uv

If you have an existing pip-based Axolotl installation, you can migrate to uv:

# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env

# Create a fresh venv (recommended for a clean start)
export UV_TORCH_BACKEND=cu128  # or cu130
uv venv --no-project --relocatable
source .venv/bin/activate

# Reinstall axolotl
uv pip install --no-build-isolation axolotl[flash-attn,deepspeed]

6 Using pip (Alternative)

If you are unable to install uv, you can still use pip directly.

Important

Please make sure to have PyTorch installed before installing Axolotl with pip.

Follow the instructions at: https://pytorch.org/get-started/locally/

pip3 install -U packaging setuptools wheel ninja
pip3 install --no-build-isolation axolotl[flash-attn,deepspeed]

For editable/development installs:

pip3 install -U packaging setuptools wheel ninja
pip3 install --no-build-isolation -e '.[flash-attn,deepspeed]'

7 Troubleshooting

If you encounter installation issues, see our FAQ and Debugging Guide.