Skip to content

Quickstart

dstack is an open-source platform that simplifies training, fine-tuning, and deploying generative AI models, leveraging the open-source ecosystem.

Installation

To use dstack, either set up the open-source server (and configure your own cloud accounts) or use the cloud version (which provides GPU out of the box).

Open-source

If you wish to use dstack with your own cloud accounts, you can do it by using the open-source server.

Install the server

The easiest way to install the server, is via pip:

$ pip install "dstack[all]" -U

Another way to install the server is through Docker.

Configure the server

If you have default AWS, GCP, or Azure credentials on your machine, the dstack server will pick them up automatically.

Otherwise, you need to manually specify the cloud credentials in ~/.dstack/server/config.yml. For further details, refer to server configuration.

Start the server

To start the server, use the dstack server command:

$ dstack server

Applying configuration...
---> 100%

The server is running at http://127.0.0.1:3000/.
The admin token is bbae0f28-d3dd-4820-bf61-8f4bb40815da

Client configuration

At startup, the server automatically configures CLI and API with the server address, user token, and the default project name (main).

To use CLI and API on different machines or projects, use dstack config.

$ dstack config --server <your server adddress> \
    --project <your project name> \
    --token <your user token>

The client configuration is stored via ~/.dstack/config.yml.

dstack Cloud

If you want to use the cloud version of dstack, sign up, and configure the client with server address, user token, and project name using dstack config.

$ dstack config --server https://cloud.dstack.ai \
    --project <your project name> \
    --token <your user token>

The client configuration is stored via ~/.dstack/config.yml.

Once dstack is set up, you can use CLI or API.

Using the CLI

The CLI allows you to define configurations (what you want to run) as YAML files and run them using the dstack run command.

Define a configuration

First, create a YAML file in your project folder. Its name must end with .dstack.yml (e.g. .dstack.yml or train.dstack.yml are both acceptable).

type: dev-environment

python: "3.11" # (Optional) If not specified, your local version is used

ide: vscode

A dev environments is a perfect tool for interactive experimentation with your IDE. It allows to pre-configure the Python version or a Docker image, etc. Go to Dev environments to learn more.

type: task

ports:
  - 7860

python: "3.11" # (Optional) If not specified, your local version is used.

commands:
  - pip install -r requirements.txt
  - gradio app.py

A task may run training scripts, batch jobs, or web apps. It allows to specify the commands, ports, and pre-configure the Python version or a Docker image, etc. Go to Tasks to learn more.

type: service

image: ghcr.io/huggingface/text-generation-inference:latest

env: 
  - MODEL_ID=TheBloke/Llama-2-13B-chat-GPTQ 

port: 80

commands:
  - text-generation-launcher --hostname 0.0.0.0 --port 80 --trust-remote-code

A service makes it very easy to deploy models or web apps. It allows to specify the commands, and the Python version or a Docker image, etc. Go to Services to learn more.

Run configuration

To run a configuration, use the dstack run command followed by the working directory path, configuration file path, and any other options (e.g., for requesting hardware resources).

$ dstack run . -f train.dstack.yml --gpu A100

 BACKEND     REGION         RESOURCES                     SPOT  PRICE
 tensordock  unitedkingdom  10xCPU, 80GB, 1xA100 (80GB)   no    $1.595
 azure       westus3        24xCPU, 220GB, 1xA100 (80GB)  no    $3.673
 azure       westus2        24xCPU, 220GB, 1xA100 (80GB)  no    $3.673

Continue? [y/n]: y

Provisioning...
---> 100%

Epoch 0:  100% 1719/1719 [00:18<00:00, 92.32it/s, loss=0.0981, acc=0.969]
Epoch 1:  100% 1719/1719 [00:18<00:00, 92.32it/s, loss=0.0981, acc=0.969]
Epoch 2:  100% 1719/1719 [00:18<00:00, 92.32it/s, loss=0.0981, acc=0.969]

No need to worry about copying code, setting up environment, IDE, etc. dstack handles it all automatically.