Quickstart¶
dstack is an open-source platform that simplifies
training, fine-tuning, and deploying generative AI models, leveraging the open-source
ecosystem.
Installation¶
To use dstack, either set up the open-source server (and configure your own cloud accounts)
or use the cloud version (which provides GPU out of the box).
Open-source
If you wish to use dstack with your own cloud accounts, you can do it
by using the open-source server.
Install the server¶
The easiest way to install the server, is via pip:
$ pip install "dstack[all]" -U
Another way to install the server is through Docker.
Configure the server¶
If you have default AWS, GCP, or Azure credentials on your machine, the dstack server will pick them up automatically.
Otherwise, you need to manually specify the cloud credentials in ~/.dstack/server/config.yml.
For further details, refer to server configuration.
Start the server¶
To start the server, use the dstack server command:
$ dstack server
Applying configuration...
---> 100%
The server is running at http://127.0.0.1:3000/.
The admin token is bbae0f28-d3dd-4820-bf61-8f4bb40815da
Client configuration¶
At startup, the server automatically configures CLI and API with the server address, user token, and
the default project name (main).
To use CLI and API on different machines or projects,
use dstack config.
$ dstack config --server <your server adddress> \
--project <your project name> \
--token <your user token>
The client configuration is stored via ~/.dstack/config.yml.
dstack Cloud
If you want to use the cloud version of dstack,
sign up, and configure the client
with server address, user token, and project name using dstack config.
$ dstack config --server https://cloud.dstack.ai \
--project <your project name> \
--token <your user token>
The client configuration is stored via ~/.dstack/config.yml.
Once dstack is set up, you can use CLI or API.
Using the CLI¶
The CLI allows you to define configurations (what you want to run) as YAML files and run them using the dstack run
command.
Define a configuration¶
First, create a YAML file in your project folder. Its name must end with .dstack.yml (e.g. .dstack.yml or train.dstack.yml
are both acceptable).
type: dev-environment
python: "3.11" # (Optional) If not specified, your local version is used
ide: vscode
A dev environments is a perfect tool for interactive experimentation with your IDE. It allows to pre-configure the Python version or a Docker image, etc. Go to Dev environments to learn more.
type: task
ports:
- 7860
python: "3.11" # (Optional) If not specified, your local version is used.
commands:
- pip install -r requirements.txt
- gradio app.py
A task may run training scripts, batch jobs, or web apps. It allows to specify the commands, ports, and pre-configure the Python version or a Docker image, etc. Go to Tasks to learn more.
type: service
image: ghcr.io/huggingface/text-generation-inference:latest
env:
- MODEL_ID=TheBloke/Llama-2-13B-chat-GPTQ
port: 80
commands:
- text-generation-launcher --hostname 0.0.0.0 --port 80 --trust-remote-code
A service makes it very easy to deploy models or web apps. It allows to specify the commands, and the Python version or a Docker image, etc. Go to Services to learn more.
Run configuration¶
To run a configuration, use the dstack run command followed by the working directory path,
configuration file path, and any other options (e.g., for requesting hardware resources).
$ dstack run . -f train.dstack.yml --gpu A100
BACKEND REGION RESOURCES SPOT PRICE
tensordock unitedkingdom 10xCPU, 80GB, 1xA100 (80GB) no $1.595
azure westus3 24xCPU, 220GB, 1xA100 (80GB) no $3.673
azure westus2 24xCPU, 220GB, 1xA100 (80GB) no $3.673
Continue? [y/n]: y
Provisioning...
---> 100%
Epoch 0: 100% 1719/1719 [00:18<00:00, 92.32it/s, loss=0.0981, acc=0.969]
Epoch 1: 100% 1719/1719 [00:18<00:00, 92.32it/s, loss=0.0981, acc=0.969]
Epoch 2: 100% 1719/1719 [00:18<00:00, 92.32it/s, loss=0.0981, acc=0.969]
No need to worry about copying code, setting up environment, IDE, etc. dstack handles it all
automatically.