Skip to content

task

The task configuration type allows running tasks.

Filename

Configuration files must have a name ending with .dstack.yml (e.g., .dstack.yml or serve.dstack.yml are both acceptable) and can be located in the project's root directory or any nested folder. Any configuration can be run via dstack run.

Examples

Python version

If you don't specify image, dstack uses the default Docker image pre-configured with python, pip, conda (Miniforge), and essential CUDA drivers. The python property determines which default Docker image is used.

type: task

python: "3.11"

commands:
  - pip install -r fine-tuning/qlora/requirements.txt
  - python fine-tuning/qlora/train.py

nvcc

Note that the default Docker image doesn't bundle nvcc, which is required for building custom CUDA kernels. To install it, use conda install cuda.

Ports

A task can configure ports. In this case, if the task is running an application on a port, dstack run will securely allow you to access this port from your local machine through port forwarding.

type: task

python: "3.11"

commands:
  - pip install -r fine-tuning/qlora/requirements.txt
  - tensorboard --logdir results/runs &
  - python fine-tuning/qlora/train.py

ports:
  - 6000

When running it, dstack run forwards 6000 port to localhost:6000, enabling secure access. See tasks for more detail.

Docker image

type: dev-environment

image: dstackai/base:py3.11-0.4rc4-cuda-12.1

commands:
  - pip install -r fine-tuning/qlora/requirements.txt
  - python fine-tuning/qlora/train.py
Private registry

Use the registry_auth property to provide credentials for a private Docker registry.

type: dev-environment

image: dstackai/base:py3.11-0.4rc4-cuda-12.1
registry_auth:
  username: peterschmidt85
  password: ghp_e49HcZ9oYwBzUbcSk2080gXZOU2hiT9AeSR5

commands:
  - pip install -r fine-tuning/qlora/requirements.txt
  - python fine-tuning/qlora/train.py

Resources

If you specify memory size, you can either specify an explicit size (e.g. 24GB) or a range (e.g. 24GB.., or 24GB..80GB, or ..80GB).

type: task

commands:
  - pip install -r fine-tuning/qlora/requirements.txt
  - python fine-tuning/qlora/train.py

resources:
  # 200GB or more RAM
  memory: 200GB..

  # 4 GPUs from 40GB to 80GB
  gpu: 40GB..80GB:4

  # Shared memory
  shm_size: 16GB

  disk: 500GB

The gpu property allows specifying not only memory size but also GPU names and their quantity. Examples: A100 (one A100), A10G,A100 (either A10G or A100), A100:80GB (one A100 of 80GB), A100:2 (two A100), 24GB..40GB:2 (two GPUs between 24GB and 40GB), A100:40GB:2 (two A100 GPUs of 40GB).

Shared memory

If you are using parallel communicating processes (e.g., dataloaders in PyTorch), you may need to configure shm_size, e.g. set it to 16GB.

Environment variables

type: task

python: "3.11"

env:
  - HUGGING_FACE_HUB_TOKEN
  - HF_HUB_ENABLE_HF_TRANSFER=1

commands:
  - pip install -r fine-tuning/qlora/requirements.txt
  - python fine-tuning/qlora/train.py

If you don't assign a value to an environment variable (see HUGGING_FACE_HUB_TOKEN above), dstack will require the value to be passed via the CLI or set in the current process.

For instance, you can define environment variables in a .env file and utilize tools like direnv.

Default environment variables

The following environment variables are available in any run and are passed by dstack by default:

Name Description
DSTACK_RUN_NAME The name of the run
DSTACK_REPO_ID The ID of the repo
DSTACK_GPUS_NUM The total number of GPUs in the run
DSTACK_NODES_NUM The number of nodes in the run
DSTACK_NODE_RANK The rank of the node
DSTACK_MASTER_NODE_IP The internal IP address the master node

Nodes

By default, the task runs on a single node. However, you can run it on a cluster of nodes.

type: task

# The size of the cluster
nodes: 2

python: "3.11"
env:
  - HF_HUB_ENABLE_HF_TRANSFER=1
commands:
  - pip install -r requirements.txt
  - torchrun
    --nproc_per_node=$DSTACK_GPUS_PER_NODE
    --node_rank=$DSTACK_NODE_RANK
    --nnodes=$DSTACK_NODES_NUM
    --master_addr=$DSTACK_MASTER_NODE_IP
    --master_port=8008 resnet_ddp.py
    --num_epochs 20

resources:
  gpu: 24GB

If you run the task, dstack first provisions the master node and then runs the other nodes of the cluster. All nodes are provisioned in the same region.

dstack is easy to use with accelerate, torchrun, and other distributed frameworks. All you need to do is pass the corresponding environment variables such as DSTACK_GPUS_PER_NODE, DSTACK_NODE_RANK, DSTACK_NODES_NUM, DSTACK_MASTER_NODE_IP, and DSTACK_GPUS_NUM (see System environment variables).

Backends

Running on multiple nodes is supported only with AWS, GCP, and Azure.

Arguments

You can parameterize tasks with user arguments using ${{ run.args }} in the configuration.

type: task

python: "3.11"

commands:
  - pip install -r fine-tuning/qlora/requirements.txt
  - python fine-tuning/qlora/train.py ${{ run.args }}

Now, you can pass your arguments to the dstack run command:

$ dstack run . -f train.dstack.yml --train_batch_size=1 --num_train_epochs=100

Web applications

Here's an example of using ports to run web apps with tasks.

type: task

python: "3.11"

commands:
  - pip3 install streamlit
  - streamlit hello

ports: 
  - 8501

Spot policy

You can choose whether to use spot instances, on-demand instances, or any available type.

type: task

commands:
  - pip install -r fine-tuning/qlora/requirements.txt
  - python fine-tuning/qlora/train.py

spot_policy: auto

The spot_policy accepts spot, on-demand, and auto. The default for tasks is auto.

Backends

By default, dstack provisions instances in all configured backends. However, you can specify the list of backends:

type: task

commands:
  - pip install -r fine-tuning/qlora/requirements.txt
  - python fine-tuning/qlora/train.py

backends: [aws, gcp]

Regions

By default, dstack uses all configured regions. However, you can specify the list of regions:

type: task

commands:
  - pip install -r fine-tuning/qlora/requirements.txt
  - python fine-tuning/qlora/train.py

regions: [eu-west-1, eu-west-2]

The task configuration type supports many other options. See below.

Root reference

nodes - (Optional) Number of nodes. Defaults to 1.

image - (Optional) The name of the Docker image to run.

entrypoint - (Optional) The Docker entrypoint.

home_dir - (Optional) The absolute path to the home directory inside the container. Defaults to /root.

registry_auth - (Optional) Credentials for pulling a private Docker image.

python - (Optional) The major version of Python. Mutually exclusive with image.

env - (Optional) The mapping or the list of environment variables.

setup - (Optional) The bash commands to run on the boot.

resources - (Optional) The resources requirements to run the configuration.

ports - (Optional) Port numbers/mapping to expose.

commands - (Optional) The bash commands to run.

backends - (Optional) The backends to consider for provisionig (e.g., [aws, gcp]).

regions - (Optional) The regions to consider for provisionig (e.g., [eu-west-1, us-west4, westeurope]).

instance_types - (Optional) The cloud-specific instance types to consider for provisionig (e.g., [p3.8xlarge, n1-standard-4]).

spot_policy - (Optional) The policy for provisioning spot or on-demand instances: spot, on-demand, or auto.

retry_policy - (Optional) The policy for re-submitting the run.

max_duration - (Optional) The maximum duration of a run (e.g., 2h, 1d, etc). After it elapses, the run is forced to stop. Defaults to off.

max_price - (Optional) The maximum price per hour, in dollars.

pool_name - (Optional) The name of the pool. If not set, dstack will use the default name.

instance_name - (Optional) The name of the instance.

creation_policy - (Optional) The policy for using instances from the pool. Defaults to reuse-or-create.

termination_policy - (Optional) The policy for termination instances. Defaults to destroy-after-idle.

termination_idle_time - (Optional) Time to wait before destroying the idle instance. Defaults to 5m for dstack run and to 3d for dstack pool add.

resources

cpu - (Optional) The number of CPU cores. Defaults to 2...

memory - (Optional) The RAM size (e.g., 8GB). Defaults to 8GB...

shm_size - (Optional) The size of shared memory (e.g., 8GB). If you are using parallel communicating processes (e.g., dataloaders in PyTorch), you may need to configure this.

gpu - (Optional) The GPU requirements. Can be set to a number, a string (e.g. A100, 80GB:2, etc.), or an object; see examples.

disk - (Optional) The disk resources.

resouces.gpu

name - (Optional) The GPU name or list of names.

count - (Optional) The number of GPUs. Defaults to 1.

memory - (Optional) The VRAM size (e.g., 16GB). Can be set to a range (e.g. 16GB.., or 16GB..80GB).

total_memory - (Optional) The total VRAM size (e.g., 32GB). Can be set to a range (e.g. 16GB.., or 16GB..80GB).

compute_capability - (Optional) The minimum compute capability of the GPU (e.g., 7.5).

resouces.disk

size - The disk size. Can be a string (e.g., 100GB or 100GB..) or an object; see examples.

registry_auth

username - The username.

password - The password or access token.