Volumes¶
Volumes allow you to persist data between runs. dstack
allows to create and attach volumes to
dev environments, tasks, and services.
Volumes are currently supported with the
aws
,gcp
, andrunpod
backends. Support for other backends and on-prem fleets is coming soon.
Define a configuration¶
First, create a YAML file in your project folder. Its name must end with .dstack.yml
(e.g. .dstack.yml
or vol.dstack.yml
are both acceptable).
type: volume
# A name of the volume
name: my-new-volume
# Volumes are bound to a specific backend and region
backend: aws
region: eu-central-1
# Required size
size: 100GB
If you use this configuration, dstack
will create a new volume based on the specified options.
Registering existing volumes
If you prefer not to create a new volume but to reuse an existing one (e.g., created manually), you can
specify its ID via volume_id
. In this case, dstack
will register the specified volume so that you can use it with dev environments, tasks, and services.
Reference
See .dstack.yml for all the options supported by volumes, along with multiple examples.
Create, register, or update a volume¶
To create or register the volume, simply call the dstack apply
command:
$ dstack apply -f volume.dstack.yml
Volume my-new-volume does not exist yet. Create the volume? [y/n]: y
NAME BACKEND REGION STATUS CREATED
my-new-volume aws eu-central-1 submitted now
When creating the volume
dstack
automatically creates anext4
file system on it.
Once created, the volume can be attached with dev environments, tasks, and services.
Attach a volume¶
Dev environments, tasks, and services let you attach any number of volumes.
To attach a volume, simply specify its name using the volumes
property and specify where to mount its contents:
type: dev-environment
# A name of the dev environment
name: vscode-vol
ide: vscode
# Map the name of the volume to any path
volumes:
- name: my-new-volume
path: /volume_data
Once you run this configuration, the contents of the volume will be attached to /volume_data
inside the dev environment,
and its contents will persist across runs.
Limitations
When you're running a dev environment, task, or service with dstack
, it automatically mounts the project folder contents
to /workflow
(and sets that as the current working directory). Right now, dstack
doesn't allow you to
attach volumes to /workflow
or any of its subdirectories.
Manage volumes¶
List volumes¶
The dstack volume list
command lists created and registered volumes:
$ dstack volume list
NAME BACKEND REGION STATUS CREATED
my-new-volume aws eu-central-1 active 3 weeks ago
Delete volumes¶
When the volume isn't attached to any active dev environment, task, or service, you can delete it using dstack delete
:
$ dstack delete -f vol.dstack.yaml
If the volume was created using dstack
, it will be physically destroyed along with the data.
If you've registered an existing volume, it will be de-registered with dstack
but will keep the data.
FAQ¶
Can I use volumes across backends?¶
Since volumes are backed up by cloud network disks, you can only use them within the same cloud. If you need to access data across different backends, you should either use object storage or replicate the data across multiple volumes.
Can I use volumes across regions?¶
Typically, network volumes are associated with specific regions, so you can't use them in other regions. Often, volumes are also linked to availability zones, but some providers support volumes that can be used across different availability zones within the same region.
Can I attach volumes to multiple runs or instances?¶
You can mount a volume in multiple runs. This feature is currently supported only by the runpod
backend.