Skip to content

Installation

To install dstack, use pip:

$ pip install "dstack[all]" -U

To use the open-source version of dstack, you have to configure your cloud accounts via ~/.dstack/server/config.yml and start the dstack server.

Configure backends

To configure cloud accounts, create the
~/.dstack/server/config.yml file, and configure each cloud under the backends property.

projects:
- name: main
  backends:
  - type: aws
    creds:
      type: access_key
      access_key: AIZKISCVKUKO5AAKLAEH
      secret_key: QSbmpqJIUBn1V5U3pyM9S6lwwiu8/fOJ2dgfwFdW

Refer below for examples on how to configure a specific cloud provider.

Projects

For flexibility, dstack server permits you to configure backends for multiple projects. If you intend to use only one project, name it main.

AWS

There are two ways to configure AWS: using an access key or using the default credentials.

Create an access key by following the this guide . Once you've downloaded the .csv file with your IAM user's Access key ID and Secret access key, proceed to configure the backend.

projects:
- name: main
  backends:
  - type: aws
    creds:
      type: access_key
      access_key: KKAAUKLIZ5EHKICAOASV
      secret_key: pn158lMqSBJiySwpQ9ubwmI6VUU3/W2fdJdFwfgO

If you have default credentials set up (e.g. in ~/.aws/credentials), configure the backend like this:

projects:
- name: main
  backends:
  - type: aws
    creds:
      type: default
Required AWS permissions

The following AWS policy permissions are sufficient for dstack to work:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:*"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "servicequotas:*"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:GetRole",
                "iam:CreateRole",
                "iam:AttachRolePolicy",
                "iam:TagRole"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:CreatePolicy",
                "iam:TagPolicy"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:GetInstanceProfile",
                "iam:CreateInstanceProfile",
                "iam:AddRoleToInstanceProfile",
                "iam:TagInstanceProfile",
                "iam:PassRole"
            ],
            "Resource": "*"
        }
    ]
}

Azure

There are two ways to configure Azure: using a client secret or using the default credentials.

A client secret can be created using the Azure CLI :

SUBSCRIPTION_ID=...
az ad sp create-for-rbac 
    --name dstack-app \
    --role $DSTACK_ROLE \ 
    --scopes /subscriptions/$SUBSCRIPTION_ID \ 
    --query "{ tenant_id: tenant, client_id: appId, client_secret: password }"

Once you have tenant_id, client_id, and client_secret, go ahead and configure the backend.

projects:
- name: main
  backends:
  - type: azure
    subscription_id: 06c82ce3-28ff-4285-a146-c5e981a9d808
    tenant_id: f84a7584-88e4-4fd2-8e97-623f0a715ee1
    creds:
      type: client
      client_id: acf3f73a-597b-46b6-98d9-748d75018ed0
      client_secret: 1Kb8Q~o3Q2hdEvrul9yaj5DJDFkuL3RG7lger2VQ

Obtain the subscription_id and tenant_id via the Azure CLI :

az account show --query "{subscription_id: id, tenant_id: tenantId}"

Then proceed to configure the backend:

projects:
- name: main
  backends:
  - type: azure
    subscription_id: 06c82ce3-28ff-4285-a146-c5e981a9d808
    tenant_id: f84a7584-88e4-4fd2-8e97-623f0a715ee1
    creds:
      type: default

NOTE:

If you don't know your subscription_id, run

az account show --query "{subscription_id: id}"
Required Azure permissions

The following Azure permissions are sufficient for dstack to work: 

{
    "properties": {
        "roleName": "dstack-role",
        "description": "Minimal reqired permissions for using Azure with dstack",
        "assignableScopes": [
            "/subscriptions/${YOUR_SUBSCRIPTION_ID}"
        ],
        "permissions": [
            {
              "actions": [
                  "Microsoft.Authorization/*/read",
                  "Microsoft.Compute/availabilitySets/*",
                  "Microsoft.Compute/locations/*",
                  "Microsoft.Compute/virtualMachines/*",
                  "Microsoft.Compute/virtualMachineScaleSets/*",
                  "Microsoft.Compute/cloudServices/*",
                  "Microsoft.Compute/disks/write",
                  "Microsoft.Compute/disks/read",
                  "Microsoft.Compute/disks/delete",
                  "Microsoft.Network/networkSecurityGroups/*",
                  "Microsoft.Network/locations/*",
                  "Microsoft.Network/virtualNetworks/*",
                  "Microsoft.Network/networkInterfaces/*",
                  "Microsoft.Network/publicIPAddresses/*",
                  "Microsoft.Resources/subscriptions/resourceGroups/read",
                  "Microsoft.Resources/subscriptions/resourceGroups/write",
                  "Microsoft.Resources/subscriptions/read"
              ],
              "notActions": [],
              "dataActions": [],
              "notDataActions": []
            }
        ]
    }
}

GCP

Enable APIs

First, ensure the required APIs are enabled in your GCP project_id.

PROJECT_ID=...
gcloud config set project $PROJECT_ID
gcloud services enable cloudapis.googleapis.com
gcloud services enable compute.googleapis.com

There are two ways to configure GCP: using a service account or using the default credentials.

To create a service account, follow this guide . Make sure to grant it the Service Account User and Compute Admin roles.

After setting up the service account create a key for it and download the corresponding JSON file.

Then go ahead and configure the backend by specifying the downloaded file path.

projects:
- name: main
  backends:
  - type: gcp
    project_id: gcp-project-id
    creds:
      type: service_account
      filename: ~/.dstack/server/gcp-024ed630eab5.json

projects:
- name: main
  backends:
  - type: gcp
    project_id: gcp-project-id
    creds:
      type: default

NOTE:

If you don't know your GCP project ID, run 

gcloud projects list --format="json(projectId)"
Required GCP permissions

The following GCP permissions are sufficient for dstack to work:

compute.disks.create
compute.firewalls.create
compute.images.useReadOnly
compute.instances.create
compute.instances.delete
compute.instances.get
compute.instances.setLabels
compute.instances.setMetadata
compute.instances.setTags
compute.networks.updatePolicy
compute.regions.list
compute.subnetworks.use
compute.subnetworks.useExternalIp
compute.zoneOperations.get

Lambda

Log into your Lambda Cloud account, click API keys in the sidebar, and then click the Generate API key button to create a new API key.

Then, go ahead and configure the backend:

projects:
- name: main
  backends:
  - type: lambda
    creds:
      type: api_key
      api_key: eersct_yrpiey-naaeedst-tk-_cb6ba38e1128464aea9bcc619e4ba2a5.iijPMi07obgt6TZ87v5qAEj61RVxhd0p

TensorDock

Log into your TensorDock account, click API in the sidebar, and use the Create an Authorization section to create a new authorization key.

Then, go ahead and configure the backend:

projects:
- name: main
  backends:
  - type: tensordock
    creds:
      type: api_key
      api_key: 248e621d-9317-7494-dc1557fa5825b-98b
      api_token: FyBI3YbnFEYXdth2xqYRnQI7hiusssBC

NOTE:

The tensordock backend supports on-demand instances only. Spot instance support coming soon.

Vast.ai

Log into your Vast.ai account, click Account in the sidebar, and copy your API Key.

Then, go ahead and configure the backend:

projects:
- name: main
  backends:
  - type: vastai
    creds:
      type: api_key
      api_key: d75789f22f1908e0527c78a283b523dd73051c8c7d05456516fc91e9d4efd8c5

NOTE:

Also, the vastai backend supports on-demand instances only. Spot instance support coming soon.

CUDO

Log into your CUDO Compute account, click API keys in the sidebar, and click the Create an API key button.

Ensure you've created a project with CUDO Compute, then proceed to configuring the backend.

projects:
- name: main
  backends:
  - type: cudo
    project_id: my-cudo-project
    creds:
      type: api_key
      api_key: 7487240a466624b48de22865589

RunPod

Log into your RunPod console, click Settings in the sidebar, expand the API Keys section, and click the button to create a key.

Then proceed to configuring the backend.

projects:
- name: main
  backends:
  - type: runpod
    creds:
      type: api_key
      api_key: US9XTPDIV8AR42MMINY8TCKRB8S4E7LNRQ6CAUQ9

NOTE:

If you're using a custom Docker image, its entrypoint cannot be anything other than /bin/bash or /bin/sh. See the issue for more details.

NOTE:

The runpod backend supports on-demand instances only. Spot instance support coming soon.

DataCrunch

Log into your DataCrunch account, click Account Settings in the sidebar, find REST API Credentials area and then click the Generate Credentials button.

Then, go ahead and configure the backend:

projects:
- name: main
  backends:
  - type: datacrunch
    creds:
      type: api_key
      client_id: xfaHBqYEsArqhKWX-e52x3HH7w8T
      client_secret: B5ZU5Qx9Nt8oGMlmMhNI3iglK8bjMhagTbylZy4WzncZe39995f7Vxh8

Kubernetes

dstack supports both self-managed, and managed Kubernetes clusters.

Prerequisite

To use GPUs with Kubernetes, the cluster must be installed with the NVIDIA GPU Operator .

To configure a Kubernetes backend, specify the path to the kubeconfig file, and the port that dstack can use for proxying SSH traffic. In case of a self-managed cluster, also specify the IP address of any node in the cluster.

Here's how to configure the backend to use a self-managed cluster.

projects:
- name: main
  backends:
  - type: kubernetes
    kubeconfig:
      filename: ~/.kube/config
    networking:
      ssh_host: localhost # The external IP address of any node
      ssh_port: 32000 # Any port accessible outside of the cluster

The port specified to ssh_port must be accessible outside of the cluster.

For example, if you are using Kind, make sure to add it via extraPortMappings:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  extraPortMappings:
  - containerPort: 32000 # Must be same as `ssh_port`
    hostPort: 32000 # Must be same as `ssh_port`

Here's how to configure the backend to use a managed cluster (AWS, GCP, Azure).

projects:
- name: main
  backends:
  - type: kubernetes
    kubeconfig:
      filename: ~/.kube/config
    networking:
      ssh_port: 32000 # Any port accessible outside of the cluster

The port specified to ssh_port must be accessible outside of the cluster.

For example, if you are using EKS, make sure to add it via an ingress rule of the corresponding security group:

aws ec2 authorize-security-group-ingress --group-id <cluster-security-group-id> --protocol tcp --port 32000 --cidr 0.0.0.0/0

Start the server

Once the ~/.dstack/server/config.yml file is configured, proceed to start the server:

$ dstack server

Applying ~/.dstack/server/config.yml...

The admin token is "bbae0f28-d3dd-4820-bf61-8f4bb40815da"
The server is running at http://127.0.0.1:3000/

$ docker run -p 3000:3000 -v $HOME/.dstack/server/:/root/.dstack/server dstackai/dstack

Applying ~/.dstack/server/config.yml...

The admin token is "bbae0f28-d3dd-4820-bf61-8f4bb40815da"
The server is running at http://127.0.0.1:3000/

Configure the CLI

To point the CLI to the dstack server, you need to configure ~/.dstack/config.yml with the server address, user token and project name.

$ dstack config --url http://127.0.0.1:3000 \
    --project main \
    --token bbae0f28-d3dd-4820-bf61-8f4bb40815da

Configuration is updated at ~/.dstack/config.yml

What's next?

  1. Follow quickstart
  2. Browse examples
  3. Join the community via Discord