Skip to content

~/.dstack/server/config.yml

The ~/.dstack/server/config.yml file is used by the dstack server to configure cloud accounts.

The dstack server allows you to configure backends for multiple projects. If you don't need multiple projects, use only the main project.

Each cloud account must be configured under the backends property of the respective project. See the examples below.

Examples

AWS

There are two ways to configure AWS: using an access key or using the default credentials.

Create an access key by following the this guide . Once you've downloaded the .csv file with your IAM user's Access key ID and Secret access key, proceed to configure the backend.

projects:
- name: main
  backends:
    - type: aws
      creds:
        type: access_key
        access_key: KKAAUKLIZ5EHKICAOASV
        secret_key: pn158lMqSBJiySwpQ9ubwmI6VUU3/W2fdJdFwfgO

If you have default credentials set up (e.g. in ~/.aws/credentials), configure the backend like this:

projects:
  - name: main
    backends:
      - type: aws
        creds:
          type: default
VPC

By default, dstack uses the default VPC. It's possible to customize it:

projects:
  - name: main
    backends:
      - type: aws
        creds:
          type: default

        vpc_name: my-vpc
projects:
  - name: main
    backends:
      - type: aws
        creds:
          type: default

        default_vpcs: true
        vpc_ids:
          us-east-1: vpc-0a2b3c4d5e6f7g8h
          us-east-2: vpc-9i8h7g6f5e4d3c2b
          us-west-1: vpc-4d3c2b1a0f9e8d7

For the regions without configured vpc_ids, enable default VPCs by setting default_vpcs to true.

Required AWS permissions

The following AWS policy permissions are sufficient for dstack to work:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:AuthorizeSecurityGroupEgress",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:CancelSpotInstanceRequests",
                "ec2:CreateSecurityGroup",
                "ec2:CreateTags",
                "ec2:DescribeImages",
                "ec2:DescribeInstances",
                "ec2:DescribeRouteTables",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSubnets",
                "ec2:DescribeVpcs",
                "ec2:RunInstances",
                "ec2:TerminateInstances"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "servicequotas:ListServiceQuotas",
                "servicequotas:GetServiceQuota"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:CreateLoadBalancer",
                "elasticloadbalancing:CreateTargetGroup",
                "elasticloadbalancing:CreateListener",
                "elasticloadbalancing:RegisterTargets",
                "elasticloadbalancing:AddTags",
                "elasticloadbalancing:DeleteLoadBalancer",
                "elasticloadbalancing:DeleteTargetGroup",
                "elasticloadbalancing:DeleteListener",
                "elasticloadbalancing:DeregisterTargets"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "acm:DescribeCertificate",
                "acm:ListCertificates"
            ],
            "Resource": "*"
        }
    ]
}

The elasticloadbalancing:* and acm:* permissions are only needed for provisioning gateways with ACM (AWS Certificate Manager) certificates.

Private subnets

By default, dstack utilizes public subnets and permits inbound SSH traffic exclusively for any provisioned instances. If you want dstack to use private subnets, set public_ips to false.

projects:
  - name: main
    backends:
      - type: aws
        creds:
          type: default

        public_ips: false

Using private subnets assumes that both the dstack server and users can access the configured VPC's private subnets (e.g., through VPC peering).

Azure

There are two ways to configure Azure: using a client secret or using the default credentials.

A client secret can be created using the Azure CLI :

SUBSCRIPTION_ID=...
az ad sp create-for-rbac
    --name dstack-app \
    --role $DSTACK_ROLE \
    --scopes /subscriptions/$SUBSCRIPTION_ID \
    --query "{ tenant_id: tenant, client_id: appId, client_secret: password }"

Once you have tenant_id, client_id, and client_secret, go ahead and configure the backend.

projects:
- name: main
  backends:
  - type: azure
    subscription_id: 06c82ce3-28ff-4285-a146-c5e981a9d808
    tenant_id: f84a7584-88e4-4fd2-8e97-623f0a715ee1
    creds:
      type: client
      client_id: acf3f73a-597b-46b6-98d9-748d75018ed0
      client_secret: 1Kb8Q~o3Q2hdEvrul9yaj5DJDFkuL3RG7lger2VQ

Obtain the subscription_id and tenant_id via the Azure CLI :

az account show --query "{subscription_id: id, tenant_id: tenantId}"

Then proceed to configure the backend:

projects:
  - name: main
    backends:
      - type: azure
        subscription_id: 06c82ce3-28ff-4285-a146-c5e981a9d808
        tenant_id: f84a7584-88e4-4fd2-8e97-623f0a715ee1
        creds:
          type: default

If you don't know your subscription_id, run

az account show --query "{subscription_id: id}"
Required Azure permissions

The following Azure permissions are sufficient for dstack to work:

{
    "properties": {
        "roleName": "dstack-role",
        "description": "Minimal required permissions for using Azure with dstack",
        "assignableScopes": [
            "/subscriptions/${YOUR_SUBSCRIPTION_ID}"
        ],
        "permissions": [
            {
            "actions": [
                "Microsoft.Authorization/*/read",
                "Microsoft.Compute/availabilitySets/*",
                "Microsoft.Compute/locations/*",
                "Microsoft.Compute/virtualMachines/*",
                "Microsoft.Compute/virtualMachineScaleSets/*",
                "Microsoft.Compute/cloudServices/*",
                "Microsoft.Compute/disks/write",
                "Microsoft.Compute/disks/read",
                "Microsoft.Compute/disks/delete",
                "Microsoft.Network/networkSecurityGroups/*",
                "Microsoft.Network/locations/*",
                "Microsoft.Network/virtualNetworks/*",
                "Microsoft.Network/networkInterfaces/*",
                "Microsoft.Network/publicIPAddresses/*",
                "Microsoft.Resources/subscriptions/resourceGroups/read",
                "Microsoft.Resources/subscriptions/resourceGroups/write",
                "Microsoft.Resources/subscriptions/read"
            ],
            "notActions": [],
            "dataActions": [],
            "notDataActions": []
            }
        ]
    }
}

GCP

Enable APIs

First, ensure the required APIs are enabled in your GCP project_id.

PROJECT_ID=...
gcloud config set project $PROJECT_ID
gcloud services enable cloudapis.googleapis.com
gcloud services enable compute.googleapis.com

There are two ways to configure GCP: using a service account or using the default credentials.

To create a service account, follow this guide . After setting up the service account create a key for it and download the corresponding JSON file.

Then go ahead and configure the backend by specifying the downloaded file path.

projects:
- name: main
  backends:
    - type: gcp
      project_id: gcp-project-id
      creds:
        type: service_account
        filename: ~/.dstack/server/gcp-024ed630eab5.json

Enable GCP application default credentials:

gcloud auth application-default login 

Then configure the backend like this:

projects:
- name: main
  backends:
    - type: gcp
      project_id: gcp-project-id
      creds:
        type: default

If you don't know your GCP project ID, run

gcloud projects list --format="json(projectId)"
projects:
- name: main
  backends:
    - type: gcp
      project_id: gcp-project-id
      creds:
        type: default

      vpc_name: my-custom-vpc
projects:
- name: main
  backends:
    - type: gcp
      project_id: gcp-project-id
      creds:
        type: default

      vpc_name: my-custom-vpc
      vpc_project_id: another-project-id

To use a shared VPC, that VPC has to be configured with two additional firewall rules:

  • Allow INGRESS traffic on port 22, with the target tag dstack-runner-instance
  • Allow INGRESS traffic on ports 22, 80, 443, with the target tag dstack-gateway-instance
Required GCP permissions

The following GCP permissions are sufficient for dstack to work:

compute.disks.create
compute.firewalls.create
compute.images.useReadOnly
compute.instances.create
compute.instances.delete
compute.instances.get
compute.instances.setLabels
compute.instances.setMetadata
compute.instances.setTags
compute.networks.get
compute.networks.updatePolicy
compute.regions.list
compute.subnetworks.list
compute.subnetworks.use
compute.subnetworks.useExternalIp
compute.zoneOperations.get
Permissions for running TPUs

Running TPUs also requires the following permissions:

tpu.nodes.create
tpu.nodes.delete
tpu.nodes.get
tpu.operations.get
tpu.operations.list

You also need to have the serviceAccountUser role granted. dstack will run TPUs under the default service account, so you don't need to create one.

Private subnets

By default, dstack utilizes public subnets and permits inbound SSH traffic exclusively for any provisioned instances. If you want dstack to use private subnets, set public_ips to false.

projects:
  - name: main
    backends:
      - type: gcp
        creds:
          type: default

        public_ips: false

Using private subnets assumes that both the dstack server and users can access the configured VPC's private subnets (e.g., through VPC peering). Additionally, Cloud NAT must be configured to provide access to external resources for provisioned instances.

OCI

There are two ways to configure OCI: using client credentials or using the default credentials.

Log into the OCI Console , go to My profile, select API keys, and click Add API key.

Once you add a key, you'll see the configuration file. Copy its values to configure the backend as follows:

projects:
- name: main
  backends:
  - type: oci
    creds:
      type: client
      user: ocid1.user.oc1..g5vlaeqfu47akmaafq665xsgmyaqjktyfxtacfxc4ftjxuca7aohnd2ev66m
      tenancy: ocid1.tenancy.oc1..ajqsftvk4qarcfaak3ha4ycdsaahxmaita5frdwg3tqo2bcokpd3n7oizwai
      region: eu-frankfurt-1
      fingerprint: 77:32:77:00:49:7c:cb:56:84:75:8e:77:96:7d:53:17
      key_file: ~/.oci/private_key.pem

Make sure to include either the path to your private key via key_file or the contents of the key via key_content.

If you have default credentials set up in ~/.oci/config, configure the backend like this:

projects:
- name: main
  backends:
  - type: oci
    creds:
      type: default
Required OCI permissions

This is an example of a restrictive policy for a group of dstack users:

Allow group <dstack-users> to read compartments in tenancy where target.compartment.name = '<dstack-compartment>'
Allow group <dstack-users> to read marketplace-community-listings in compartment <dstack-compartment>
Allow group <dstack-users> to manage app-catalog-listing in compartment <dstack-compartment>
Allow group <dstack-users> to manage instances in compartment <dstack-compartment>
Allow group <dstack-users> to manage compute-capacity-reports in compartment <dstack-compartment>
Allow group <dstack-users> to manage volumes in compartment <dstack-compartment>
Allow group <dstack-users> to manage volume-attachments in compartment <dstack-compartment>
Allow group <dstack-users> to manage virtual-network-family in compartment <dstack-compartment>

To use this policy, create a compartment for dstack and specify it in ~/.dstack/server/config.yml.

projects:
- name: main
  backends:
  - type: oci
    creds:
      type: default
    compartment_id: ocid1.compartment.oc1..aaaaaaaa

Lambda

Log into your Lambda Cloud account, click API keys in the sidebar, and then click the Generate API key button to create a new API key.

Then, go ahead and configure the backend:

projects:
- name: main
  backends:
    - type: lambda
      creds:
        type: api_key
        api_key: eersct_yrpiey-naaeedst-tk-_cb6ba38e1128464aea9bcc619e4ba2a5.iijPMi07obgt6TZ87v5qAEj61RVxhd0p

TensorDock

Log into your TensorDock account, click API in the sidebar, and use the Create an Authorization section to create a new authorization key.

Then, go ahead and configure the backend:

projects:
  - name: main
    backends:
      - type: tensordock
        creds:
          type: api_key
          api_key: 248e621d-9317-7494-dc1557fa5825b-98b
          api_token: FyBI3YbnFEYXdth2xqYRnQI7hiusssBC

The tensordock backend supports on-demand instances only. Spot instance support coming soon.

Vast.ai

Log into your Vast.ai account, click Account in the sidebar, and copy your API Key.

Then, go ahead and configure the backend:

projects:
- name: main
  backends:
    - type: vastai
      creds:
        type: api_key
        api_key: d75789f22f1908e0527c78a283b523dd73051c8c7d05456516fc91e9d4efd8c5

Also, the vastai backend supports on-demand instances only. Spot instance support coming soon.

RunPod

Log into your RunPod console, click Settings in the sidebar, expand the API Keys section, and click the button to create a key.

Then proceed to configuring the backend.

projects:
  - name: main
    backends:
      - type: runpod
        creds:
          type: api_key
          api_key: US9XTPDIV8AR42MMINY8TCKRB8S4E7LNRQ6CAUQ9

CUDO

Log into your CUDO Compute account, click API keys in the sidebar, and click the Create an API key button.

Ensure you've created a project with CUDO Compute, then proceed to configuring the backend.

projects:
  - name: main
    backends:
      - type: cudo
        project_id: my-cudo-project
        creds:
          type: api_key
          api_key: 7487240a466624b48de22865589

DataCrunch

Log into your DataCrunch account, click Account Settings in the sidebar, find REST API Credentials area and then click the Generate Credentials button.

Then, go ahead and configure the backend:

projects:
  - name: main
    backends:
      - type: datacrunch
        creds:
          type: api_key
          client_id: xfaHBqYEsArqhKWX-e52x3HH7w8T
          client_secret: B5ZU5Qx9Nt8oGMlmMhNI3iglK8bjMhagTbylZy4WzncZe39995f7Vxh8

Kubernetes

dstack supports both self-managed, and managed Kubernetes clusters.

Prerequisite

To use GPUs with Kubernetes, the cluster must be installed with the NVIDIA GPU Operator .

To configure a Kubernetes backend, specify the path to the kubeconfig file, and the port that dstack can use for proxying SSH traffic. In case of a self-managed cluster, also specify the IP address of any node in the cluster.

Here's how to configure the backend to use a self-managed cluster.

projects:
- name: main
  backends:
    - type: kubernetes
      kubeconfig:
      filename: ~/.kube/config
      networking:
      ssh_host: localhost # The external IP address of any node
      ssh_port: 32000 # Any port accessible outside of the cluster

The port specified to ssh_port must be accessible outside of the cluster.

For example, if you are using Kind, make sure to add it via extraPortMappings:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
    extraPortMappings:
  - containerPort: 32000 # Must be same as `ssh_port`
    hostPort: 32000 # Must be same as `ssh_port`

Here's how to configure the backend to use a managed cluster (AWS, GCP, Azure).

projects:
  - name: main
    backends:
      - type: kubernetes
        kubeconfig:
        filename: ~/.kube/config
        networking:
        ssh_port: 32000 # Any port accessible outside of the cluster

The port specified to ssh_port must be accessible outside of the cluster.

For example, if you are using EKS, make sure to add it via an ingress rule of the corresponding security group:

aws ec2 authorize-security-group-ingress --group-id <cluster-security-group-id> --protocol tcp --port 32000 --cidr 0.0.0.0/0

Root reference

projects - The list of projects.

projects[n]

name - The name of the project.

backends - The list of backends.

projects[n].backends[type=aws]

type - The type of the backend. Must be aws.

regions - (Optional) The list of AWS regions.

vpc_name - (Optional) The VPC name. All configured regions must have a VPC with this name.

vpc_ids - (Optional) The mapping from AWS regions to VPC IDs. If default_vpcs: true, omitted regions will use default VPCs.

default_vpcs - (Optional) A flag to enable/disable using default VPCs in regions not configured by vpc_ids. Set to false if default VPCs should never be used. Defaults to true.

public_ips - (Optional) A flag to enable/disable public IP assigning on instances. Defaults to true.

creds - The credentials.

projects[n].backends[type=aws].creds

type - The type of credentials. Must be access_key.

access_key - The access key.

secret_key - The secret key.

type - The type of credentials. Must be default.

projects[n].backends[type=azure]

type - The type of the backend. Must be azure.

tenant_id - The tenant ID.

subscription_id - The subscription ID.

creds - The credentials.

projects[n].backends[type=azure].creds

type - The type of credentials. Must be client.

client_id - The client ID.

client_secret - The client secret.

type - The type of credentials. Must be default.

projects[n].backends[type=datacrunch]

type - The type of backend. Must be datacrunch.

creds - The credentials.

projects[n].backends[type=datacrunch].creds

type - The type of credentials. Must be api_key.

client_id - The client ID.

client_secret - The client secret.

projects[n].backends[type=gcp]

type - The type of backend. Must be gcp.

project_id - The project ID.

vpc_name - (Optional) The VPC name.

vpc_project_id - (Optional) The shared VPC hosted project ID. Required for shared VPC only.

public_ips - (Optional) A flag to enable/disable public IP assigning on instances. Defaults to true.

creds - The credentials.

projects[n].backends[type=gcp].creds

type - The type of credentials. Must be service_account.

filename - The path to the service account file.

data - (Optional) The contents of the service account file.

type - The type of credentials. Must be default.

projects[n].backends[type=lambda]

type - The type of backend. Must be lambda.

creds - The credentials.

projects[n].backends[type=lambda].creds

type - The type of credentials. Must be api_key.

api_key - The API key.

projects[n].backends[type=oci]

type - The type of backend. Must be oci.

creds - The credentials.

regions - (Optional) List of region names for running dstack jobs. Omit to use all regions.

compartment_id - (Optional) Compartment where dstack will create all resources. Omit to instruct dstack to create a new compartment.

projects[n].backends[type=oci].creds

type - The type of credentials. Must be client.

user - User OCID.

tenancy - Tenancy OCID.

key_file - (Optional) Path to the user's private PEM key. Either this or key_content should be set.

key_content - (Optional) Content of the user's private PEM key. Either this or key_file should be set.

pass_phrase - (Optional) Passphrase for the private PEM key if it is encrypted.

fingerprint - User's public key fingerprint.

region - Name or key of any region the tenancy is subscribed to.

type - The type of credentials. Must be default.

file - (Optional) Path to the OCI CLI-compatible config file. Defaults to ~/.oci/config.

profile - (Optional) Profile to load from the config file. Defaults to DEFAULT.

projects[n].backends[type=tensordock]

type - The type of backend. Must be tensordock.

creds - The credentials.

projects[n].backends[type=tensordock].creds

type - The type of credentials. Must be api_key.

api_key - The API key.

api_token - The API token.

projects[n].backends[type=vastai]

type - The type of backend. Must be vastai.

creds - The credentials.

projects[n].backends[type=vastai].creds

type - The type of credentials. Must be api_key.

api_key - The API key.

projects[n].backends[type=kubernetes]

type - The type of backend. Must be kubernetes.

kubeconfig - The kubeconfig configuration.

networking - (Optional) The networking configuration.

projects[n].backends[type=kubernetes].kubeconfig

filename - The path to the kubeconfig file.

data - (Optional) The contents of the kubeconfig file.

projects[n].backends[type=kubernetes].networking

ssh_host - (Optional) The external IP address of any node.

ssh_port - (Optional) Any port accessible outside of the cluster.