Python API¶

The Python API enables running tasks, services, and managing runs programmatically.

Usage example¶

Below is a quick example of submitting a task for running and displaying its logs.

import sys

from dstack.api import Task, GPU, Client, Resources

client = Client.from_config()

task = Task(
    image="ghcr.io/huggingface/text-generation-inference:latest",
    env={"MODEL_ID": "TheBloke/Llama-2-13B-chat-GPTQ"},
    commands=[
        "text-generation-launcher --trust-remote-code --quantize gptq",
    ],
    ports=["80"],
    resources=Resources(gpu=GPU(memory="24GB")),
)

run = client.runs.submit(
    run_name="my-awesome-run",  # If not specified, a random name is assigned 
    configuration=task,
    repo=None, # Specify to mount additional files
)

run.attach()

try:
    for log in run.logs():
        sys.stdout.buffer.write(log)
        sys.stdout.buffer.flush()
except KeyboardInterrupt:
    run.stop(abort=True)
finally:
    run.detach()

NOTE:

The configuration argument in the submit method can be either dstack.api.Task or dstack.api.Service.
If you create dstack.api.Task or dstack.api.Service, you may specify the image argument. If image isn't specified, the default image will be used. For a private Docker registry, ensure you also pass the registry_auth argument.
The repo argument in the submit method allows the mounting of a local folder, a remote repo, or a programmatically created repo. In this case, the commands argument can refer to the files within this repo.
The attach method waits for the run to start and, for dstack.api.Task sets up an SSH tunnel and forwards configured ports to localhost.

`dstack.api`¶

`dstack.api.Client`¶

High-level API client for interacting with dstack server

Attributes:

Name	Type	Description
`runs`	`RunCollection`	Operations with runs.
`repos`	`RepoCollection`	Operations with repositories.
`backends`	`BackendCollection`	Operations with backends.

`from_config(project_name=None, server_url=None, user_token=None, ssh_identity_file=None)` `staticmethod` ¶

Creates a Client using the default configuration from ~/.dstack/config.yml if it exists.

Parameters:

Name	Type	Description	Default
`project_name`	`Optional[str]`	The name of the project, required if `server_url` and `user_token` are specified	`None`
`server_url`	`Optional[str]`	The dstack server URL (e.g. `http://localhost:3000/` or `https://sky.dstack.ai`)	`None`
`user_token`	`Optional[str]`	The dstack user token	`None`
`ssh_identity_file`	`Optional[PathLike]`	The private SSH key path for SSH tunneling	`None`

Returns:

Type	Description
`Client`	A client instance

`dstack.api.RunCollection`¶

Operations with runs

`get(run_name)` ¶

Get run by run name

Parameters:

Name	Type	Description	Default
`run_name`	`str`	run name	required

Returns:

Type	Description
`Optional[Run]`	The run or `None` if not found

`list(all=False)` ¶

List runs

Parameters:

Name	Type	Description	Default
`all`	`bool`	show all runs (active and finished) if `True`	`False`

Returns:

Type	Description
`List[Run]`	list of runs

`submit(configuration, configuration_path=None, repo=None, backends=None, regions=None, instance_types=None, resources=None, spot_policy=None, retry_policy=None, max_duration=None, max_price=None, working_dir=None, run_name=None, reserve_ports=True)` ¶

Submit a run

Parameters:

Name	Type	Description	Default
`configuration`	`Union[Task, Service]`	A run configuration.	required
`configuration_path`	`Optional[str]`	The path to the configuration file, relative to the root directory of the repo.	`None`
`repo`	`Union[LocalRepo, RemoteRepo, VirtualRepo]`	A repo to mount to the run.	`None`
`backends`	`Optional[List[BackendType]]`	A list of allowed backend for provisioning.	`None`
`regions`	`Optional[List[str]]`	A list of cloud regions for provisioning.	`None`
`resources`	`Optional[ResourcesSpec]`	The requirements to run the configuration. Overrides the configuration's resources.	`None`
`spot_policy`	`Optional[SpotPolicy]`	A spot policy for provisioning.	`None`
`retry_policy`	`RetryPolicy`	A retry policy.	`None`
`max_duration`	`Optional[Union[int, str]]`	The max instance running duration in seconds.	`None`
`max_price`	`Optional[float]`	The max instance price in dollars per hour for provisioning.	`None`
`working_dir`	`Optional[str]`	A working directory relative to the repo root directory	`None`
`run_name`	`Optional[str]`	A desired name of the run. Must be unique in the project. If not specified, a random name is assigned.	`None`
`reserve_ports`	`bool`	Whether local ports should be reserved in advance.	`True`

Returns:

Type	Description
`Run`	submitted run

`dstack.api.RepoCollection`¶

Operations with repos

`init(repo, git_identity_file=None, oauth_token=None)` ¶

Initializes the repo and configures its credentials in the project. Must be invoked before mounting the repo to a run.

Example:

repo=RemoteRepo.from_url(
    repo_url="https://github.com/dstackai/dstack-examples",
    repo_branch="main",
)
client.repos.init(repo)

By default, it uses the default Git credentials configured on the machine. You can override these credentials via the git_identity_file or oauth_token arguments of the init method.

Once the repo is initialized, you can pass the repo object to the run:

run = client.runs.submit(
    configuration=...,
    repo=repo,
)

Parameters:

Name	Type	Description	Default
`repo`	`Repo`	The repo to initialize.	required
`git_identity_file`	`Optional[PathLike]`	The private SSH key path for accessing the remote repo.	`None`
`oauth_token`	`Optional[str]`	The GitHub OAuth token to access the remote repo.	`None`

`dstack.api.Task`¶

`nodes` - (Optional) Number of nodes. Defaults to `1`.¶

`name` - (Optional) The run name.¶

`image` - (Optional) The name of the Docker image to run.¶

`entrypoint` - (Optional) The Docker entrypoint.¶

`working_dir` - (Optional) The path to the working directory inside the container. It's specified relative to the repository directory (`/workflow`) and should be inside it. Defaults to `"."` .¶

`home_dir` - (Optional) The absolute path to the home directory inside the container. Defaults to `/root`. Defaults to `/root`.¶

`registry_auth` - (Optional) Credentials for pulling a private Docker image.¶

`python` - (Optional) The major version of Python. Mutually exclusive with `image`.¶

`env` - (Optional) The mapping or the list of environment variables.¶

`setup` - (Optional) The bash commands to run on the boot.¶

`resources` - (Optional) The resources requirements to run the configuration.¶

`volumes` - (Optional) The volumes mount points.¶

`ports` - (Optional) Port numbers/mapping to expose.¶

`commands` - (Optional) The bash commands to run.¶

`backends` - (Optional) The backends to consider for provisioning (e.g., `[aws, gcp]`).¶

`regions` - (Optional) The regions to consider for provisioning (e.g., `[eu-west-1, us-west4, westeurope]`).¶

`instance_types` - (Optional) The cloud-specific instance types to consider for provisioning (e.g., `[p3.8xlarge, n1-standard-4]`).¶

`spot_policy` - (Optional) The policy for provisioning spot or on-demand instances: `spot`, `on-demand`, or `auto`.¶

`retry` - (Optional) The policy for resubmitting the run. Defaults to `false`.¶

`retry_policy` - (Optional) The policy for resubmitting the run. Deprecated in favor of `retry`.¶

`max_duration` - (Optional) The maximum duration of a run (e.g., `2h`, `1d`, etc). After it elapses, the run is forced to stop. Defaults to `off`.¶

`max_price` - (Optional) The maximum instance price per hour, in dollars.¶

`pool_name` - (Optional) The name of the pool. If not set, dstack will use the default name.¶

`instance_name` - (Optional) The name of the instance.¶

`creation_policy` - (Optional) The policy for using instances from the pool. Defaults to `reuse-or-create`.¶

`termination_policy` - (Optional) The policy for instance termination. Defaults to `destroy-after-idle`.¶

`termination_idle_time` - (Optional) Time to wait before destroying the idle instance. Defaults to `5m` for `dstack run` and to `3d` for `dstack pool add`.¶

`dstack.api.Service`¶

`port` - The port, that application listens on or the mapping.¶

`model` - (Optional) Mapping of the model for the OpenAI-compatible endpoint.¶

`https` - (Optional) Enable HTTPS. Defaults to `True`.¶

`auth` - (Optional) Enable the authorization. Defaults to `True`.¶

`replicas` - (Optional) The number of replicas. Can be a number (e.g. `2`) or a range (`0..4` or `1..8`). If it's a range, the `scaling` property is required. Defaults to `1`.¶

`scaling` - (Optional) The auto-scaling rules. Required if `replicas` is set to a range.¶

`name` - (Optional) The run name.¶

`image` - (Optional) The name of the Docker image to run.¶

`entrypoint` - (Optional) The Docker entrypoint.¶

`working_dir` - (Optional) The path to the working directory inside the container. It's specified relative to the repository directory (`/workflow`) and should be inside it. Defaults to `"."` .¶

`home_dir` - (Optional) The absolute path to the home directory inside the container. Defaults to `/root`. Defaults to `/root`.¶

`registry_auth` - (Optional) Credentials for pulling a private Docker image.¶

`python` - (Optional) The major version of Python. Mutually exclusive with `image`.¶

`env` - (Optional) The mapping or the list of environment variables.¶

`setup` - (Optional) The bash commands to run on the boot.¶

`resources` - (Optional) The resources requirements to run the configuration.¶

`volumes` - (Optional) The volumes mount points.¶

`commands` - (Optional) The bash commands to run.¶

`backends` - (Optional) The backends to consider for provisioning (e.g., `[aws, gcp]`).¶

`regions` - (Optional) The regions to consider for provisioning (e.g., `[eu-west-1, us-west4, westeurope]`).¶

`instance_types` - (Optional) The cloud-specific instance types to consider for provisioning (e.g., `[p3.8xlarge, n1-standard-4]`).¶

`spot_policy` - (Optional) The policy for provisioning spot or on-demand instances: `spot`, `on-demand`, or `auto`.¶

`retry` - (Optional) The policy for resubmitting the run. Defaults to `false`.¶

`retry_policy` - (Optional) The policy for resubmitting the run. Deprecated in favor of `retry`.¶

`max_duration` - (Optional) The maximum duration of a run (e.g., `2h`, `1d`, etc). After it elapses, the run is forced to stop. Defaults to `off`.¶

`max_price` - (Optional) The maximum instance price per hour, in dollars.¶

`pool_name` - (Optional) The name of the pool. If not set, dstack will use the default name.¶

`instance_name` - (Optional) The name of the instance.¶

`creation_policy` - (Optional) The policy for using instances from the pool. Defaults to `reuse-or-create`.¶

`termination_policy` - (Optional) The policy for instance termination. Defaults to `destroy-after-idle`.¶

`termination_idle_time` - (Optional) Time to wait before destroying the idle instance. Defaults to `5m` for `dstack run` and to `3d` for `dstack pool add`.¶

`dstack.api.Run`¶

Attributes:

Name	Type	Description
`name`	`str`	run name
`ports`	`Optional[Dict[int, int]]`	ports mapping, if run is attached
`backend`	`Optional[BackendType]`	backend type
`status`	`RunStatus`	run status
`hostname`	`str`	instance hostname

`attach(ssh_identity_file=None)` ¶

Establish an SSH tunnel to the instance and update SSH config

Parameters:

Name	Type	Description	Default
`ssh_identity_file`	`Optional[PathLike]`	SSH keypair to access instances	`None`

Raises:

Type	Description
`PortUsedError`	If ports are in use or the run is attached by another process.

`detach()` ¶

Stop the SSH tunnel to the instance and update SSH config

`logs(start_time=None, diagnose=False, replica_num=0, job_num=0)` ¶

Iterate through run's log messages

Parameters:

Name	Type	Description	Default
`start_time`	`Optional[datetime]`	minimal log timestamp	`None`
`diagnose`	`bool`	return runner logs if `True`	`False`

Yields:

Type	Description
`Iterable[bytes]`	log messages

`refresh()` ¶

Get up-to-date run info

`stop(abort=False)` ¶

Terminate the instance and detach

Parameters:

Name	Type	Description	Default
`abort`	`bool`	gracefully stop the run if `False`	`False`

`dstack.api.Resources`¶

`cpu` - (Optional) The number of CPU cores. Defaults to `2..`.¶

`memory` - (Optional) The RAM size (e.g., `8GB`). Defaults to `8GB..`.¶

`shm_size` - (Optional) The size of shared memory (e.g., `8GB`). If you are using parallel communicating processes (e.g., dataloaders in PyTorch), you may need to configure this.¶

`gpu` - (Optional) The GPU requirements.¶

`disk` - (Optional) The disk resources.¶

`dstack.api.GPU`¶

`name` - (Optional) The name of the GPU (e.g., `A100` or `H100`).¶

`count` - (Optional) The number of GPUs. Defaults to `1`.¶

`memory` - (Optional) The RAM size (e.g., `16GB`). Can be set to a range (e.g. `16GB..`, or `16GB..80GB`).¶

`total_memory` - (Optional) The total RAM size (e.g., `32GB`). Can be set to a range (e.g. `16GB..`, or `16GB..80GB`).¶

`compute_capability` - (Optional) The minimum compute capability of the GPU (e.g., `7.5`).¶

`dstack.api.Disk`¶

`size` - Disk size.¶

`dstack.api.LocalRepo`¶

Creates an instance of a local repo from a local path.

Example:

run = client.runs.submit(
    configuration=...,
    repo=LocalRepo.from_dir("."), # Mount the current folder to the run
)

`from_dir(repo_dir)` `staticmethod` ¶

Creates an instance of a local repo from a local path.

Parameters:

Name	Type	Description	Default
`repo_dir`	`PathLike`	The path to a local folder	required

Returns:

Type	Description
`LocalRepo`	A local repo instance

`dstack.api.RemoteRepo`¶

Creates an instance of a remote Git repo for mounting to a submitted run.

Using a locally checked-out remote Git repo:

repo=RemoteRepo.from_dir(repo_dir=".")

Using a remote Git repo by a URL:

repo=RemoteRepo.from_url(
    repo_url="https://github.com/dstackai/dstack-examples",
    repo_branch="main"
)

Initialize the repo before mounting it.

client.repos.init(repo)

By default, it uses the default Git credentials configured on the machine. You can override these credentials via the git_identity_file or oauth_token arguments of the init method.

Finally, you can pass the repo object to the run:

run = client.runs.submit(
    configuration=...,
    repo=repo,
)

`from_dir(repo_dir)` `staticmethod` ¶

Creates an instance of a remote repo from a local path.

Parameters:

Name	Type	Description	Default
`repo_dir`	`PathLike`	The path to a local folder	required

Returns:

Type	Description
`RemoteRepo`	A remote repo instance

`from_url(repo_url, repo_branch=None, repo_hash=None)` `staticmethod` ¶

Creates an instance of a remote repo from a URL.

Parameters:

Name	Type	Description	Default
`repo_url`	`str`	The URL of a remote Git repo	required
`repo_branch`	`Optional[str]`	The name of the remote branch. Must be specified if `hash` is not specified.	`None`
`repo_hash`	`Optional[str]`	The hash of the revision. Must be specified if `branch` is not specified.	`None`

Returns:

Type	Description
`RemoteRepo`	A remote repo instance

`dstack.api.VirtualRepo`¶

Allows mounting a repo created programmatically.

Example:

virtual_repo = VirtualRepo(repo_id="some-unique-repo-id")
virtual_repo.add_file_from_package(package=some_package, path="requirements.txt")
virtual_repo.add_file_from_package(package=some_package, path="train.py")

run = client.runs.submit(
    configuration=...,
    repo=virtual_repo,
)

Attributes:

Name	Type	Description
`repo_id`		A unique identifier of the repo

`add_file(path, content)` ¶

Adds a given file to the repo.

Attributes:

Name	Type	Description
`path`	`str`	The path inside the repo to add the file.
`content`	`bytes`	The contents of the file.

`add_file_from_package(package, path)` ¶

Includes a file from a given package to the repo.

Attributes:

Name	Type	Description
`package`	`Union[ModuleType, str]`	A package to include the file from.
`path`	`str`	The path to the file to include to the repo. Must be relative to the package directory.

`dstack.api.RegistryAuth`¶

`username` - The username.¶

`password` - The password or access token.¶

`dstack.api.Scaling`¶

`metric` - The target metric to track. Currently, the only supported value is `rps` (meaning requests per second).¶

`target` - The target value of the metric. The number of replicas is calculated based on this number and automatically adjusts (scales up or down) as this metric changes.¶

`scale_up_delay` - (Optional) The delay in seconds before scaling up. Defaults to `300`.¶

`scale_down_delay` - (Optional) The delay in seconds before scaling down. Defaults to `600`.¶

`dstack.api.BackendType`¶

Attributes:

Name	Type	Description
`AWS`	`BackendType`	Amazon Web Services
`AZURE`	`BackendType`	Microsoft Azure
`CUDO`	`BackendType`	Cudo
`DSTACK`	`BackendType`	dstack Sky
`GCP`	`BackendType`	Google Cloud Platform
`DATACRUNCH`	`BackendType`	DataCrunch
`KUBERNETES`	`BackendType`	Kubernetes
`LAMBDA`	`BackendType`	Lambda Cloud
`RUNPOD`	`BackendType`	Runpod Cloud
`TENSORDOCK`	`BackendType`	TensorDock Marketplace
`VASTAI`	`BackendType`	Vast.ai Marketplace

Python API¶

Usage example¶

dstack.api¶

dstack.api.Client¶

from_config(project_name=None, server_url=None, user_token=None, ssh_identity_file=None) staticmethod ¶

dstack.api.RunCollection¶

get(run_name) ¶

list(all=False) ¶

submit(configuration, configuration_path=None, repo=None, backends=None, regions=None, instance_types=None, resources=None, spot_policy=None, retry_policy=None, max_duration=None, max_price=None, working_dir=None, run_name=None, reserve_ports=True) ¶

dstack.api.RepoCollection¶

init(repo, git_identity_file=None, oauth_token=None) ¶

dstack.api.Task¶

nodes - (Optional) Number of nodes. Defaults to 1.¶

name - (Optional) The run name.¶

image - (Optional) The name of the Docker image to run.¶

entrypoint - (Optional) The Docker entrypoint.¶

working_dir - (Optional) The path to the working directory inside the container. It's specified relative to the repository directory (/workflow) and should be inside it. Defaults to "." .¶

home_dir - (Optional) The absolute path to the home directory inside the container. Defaults to /root. Defaults to /root.¶

registry_auth - (Optional) Credentials for pulling a private Docker image.¶

python - (Optional) The major version of Python. Mutually exclusive with image.¶

env - (Optional) The mapping or the list of environment variables.¶

setup - (Optional) The bash commands to run on the boot.¶

resources - (Optional) The resources requirements to run the configuration.¶

volumes - (Optional) The volumes mount points.¶

ports - (Optional) Port numbers/mapping to expose.¶

commands - (Optional) The bash commands to run.¶

backends - (Optional) The backends to consider for provisioning (e.g., [aws, gcp]).¶

regions - (Optional) The regions to consider for provisioning (e.g., [eu-west-1, us-west4, westeurope]).¶

instance_types - (Optional) The cloud-specific instance types to consider for provisioning (e.g., [p3.8xlarge, n1-standard-4]).¶

spot_policy - (Optional) The policy for provisioning spot or on-demand instances: spot, on-demand, or auto.¶

retry - (Optional) The policy for resubmitting the run. Defaults to false.¶

retry_policy - (Optional) The policy for resubmitting the run. Deprecated in favor of retry.¶

max_duration - (Optional) The maximum duration of a run (e.g., 2h, 1d, etc). After it elapses, the run is forced to stop. Defaults to off.¶

max_price - (Optional) The maximum instance price per hour, in dollars.¶

pool_name - (Optional) The name of the pool. If not set, dstack will use the default name.¶

instance_name - (Optional) The name of the instance.¶

creation_policy - (Optional) The policy for using instances from the pool. Defaults to reuse-or-create.¶

termination_policy - (Optional) The policy for instance termination. Defaults to destroy-after-idle.¶

termination_idle_time - (Optional) Time to wait before destroying the idle instance. Defaults to 5m for dstack run and to 3d for dstack pool add.¶

dstack.api.Service¶

port - The port, that application listens on or the mapping.¶

model - (Optional) Mapping of the model for the OpenAI-compatible endpoint.¶

https - (Optional) Enable HTTPS. Defaults to True.¶

auth - (Optional) Enable the authorization. Defaults to True.¶

replicas - (Optional) The number of replicas. Can be a number (e.g. 2) or a range (0..4 or 1..8). If it's a range, the scaling property is required. Defaults to 1.¶

scaling - (Optional) The auto-scaling rules. Required if replicas is set to a range.¶

name - (Optional) The run name.¶

image - (Optional) The name of the Docker image to run.¶

entrypoint - (Optional) The Docker entrypoint.¶

working_dir - (Optional) The path to the working directory inside the container. It's specified relative to the repository directory (/workflow) and should be inside it. Defaults to "." .¶

home_dir - (Optional) The absolute path to the home directory inside the container. Defaults to /root. Defaults to /root.¶

registry_auth - (Optional) Credentials for pulling a private Docker image.¶

python - (Optional) The major version of Python. Mutually exclusive with image.¶

env - (Optional) The mapping or the list of environment variables.¶

setup - (Optional) The bash commands to run on the boot.¶

resources - (Optional) The resources requirements to run the configuration.¶

volumes - (Optional) The volumes mount points.¶

commands - (Optional) The bash commands to run.¶

backends - (Optional) The backends to consider for provisioning (e.g., [aws, gcp]).¶

regions - (Optional) The regions to consider for provisioning (e.g., [eu-west-1, us-west4, westeurope]).¶

instance_types - (Optional) The cloud-specific instance types to consider for provisioning (e.g., [p3.8xlarge, n1-standard-4]).¶

spot_policy - (Optional) The policy for provisioning spot or on-demand instances: spot, on-demand, or auto.¶

retry - (Optional) The policy for resubmitting the run. Defaults to false.¶

retry_policy - (Optional) The policy for resubmitting the run. Deprecated in favor of retry.¶

max_duration - (Optional) The maximum duration of a run (e.g., 2h, 1d, etc). After it elapses, the run is forced to stop. Defaults to off.¶

max_price - (Optional) The maximum instance price per hour, in dollars.¶

pool_name - (Optional) The name of the pool. If not set, dstack will use the default name.¶

instance_name - (Optional) The name of the instance.¶

creation_policy - (Optional) The policy for using instances from the pool. Defaults to reuse-or-create.¶

termination_policy - (Optional) The policy for instance termination. Defaults to destroy-after-idle.¶

termination_idle_time - (Optional) Time to wait before destroying the idle instance. Defaults to 5m for dstack run and to 3d for dstack pool add.¶

dstack.api.Run¶

attach(ssh_identity_file=None) ¶

detach() ¶

logs(start_time=None, diagnose=False, replica_num=0, job_num=0) ¶

refresh() ¶

stop(abort=False) ¶

dstack.api.Resources¶

cpu - (Optional) The number of CPU cores. Defaults to 2...¶

memory - (Optional) The RAM size (e.g., 8GB). Defaults to 8GB...¶

`dstack.api`¶

`dstack.api.Client`¶

`from_config(project_name=None, server_url=None, user_token=None, ssh_identity_file=None)` `staticmethod` ¶

`dstack.api.RunCollection`¶

`get(run_name)` ¶

`list(all=False)` ¶

`submit(configuration, configuration_path=None, repo=None, backends=None, regions=None, instance_types=None, resources=None, spot_policy=None, retry_policy=None, max_duration=None, max_price=None, working_dir=None, run_name=None, reserve_ports=True)` ¶

`dstack.api.RepoCollection`¶

`init(repo, git_identity_file=None, oauth_token=None)` ¶

`dstack.api.Task`¶

`nodes` - (Optional) Number of nodes. Defaults to `1`.¶

`name` - (Optional) The run name.¶

`image` - (Optional) The name of the Docker image to run.¶

`entrypoint` - (Optional) The Docker entrypoint.¶

`working_dir` - (Optional) The path to the working directory inside the container. It's specified relative to the repository directory (`/workflow`) and should be inside it. Defaults to `"."` .¶

`home_dir` - (Optional) The absolute path to the home directory inside the container. Defaults to `/root`. Defaults to `/root`.¶

`registry_auth` - (Optional) Credentials for pulling a private Docker image.¶

`python` - (Optional) The major version of Python. Mutually exclusive with `image`.¶

`env` - (Optional) The mapping or the list of environment variables.¶

`setup` - (Optional) The bash commands to run on the boot.¶

`resources` - (Optional) The resources requirements to run the configuration.¶

`volumes` - (Optional) The volumes mount points.¶

`ports` - (Optional) Port numbers/mapping to expose.¶

`commands` - (Optional) The bash commands to run.¶

`backends` - (Optional) The backends to consider for provisioning (e.g., `[aws, gcp]`).¶

`regions` - (Optional) The regions to consider for provisioning (e.g., `[eu-west-1, us-west4, westeurope]`).¶

`instance_types` - (Optional) The cloud-specific instance types to consider for provisioning (e.g., `[p3.8xlarge, n1-standard-4]`).¶

`spot_policy` - (Optional) The policy for provisioning spot or on-demand instances: `spot`, `on-demand`, or `auto`.¶

`retry` - (Optional) The policy for resubmitting the run. Defaults to `false`.¶

`retry_policy` - (Optional) The policy for resubmitting the run. Deprecated in favor of `retry`.¶

`max_duration` - (Optional) The maximum duration of a run (e.g., `2h`, `1d`, etc). After it elapses, the run is forced to stop. Defaults to `off`.¶

`max_price` - (Optional) The maximum instance price per hour, in dollars.¶

`pool_name` - (Optional) The name of the pool. If not set, dstack will use the default name.¶

`instance_name` - (Optional) The name of the instance.¶

`creation_policy` - (Optional) The policy for using instances from the pool. Defaults to `reuse-or-create`.¶

`termination_policy` - (Optional) The policy for instance termination. Defaults to `destroy-after-idle`.¶

`termination_idle_time` - (Optional) Time to wait before destroying the idle instance. Defaults to `5m` for `dstack run` and to `3d` for `dstack pool add`.¶

`dstack.api.Service`¶

`port` - The port, that application listens on or the mapping.¶

`model` - (Optional) Mapping of the model for the OpenAI-compatible endpoint.¶

`https` - (Optional) Enable HTTPS. Defaults to `True`.¶

`auth` - (Optional) Enable the authorization. Defaults to `True`.¶

`replicas` - (Optional) The number of replicas. Can be a number (e.g. `2`) or a range (`0..4` or `1..8`). If it's a range, the `scaling` property is required. Defaults to `1`.¶

`scaling` - (Optional) The auto-scaling rules. Required if `replicas` is set to a range.¶

`name` - (Optional) The run name.¶

`image` - (Optional) The name of the Docker image to run.¶

`entrypoint` - (Optional) The Docker entrypoint.¶

`working_dir` - (Optional) The path to the working directory inside the container. It's specified relative to the repository directory (`/workflow`) and should be inside it. Defaults to `"."` .¶

`home_dir` - (Optional) The absolute path to the home directory inside the container. Defaults to `/root`. Defaults to `/root`.¶

`registry_auth` - (Optional) Credentials for pulling a private Docker image.¶

`python` - (Optional) The major version of Python. Mutually exclusive with `image`.¶

`env` - (Optional) The mapping or the list of environment variables.¶

`setup` - (Optional) The bash commands to run on the boot.¶

`resources` - (Optional) The resources requirements to run the configuration.¶

`volumes` - (Optional) The volumes mount points.¶

`commands` - (Optional) The bash commands to run.¶

`backends` - (Optional) The backends to consider for provisioning (e.g., `[aws, gcp]`).¶

`regions` - (Optional) The regions to consider for provisioning (e.g., `[eu-west-1, us-west4, westeurope]`).¶

`instance_types` - (Optional) The cloud-specific instance types to consider for provisioning (e.g., `[p3.8xlarge, n1-standard-4]`).¶

`spot_policy` - (Optional) The policy for provisioning spot or on-demand instances: `spot`, `on-demand`, or `auto`.¶

`retry` - (Optional) The policy for resubmitting the run. Defaults to `false`.¶

`retry_policy` - (Optional) The policy for resubmitting the run. Deprecated in favor of `retry`.¶

`max_duration` - (Optional) The maximum duration of a run (e.g., `2h`, `1d`, etc). After it elapses, the run is forced to stop. Defaults to `off`.¶

`max_price` - (Optional) The maximum instance price per hour, in dollars.¶

`pool_name` - (Optional) The name of the pool. If not set, dstack will use the default name.¶

`instance_name` - (Optional) The name of the instance.¶

`creation_policy` - (Optional) The policy for using instances from the pool. Defaults to `reuse-or-create`.¶

`termination_policy` - (Optional) The policy for instance termination. Defaults to `destroy-after-idle`.¶

`termination_idle_time` - (Optional) Time to wait before destroying the idle instance. Defaults to `5m` for `dstack run` and to `3d` for `dstack pool add`.¶

`dstack.api.Run`¶

`attach(ssh_identity_file=None)` ¶

`detach()` ¶

`logs(start_time=None, diagnose=False, replica_num=0, job_num=0)` ¶

`refresh()` ¶

`stop(abort=False)` ¶

`dstack.api.Resources`¶

`cpu` - (Optional) The number of CPU cores. Defaults to `2..`.¶

`memory` - (Optional) The RAM size (e.g., `8GB`). Defaults to `8GB..`.¶

`shm_size` - (Optional) The size of shared memory (e.g., `8GB`). If you are using parallel communicating processes (e.g., dataloaders in PyTorch), you may need to configure this.¶

`gpu` - (Optional) The GPU requirements.¶