Skip to content

dstack 0.16.0: Pools

Previously, when running a dev environment, task, or service, dstack provisioned an instance in a configured backend, and upon completion of the run, deleted the instance.

In the latest update, we introduce pools, a significantly more efficient way to manage instance lifecycles and reuse instances across runs.

dstack run

Now, when using the dstack run command, it tries to reuse an instance from a pool. If no ready instance meets the requirements, dstack automatically provisions a new one and adds it to the pool.

Once the workload finishes, the instance is marked as ready (to run other workloads). If the instance remains idle for the configured duration, dstack tears it down.

Idle duration

By default, if dstack run provisions a new instance, its idle duration is set to 5m. This means the instance waits for a new workload for only five minutes before getting torn down. To override it, use the --idle-duration DURATION argument.

dstack pool

The dstack pool command allows for managing instances within pools.

To manually add an instance to a pool, use dstack pool add:

$ dstack pool add --gpu 80GB --idle-duration 1d

 BACKEND     REGION         RESOURCES                     SPOT  PRICE
 tensordock  unitedkingdom  10xCPU, 80GB, 1xA100 (80GB)   no    $1.595
 azure       westus3        24xCPU, 220GB, 1xA100 (80GB)  no    $3.673
 azure       westus2        24xCPU, 220GB, 1xA100 (80GB)  no    $3.673

Continue? [y/n]: y

The dstack pool add command allows specifying resource requirements, along with the spot policy, idle duration, max price, retry policy, and other policies.

If no idle duration is configured, by default, dstack sets it to 72h. To override it, use the --idle-duration DURATION argument.

Limitations

The dstack pool add command is not yet supported for Lambda, Azure, TensorDock, Kubernetes, and VastAI backends. Support for them is coming in version 0.16.1.

Refer to pools for more details on the new feature and how to use it.

Why does this matter?

With this new feature, using the cloud can be a lot more predictable and convenient:

  1. Now, you can provision instances in advance and ensure they are available for the entire duration of the project. This saves you from the risk of not having a GPU when you need it most.
  2. If you reuse an instance from a pool, dstack run starts much faster. For example, you can provision an instance and reuse it for running a dev environment, task, or service.

Feedback

Have questions or need help? Drop us a message on our Discord server. See a bug? Report it to GitHub issues.