Dev environments¶

A dev environment is a virtual machine that includes the environment and an interactive IDE or notebook setup based on a pre-defined configuration.

With dstack, you can define such configurations as code and launch your dev environments with a single command, either locally or in any cloud.

Creating a configuration file¶

A configuration can be defined as a YAML file (under the .dstack/workflows directory).

workflows:
  - name: code-gpu
    provider: code
    setup:
      - pip install -r dev-environments/requirements.txt
    resources:
      gpu:
        count: 1

The configuration allows you to customize hardware resources, set up the Python environment, expose ports, configure cache, and more.

Running a dev environment¶

Once a configuration is defined, you can run it using the dstack run command:

$ dstack run code-gpu

RUN      WORKFLOW  SUBMITTED  STATUS     TAG
shady-1  code-gpu   now        Submitted  

Starting SSH tunnel...

To exit, press Ctrl+C.

Web UI available at http://127.0.0.1:10000/?tkn=4d9cc05958094ed2996b6832f899fda1

If we click the URL, it will open the dev environment.

For convenience, dstack uses an exact copy of the source code that is locally present in the folder where you use the dstack command.

Using .gitignore

If you don't want the dev environment to sync certain files (especially large local files that are not needed for the dev environment), feel free to add them to the .gitignore file. In this case, dstack will ignore them, even if you aren't using Git.

If you configure a project to run dev environments in the cloud, dstack will automatically provision the required cloud resources, and forward ports of the dev environment to your local machine.

Projects

The default project runs dev environments locally. However, you can log into Hub and configure additional projects to run dev environments in a cloud account of your choice.

Learn more →

Stopping a dev environment¶

To stop the dev environment, click Ctrl+C while the dstack run command is running, or use the dstack stop command. dstack will automatically clean up any cloud resources if they are used.

Supported IDEs¶

Out of the box, dstack allows you to launch VS Code, JupyterLab, and Jupyter notebooks. All you have to do is to set the provider property in the YAML configuration accordingly:

code – VS Code
lab - JupyterLab
notebook – Jupyter notebook

Connecting via SSH¶

Alternatively, you can use the bash provider with the ssh property set to true to attach any desktop IDE to the dev environment via SSH.

  workflows:
  - name: ssh
    provider: bash
    ssh: true 
    commands:
      - tail -f /dev/null

Configuring resources¶

If your project is configured to run dev environments in the cloud, you can use the resources property in the YAML file to request hardware resources like memory, GPUs, and shared memory size.

workflows:
  - name: code-v100-spot
    provider: code
    resources:
      gpu:
        name: V100
        count: 8
      interruptible: true

NOTE:

The interruptible property instructs dstack to use spot instances, which may not always be available. However, when they are, they are significantly cheaper.

Setting up the environment¶

You can use the setup property in the YAML file to pre-install packages or run other bash commands during environment startup.

workflows:
  - name: code-conda
    provider: code
    python: 3.11
    setup:
      - conda env create -f environment.yml

The python property specifies the version of Python. If not specified, dstack uses your current version.

You can use pip and conda executables to install packages and set up the environment.

NOTE:

Refer to Configuring cache to speed up the setup process.

Exposing ports¶

If you intend to run web apps from the dev environment, specify the list of ports using the ports property.

dstack will automatically forward these ports to your local machine. You'll see the URLs to access each port in the output.

Configuring cache¶

To avoid downloading files (e.g. pre-trained models, external data, or Python packages) every time you restart your dev environment, you can selectively cache the desired paths between runs.

workflows:
  - name: code-cached
    provider: code
    cache:
      - path: ~/.cache/pip
      - path: ./data
      - path: ./models

NOTE:

Caching enhances performance and helps you avoid unnecessary data transfer costs.

Cleaning up the cache¶

To clean up the cache, you can delete the files directly from the dev environment or use the dstack prune cache CLI command, followed by the name of the configuration.

NOTE:

Check out the dstackai/dstack-examples repo for source code and other examples.