Dev environments¶
A dev environment is a virtual machine that includes the environment and an interactive IDE or notebook setup based on a pre-defined configuration.
With dstack
, you can define such configurations as code and launch your dev environments with a single command,
either locally or in any cloud.
Creating a configuration file¶
A configuration can be defined as a YAML file (under the .dstack/workflows
directory).
workflows:
- name: code-gpu
provider: code
setup:
- pip install -r dev-environments/requirements.txt
resources:
gpu:
count: 1
The configuration allows you to customize hardware resources, set up the Python environment, expose ports, configure cache, and more.
Running a dev environment¶
Once a configuration is defined, you can run it using the dstack run
command:
$ dstack run code-gpu
RUN WORKFLOW SUBMITTED STATUS TAG
shady-1 code-gpu now Submitted
Starting SSH tunnel...
To exit, press Ctrl+C.
Web UI available at http://127.0.0.1:10000/?tkn=4d9cc05958094ed2996b6832f899fda1
If we click the URL, it will open the dev environment.
For convenience, dstack
uses an exact copy of the source code that is locally present in the folder where you use the dstack
command.
Using .gitignore
If you don't want the dev environment to sync certain files (especially large local files that are not needed
for the dev environment), feel free to add them to the .gitignore
file. In this case, dstack
will ignore them,
even if you aren't using Git.
If you configure a project to run dev environments in the cloud, dstack
will automatically provision the
required cloud resources, and forward ports of the dev environment to your local machine.
Projects
The default project runs dev environments locally. However, you can log into Hub and configure additional projects to run dev environments in a cloud account of your choice.
Stopping a dev environment¶
To stop the dev environment, click Ctrl
+C
while the dstack run
command is running,
or use the dstack stop
command. dstack
will automatically clean up any cloud resources if they are used.
Supported IDEs¶
Out of the box, dstack
allows you to launch VS Code, JupyterLab, and Jupyter notebooks.
All you have to do is to set the provider
property in the YAML configuration accordingly:
Connecting via SSH¶
Alternatively, you can use the bash
provider with the
ssh
property set to true
to attach any desktop IDE to the dev
environment via SSH.
workflows:
- name: ssh
provider: bash
ssh: true
commands:
- tail -f /dev/null
Configuring resources¶
If your project is configured to run dev environments in the cloud, you can use the
resources
property in the YAML
file to request hardware resources like memory, GPUs, and shared memory size.
workflows:
- name: code-v100-spot
provider: code
resources:
gpu:
name: V100
count: 8
interruptible: true
NOTE:
The interruptible
property instructs dstack
to use spot instances, which may not always be available. However, when they
are, they are significantly cheaper.
Setting up the environment¶
You can use the setup
property in the YAML file to pre-install packages or run other bash commands during environment
startup.
workflows:
- name: code-conda
provider: code
python: 3.11
setup:
- conda env create -f environment.yml
The python
property specifies the version of Python. If not specified, dstack
uses
your current version.
You can use pip
and conda
executables to install packages and set up the environment.
NOTE:
Refer to Configuring cache to speed up the setup process.
Exposing ports¶
If you intend to run web apps from the dev environment, specify the list of ports using the
ports
property.
dstack
will automatically forward these ports to your local machine. You'll see the URLs to access each port in the
output.
Configuring cache¶
To avoid downloading files (e.g. pre-trained models, external data, or Python packages) every time you restart your dev environment, you can selectively cache the desired paths between runs.
workflows:
- name: code-cached
provider: code
cache:
- path: ~/.cache/pip
- path: ./data
- path: ./models
NOTE:
Caching enhances performance and helps you avoid unnecessary data transfer costs.
Cleaning up the cache¶
To clean up the cache, you can delete the files directly from the dev environment or use the
dstack prune cache
CLI command, followed by the name of the configuration.
NOTE:
Check out the dstackai/dstack-examples
repo for source code and other examples.