An app can be either a web application (such as Streamlit, Gradio, etc.) or an API endpoint (like FastAPI, Flask, etc.) setup based on a pre-defined configuration.
dstack, you can define such configurations as code and run your apps with a single command, either
locally or in any cloud.
Creating a configuration file¶
A configuration can be defined as a YAML file (under the
workflows: - name: fastapi-gpu provider: bash ports: - 3000 commands: - pip install -r apps/requirements.txt - uvicorn apps.main:app --port 3000 --host 0.0.0.0 resources: gpu: count: 1
The configuration allows you to customize hardware resources, set up the Python environment, and more.
To configure ports, you have to specify the list of ports via the
Running an app¶
Once a configuration is defined, you can run it using the
dstack run command:
$ dstack run fastapi-gpu RUN WORKFLOW SUBMITTED STATUS TAG silly-dodo-1 fastapi-gpu now Submitted Starting SSH tunnel... To interrupt, press Ctrl+C. INFO: Started server process  INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://127.0.0.1:3000 (Press CTRL+C to quit)
dstack uses an exact copy of the source code that is locally present in the folder where you use the
If you don't want the app to sync certain files (especially large local files that are not needed
for the app), feel free to add them to the
.gitignore file. In this case,
dstack will ignore them,
even if you aren't using Git.
If you configure a project to run apps in the cloud,
dstack will automatically provision the
required cloud resources, and forward ports of the app to your local machine.
dstack tries to map the application ports to the same local ports. However, you can specify which local
ports to map the application ports to. For example, if you want to access the application port
127.0.0.1:3001, you have to pass
The default project runs apps locally. However, you can log into Hub and configure additional projects to run apps in a cloud account of your choice.
Stopping a run¶
To stop the app, click
C while the
dstack run command is running,
or use the
dstack stop command.
dstack will automatically clean up any cloud resources
if they are used.
If your project is configured to run apps in the cloud, you can use the
resources property in the YAML file to
request hardware resources like memory, GPUs, and shared memory size.
workflows: - name: fastapi-gpu-i provider: bash ports: - 3000 commands: - pip install -r apps/requirements.txt - uvicorn apps.main:app --port 3000 --host 0.0.0.0 resources: gpu: count: 1 interruptible: true
interruptible property tells
dstack to utilize spot instances. Spot instances may be not always available.
But when they are available, they are significantly cheaper.
Setting up the environment¶
You can use
conda executables to install packages and set up the environment.
python property to specify a version of Python for pre-installation. Otherwise,
dstack uses the local version.
To run the app with your custom Docker image, you can use the
workflows: - name: fastapi-docker provider: docker image: python:3.11 ports: - 3000 commands: - pip install -r apps/requirements.txt - uvicorn apps.main:app --port 3000 --host 0.0.0.0
Apps may download files like pre-trained models, external data, or Python packages. To avoid downloading them on each run, you can choose which paths to cache between runs.
workflows: - name: fastapi-cached provider: bash ports: - 3000 commands: - pip install -r apps/requirements.txt - uvicorn apps.main:app --port 3000 --host 0.0.0.0 cache: - path: ~/.cache/pip
Cache saves files in the configured storage and downloads them at startup. This improves performance and saves you from data transfer costs.
Cleaning up the cache¶
To clean up the cache, use the
dstack prune cache CLI command, followed by the name of the configuration.
Check out the
dstackai/dstack-examples repo for source code and other examples.