Orchestrate GPU workloads effortlessly
dstack is an open-source orchestration engine for cost-effectively running AI workloads in the cloud as well as on-premises.
Dev environments
Before submitting a task or deploying a model, you may want to run code interactively. Dev environments allow you to do exactly that.
You specify the required environment and resources, then run it. dstack provisions the dev environment in the cloud and enables access via your desktop IDE.
Tasks
Tasks allow for convenient scheduling of any kind of batch jobs, such as training, fine-tuning, or data processing, as well as running web applications.
Specify the environment and resources, then run it. dstack executes the task in the cloud, enabling port forwarding to your local machine for convenient access.
Services
Services make it very easy to deploy any kind of model or web application as public endpoints.
Use any serving frameworks and specify required resources. dstack deploys it in the configured backend, handles authentication, auto-scaling, and provides an OpenAI-compatible interface if needed.
Pools
You can have instances provisioned in the cloud automatically, or add them manually, configuring the required resources, idle duration, etc.
Pools simplify managing the lifecycle of cloud instances and enable their efficient reuse across runs.
Featured examples
Mixtral 8x7B
Deploy Mixtral 8x7B as a service using vLLM, an open-source serving library.
Text Embeddings Inference
Deploy text embeddings models using Services and TEI, an open-source text embeddings toolkit by Hugging Face.
Llama Index
Use Llama Index and Weaviate to enhance the capabilities of LLMs with the context of your data.
QLoRA
Fine-tune Llama 2 on a custom dataset, with QLoRA and your own script, using Tasks.
Text Generation Inference
Deploy LLMs using Services and TGI, an open-source serving framework by Hugging Face.
vLLM
Deploy LLMs with Services and vLLM,
an open-source serving library.
Get started in a minute
Looking for the enterprise version? Contact our sales.
FAQ
The open-source version allows you to run workloads using your own cloud accounts or on-premise servers.
dstack Sky is a fully managed service that enables you to run workloads across multiple cloud providers, guaranteeing optimal GPU pricing and availability. You don't need individual accounts with each provider – dstack Sky manages everything for you.