Skip to content

Blog

Managed gateways for serving LLMs across clouds

The dstack 0.11.1 update makes it easier to serve LLMs via public endpoints.

Services is a preview feature that allows serving LLMs via public endpoints using multiple clouds (AWS, GCP, Azure, Lambda, etc.) The latest update of dstack makes the use of services much easier by introducing default gateways and wildcard domains.

Automatic GPU discovery across clouds

The 0.11 update significantly cuts GPU costs and boosts their availability.

The latest release of dstack enables the automatic discovery of the best GPU price and availability across multiple configured cloud providers and regions.

An early preview of services

The 0.10.7 update introduces a new configuration type for serving.

Until now, dstack has supported dev-environment and task as configuration types. Even though task may be used for basic serving use cases, it lacks crucial serving features. With the new update, we introduce service, a dedicated configuration type for serving.

Port mapping, max duration, and more

The 0.10.6 release introduces port mapping, max duration, more supported GPUs, etc.

The latest release of dstack brings numerous improvements in many areas: from support for more GPU types to better mapping of ports and monitoring running workloads. Read below to learn more.

Lambda Cloud GA, and Docker support

The 0.10.5 release improves Lambda Cloud integration and adds support for Docker.

In the previous update, we added initial integration with Lambda Cloud. With today's release, this integration has significantly improved and finally goes generally available. Additionally, the latest release adds support for custom Docker images.

An early preview of Lambda Cloud support

Check out the 0.10.3 update with initial support for Lambda Cloud.

dstack has two key features. Firstly, it simplifies the running of ML workloads in the cloud. Secondly, it supports multiple clouds, allowing to stay independent of a particular vendor and reduce costs. Our latest update represents a significant stride in this direction.

Say goodbye to managed notebooks

Why managed notebooks are losing ground to cloud dev environments.

Data science and ML tools have made significant advancements in recent years. This blog post aims to examine the advantages of cloud dev environments (CDE) for ML engineers and compare them with web-based managed notebooks.

An early preview of the build command

The 0.10.2 update adds a command that allows for pre-building environments.

We are continuously striving to make running dev environments and ML tasks in the cloud even easier. With the new release, we have added two new features that we believe radically improve the developer experience.

Using OSS LLMs with LangChain in the cloud

A tutorial explaining how to build a chatbot using OSS LLMs on your cloud.

LangChain makes it easier to use LLMs for app development, which is why many people want to use it. This tutorial shows how to use LangChain with OSS LLMs on your own cloud, using the example of building a chatbot with Falcon.

New configuration format and CLI experience

The 0.10 update is out with refined configuration format and CLI experience.

This is the most massive and feature-loaded update of dstack in the last several months. The update introduces a brand-new way to configure and run dev environments and tasks, making it a lot more convenient to use.