gateway

The gateway configuration type allows creating and updating gateways.

Root reference

name - (Optional) str The gateway name.

default - (Optional) bool Make the gateway default.

backend - (Required) "amddevcloud" | "aws" | "azure" | "cloudrift" | "cudo" | "datacrunch" | "digitalocean" | "dstack" | "gcp" | "hotaisle" | "kubernetes" | "lambda" | "local" | "remote" | "nebius" | "oci" | "runpod" | "tensordock" | "vastai" | "verda" | "vultr" The gateway backend.

region - (Required) str The gateway region.

instance_type - (Optional) str Backend-specific instance type to use for the gateway instance. Omit to use the backend's default, which is typically a small non-GPU instance.

router - (Optional) object The router configuration.

domain - (Optional) str The gateway domain, e.g. example.com.

public_ip - (Optional) bool Allocate public IP for the gateway. Defaults to True.

certificate - (Optional) object The SSL certificate configuration. Defaults to type: lets-encrypt.

tags - (Optional) dict The custom tags to associate with the gateway. The tags are also propagated to the underlying backend resources. If there is a conflict with backend-level tags, does not override them.

router

SGLang Model Gateway

type - (Required) "sglang" The router type. Must be sglang.

policy - (Optional) "random" | "round_robin" | "cache_aware" | "power_of_two" The routing policy. Options: random, round_robin, cache_aware, power_of_two. Defaults to cache_aware.

certificate

Let's encryptACM

type - (Required) "lets-encrypt" Automatic certificates by Let's Encrypt. Must be lets-encrypt.

type - (Required) "acm" Certificates by AWS Certificate Manager (ACM). Must be acm.

arn - (Required) str The ARN of the wildcard ACM certificate for the domain.