AWS¶
The AWS
backend type allows provisioning infrastructure and storing artifacts in an AWS account.
Follow the step-by-step guide below to configure this backend in your project.
Create an S3 bucket¶
First, you need to create an S3 bucket. dstack
will use this bucket to store state and artifacts.
NOTE:
Make sure that the bucket is created in the same region where you plan to provision infrastructure.
Create an IAM user¶
The next step is to create an IAM user and
grant this user permissions to perform actions on the s3
, logs
, secretsmanager
, ec2
, and iam
services.
IAM policy template
If you'd like to limit the permissions to the most narrow scope, feel free to use the IAM policy template below.
Replace {bucket_name}
and {bucket_name_under_score}
variables in the template below
with the values that correspond to your S3 bucket.
For {bucket_name}
, use the name of the S3 bucket.
For {bucket_name_under_score}
, use the same but with dash characters replaced to underscores
(e.g. if {bucket_name}
is my-awesome-project
, then {bucket_name_under_score}
must be my_awesome_project
.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListAllMyBuckets",
"s3:GetBucketLocation"
],
"Resource": [
"arn:aws:s3:::*"
]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:ListBucket",
"s3:GetLifecycleConfiguration",
"s3:PutLifecycleConfiguration",
"s3:PutObjectTagging",
"s3:GetObjectTagging",
"s3:DeleteObjectTagging",
"s3:GetBucketAcl"
],
"Resource": [
"arn:aws:s3:::{bucket_name}",
"arn:aws:s3:::{bucket_name}/*"
]
},
{
"Effect": "Allow",
"Action": [
"logs:DescribeLogGroups"
],
"Resource": [
"arn:aws:logs:*:*:log-group:*"
]
},
{
"Effect": "Allow",
"Action": [
"logs:FilterLogEvents",
"logs:TagLogGroup",
"logs:CreateLogGroup",
"logs:CreateLogStream"
],
"Resource": [
"arn:aws:logs:*:*:log-group:/dstack/jobs/{bucket_name}*:*",
"arn:aws:logs:*:*:log-group:/dstack/runners/{bucket_name}*:*"
]
},
{
"Effect": "Allow",
"Action": [
"secretsmanager:UpdateSecret",
"secretsmanager:GetSecretValue",
"secretsmanager:CreateSecret",
"secretsmanager:PutSecretValue",
"secretsmanager:PutResourcePolicy",
"secretsmanager:TagResource",
"secretsmanager:DeleteSecret"
],
"Resource": [
"arn:aws:secretsmanager:*:*:secret:/dstack/{bucket_name}/credentials/*",
"arn:aws:secretsmanager:*:*:secret:/dstack/{bucket_name}/secrets/*"
]
},
{
"Effect": "Allow",
"Action": [
"ec2:DescribeInstanceTypes",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeImages",
"ec2:DescribeInstances",
"ec2:DescribeSpotInstanceRequests",
"ec2:DescribeSpotPriceHistory",
"ec2:RunInstances",
"ec2:CreateTags",
"ec2:CreateSecurityGroup",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:AuthorizeSecurityGroupEgress"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"ec2:CancelSpotInstanceRequests",
"ec2:TerminateInstances"
],
"Resource": "*",
"Condition": {
"StringEquals": {
"aws:ResourceTag/dstack_bucket": "{bucket_name}"
}
}
},
{
"Effect": "Allow",
"Action": [
"iam:GetRole",
"iam:CreateRole",
"iam:AttachRolePolicy",
"iam:TagRole"
],
"Resource": "arn:aws:iam::*:role/dstack_role_{bucket_name_under_score}*"
},
{
"Effect": "Allow",
"Action": [
"iam:CreatePolicy",
"iam:TagPolicy"
],
"Resource": "arn:aws:iam::*:policy/dstack_policy_{bucket_name_under_score}*"
},
{
"Effect": "Allow",
"Action": [
"iam:GetInstanceProfile",
"iam:CreateInstanceProfile",
"iam:AddRoleToInstanceProfile",
"iam:TagInstanceProfile",
"iam:PassRole"
],
"Resource": [
"arn:aws:iam::*:instance-profile/dstack_role_{bucket_name_under_score}*",
"arn:aws:iam::*:role/dstack_role_{bucket_name_under_score}*"
]
}
]
}
Set up AWS credentials¶
dstack
support two methods to authenticate with AWS: Default credentials and Access key.
Default credentials¶
dstack
can automatically pick up AWS credentials set up on your machine
(e.g. credentials stored as AWS profiles or environment variables).
You can use default credentials if you don't want to enter and store AWS credentials in dstack
.
Access key¶
dstack
also support authentication using an access key. To create an access key,
follow this guide. Once the access key is created, make sure to download the .csv
file containing your IAM user's
Access key ID
and Secret access key
.
Configure the backend¶
Now, log in to the UI, open the project's settings,
click Edit
, then click Add backend
, and select AWS
in the Type
field.
Fields reference¶
The following fields are required:
Region
- (Required) The region wheredstack
will provision infrastructure and store state and artifactsBucket
- (Required) The S3 bucket to store state and artifacts (must be in the same region)
The following arguments are optional:
Access key ID
- (Optional) The Access key ID to authenticatedstack
Secret access key
- (Optional) The Secret access key to authenticatedstack
Subnet
- (Optional) The EC2 subnet is required to provision infrastructure using a non-default VPC and subnet. If not specified, dstack will use the default VPC and subnet.