Applying DRY to AWS policies in Terraform

Photo by Dino Reichmuth on Unsplash

I have been working on Terraform for over 7 years now but never thought of applying DRY (Don't repeat yourself) concept on policies until recently. Three years ago, I started working for a startup, where I was solely responsible for architecting, deploying and managing multi-account infrastructure on AWS using Terraform for a single-tenant SaaS product. Little I knew back then that it would grow so big and complex that down the line I’ll have to refactor the code base to even support multi-region deployment as the client base grow. Its all together a different story why I had to refactor the code base to support multi-region.

While refactoring the code, I noticed there was a lot of repetition of statement blocks in IAM policies and it was degrading the readability and that’s when I decided to apply DRY. Whether it’s a trust relationship policy, resource policy or identity policy I started modifying the implementation one by one for all the policies to achieve DRY.

Enough of talking. Let’s see in action how I did it.

---

Trust Relationship Polices

Trust relationship policies are attached to IAM entities like user and role so that the permissions associated to these entities can be assumed by the IAM principal.

I created a few roles that will be used by GitHub Actions workflows and all these workflows require different set of permissions. Prioritising security like always, I decided to create individual role for each workflow rather than creating a shared role. Initially, the thought to implement DRY did not cross my mind and later that resulted in degraded code quality and that’s when I decided to optimise it.

Initially, I thought of creating a local variable but then I realised that it wouldn’t work because the policy isn’t static. So, I decided to use templatefile function and created a file which will contain the skeleton of trust policy for roles that will be used for GitHub workflow.

github-oidc-trust-policy.tftpl

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Principal": {
        "Federated": "${github_oidc_arn}"
      },
      "Condition": {
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
        },
        "StringLike": {
          "token.actions.githubusercontent.com:sub": ${jsonencode([
            for sub in subjects: "repo:${sub.repo_org}/${sub.repo_name}:*:workflow:${sub.workflow_name}"
          ])}
        }
      }
    }
  ]
}

Note: The above template file contains variables within interpolation syntax which will be passed while creating an IAM role.

resource "aws_iam_role" "github_runner_oidc" {
  name                 = "${local.prefix}-github-runner-oidc"
  max_session_duration = 21600 # 6 hours

  assume_role_policy = templatefile("${path.module}/github-oidc-trust-policy.tftpl", {
    github_oidc_arn = aws_iam_openid_connect_provider.github.arn,
    subjects = [
      {
        repo_org      = "paliwalvimal"
        repo_name     = "example"
        workflow_name = "deploy"
      }
    ]
  })

  tags = local.tags
}

Note: By default, the JWT token received from GitHub does not contain workflow information in the subject, so we need to modify the subject claim to include the workflow name.

With just one role, the above implementation might not provide much benefit and might feel an overhead but in case of multiple roles you will notice a huge impact just like I did. Moreover, in future whenever I have to modify the trust policy I don’t have to do it individually for each of the roles.

After noticing the above improvements, I decided to optimise identity and resource policies too.

---

Resource Policy

It can be different in case of resource policies as compared to trust policy because sometimes in case a resource policy there might be just one of the statement blocks rather than the entire policy that we want to manage as a template. The same happened with me too. In few scenarios, I was able to manage resource policies entirely using templatefile while other times I had to extract one or few statement blocks out of an entire policy.

After browsing through the entire codebase, I noticed I can optimise S3 bucket policy as all the bucket policies had at least one statement block in common which was about denying insecure connections. So, I created another file which contains only the deny statement block.

s3-ssl-deny-statement.tftpl

{
  "Sid": "AllowSSLRequestsOnly",
  "Effect": "Deny",
  "Action": "s3:*",
  "Principal": "*",
  "Resource": [
    "${bucket_arn}",
    "${bucket_arn}/*"
  ],
  "Condition": {
    "Bool": {
      "aws:SecureTransport": "false"
    }
  }
}

Note: Just like the trust policy we created earlier, the above statement contains a variable bucket_arn that needs to be passed when we refer this file.

resource "aws_s3_bucket_policy" "example" {
  bucket = aws_s3_bucket.example.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      jsondecode(templatefile("s3-ssl-deny-statement.tftpl", {
        bucket_arn = aws_s3_bucket.example.arn
      }))
    ]
  })
}

It was going all smooth until I came across another hurdle where I had to extract multiple statement blocks and construct a policy. It was a bit tricky and took me a while to come up with a solution. Let me show you what I’m talking about.

aws-log-s3-access.tftpl

[
  {
    "Sid": "AWSLogDeliveryWrite",
    "Effect": "Allow",
    "Principal": {
      "Service": "delivery.logs.amazonaws.com"
    },
    "Action": "s3:PutObject",
    "Resource": "${bucket_arn}/*",
    "Condition": {
      "StringEquals": {
        "s3:x-amz-acl": "bucket-owner-full-control",
        "aws:SourceAccount": "${account_id}"
      },
      "ArnLike": {
        "aws:SourceArn": "arn:aws:logs:${region}:${account_id}:*"
      }
    }
  },
  {
    "Sid": "AWSLogDeliveryAclCheck",
    "Effect": "Allow",
    "Principal": {
      "Service": "delivery.logs.amazonaws.com"
    },
    "Action": "s3:GetBucketAcl",
    "Resource": "${bucket_arn}",
    "Condition": {
      "StringEquals": {
        "aws:SourceAccount": "${account_id}"
      },
      "ArnLike": {
        "aws:SourceArn": "arn:aws:logs:${region}:${account_id}:*"
      }
    }
  }
]

In the above file, we have two statement blocks and these needs to be attached to an S3 bucket policy along with the deny statement block that we created earlier. If you notice, the above statement blocks are within an array because we cannot have two root map blocks in a single JSON object and hence, I had to put them in an array.

The fun part is that within the resource policy Statement is itself an array and now I have an array of two statement blocks and an additional map of statement block which needs to be tied together to complete the entire bucket policy. Let’s see how I decided to accomplish this after scratching my head for a while.

resource "aws_s3_bucket_policy" "example" {
  bucket = aws_s3_bucket.example.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = flatten([
      jsondecode(templatefile("aws-log-s3-access.tftpl", {
        bucket_arn = aws_s3_bucket.example.arn
        region     = "us-east-1"
        account_id = "1111111111"
      })),
      [jsondecode(templatefile("s3-ssl-deny-statement.tftpl", {
        bucket_arn = aws_s3_bucket.example.arn
      }))]
    ])
  })
}

That’s a lot of use of templatefile! But overall, it looks neat and easy to read too. Isn’t it?

This approach improved the readability of code for me to a great extent and reduced the overall line count by almost 40%.

Let’s also understand that we don’t have to make use of templatefile everywhere in our Terraform code to apply DRY. For example, in case of a static policy, we can simple create a local variable and refer to it wherever needed.

Thanks for reading the article and hope you enjoyed reading it and learnt something new! See you in the next article.

---

Understanding the basics

• How to do a dry run in Terraform?

  You can fire the command terraform plan to perform a dry run that will refresh the state and present you with all the modifications if any that will be performed when you run terraform apply. Note: this dry run is different from the DRY concept discussed in this article.

• What is the DRY principle in Terraform?

  DRY stands for Don’t Repeat Yourself. It isn’t a Terraform specific principle but is rather an engineering principle which simply means you should avoid duplicating any piece of code as this increases complexity and requires extra effort for maintenance.

• How do I pass a Terraform policy?

  There are multiple ways through which you can create an IAM policy in Terraform. You can either choose to opt for inline option just as we have done in this article or use the data source aws_iam_policy_document to generate a policy.