AWS Lambda for CloudFormation Stack Dependencies

Kongregate’s infrastructure is represented entirely in code via AWS CloudFormation. CloudFormation is a templating system built by AWS for managing their infrastructure as code; you can read more about CloudFormation here. Since Kongregate’s infrastructure is entirely in code, it has been necessary to solve a number of limitations in the CloudFormation structure.

One such limitation in CloudFormation is cross stack dependencies. When defining entire infrastructure as code, it becomes a requirement to break down the architecture into logical components (e.g. DNS, web nodes, security groups, etc.). CloudFormation can handle this by nesting a stack within another stack, aptly named "nested stacks" (you can read more here about nested stacks). The major downside to CloudFormation’s nested stacks is its limited flexibility.

The Sony Xperia team came up with a great workaround. Using AWS Lambda as an intermediary layer, CloudFormation can reference another stack’s output as an input without a rigid code structure to maintain. Check out their repo here. This tool allows for simpler CloudFormation templates, better code maintainability, and improved overall readability.

To get started with stack dependencies in CloudFormation, follow these steps:

You will need an AWS account and git installed locally
Download the Sony Xperia repo locally (GitHub repo)
Install and setup AWS CLI (how-to here)
Install jq (how-to here)
Change your working directory to the stack-dependency directory in the repo directory
Run ./create-role.sh to create the IAM role that Lambda will use
Run ./deploy-lambda.sh to create the Lambda function

Build a dependency stack (you will need to change ‘myamazingbucket’):

 aws cloudformation create-stack --stack-name MyS3Stack --template-body '
   {
     "AWSTemplateFormatVersion" : "2010-09-09",
     "Resources" : {
       "S3Bucket": {
         "Type" : "AWS::S3::Bucket",
         "Properties" : {
           "BucketName" : "myamazingbucket"
         }
       }
     },
     "Outputs" : {
       "MyBucket" : {
         "Description" : "All your bucket are belong to me",
         "Value" : { "Ref" : "S3Bucket" }
       }
     }
   }'

Build a referring stack:

 aws cloudformation create-stack --stack-name MyS3BucketPolicyStack --template-body '
   {
     "AWSTemplateFormatVersion": "2010-09-09",
     "Resources": {
       "MyStackDependency": {
         "Type": "Custom::StackDependency",
         "Properties": {
           "ServiceToken": {
             "Fn::Join": ["", [
               "arn:aws:lambda:", {
                 "Ref": "AWS::Region"
               },
               ":", {
                 "Ref": "AWS::AccountId"
               },
               ":function:stackDependency"
             ]]
           },
           "StackName": "MyS3Stack"
         }
       },
       "MyS3BucketPolicy": {
         "Type": "AWS::S3::BucketPolicy",
         "Properties": {
           "Bucket": {
             "Fn::GetAtt": ["MyStackDependency", "MyBucket"]
           },
           "PolicyDocument": {
             "Statement": [{
               "Action": ["s3:GetObject"],
               "Effect": "Allow",
               "Resource": {
                 "Fn::Join": ["", ["arn:aws:s3:::", {
                     "Fn::GetAtt": ["MyStackDependency", "MyBucket"]
                   },
                   "/*"
                 ]]
               },
               "Principal": "*",
               "Condition": {
                 "StringLike": {
                   "aws:Referer": [
                     "http://www.example.com/*",
                     "http://example.com/*"
                   ]
                 }
               }
             }]
           }
         }
       }
     }
   }'

AWS CloudFormation is an extremely powerful tool that can be further extended using Lambda. Using Lambda can give CloudFormation a more robust and powerful DSL for infrastructure provisioning and management.