Home

Awesome

Cloud Infrastructure Manager (CIM)

version downloads license dependencies

CIM takes the pain out of Infrastructure as Code and CloudFormation!

CIM is a simple command line utility that bootstraps your CloudFormation CRUD operations, making them easier to execute, repeatable, and less error-prone. CIM separates out the stack template (YAML file) from the stack configuration (CLI options) so both can be stored safely in your project and executed again-and-again for stack updates. 

CIM is not a CloudFormation abstraction.

So what’s the problem? Why did you build CIM? The problem I was having with the AWS CloudFormation cli was remembering the exact cli options used in previous executions. Plus I wanted support for things like nested stacks, variable resolution, environments, encryption, Lambda deployments, etc…

Easily create your stack, build and deploy your code, and view your logs.

Table of contents:

Usage

# Install CIM
npm install cim -g

# See the available templates
cim templates

# Create your first stack using the lambda template
mkdir app
cd app
cim create --template=lambda-node

# Deploy your stack
cim stack-up

# Deploy your code
cim lambda-deploy

# View logs
cim lambda-logs --function=<function-name> --tail=true

# Delete you stack
cim stack-delete

Setup

Prerequisites

Install CIM

Use npm to install CIM globally.

npm install cim -g

Commands

create

Create a new CIM package based on a give template.

At minimum a CIM package contains the following files:

Usage

mkdir app
cd app
cim create --template=<template>

Where <template> is equal to one of the templates from cim templates.

For examples see the templates.

Options

templates

View all the available templates. Templates are used in the create command.

Usage

cim templates

stack-up

Create or update your stack. Sends a create or update command to CloudFormation using the properties defined in your _cim.yml.

Usage

cim stack-up {OPTIONS}

Options

stack-show

Show all the details about your CloudFormation stack. Helper method to see the status of your stack.

Usage

cim stack-show {OPTIONS}

Options

stack-delete

Delete your stack. Sends a delete command to CloudFormation using the properties defined in your _cim.yml.

Usage

cim stack-delete {OPTIONS}

Options

lambda-deploy

Deploy your Lambda functions.

If alias and lambda-version are used then alias is simply updated to point to the specified lambda-version. Read more about Versions and Aliases here.

If alias and lambda-version are omitted then a new version of the code is uploaded and the '$LATEST' alias is updated to point to this new version.

Usage

cim lambda-deploy {OPTIONS}

Options

Versions and Aliases

AWS recommends using Lambda Versions and Aliases in production. This is not required though

Here is an example CloudFormation template that uses a Lambda version and alias. The alias is then used by the S3 trigger.

  #
  # Our Lambda function.
  #
  LambdaFunction:
    Type: AWS::Lambda::Function
    Properties:
      Handler: index.handler
      Timeout: 5
      Role:
        Fn::GetAtt:
          - IamRoleLambdaExecution
          - Arn
      Code:
        ZipFile: !Sub |
          'use strict';

          exports.handler = function(event, context) {
              console.log(JSON.stringify(event));
              context.succeed('Hello CIM!');
          };
      Runtime: nodejs6.10

  #
  # Version 1 of our function.
  #
  LambdaFunctionVersion:
    Type: AWS::Lambda::Version
    Properties:
      FunctionName: !Ref LambdaFunction

  #
  # 'PROD' Alias -> Version 1
  #
  LambdaFunctionAlias:
    Type: AWS::Lambda::Alias
    Properties:
      FunctionName: !Ref LambdaFunction
      FunctionVersion: !GetAtt LambdaFunctionVersion.Version
      Name: 'PROD'

  #
  # S3 bucket
  #
  S3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName:
        Fn::Join:
          - ''
          - - !Ref AWS::StackName
            - '-s3'
      AccessControl: 'Private'
      NotificationConfiguration:
        LambdaConfigurations:
          -
            Function: !Ref LambdaFunctionAlias
            Event: 's3:ObjectCreated:*'
            Filter:
              S3Key:
                Rules:
                  -
                    Name: suffix
                    Value: .jpg

Here is the _cim.yml lambda section that uses the Alias:

lambda:
  functions:
    -
      function: ${stack.outputs.LambdaFunction}
      aliases:
        PROD: ${stack.outputs.LambdaFunctionAlias}
      zip_file: index.zip
  deploy:
    phases:
      pre_deploy:
        commands:
          # Install all npm packages including dev packages.
          - npm install

          # Run the tests
          # - npm test

          # Remove all the npm packages.
          - rm -Rf node_modules

          # Only install the non-dev npm packages.  We don't want to bloat our Lambda with dev packages.
          - npm install --production

          # Zip the Lambda for upload to S3.
          - zip -r index.zip .
      post_deploy:
        commands:
          # Remove the zip file.
          - rm -Rf index.zip

          # Reinstall the dev npm packages.
          - npm install

We can now deploy additional versions to our Lambda without affecting the PROD alias. Once we have tested our code and are ready to go live we simply update the PROD alias to point to the new version.

Don't forget to prune your unused versions so you don't run out of space.

If you decide to use versions and aliases your deployment becomes two steps.

Deployment with versions and aliases:

Deployment without versions and aliases:

lambda-publish

Publish a new lambda-version of this function.

Usage

cim lambda-publish {OPTIONS}

Options

lambda-versions

Show all the lambda function versions and associated aliases.

Usage

cim lambda-versions {OPTIONS}

Options

lambda-prune

Delete one or more unused version's of this function.

Usage

cim lambda-prune {OPTIONS}

Options

lambda-logs

Show the CloudWatch Logs for a single Lambda function.

Usage

cim lambda-logs {OPTIONS}

Options

help

Usage

View all the available commands.

cim help

View a single command.

cim <command> --help

_cim.yml

The _cim.yml file is where details about your stack are stored. Lets take a look at base template.

Basic

version: 0.1
stack:
  name: 'base'
  template:
    file: 'cloudformation.yml'
    bucket: 'base-templates'

The stack name will be used when creating your CloudFormation stack. Shen you view all your CloudFormation stacks through the AWS console, this will be the name.
A lot of the AWS resources created within this stack will also have this name as a prefix within their name.

The template defines the local CloudFormation file to use for this stack. When calling CloudFormation the CloudFormation file needs to be on S3. The S3 bucket is where CIM stores the CloudFormation file prior to calling CloudFormation.

Parent stacks

In most cases you will have multiple stacks, and these stacks will be nested. For example, lets say you have a base stack for your VPC and other shared resources, and then a stack for your api application. Your project structure might look like:

/iac
    /base
        - _cim.yml
        - cloudformation.yml
    /api-app
        - _cim.yml
        - cloudformation.yml

Through the use of the parents field you can reference and reuse items in a parent stack. In our api app example we reuse the parent stack name and template bucket.

version: 0.1
stack:
  name: '${stack.parents.base.stack.name}-api'
  template:
    file: 'cloudformation.yml'
    bucket: '${stack.parents.base.stack.template.bucket}'
  parents:
    base: '../base'

You can reference multiple parents by key.

Parameters

The parameters field is used to define the input parameter values sent to CloudFormation during a stack-up command.

Continuing our example above lets say we also want to pass in the base stack name as an input parameter for cross stack parameter referencing.

version: 0.1
stack:
  name: '${stack.parents.base.stack.name}-api'
  template:
    file: 'cloudformation.yml'
    bucket: '${stack.parents.base.stack.template.bucket}'
  parents:
    base: '../base'
  parameters:
    BaseStackName: '${stack.parents.base.stack.name}'

Capabilities

If you have IAM resources, you can specify either capability. If you have IAM resources with custom names, you must specify CAPABILITY_NAMED_IAM. If you don't specify this parameter, this action returns an InsufficientCapabilities error.

Valid Values: CAPABILITY_IAM | CAPABILITY_NAMED_IAM Continuing our example above lets say we also want to pass in the base stack name as an input parameter for cross stack parameter referencing.

version: 0.1
stack:
  name: '${stack.parents.base.stack.name}-api'
  template:
    file: 'cloudformation.yml'
    bucket: '${stack.parents.base.stack.template.bucket}'
  parents:
    base: '../base'
  parameters:
    BaseStackName: '${stack.parents.base.stack.name}'
  capabilities:
    - 'CAPABILITY_IAM'

Tags

Tags are used to not only tag your CloudFormation stack but to also tag all resources created by the stack given that those resources support tags.

version: 0.1
stack:
  name: '${stack.parents.base.stack.name}-api'
  template:
    file: 'cloudformation.yml'
    bucket: '${stack.parents.base.stack.template.bucket}'
  parents:
    base: '../base'
  parameters:
    BaseStackName: '${stack.parents.base.stack.name}'
  capabilities:
    - 'CAPABILITY_IAM'
  tags:
    app: 'api-app'
    owner: 'John Doe'

Policy and PolicyDuringUpdate

CIM supports both the Policy and PolicyDuringUpdate CloudFormation params.

version: 0.1
stack:
  name: 'test'
  template:
    file: 'cloudformation.yml'
    bucket: 'test-bucket'
  policy:
    file: 'policy.json'
    bucket: 'test-bucket'
  policyDuringUpdate:
    file: 'policyDuringUpdat'
    bucket: 'test-bucket'

For more information about Policy and PolicyDuringUpdate see here: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/protect-stack-resources.html#protect-stack-resources-modifying

All CloudFormation Params

All AWS SDK CloudFormation createStack and updateStack params are supported. If you add them to your _cim.yml config file they will be used when creating and updating your stack.

Stage

The stage object is used to override any part of the configuration file when that --stage is used as a command line option. For example if we have the following dev stage:

version: 0.1
stack:
  name: 'base-prod'
  template:
    file: 'cloudformation.yml'
    bucket: 'base-templates'
  parameters:
    param1: 'prod-param'
stage:
  dev:
    stack:
      name: 'base-dev'
      parameters:
        param1: 'dev-param'

Now when we use the --stage=dev command line option, our stack name will be 'base-dev' and our param1 will be 'dev-param'. Any field can be overridden.

opt

You can reference any cli option and use it in a variable.

For example, we use the profile cli option in this command below: cim stack-up --profile=bluefin

version: 0.1
stack:
  name: 'base-prod'
  template:
    file: 'cloudformation.yml'
    bucket: 'base-templates'
  parameters:
    param1: '${opt.profile}'

env

You can reference any environment var in the config file and use it in a variable.

version: 0.1
stack:
  name: 'base-prod'
  template:
    file: 'cloudformation.yml'
    bucket: 'base-templates'
  parameters:
    param1: '${env.param1}'

kms.decrypt

You can include any kms encrypted string (base64 encoded) and CIM will decrypt prior to running the commend.

version: 0.1
stack:
  name: 'base-prod'
  template:
    file: 'cloudformation.yml'
    bucket: 'base-templates'
  parameters:
    param1: '${kms.decrypt(<kms encrypted and bas64 encoded string)}'

Lambda

If your stack includes one or more lambda's you can add the lambda section to your _cim.yml to enable Lambda support (lambda-deploy, lambda-logs).

In this example we have two Lambda functions. The function will be an output param from our stack. The function is used by the lambda-deploy and lambda-logs commands to specify a single function.

The deploy section which is broken up into two parts is used by the lambda-deploy command.

The Lambda zip that is packaged in the pre_deploy phase must match the zip_file under each function. When a function is deployed it uses the zip_file as the deployment artifact.

lambda:
  functions:
    -
      function: ${stack.outputs.LambdaFunction}
      zip_file: index.zip
    -
      function: ${stack.outputs.LambdaFunctionSecond}
      zip_file: index.zip
  deploy:
    phases:
      pre_deploy:
        commands:
          # Install all npm packages including dev packages.
          - npm install

          # Run the tests
          # - npm test

          # Remove all the npm packages.
          - rm -Rf node_modules

          # Only install the non-dev npm packages.  We don't want to bloat our Lambda with dev packages.
          - npm install --production

          # Zip the Lambda for upload to S3.
          - zip -r index.zip .
      post_deploy:
        commands:
          # Remove the zip file.
          - rm -Rf index.zip

          # Reinstall the dev npm packages.
          - npm install

Lambda versions and aliases are also supported. If your cloudformation template includes an alias, you can include it in your _cim.yml file like so:

lambda:
  functions:
    -
      function: ${stack.outputs.LambdaFunction}
      aliases:
        PROD: ${stack.outputs.LambdaFunctionProdAlias}
      zip_file: index.zip

Use the lambda-publish and lambda-unpublish commands to upload and delete new versions.

The lambda-deploy command can be used to deploy new versions, by updating the alias to point to the specified version.

Templates

The templates are used when creating a new package.

cim create --template=<template>
NameDescription
cloudformationBasic setup.
serverless-web-appStatic S3 website with SSL, CDN, and CI/CD.
serverless-apiAPI Gateway proxying calls to a Lambda backend. Optional custom domain.
lambda-nodeA single Lambda function.
lambda-node-s3A single Lambda function with an S3 event trigger.
lambda-node-dynamodbA single Lambda function with a DynamoDB stream event trigger.
lambda-node-kinesisA single Lambda function with a Kinesis stream event trigger.
lambda-node-snsA single Lambda function with an SNS event trigger.
lambda-node-cloudwatch-cronA single Lambda function with a scheduled CloudWatch cron event trigger.
lambda-node-cloudwatch-logsA single Lambda function with a CloudWatch Logs event trigger.
vpcVPC - Modular and scalable virtual networking foundation on the AWS Cloud.
ecrECR - AWS Docker Container Registry.
ecsECS - AWS EC2 Docker Container Service.
ecs-serviceExample ECS Service.
ecs-service-ciExample ECS Service with continuous integration.

Plugin Framework

Do you want to create additional CIM commands? Or do you want to create before and after hooks for any CIM command? Or do you just want to create a new template?

There are two ways to contribute to CIM:

  1. Add a new Plugin and create a PR.
  2. Create your own 3rd party CIM plugin. Here is an example. Install these plugins globally. CIM searches the global npm directory for packages starting with cim- or cim_.

Coming soon...