Home

Awesome

dynamodb-lambda-autoscale

Autoscale AWS DynamoDB using an AWS Lambda function

Disclaimer

Any reliance you place on dynamodb-lambda-autoscale is strictly at your own risk.

In no event will we be liable for any loss or damage including without limitation, indirect or consequential loss or damage, or any loss or damage whatsoever arising from loss of data or profits arising out of, or in connection with, the use of this code.

Getting started

Note: dynamodb-lambda-autoscale uses Flow extensively for static type checking, we highly recommend you use Nuclide when making modification to code / configuration. Please see the respective websites for advantages / reasons.

  1. Build and package the code
  2. Fork the repo
  3. Clone your fork
  4. Create a new file in the root folder called 'config.env.production'
  5. Put your AWS credentials into the file in the following format, only if you want to run a local test (not needed for lambda)
```javascript

AWS_ACCESS_KEY_ID="###################"
AWS_SECRET_ACCESS_KEY="###############"
```

5. Update Region.json to match the region of your DynamoDB instance 6. Run 'npm install' 7. Run 'npm run build' 8. Verify this has created a 'dist.zip' file 9. Optionally, run a local test by running 'npm run start'

Running on AWS Lambda

  1. Follow the steps in 'Running locally'

  2. Create an AWS Policy and Role

  3. Create a policy called 'DynamoDBLambdaAutoscale'

  4. Use the following content to give access to dynamoDB, cloudwatch and lambda logging

    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "dynamodb:ListTables",
        "dynamodb:DescribeTable",
        "dynamodb:UpdateTable",
        "cloudwatch:GetMetricStatistics",
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}

  5. Create a role called 'DynamoDBLambdaAutoscale'

  6. Attach the newly created policy to the role

  7. Create a AWS Lambda function

  8. Skip the pre defined functions step

  9. Set the name to 'DynamoDBLambdaAutoscale'

  10. Set the runtime to 'Node.js 4.3'

  11. Select upload a zip file and select 'dist.zip' which you created earlier

  12. Set the handler to 'index.handler'

  13. Set the Role to 'DynamoDBLambdaAutoscale'

  14. Set the Memory to the lowest value initially but test different values at a later date to see how it affects performance

  15. Set the Timeout to approximately 5 seconds (higher or lower depending on the amount of tables you have and the selected memory setting)

  16. Once the function is created, attach a 'scheduled event' event source and make it run every minute. Event Sources > Add Event Source > Event Type = Cloudwatch Events - Schedule. Set the name to 'DynamoDBLambdaAutoscale' and the schedule expression to 'rate(1 minute)'

Configuration

The default setup in the Provisioner.js allows for a quick no touch setup. A breakdown of the configuration behaviour is as follows:

Strategy Settings

The strategy settings described above uses a simple schema which applies to both Read/Write and to both the Increment/Decrement. Using the options below many different strategies can be constructed:

A sample of the strategy setting json is...

{
  "ReadCapacity": {
    "Min": 1,
    "Max": 100,
    "Increment": {
      "When": {
        "UtilisationIsAbovePercent": 75,
        "ThrottledEventsPerMinuteIsAbove": 25
      },
      "By": {
        "Units": 3,
        "ProvisionedPercent": 30,
        "ThrottledEventsWithMultiplier": 0.7
      },
      "To": {
        "ConsumedPercent": 130
      }
    },
    "Decrement": {
      "When": {
        "UtilisationIsBelowPercent": 30,
        "AfterLastIncrementMinutes": 60,
        "AfterLastDecrementMinutes": 60,
        "UnitAdjustmentGreaterThan": 5
      },
      "To": {
        "ConsumedPercent": 100
      }
    }
  },
  "WriteCapacity": {
    "Min": 1,
    "Max": 100,
    "Increment": {
      "When": {
        "UtilisationIsAbovePercent": 75,
        "ThrottledEventsPerMinuteIsAbove": 25
      },
      "By": {
        "Units": 3,
        "ProvisionedPercent": 30,
        "ThrottledEventsWithMultiplier": 0.7
      },
      "To": {
        "ConsumedPercent": 130
      }
    },
    "Decrement": {
      "When": {
        "UtilisationIsBelowPercent": 30,
        "AfterLastIncrementMinutes": 60,
        "AfterLastDecrementMinutes": 60,
        "UnitAdjustmentGreaterThan": 5
      },
      "To": {
        "ConsumedPercent": 100
      }
    }
  }
}

Advanced Configuration

This project takes a 'React' style code first approach over declarative configuration traditionally used by other autoscaling community projects. Rather than being limited to a structured configuration file or even the 'strategy' settings above you have the option to extend the ProvisionerBase.js abstract base class for yourself and programmatically implement any desired logic.

The following three functions are all that is required to complete the provisioning functionality.
As per the 'React' style, only actual updates to the ProvisionedCapacity will be sent to AWS.

getDynamoDBRegion(): string {
  // Return the AWS region as a string
}

async getTableNamesAsync(): Promise<string[]> {
  // Return the table names to apply autoscaling to as a string array promise
}

async getTableUpdateAsync(
  tableDescription: TableDescription,
  tableConsumedCapacityDescription: TableConsumedCapacityDescription):
  Promise<?UpdateTableRequest> {
  // Given an AWS DynamoDB TableDescription and AWS CloudWatch ConsumedCapacity metrics
  // return an AWS DynamoDB UpdateTable request
}

DescribeTable.ResponseSyntax UpdateTable.RequestSyntax

Flexibility is great, but implementing all the logic required for a robust autoscaling strategy isn't something everyone wants to do. Hence, the default 'Provisioner' builds upon the base class in a layered approach. The layers are as follows:

Throttled Events

Throttled events are now taken into account as part of the provisioning calculation. A multiple of the events can be added to the existing calculation so that both large spikes in usage and hot key issues are both dealt with.

Rate Limited Decrement

AWS only allows 4 table decrements in a calendar day. To account for this we have included an algorithm which segments the remaining time to midnight by the amount of decrements we have left. This logic allows us to utilise each 4 decrements as efficiently as possible. The increments on the other hand are unlimited, so the algorithm follows a unique 'sawtooth' profile, dropping the provisioned capacity all the way down to the consumed throughput rather than gradually. Please see RateLimitedDecrement.js for full implementation.

Capacity Calculation

As well as implementing the correct Provisioning logic it is also important to calculate the ConsumedCapacity for the current point in time. We have provided a default algorithm in CapacityCalculator.js which should be good enough for most purposes but it could be swapped out with perhaps an improved version. The newer version could potentially take a series of data points and plot a linear regression line through them for example.

Dependencies

This project has the following main dependencies (n.b. all third party dependencies are compiled into a single javascript file before being zipped and uploaded to lambda):

Licensing

The source code is licensed under the MIT license found in the LICENSE file in the root directory of this source tree.