Awesome
<div align="center">Elixir
Invoke Lambda
Example λ
A basic example showing how to invoke AWS Lambda functions from Elixir/Phoenix Apps.
</div> <br />Why? 🤷
To keep our Elixir
/Phoenix
App
as focussed as possible,
we are delegating all
of the non-core functionality
to AWS Lambda functions.
AWS Lambda allows us
to offload specific non-core functionality
such as sending/receiving emails and
uploading/resizing/transcoding images/video.
This non-core functionality
still needs to work flawlessly
but it is not invoked directly by end-users.
Rather the Lambda functions
are called asynchronously and transparently
by Elixir
with as little overhead as possible.
If keeping your app focussed
on it's core business logic
sounds like a good idea to you, <br />
follow along with us on the
Elixir
invoke Lambda
quest! 🏔️
What? 💭
This example invokes our
aws-ses-lambda
function that handles all our email
needs.
The example is a step-by-step implementation, designed to help anyone follow along.
Who? 👤
This example is targeted at Elixir/Phoenix novices who are hoping to leverage the power of "serverless", to run specific bits of non-core functionality.
How? 👩💻
This is a complete build log for getting this working. We hope that it's useful to others.
0. Prerequisites? ✅
If you already have a bit of Elixir/Phoenix knowledge/experience and some basic JavaScript exposure, you will be able to dive straight into the example below!
Just ensure that you have
the latest
Elixir,
Phoenix
and
Postgres
installed on your localhost
before beginning.
elixir -v
Elixir 1.10.1 (compiled with Erlang/OTP 22)
mix phx.new -v
v1.4.13
psql --version
psql (PostgreSQL) 12.1
If you are new to (or rusty on) Elixir/Phoenix, we recommend reading dwyl/learn-elixir <br /> and following the dwyl/phoenix-chat-example which is a "my first phoenix app".
You don't need to have any knowledge of AWS Lambda, just treat it as any other function call. <br /> If you are curious to learn more about Lambda, read our beginner's guide: dwyl/learn-aws-lambda
Ensure you have aws-ses-lambda
running!
This example invokes our
aws-ses-lambda
,
which as it's name suggests is a AWS Lambda function
that handles sending email
using AWS Simple Email Service (SES).
You need to deploy the Lambda function
and test it in the AWS console
ensuring that it's working before
attempting to invoke it from Elixir
.
The setup and deployment instructions
are all included in
How? section.
This is what success looks like in AWS Lambda console:
Our Lambda function responds with the following JSON
:
{
"ResponseMetadata": {
"RequestId": "f43c4f3d-1d9b-4646-bb27-8c3a8a7ad674"
},
"MessageId": "010201703f49f928-6860c2f3-5b6d-474a-be93-3faecefb1b3a-000000"
}
With the Lambda working, let's get back to our quest!
1. Create a Phoenix Project 🆕
In your terminal, create a new Phoenix app using the command:
mix phx.new app
Ensure you install all the dependencies:
mix deps.get
cd assets && npm install && cd ..
Setup the database:
mix ecto.setup
Start the Phoenix server:
mix phx.server
Now you can visit
localhost:4000
from your web browser.
Also make sure you run the tests to ensure everything works as expected:
mix test
You should see:
Compiling 16 files (.ex)
Generated app app
17:49:40.111 [info] Already up
...
Finished in 0.04 seconds
3 tests, 0 failures
Having established that your Phoenix App works as expected, let's dive into the fun part!
2. Add ex_aws_lambda
to deps
🎁
We are using
ex_aws_lambda
which depends on
ex_aws_lambda
, <br />
which in turn requires an HTTP library
hackney
and JSON library
poison
.
Add the following lines to the deps
list
in the mix.exs
file:
{:ex_aws, "~> 2.1.0"},
{:ex_aws_lambda, "~> 2.0"},
{:hackney, "~> 1.9"},
{:poison, "~> 3.0"},
e.g: mix.exs#L47-L52
Then run:
mix deps.get
3. Environment Variables 🔐
In order to invoke a AWS Lambda function
(and specifically our aws-ses-lambda
), <br />
we need three Environment Variables to be defined.
To speed this up, we created an
.env_sample
file that has all the Environment Variables you need:
export AWS_REGION=eu-west-1
export AWS_ACCESS_KEY_ID=YOURACCESSKEYID
export AWS_SECRET_ACCESS_KEY=SUPERSECRETACCESSKEY
Copy this file into a new file called .env
.
e.g:
cp .env_sample .env && echo ".env\n" > .gitignore
Then update the values to your real ones!
Note: we added a
RECIPIENT_EMAIL_ADDRESS
environment variable to store the email address of the person we are sending our test email to, just so that we don't hard code our personal email address into code on GitHub. 💭
Finally run source .env
in your terminal
to load the environment variables. <br />
Confirm that the environment variables are loaded by
running the printenv
command.
💡 Tip: If you are new to Environment Variables, see: https://github.com/dwyl/learn-environment-variables
4. Write a Test! 😮
Yes, even in these simple examples, we can still follow Test Driven Development (TDD), in fact it's a really good idea to always write tests! This way you know the Lambda invocation works exactly the way you expect it to!
Create a new file called
test/app_web/controllers/invoke_lambda_test.exs
In that test file type (or, let's be honest, copy-paste) the following code:
defmodule AppWeb.InvokeLambdaControllerTest do
use ExUnit.Case
test "Invoke the aws-ses-lambda-v1 Lambda Function!" do
payload = %{
name: "Elixir Lover",
email: System.get_env("RECIPIENT_EMAIL_ADDRESS"),
template: "welcome",
id: "1"
}
{:ok, response} = AppWeb.InvokeLambdaController.invoke(payload)
# IO.inspect(response, label: "response")
message_id = Map.get(response, "message_id")
assert String.length(message_id) == 60 end
end
We know from reading the ex_aws
tests
and from running our lambda function
that the Lambda SES response Map
has the following format:
{:ok, %{
"email" => "testy.mctestface@gmail.com",
"id" => 42,
"message_id" => "0102017103df5cb1-27a0d3b3-bf06-42f9-924b-51df72e096da",
"name" => "Elixir Lover",
"request_id" => "28375a56-edf1-40a1-b2ee-cf42631391c2",
"status" => "Sent",
"template" => "welcome"
}}
So that's what we are expecting in the test above.
4.1 Run the Test and Watch it Fail! 🔴
Now that we have written our test for the invoke
function,
we can run the test an watch it fail:
mix test test/app_web/controllers/invoke_lambda_test.exs
You should see output similar to the following:
Compiling 1 file (.ex)
15:51:10.166 [info] Already up
warning: AppWeb.InvokeLambdaController.invoke/1 is undefined (module AppWeb.InvokeLambdaController is not available or is yet to be defined)
test/app_web/controllers/invoke_lambda_test.exs:19: AppWeb.InvokeLambdaControllerTest."test Invoke the aws-ses-lambda-v1 Lambda Function!"/1
1) test Invoke the aws-ses-lambda-v1 Lambda Function! (AppWeb.InvokeLambdaControllerTest)
test/app_web/controllers/invoke_lambda_test.exs:4
** (UndefinedFunctionError) function AppWeb.InvokeLambdaController.invoke/1 is undefined (module AppWeb.InvokeLambdaController is not available)
code: {:ok, %{"MessageId" => mid}} = AppWeb.InvokeLambdaController.invoke(payload)
stacktrace:
AppWeb.InvokeLambdaController.invoke(%{email: "nelson+elixir.invoke@dwyl.com", name: "Elixir Lover", template: "welcome"})
test/app_web/controllers/invoke_lambda_test.exs:19: (test)
Finished in 0.04 seconds
1 test, 1 failure
This is just telling us that the
AppWeb.InvokeLambdaController.invoke
function does not exist. <br />
This is not "news" as we have not yet created it!
But it's good to know that the test runs. <br/>
We feel satisfied that we've completed the "Red" stage of the TDD
"Red, Green, Refactor"
cycle. 🔴
1 test, 1 failure
<!--
> **Side note**:
I did't get much out of reading the
[Docs](https://hexdocs.pm/ex_aws/ExAws.html)
for [`ex_aws`](https://github.com/ex-aws/ex_aws) <br />
so I ended up reading the _tests_
in order to undestand how the package works:
[/test/ex_aws/auth_test.exs](https://github.com/ex-aws/ex_aws/blob/ecd51b1965909119ee597d6c0783334e30e59e58/test/ex_aws/auth_test.exs) <br />
Don't bother reading the tests for
[`ex_aws_lambda`](https://github.com/ex-aws/ex_aws_lambda)
they are
["incomplete"](https://github.com/dwyl/aws-ses-lambda/issues/8#issuecomment-585360225)
... 😞 <br />
Moral of the story:
**_always_ write good tests** for your code.
Other people will read them
and ~~_totally_ judge you as a developer~~
learn how you implement things. 😜
-->
5. Write the invoke
Function to Make the Test Pass! ✅
Create a new file called
lib/app_web/controllers/invoke_lambda_controller.ex
And add the following code to the file:
defmodule AppWeb.InvokeLambdaController do
@doc """
`invoke/1` uses ExAws.Lambda.invoke to invoke our aws-ses-lambda-v1 function.
"""
def invoke(payload) do
ExAws.Lambda.invoke("aws-ses-lambda-v1", payload, "no_context")
|> ExAws.request(region: System.get_env("AWS_REGION"))
end
end
Re-run the test:
mix test test/app_web/controllers/invoke_lambda_test.exs
You should see the following output indicating success:
Compiling 1 file (.ex)
Generated app app
16:36:14.994 [info] Already up
MessageId: "010201703f687a8b-331c3cf8-853e-4bac-850f-51ab5b2a7474-000000"
.
Finished in 1.6 seconds
1 test, 0 failures
The test passes using the
success@simulator.amazonses.com
email address. <br />
Next let's try sending an email to a real email address!
5.1 Invoke in iex
✉️
In your terminal, open iex
:
iex -S mix
Paste the following payload
variable:
payload = %{
name: "Elixir Lover",
email: System.get_env("RECIPIENT_EMAIL_ADDRESS"),
template: "welcome",
id: 42
}
Make sure you have the RECIPIENT_EMAIL_ADDRESS
environment variable
defined from step 2 above.
Then invoke the function:
AppWeb.InvokeLambdaController.invoke(payload)
Sample output from iex
:
iex(1)> payload = %{
...(1)> name: "Elixir Lover",
...(1)> email: System.get_env("RECIPIENT_EMAIL"),
...(1)> template: "welcome",
...(1)> id: 42
...(1)> }
%{
email: "nelson+elixir.invoke@gmail.com",
name: "Elixir Lover",
template: "welcome",
id: 42
}
iex(2)> AppWeb.InvokeLambdaController.invoke(payload)
{:ok, %{
"email" => "testy.mctestface@gmail.com",
"id" => 42,
"message_id" => "0102017103df5cb1-27a0d3b3-bf06-42f9-924b-51df72e096da",
"name" => "Elixir Lover",
"request_id" => "28375a56-edf1-40a1-b2ee-cf42631391c2",
"status" => "Sent",
"template" => "welcome"
}}
Check your email inbox, you should expect to see something like this:
<br />Congratulations! You just invoked an AWS Lambda Function from Elixir
! 🎉
<br /><br />
7. Conclusion!
If you distil the code required to invoke an AWS Lambda function from Elixir, there are fewer than 10 lines.
4 lines added to mix.exs
:
{:ex_aws, "~> 2.1.0"},
{:ex_aws_lambda, "~> 2.0"},
{:hackney, "~> 1.9"},
{:poison, "~> 3.0"},
3 environment variables added to .env
:
export AWS_REGION=eu-west-1
export AWS_ACCESS_KEY_ID=YOURACCESSKEYID
export AWS_SECRET_ACCESS_KEY=SUPERSECRETACCESSKEY
If you already had these environment variables on in your Production environment for any other reason, it's less to add!
2 lines of Elixir
code
to invoke the function
from anywhere in your Phoenix
app:
ExAws.Lambda.invoke("aws-ses-lambda-v1", payload, "no_context")
|> ExAws.request(region: System.get_env("AWS_REGION"))
Where the payload
is whatever Map
of data
your Lambda expects to receive. <br />
Or nothing at all if the Lambda function takes no input.
We believe this is a very viable way to offload specific bits of functionality to AWS Lambda from our Elixir/Phoenix apps! 🚀
<br />Thanks for learning with us! If you enjoyed this quest, please ⭐️ the GitHub repo to show your delight!
<br /><br />
Continuous Integration
This wouldn't be a dwyl example without independent verification that it works from our friends at Travis-CI! 😉
If you're new to Travis-CI or Continuous Integration, see: https://github.com/dwyl/learn-travis
The only thing special about running at CI test that invokes a Lambda function that sends an email, is that we want to use the AWS SES mailbox simulator instead of sending lots of email to a real address. see: https://docs.aws.amazon.com/ses/latest/DeveloperGuide/mailbox-simulator.html
Set the RECIPIENT_EMAIL_ADDRESS
to "success@simulator.amazonses.com"
e.g:
.travis.yml#L20
<br /><br />
Trouble Shooting 🤷
If you forget to include some data
you will get a friendly error message. <br />
e.g: In this case I didn't have
the RECIPIENT_EMAIL_ADDRESS
environment variable defined <br />
so there was no "To" (email address) defined in the event
:
{:ok,
%{
"errorMessage" => "Missing required header 'To'.",
"errorType" => "InvalidParameterValue",
"trace" => ["InvalidParameterValue: Missing required header 'To'.",
" at Request.extractError (/var/task/node_modules/aws-sdk/lib/protocol/query.js:50:29)",
" at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:106:20)",
" at Request.emit (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:78:10)",
" at Request.emit (/var/task/node_modules/aws-sdk/lib/request.js:683:14)",
" at Request.transition (/var/task/node_modules/aws-sdk/lib/request.js:22:10)",
" at AcceptorStateMachine.runTo (/var/task/node_modules/aws-sdk/lib/state_machine.js:14:12)",
" at /var/task/node_modules/aws-sdk/lib/state_machine.js:26:10",
" at Request.<anonymous> (/var/task/node_modules/aws-sdk/lib/request.js:38:9)",
" at Request.<anonymous> (/var/task/node_modules/aws-sdk/lib/request.js:685:12)",
" at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:116:18)"]
}}
Note: Obviously we don't like the fact that the
ex_aws
package returned an <br />{:ok, %{"errorMessage" => "Missing required header 'To'."}
... an:error
should not be ":ok
" ... 🙄 <br /> but let's not get hung up on it. Theex_aws
package works! 👍
When we did correctly set
the RECIPIENT_EMAIL_ADDRESS
environment variable, <br />
we got the following success message confirming the email was sent:
{:ok, %{
"email" => "testy.mctestface@gmail.com",
"id" => 42,
"message_id" => "0102017103df5cb1-27a0d3b3-bf06-42f9-924b-51df72e096da",
"name" => "Elixir Lover",
"request_id" => "28375a56-edf1-40a1-b2ee-cf42631391c2",
"status" => "Sent",
"template" => "welcome"
}}
<br /> <br />
TODO:
open an issue on https://github.com/ex-aws/ex_aws_lambda/issues
sharing a link to this repo for anyone considering using the package!