Messing with Swift AWS Lambda Runtime (Part 1)

This article has been updated since the release of v0.2.0 of swift-aws-lambda-runtime

Less than two weeks ago Apple released the Swift AWS Lambda Runtime. The runtime allows us to develop Swift based serverless functions to run on the Amazon Web Services (AWS) Lambda platform. Given my time spent on AWS SDK Swift this has piqued my curiosity. So I spent the last weekend trying to implement my first Swift Lambdas. What would I develop though? I decided I would try to replace a couple of the Node.js Lambdas that I use in managing my AWS setup. For this article I will discuss my Simple Notification Service Slack publisher. I'll leave the other Lambda for a later article.

SNS to Slack

For those who don't know the Simple Notification Service (SNS) is an AWS web service that manages the sending of messages to subscribing clients. In SNS you create a topic. Services can publish messages to the topic and then clients who are subscribed to the topic will have these messages passed onto them. Clients can include email, web servers, Amazon SQS queues or AWS Lambda functions. For example messages can be sent when someone puts an object in an S3 bucket, when a server shuts down or when an email is received. Pretty much every AWS service can send notifications of events to SNS topics.

I wrote a small Node.js Lambda which took these messages and formatted them a bit and sent them to a Slack webhook, which would then post the message in a Slack channel. The Lambda involved about 50 lines of js code and was fairly simple. This should be a good start in my Swift AWS Lambda runtime investigations.

During this article I'm not going to give you much detail about setting up the Slack side as this isn't what it is about. There is more details in the README.md from the Git Hub repository containing the sns-to-slack code.

Kicking off

First you need to setup a project. Fabian Fett details this in his great blob post Getting started with Swift on AWS Lambda. I'm going to assume you have read that already and can already create a basic Swift Lambda project. The only exception to Fabian's setup is we need to add an additional dependency. We are going to be posting to a webhook so we need an http client. For this example I'm using the swift-server AsyncHTTPClient. Add the package in the dependencies section and the add the product in the dependencies section of the target.

...
dependencies: [
    .package(url: "https://github.com/swift-server/swift-aws-lambda-runtime", from: "0.1.0"),
    .package(url: "https://github.com/swift-server/async-http-client", from: "1.0.0")
],
targets: [
    .target(name: "SNSToSlack", dependencies: [
        .product(name: "AWSLambdaRuntime", package: "swift-aws-lambda-runtime"),
        .product(name: "AWSLambdaEvents", package: "swift-aws-lambda-runtime"),
        .product(name: "AsyncHTTPClient", package: "async-http-client"),
    ])
]

You will also notice I have included the product AWSLambdaEvents in the list of dependencies. This library contains a list of input structures for when a Lambda has been triggered by another AWS system. We are triggering our Lambda from SNS so will need the event SNS.Event.

Attempt #1

So far the only code examples I had seen were structured as follows. They used a Lambda.run function which you passed a closure which accepted a context object, your input object and a callback to call when your Lambda was done. Given that my input is the SNS.Event object and I didn't need to return anything I started like this.

import AWSLambdaEvents
import AWSLambdaRuntime
import AsyncHTTPClient

Lambda.run { (context, input: SNS.Event, callback) in
    let httpClient = HTTPClient(eventLoopGroupProvider: .shared(context.eventLoop))
    let output = try FormatMessage(input)
    let request = try HTTPClient.Request(
        url: slackHookUrl,
        method: .POST,
        headers: ["Content-Type": "application/json"],
        body: .data(output))
    let futureResult = httpClient.execute(request: request, deadline: .now() + .seconds(10)).map { _ in }
    futureResult.whenComplete { result in
        callback(result)
    }
}

Format the event message from the SNS.Event, send the message to the Slack Hook URL and when that completes call the callback. For brevity I haven't included all the code. You can find all the code to my SNS to Slack Lambda here.

Anyway I thought job done, that was easy. I ran the code using the local test server and it immediately complained about me not calling syncShutdown on the the http client. My code was creating a HTTPClient, executing a request and then immediately deleting the HTTPClient because it went out of scope. No worries I can fix this. I add a defer block to call httpClient.syncShutdown when we exit the function and I add a wait on the futureResult to ensure the http request has finished before leaving the function.

let httpClient = HTTPClient(eventLoopGroupProvider: .shared(context.eventLoop))
defer {
    try? httpClient.syncShutdown()
}
...

...
futureResult.whenComplete { result in
    callback(result)
}
_ = try? futureResult.wait()

While this code will work something about it doesn't feel right though. I shouldn't be required to add a wait() into my code. It made me think back to one of the Swift Server working group (SSWG) requirements for their packages "Packages should be non-blocking (w/ async API) unless not possible (blocking C libs, etc)". Could a swift-server group package be forcing me into writing blocking code?

Local test server

As an aside before I continue, the local test server is a godsend. Uploading the Lambda to AWS every time I needed to test it would have slowed down development ten fold. In Xcode add an environment variable LOCAL_LAMBDA_SERVER_ENABLED set to true and you can run the Lambda locally and communicate with it via curl. I setup a file sample-event.json which contained json for an SNS.Event and called the following to test the Lambda. You should really add in a "Content-Type" header but it never complained if I didn't include it.

curl -X POST -d @sample-event.json localhost:7000/invoke

Attempt #2

Is there a way I can write asynchronous code using Swift AWS Lambda runtime? Of coures there is. Swift AWS Lambda runtime provides a EventLoopLambdaHandler protocol which can be used to manage your Lambda code. It provides opportunity for more complex code and object lifecycle management. You create a handler class that conforms to EventLoopLambdaHandler. Conforming requires you include a handle function which is called when the Lambda is invoked. This handle function is called with the Lambda context and the input object and is expected to return an EventLoopFuture which will be fulfilled with the output object. The input and output objects are defined by the In and Out typealiases.

struct SNSToSlackHandler : EventLoopLambdaHandler {
    typealias In = SNS.Event
    typealias Out = Void

    func handle(context: Lambda.Context, payload: SNS.Event) -> EventLoopFuture<Void> {
    }
}

Now I have a struct, I can manage the life cycle of my HTTPClient outside of the handle function so I don't need the call to wait. EventLoopLambdaHandler conforms to the protocol ByteBufferLambdaHandler which has a shutdown method. I can place my HTTPClient shutdown code in there.

struct SNSToSlackHandler : EventLoopLambdaHandler {
       var httpClient: HTTPClient

       init(eventLoop: EventLoop) {
           self.httpClient = HTTPClient(eventLoopGroupProvider: .shared(eventLoop))
       }

       func shutdown(context: Lambda.ShutdownContext) -> EventLoopFuture<Void> {
           try? self.httpClient.syncShutdown()
           return context.eventLoop.makeSucceededFuture(())
       }
    }

In theory I should not be able to call httpClient.syncShutdown in this function because I could be shutting down an EventLoopGroup while running on an EventLoop which causes issues. The non-sync version of the shutdown is not available at the moment (we have to wait for the next release of AsyncHTTPClient). In this situation though because I am using the EventLoop from the Lambda there is no EventLoopGroup shutdown and everything is ok.

To invoke this new style of Lambda change your Lambda.run call to the following, where your closure returns an instance of the SNSToSlackHandler class.

Lambda.run { eventLoop in
    return SNSToSlackHandler(eventLoop: eventLoop)
}

Finally fill out your handle function to actually do the work. The handle should return an EventLoopFuture that'll be fulfilled once the HTTP post has completed.

func handle(context: Lambda.Context, payload: SNS.Event) -> EventLoopFuture<Void> {
    do {
        let output = try FormatMessage(payload)
        let request = try HTTPClient.Request(
            url: slackHookUrl,
            method: .POST,
            headers: ["Content-Type": "application/json"],
            body: .data(output))
        return httpClient.execute(request: request, deadline: .now() + .seconds(10)).map { _ in }
    } catch {
        return context.eventLoop.makeFailedFuture(error)
    }
}

Run your Lambda, run the curl command from above and you should see a message posted to your Slack channel.

An added advantage of using EventLoopLambdaHandler is you can move all your object creation into the init function of the handler class. The handler is created on a "cold start" of the Lambda, but if you are invoking a Lambda that is already running, a "warm start", it will reuse the already existing handler. So I only create the HTTPClient once instead of everytime I invoke the Lambda.

Deploying your Lambda

Before you continue you need to be sure you have Docker Desktop and the aws command line interface installed. You can install Docker Desktop from here and you can install the aws cli via Homebrew with the following.

brew install awscli

Building and deploying your Lambda involves a number of steps that include various calls to docker and the aws cli. Very quickly I became bored of looking up the correct syntax for all the commands to build and deploy, so looked to write a number of scripts to simplify this process. I imagine this is going to be one of the bigger obstacles for encouraging people to develop Swift Lambdas. You can break down the process into four stages.

Build Docker image for building Lambda with amazonlinux2 Docker image
Compile Lambda code
Package Lambda code along with the libraries it needs to run
Deploy Lambda to AWS. I use the aws cli to deploy Lambdas, but you can deploy them through the AWS web console or using the AWS Serverless Application Model (SAM) cli.

I wrote/stole (mainly from the swift-aws-lambda-runtime repository) scripts to cover each of these stages. I then wrote one script install.sh to run all the stages in one go. I now run the install.sh script and have a working deployed Lambda.

Once the Lambda is deployed you can go to the AWS dashboard, select SNS, create an SNS Topic and press the "Create subscription" button. Select the protocol "AWS Lambda" and then select your Lambda from the "Endpoint" dropdown. You can test it works by pressing the "Publish message" button from the topic page. Now any service that sends messages to this topic will have these output in your Slack channel.

Final Code

You can find the final version of the SNS to Slack Lambda here. On top of what I detailed above I have added some error handling, the Slack URL is acquired via an environment variable and I support outputting multiple messages from one SNS.Event.

Finally Tom Doron, Fabian Fett and everyone who worked on Swift AWS Lambda Runtime have done a great job. It has certainly made the development of Swift based Lambdas a more positive experience. Prior to this it was quite a soul destroying process. There are still obstacles but I'm sure these will be improved upon as more people take time to develop their own Lambdas. I hope you found this article of interest. If so, please provide feedback on Twitter where you can find me @o_aberration, or on the Vapor Discord where I am @adamfowlerphoto.

#aws#lambda#slack#swift

Messing with Swift AWS Lambda Runtime (Part 2)

OpticalAberration