Slack Apps with Bolt.js & Serverless

I’ve spend the past few weeks bootstrapping the Small Batches slack app. The app is simple: post a small batch of software delivery knowledge to slack every day (ala Deming Bot). I chose to use the bolt Node JS framework because my previous approach of integrating at a low level with express and relying on some of their SDKs is end-of-lifed. Plus, I wanted to get back into Serverless after some time away. I reached a way point over the weekend. The app could be installed and interacted with by someone other than myself. Now is a good time to pause and collect my thoughts on the experience.

First, the Bolt docs are so-so. The deployment sections are not thorough enough. They don’t list what URLs are generated by the framework and which ones to configure in Slack. This required me to read the code and synthesize what I needed to do from reading other docs. Plus, going to the gemba really helps: just try in a live environment and log everything. Unfortunately, that’s the best way to really know what systems send to each other.

Speaking of that, my contract assertions saved me a ton of debugging effort. I opted to use Joi on this project for runtime contract assertions.

Sidebar here: D really turned me onto this. D has input and output contracts built into the language. They’re great and I wish that were part of more languages. Dynamic languages are great because they can do anything. If you want to add this stuff, just do it in guard clauses and before a return.

I would have reached for JSON schema in the past but I figure this is an exploratory project so what the hell. Let’s try something new. Using Joi validators as contracts was easy. The error messages on failed assertions are great too. They include the entire object and highlight the validation errors. This saved me during my spikes because the docs were wrong. I had written my test according to the docs, but those were wrong. So la-de-da, I go about my business and exercise the functionality. BOOM. A big loud error in the logs: Contract validation failed! Well, hooray this is great I thought. I can actually debug what’s wrong, update me tests, and sync the contracts. That lead to uncover another bad contract I had, which in turn led me to read more code, which led to another bad contract. Typically I don’t like to do this much code reading to understand behavior, but sometimes such is life with poor docs and dynamic languages. All in all, I was happy that my system failed fast and alerted me to problem exactly where it happened. THVE capability one in action right.

Back to Bolt. The Bolt docs make it seem like “installation management” and “authorization” is handled transparently by the framework. This is only half true.

First, if you use the Lambda receiver, then you need to explicitly add installation support which uses the HTTP receiver. The docs do not mention this at all. This was revealed to me (of course while debugging something else) by this Github Issue. That issue shows how to use HTTP receiver to configure the install handlers. After that, you still need to learn which URLs to expose as lambda triggers (since the example maps the HTTP receiver to a lambda handler).

The other half of all of this is the installation and authorizat functions. The docs say you’ll need to write your own versions of these functions but don’t quite explain when they’re called, how they fit into the flow, or what data they’re called with or supposed to return. The answer to the third part is to read the code and find the type definitions. The answer to the first and second parts is go to the gembda and try it. The default implementation use memory stores which are obviously not fit for production but that’s mentioned anywhere in the docs either. It’s alluded too but never called out specifically in some sort of “production deployment” guide. The existing deployment guides are too “Hell world” focused with no information on actual production concerns. That’s an exercise to the reader.

Of course the library mentioned in the previous issue has been transitioned to a new owner and supplanted by a newer version. It’s never a good sign when you stumble upon an issue that links to a project that solves the problem, then that project has a big ole “this project has been replaced by X” warning. Great….who knows if things work now and likely all the old docs that assumed you used this think are now incorrect because the newer version has a different API than the old version. Well, turns out it wasn’t so bad. But still, the spider sense for triggers hard in these scenarios.

Bolt uses an event driven architecture, so you create an App instance then attach all your listeners to it. This architecture is easy to unit test and difficult to integration test.

It’s easy because listeners can be declared in individual files then tested in isolation. This is made significantly easier if they work with dependency injection via a context argument. You can pass whatever you need in that object. Throw some shims in middleware to inject items into the context, mock anything else, and the unit tests are hummin.

It’s difficult because at some point you need to actually trigger events on the App instance to see how everything works from passing data to the events, what the events do, and what they integrate with. This can be hairy if there are multiple listeners for a given event. Than may create race conditions and unexpected side effects in tests.

I solved this problem with Cucumber. I really like Cucumber. It’s great for forcing the developer to take a step back and consider what should happen from a consumer’s perspective. So I wrote a test receiver that forgoes some of the HTTP stuff and just allows to invoke functions on the App instance as if they were Lambda events themselves. You see, the listeners are just functions and the App is just a lambda handler. The bolt framework wires up everything so in the incoming slack event (be it actual event, slack command, view submissions, or whatever) to it’s own internal event listeners. You don’t need a HTTP server. You just need to know the event schema which you can find with appropriate digging. After writing that all up I had tests that:

1. Simulate a user install which leverages my custom DynamoDB store
2. Simulate slack interactions by invoking the App with the relevant schema
3. Assertions on internal state changes and external calls via fakes

Now the setup is humming along. I can add more event listeners and write more acceptance tests. It’s nice to see it working, I just wish things are documented more clearly.

If you made it this far, then I won’t leave you hanging without some code snippets. Here are the relevant bits of my serverless file and relevant Lambda handlers.

# serverless.yaml

functions:
  # This is the primary "app" handler. This goes to https://yourdomain.com/slack/events.
  # That URL needs to go in the events endpoint in the Slack app admin
  events:
    handler: src/lambda/slack.handler
    events:
      - http:
          path: slack/events
          method: post

  # Bolt defaults to using the slack/oauth_redirect. This is provided
  # by the wrapped serverless-express handler.
  oauth_redirect:
    handler: src/lambda/slack.oauthHandler
    events:
      - http:
          path: slack/oauth_redirect
          method: get

  # Also provided ty the serverless-express handler. Visit this site
  # and you'll see a link to install the app.
  install:
    handler: src/lambda/slack.oauthHandler
    events:
      - http:
          path: slack/install
          method: get

And the lambda handler:

const assert = require("assert");
const { AwsLambdaReceiver, ExpressReceiver } = require("@slack/bolt");
// my own internal factory to create the app and attach all the
// listeners
const { createApp } = require("../app");
// implements fetchInstallation, storeInstallation, and
// deleteInstallation
const Store = require("../store");

assert(process.env.SLACK_SIGNING_SECRET, "SLACK_SIGNING_SECRET missing");
assert(process.env.SLACK_SCOPES, "SLACK_SCOPES missing");
assert(process.env.SLACK_CLIENT_ID, "SLACK_CLIENT_ID missing");
assert(process.env.SLACK_CLIENT_SECRET, "SLACK_CLIENT_SECRET missing");

const store = Store.connect();

// OAuth Flow
const expressReceiver = new ExpressReceiver({
  clientId: process.env.SLACK_CLIENT_ID,
  clientSecret: process.env.SLACK_CLIENT_SECRET,
  scopes: process.env.SLACK_SCOPES,
  stateSecret: "my-secret",
  installationStore: store,
});

// The new version of the old referenced package
const serverlessExpress = require("@vendia/serverless-express");

// convert the HTTP based OAuth app to a Lambda handler
module.exports.oauthHandler = serverlessExpress({
  app: expressReceiver.app,
});

const receiver = new AwsLambdaReceiver({
  signingSecret: process.env.SLACK_SIGNING_SECRET,
});

const app = createApp({
  receiver,
  // Pass an authorize function to match incoming events with an
  // installation. The function should return an a token
  authorize: async (query) => {
    return store.authorize(query);
  },
  // The `processBeforeResponse` option is required for all FaaS environments.
  // It allows Bolt methods (e.g. `app.message`) to handle a Slack request
  // before the Bolt framework responds to the request (e.g. `ack()`). This is
  // important because FaaS immediately terminate handlers after the response.
  processBeforeResponse: true,
});

// Handle the Lambda function event
module.exports.handler = async (event, context, callback) => {
  const handler = await app.start();
  return handler(event, context, callback);
};

I plan to write guides for both Bolt and Serverless once I have more experience. This is my first time using Bolt. I have much more experience with serverless but that needs be certified not stale before adding it to the guides.