Testing on Node.js with Jest and a Sequelize test database

Using a testing framework like Jest or Mocha provides a consistent way to test a web system, but there are so many ways to approach testing that the nuances make it difficult to get that consistent process ready to use. One aspect of testing which can be handled in different ways is the data store, and whether to use mock data or databases. I have done this a few different ways in the past, such as abstracting the database away as a dependency so a “fake” database can be used, or using a real development database.

At work we use a real database but a separate test one, providing a clean datastore which simplifies the testing process. Whilst integrating this setup into a project recently I ran into some issues running this test database with Sequelize, and performing tests in Jest, so I wanted to document the approach here.

My project is using Typescript but there’s not much extra that I’ve had to implement because of Typescript, so I’ll include some TS bits as well as normal JS.

Prerequisites

This post assumes prior experience with Node, Postgres, Sequelize and Jest.

Installing dependencies

If you haven’t done so already, install the dependencies required:

npm i sequelize sequelize-cli jest jest-cli faker supertest ts-jest

The ts-jest is only required if you’re running Typescript, and enables Jest to parse .ts files.

Environment setup

As mentioned above, this testing approach requires a specific database for testing. Before each test is initiated, the database will be wiped and migrations done. That means it’s ideal to have a separate database for both dev and testing. The easiest way to do this is with Docker Compose.

If you are using Docker Compose, you’ll want something like the following for setting up the dev and test databases:

version: "3"
services:
    ...
    api:
        build: ./api
        command: sh -c "npm run db-migrate && npm run run-dev"
        container_name: pc-api-dev
        environment:
            DB_USER: user
            DB_PASSWORD: password
            DB_NAME: next-ts-jwt-boilerplate
            DB_HOST: db
            DB_PORT: 5432
            DB_TEST_HOST: db-test
        ports:
            - 3001:3001
        volumes:
            - ./api:/home/app/api
            - /home/app/api/node_modules
        working_dir: /home/app/api
        restart: on-failure
        depends_on:
            - db
            - db-test
    db:
        image: postgres
        environment:
            - POSTGRES_USER=user
            - POSTGRES_DB=next-ts-jwt-boilerplate
            - POSTGRES_PASSWORD=password
            - POSTGRES_HOST_AUTH_METHOD=trust
        volumes:
            - ./db/data/postgres:/var/lib/postgresql/data
        ports:
            - 5432:5432
    db-test:
        image: postgres
        environment:
            - POSTGRES_USER=user
            - POSTGRES_DB=next-ts-jwt-boilerplate
            - POSTGRES_PASSWORD=password
            - POSTGRES_HOST_AUTH_METHOD=trust
        volumes:
            - ./db/data-test/postgres:/var/lib/postgresql/data
        ports:
            - 5430:5432

The ports in each database service definition will be the ports to enter into your database viewer to see the databases within the Docker containers from your host machine using localhost.

If you aren’t using Docker Compose, you’ll need to set up 2 databases manually on your machine.

Either way, take note of the connection env details as they will be needed next.

Sequelize setup

This post assumes you have Sequelize set up already, so I’ll give a brief overview of how I’ve integrated it. The database config and other supporting files can be found here for more detail, but the Sequelize config file will look something like this:

// src/db/models/index

import { config } from "dotenv"
config({ path: "./../.env" })

module.exports = {
  development: {
    database: process.env.DB_NAME,
    username: process.env.DB_USER,
    password: process.env.DB_PASSWORD,
    host: process.env.DB_HOST,
    dialect: "postgres",
    logging: false,
  },
  test: {
    database: process.env.DB_NAME,
    username: process.env.DB_USER,
    password: process.env.DB_PASSWORD,
    host: process.env.DB_TEST_HOST,
    dialect: "postgres",
    logging: false,
  },
  production: {},
}

The test credentials there will need to match the test database you have on your system now.

The end goal is to have the test database emptied and migrations done before the tests are ran. Sequelize allows database migrations and seeds using the sequelize-cli package (sequelize db:migrate), which is great for running migrations on dev and production, but there’s no dedicated CLI command for emptying the database first. Emptying the database is possible on the CLI but it looks very messy and is too complicated for what it’s worth.

Instead, Sequelize offers a handy sync() function which empties the database and runs all migrations, but this can only be executed from a file.

This is where Jest config comes in…

Jest config

First, you’ll want to make sure you’ve followed the official Jest advice and split out the app and server functionality into separate files. This is because tests are more efficient when ran using the bare bones of the app rather than spinning up a whole server.

// app.ts

// Get dependencies
import express from 'express';
import db from './db/models';

...

// Route handlers
const authApi = require('./v1/auth');

// Create server
const app: express.Application = express();

// API routes
app.use('/v1/auth', authApi);

export { app };

I’ve cut out a lot of the supplementary includes above, but the basic structure is important.

// server.ts

import { app } from "./app"

const server = app.listen(app.get("port"), () => {})

And the app is included in to the server and used to create the server for dev and production.

So back to the testing, and the most important part which is the test file:

// auth.test.ts

const { app } = require("../../app")
import db from "../../db/models"

import * as faker from "faker"
import supertest from "supertest"

import { Authentication } from "../../services/Authentication"

describe("test the JWT authorization middleware", () => {
  // Set the db object to a variable which can be accessed throughout the whole test file
  let thisDb: any = db

  // Before any tests run, clear the DB and run migrations with Sequelize sync()
  beforeAll(async () => {
    await thisDb.sequelize.sync({ force: true })
  })

  it("should succeed when accessing an authed route with a valid JWT", async () => {
    const authentication = new Authentication()
    const randomString = faker.random.alphaNumeric(10)
    const email = `user-${randomString}@email.com`
    const password = `password`

    await authentication.createUser({ email, password })

    const { authToken } = await authentication.loginUser({
      email,
      password,
    })

    // App is used with supertest to simulate server request
    const response = await supertest(app)
      .post("/v1/auth/protected")
      .expect(200)
      .set("authorization", `bearer ${authToken}`)

    expect(response.body).toMatchObject({
      success: true,
    })
  })

  it("should fail when accessing an authed route with an invalid JWT", async () => {
    const invalidJwt = "OhMyToken"

    const response = await supertest(app)
      .post("/v1/auth/protected")
      .expect(400)
      .set("authorization", `bearer ${invalidJwt}`)

    expect(response.body).toMatchObject({
      success: false,
      message: "Invalid token.",
    })
  })

  // After all tersts have finished, close the DB connection
  afterAll(async () => {
    await thisDb.sequelize.close()
  })
})

This solves a couple of problems - firstly, the app object is included in each test file so supertest can make a server request. Our tests look for specific response codes and messages, so testing like this is crucial.

The Sequelize sync() function mentioned earlier is executed at the start of the test, before any tests have ran. This is required because a clean and migrated database is needed to get the full benefit of the testing. It stops us from accidentally overwriting or removing information from another test, and means our tests can be ran with precision.

The thisDb variable (instance of the clean, migrated database) is then used after all of the tests have finished to close the connection. This stops that annoying message when an async action has not finished correctly:

Running tests sequentially

With the approach above, it’s not possible to run tests in parallel because the database instance is the same one being imported in to each test file. Trying to run all tests in parallel (Jest default) means the database will be closing while the next test is being executed. To get around this, the package.json file can be updated to run the following Jest script:

  "scripts": {
    ...
    "test": "NODE_ENV=test jest --runInBand",
  }

The --runInBand means the tests will be executed sequentially, so the database is cleaned, migrated and closed in between each test suite. This makes the test time a little slower, but not by much, and I think it’s worth doing so you are able to use a simple and usable testing approach with a dedicated test database.

Improvements

In a larger or production app, testing time may be an important factor. If so, the above implementation might not suit the needs. Making the tests run faster will demand parallel testing.

The reason this project doesn’t run parallel testing is because the Sequelize database is emptied and migrated at the start of each test. An alternative solution would be to empty and migrate the database once at the start of the whole test, and close that same one instance at the end of each test. This solution would mean feeding in the same database instance to all tests in between though, which is not hard to do, but it’s a different setup to what my project is running.

The above can be done using Jest global setup and global teardown.