Real-time notifications with WebSockets and Redis

I’ve recently been working on a project which requires real-time notifications to be delivered in an application. A classically fun part of web development to work with because real-time communication is not the run-of-the-mill work most developers get immersed in every day. This quick post discusses the requirements I had, technologies I decided to use, why I used them, and what off-the-shelf alternatives exist.

Real-time in-app notifications

The app I am working on is a React Native mobile app, but the fact it’s built in React Native is irrelevant for this feature as it’s technologies and methodologies can be applied on any web project. I wanted to have a server send in-app notifications to the client device while the app is being used. With that said, I want to clarify what I mean by in-app notifications, as they are different from push notifications.

In-app notifications, as I refer to them at least, are notifications which are sent to the client while the user is using the app. They appear with a nice little animation which shows their notification number increment, and perhaps a subtle slide out component to show the user what their notification contains - akin to the likes of Facebook notifications. These are different from push notifications which generally get sent received by the client while the app is in the background or closed.

In-app notifications are developed using a custom process and tools, as opposed to push notifications which require the use of the APNS (Apple’s Push Notification Service) for Apple and a similar integration for Android.

WebSockets for real-time features

Web technology has been the same protocol to define how messages are sent to and from the server since the birth of the internet, and this is the HTTP protocol. As many of you may know, the HTTP protocol is stateless because each request is completely independent, meaning someone browsing a website are sending an HTTP request to send and receive new information at almost every click. This was great for decades because of how we were using the internet - viewing simple pages with little data and little interactivity, but things have changes massively over the last 5 - 10 years.

We now have rich media around the web, many more users, and users expect a certain level of UX when it comes to sites and applications being responsive and interactive. It’s not all about sending a request for a new page and waiting a few seconds for the browser to reload. We want information and feedback instantly, which doesn’t quite fit in with the HTTP protocol.

This is where WebSockets come in. WebSockets use the existing HTTP protocol to set up a connection, and that connection is then kept open in order to provide full duplex communication between client and server. As long as the socket connection is open, both client and server can send data to the one another.

This is the perfect environment for in-app notifications, and what I decided to leverage for my project.

Real time communication can be imitated without using WebSockets and instead using a technique called Long Polling, where the client periodically sends requests to the server to look for an update, say every 5 or 10 seconds. This is not the preferred solution as it can consume battery life and introduce buggy code very easily (trust me, I’ve been there).

The setup of notifications with React, Node, and Redis

My project consists of a React Native front end and a Node back end. The front end, like with typical web applications, communicates with the server via REST API requests. I introduced WebSockets so the server could sent notifications to the app at any time while the user is in the app, so here’s a quick setup of WebSockets using socket.io.

Socket.io installation and setup

Firstly, socket.io must be installed on the client and server side. For the client side, Socket.io can be installed on any web project running Javascript, so it doesn’t have to be React or React Native:

npm i socket.io-client

In my React project, I created a function to initialise the sockets:

export const loadSockets = authToken => {
  const socket = SocketIOClient('localhost/my-site');

  socket.emit('auth', authToken);
  socket.on('authResponse', initResponse => {
    const initialNotifications = initResponse.userNotifications;
    theStore.dispatch(setInitialNotifications(initialNotifications));
  });
  ...
};

This loadSockets function takes a parameter of my users auth token, which is a JSON Web Token. This is not required, but it provides a way to identify the user initiating the socket, and means that a server side check can take place to authenticate the user as with any other JWT setup. If the JWT is invalid, you can choose not initiate the socket connection on the server and send back an error.

The socket connection is initiated using the SocketIOClient function provided by socket.io. This function opens the ‘full-duplex’ socket connection between the client and server. Once the connection is open, the client can do 2 things:

• Send socket communication using the socket.emit function.
• Listen for socket communication using the socket.on function.

In my case, I emit a request for auth, which will be picked up by the server looking to receive communication from auth. These can be called anything you want, and contain any information you want. In my case for auth I’m sendign the auth token.

Going to the server side, here’s the setup and how the server responds to socket requests. First, install socket.io:

npm i socket.io

As soon as the Node server starts, it’s set to listen for incoming connections from any client:

let io = require('socket.io')(server);
io.on('connection', socket => {
  ...
  socket.on('disconnect', function() {
    console.log('Sockets disconnected.');
  });
});
app.set('socketio', io);

This few lines of code to listen for a socket connection gives us access to 2 crucial socket instances:

• io - an instance of the socket package. Once the io here has been connected, it can be used anywhere on the server to send/listen for communication. Of course, we could just send a listen right here where the socket is connected, but you’ll likely want to keep your initiation functionality perhaps in the server setup files, and be able to send a socket message elsewhere in your app, in another file for example.
• socket - an instance of this specific connection to the server. The SocketIOClient function initiated from the client is what initiates the io.on('connection') function, letting us know that the connection is open and giving us access to the specific socket instance in the callback. This socket instance is crucial, because it contains a unique socket id of the originating user’s connection. This is how we are able to send a message to a specific user who is connected to the server in real-time.

Now we have a socket channel open, the server can listen for requests from the client. Earlier I mentioned the client first emits an auth request which contains an auth token. This can be listened for on the server, but rather than littering the initial setup with listen (on) and send(emit) requests, I have delegated them to their own functions:

let io = require("socket.io")(server)
io.on("connection", (socket) => {
  initSockets(socket) // initiate the socket functionality to listen for specific requests
  socket.on("disconnect", function () {
    console.log("Sockets disconnected.")
  })
})
app.set("io", io)

The initSockets function listens for the auth request initiated by the client:

let initSockets = (socket) => {
  // listen for the init function sent by the client
  socket.on("init", (token) => {
    // check the auth token so we can only process more information if the user
    // is authenticated
    jwt.verify(token, MY_SECRET, (jwtErr, decodedJwt) => {
      if (!jwtErr) {
        // if the user is authenticated, get the user's initial notifications
        getNotifications(decidedJwt.userId).then((userNotifications) => {
          socket.emit("initResponse", {
            userNotifications,
          })
        })
      } else {
        console.log("JWT error...")
      }
    })
  })
}

The token verification part of this post isn’t explained here, but I have a few posts going into JWT authentication which go in to detail.

Once the token is verified, we can send something back to the client for an initial response. In the case of my application, this made sense to be all currently unread in-app notifications. This would mean the app doesn’t need to send another REST request when the app opens. A function, getNotifications is called with the user ID stored in the JWT and then socket.emit is called to send the notifications back to the client just connected.

It’s important here that socket object is used, and not the io object from the initial server setup. Emitting from the socket object sends back to this specific socket instance/user, but calling io.emit will instead send a message to all currently connected users.

So at this point, we have a client and server opening a full-duplex socket connection - the client sending an auth request, and the server responding on the same connection with a response of the user’s notifications. The connection is still open, and waiting for either client or server to send information. Our goal is to send live, in-app notification for when an action happens on the server, so here’s an example of what that might look like:

router.post("/user/follow", (req, res) => {
  const { userToFollow } = req.body
  const io = req.app.get("io")

  followUser(userToFollow).then((recipientId) => {
    // this is wrong! all users will receive this, but we only want to send to one user
    io.emit(
      "liveNotification",
      "Jay, you have received a friend request from Zack."
    )
  })
})

When this example function has been executed (friend followed), we want to send a notification to the recipientId. We need to access the io object in order to do this, which I added as middleware with the app.set('io', io). The io object can therefore be accessed using req.app.get('io'), and passed around through function calls.

Earlier I mentioned that the io.emit() sends a request to all users, which is not what we want to do, obviously. We want to send to a specific user who is currently connected to the server. In order to send a socket message to a specific user, we need to know their socket ID. As I also mentioned earlier, the socket ID is only available to us on the initial connection on the socket object, which in my case passed down to the initSockets function.

The issue here is that each time a user requests to go to a new page on a website, or re-opens the app, the socket ID changes. We need a way to store the user’s socket ID, and persist that ID through some sort of session-like mechanism in order for it to be readily available to us when/where ever we want to send a socket message to the client.

Enter Redis.

Redis installation and setup

Redis is an “in-memory data structure store”, allowing us to store information in simple key-value pair storage, similar to something like browser localStorage. It saves writing code and using resources on database read/write functionality, and is extremely fast at setting and getting data. Redis is the perfect solution for this situation as it can be used to store each user’s socket ID when they first open a socket connection to the server, and that socket ID can be read from the Redis system anywhere on the server. As Redis is a completely separate system, the data persists when the server is restarted or socket connection interrupted (although in the event of a socket interruption, the client will attempt to connect to the server and create a new socket ID, which can then also be saved).

Redis needs to be installed as a separate system on your local machine or remote server. To install on a Mac:

brew install redis

Once installed, you can enter:

redis-server which will start the server. This should stay running in order to use Redis, but if you do close this server instance, a file will be stored in your project which holds the data that was stored in Redis so it can be retrieved when the system is back up again.

Now the https://github.com/NodeRedis/node_redis package needs to be installed:

npm i redis

This enables Node to communicate with the running Redis server. Similar to a socket connection, the Redis connection needs to be opened manually. You want this to be done at roughly the same time the socket connection is ready, because we want to store each user’s socket ID. With that in mind, we initiated Redis:

let initRedis = () => {
  const client = redis.createClient()
  return new Promise((res, rej) => {
    client.on("connect", function () {
      res(client)
    })
    client.on("error", function (err) {
      rej(err)
    })
  })
}

And the instance of Redis (which we use to set and get data from the Redis store) can be passed back using a Promise. This initRedis function can be called when the socket connection is first made:

let initSockets = (socket) => {
  // Redis instance is passed back to the socket init function
  RedisService.initRedis()
    .then((redisClient) => {
      socket.on("init", (token) => {
        jwt.verify(token, authConfig.secret, (jwtErr, decodedJwt) => {
          if (!jwtErr) {
            // user is verified, so store their socket id in redis hashmap
            // so we can send them notifications specific to their socket
            // id later
            redisClient.hset("socketIds", decodedJwt.id, socket.id)
            // get users initial notifications and send back via sockets
            getNotifications(decidedJwt.userId).then((userNotifications) => {
              socket.emit("initResponse", {
                userNotifications,
              })
            })
          } else {
            console.log("JWT error...")
          }
        })
      })
    })
    .catch((err) => console.log("Redis error - can't connect: " + err))
}

The hset function is perfect for this situation because it allows us to store the key-value pair with a specific key - the key here being the user ID. By storing the data this way, every time a new connection is opened to the server, the Redis store will overwrite the socket ID with the very latest ID, allowing that to be accessed from anywhere in the server!

Sending the notification to the specific user

Now we have a way to send socket messages anywhere from the server, and also a way to retrieve every user’s socket ID, here’s how the message is sent:

router.post("/user/follow", (req, res) => {
  const { userToFollow } = req.body
  const io = req.app.get("io")

  followUser(userToFollow).then((recipientId) => {
    Promise.all([
      initRedis(),
      getNotificationsByUser({ userId: recipientId }),
    ]).then(([redisClient, userNotifications]) => {
      redisClient.hgetall("socketIds", (err, result) => {
        socket
          .to(result[recipientId])
          .emit(
            "liveNotifications",
            "Jay, you have received a friend request from Zack."
          )
      })
    })
  })
})

The Redis init function is called at the same time as we get the recipient’s notifications. Once they are both finished, we get the specific user’s socket ID from Redis, and then use the socket.to(result[recipientId]).emit('message');. Back on the client, the loadSockets function can be used to add another socket listed function for liveNotifications:

export const loadSockets = authToken => {
  const socket = SocketIOClient('localhost/my-site');

  socket.emit('auth', authToken);
  socket.on('authResponse', initResponse => {
    const initialNotifications = initResponse.userNotifications;
    theStore.dispatch(setInitialNotifications(initialNotifications));
  });
  // listen for live, in-app notifications sent back from the server
  socket.on('liveNotifications', notification => {
    theStore.dispatch(setInitialNotifications(notification));
  });
  ...
};

I am using Redux here to dispatch the notification to anywhere in the app, but any client side implementation can be used. It can easily be added to local React state or anything if needed!