Using caching to limit API requests for a lightning fast app

Hitting API’s are an everyday part of web development, but often an API will have a limit, especially if it’s a public API not in your control. This post shows my approach to help prevent constant API requests to an external API from my server using caching methods, which I used for my Cinepicks app for fetching TV and movie data.

The post will cover the general process for how my caching is handled, but not the specific details about all of the implementation.

A bit about API limits

There are many ways to limit an API, and many reasons one would want to limit an API. The idea is that a server can only handle a certain number of concurrent requests from the users which may be making requests, so a server can introduce a set of rules to limit how many times the server is hit. Also referred to as “rate limiting”, the process is essential to not only control API access from a business perspective, as many APIs will charge money for accessing an API, but it’s also critical for security reasons to help prevent attacks such as DoS.

API’s can set rate limits based on different criteria such as CPU usage, access time, but the type of limiting I will be covering is transaction limits. This is where the API will stop a user’s access once a certain amount of requests are received by the server in a given time. With the API I’m accessing, it’s no more than:

• 10,000 requests per month
• 10 requests per seconds

Why use caching?

Depending on the type of app you’re making, it could be very easy to exceed 10,000 requests per month to an external API. In my case with Cinepicks, each time a user scrolls down to reveal more movies in an infinite scroll component, 10 movies are loaded, meaning the external API would be hit 10 times for that one scroll. That means in a single month, all users of my app would only need to collectively scroll down to reveal more movies 400 times to reach that API limit, which isn’t a lot.

Caching allows me to store an API response on my server so that future requests to retrieve movie data could first hit my own data source instead of always going straight to the external API. This raises a few important questions though, such as:

• Where is the data stored on my server? There could be a lot of data which needs to be stored so it needs to be structured well and easily accessible.
• How long is it stored on my server for? This is critical for my app because movies and TV shows are added and removed to streaming services like Netflix and Primne all the time, so keeping this as up-to-date as possible and not relying on my own data source for too long is key.

The caching methods

Here’s the two methods I used to help limit my requests to the external API:

Local database caching

The first approach I used is local database caching, which involves storing data from the external API in my Postgres database. Here’s the stripped down function which handles the database caching:

async function getStreamingAvailability(titles) {
  const withAvailability = await Promise.all(
    // Map over each of the 10 titles
    titles.map(async (title: any) => {
      // Check if data already exists in local database
      const localCheck = await localCheckByImdbId(title.imdbId)

      if (localCheck.exists && localCheck.needsUpdating === false) {
        // Doesn't need updating so return local data
        return localCheck.exists
      } else {
        // Hit API
        try {
          const externalApiResponse = await fetch(
            `https://external.api?imdbId=${title.imdbId}`
          )

          // Store API response in local database for future requests
          return await upsertNetflixByImdbId(title.imdbId, externalApiResponse)
        } catch (e) {
          // If API returns an error, log in database
          return await Promise.all([
            upsertNetflixByImdbId(title.imdbId, {
              status: e || "error-unknown",
            }),
          ])
        }
      }
    })
  )

  return withAvailability
}

The user scrolls down to reveal 10 movies, and those 10 movies IMDb ID is sent to the above function. For each of the 10 titles, the function first checks if it exists in the Postgres database on my server, and if it does, the database data is returned. If the data doesn’t exist, the API is hit, and the data returned from the external API is cached in the local database for use with future requests.

The key point with this process is the localCheckByImdbId(title.imdbId) part, as this decides if the API needs to be hit. As mentioned earlier, the local data source can soon become outdated as movies are added and removed from Netflix frequently. Here’s that function:

async function localCheckByImdbId(imdbId) {
  const exists = await db.table.findOne({ where: { imdbId } })
  if (!exists) {
    // Doesn't exist in local, so proceed to fetch from API
    return {
      exists: false,
      needsUpdating: true,
    }
  }

  // Decide if the local data is outdated
  let cacheTimeoutInDays
  if (exists.status === "error-timeout" || exists.status === "error-unknown") {
    cacheTimeoutInDays = 1
  } else if (exists.status === "not-present") {
    cacheTimeoutInDays = 30
  } else if (exists.status === "updated") {
    cacheTimeoutInDays = 14
  } else {
    cacheTimeoutInDays = 7
  }

  // Caclulate the exact data in which the local data source will be outdated based on above value
  const lastUpdated = exists.updatedAt
  const lastUpdatedPlusCacheTimeout = new Date(lastUpdated)
  lastUpdatedPlusCacheTimeout.setDate(
    lastUpdatedPlusCacheTimeout.getDate() + cacheTimeoutInDays
  )
  const lastPlusThreeDaysEpoch = lastUpdatedPlusCacheTimeout.valueOf()

  const nowEpoch = new Date().valueOf()

  if (lastPlusThreeDaysEpoch > nowEpoch) {
    // Local version is still good to use
    return {
      exists,
      needsUpdating: false,
    }
  } else {
    // Local version is outdated so fetch new data from API
    return {
      exists,
      needsUpdating: true,
    }
  }
}

The above function is a collection of different methods in the real system, but I’ve merged them together to make it easier to follow the process.

First, the local database is checked to see if the title exists. If it doesn’t exist, the function is exited right away so data can be fetched. If an entry does exist, we need to decide how long the existing local data is valid for.

This all hangs on the “status” column in the database, which stores if the last title fetch was successful, and if it wasn’t, what sort of error or response was received. If the previous fetch was an error or timeout, this may be because the external API server was down for a moment, or my API requests were rate limited. In this case, we’d want to try again relatively soon as it could succeed next time. If the movie data shows that the movie is not on Netflix, it’s likely it won’t be on Netflix in the next few days, so the time to check again is set to 30 days away. And if the last external API request did update the local database, it’s worth checking again in a couple of weeks.

The cache timeout values above have been edited many times during and after development to find a good balance between keeping the data as fresh as possible without hitting the API too many times. This is why I decided to control the calculation of the timeout within the code rather than adding an expiration column in the database as I may have done usually.

Then finally, depending on whether the local data is deemed out of date or not, a suitable response is sent back to the main function, getStreamingAvailability, to handle.

Redis database caching

The local database caching method above is great for limiting the API requests, but the process still involves mapping over a number of movies and looking in what turns out to be a huge database. There are obviously millions of movies and TV shows out there, a load of which end up with their own little row in my small AWS EC2 t3.micro instance Postgres database. This is where redis caching comes in:

router.post("/get-titles-flat", async (req, res) => {
  const filterData = req.body.filterData

  const cacheKey = JSON.stringify(filterData)

  const existingEntry = await redis.getCachedData(cacheKey)

  if (existingEntry) {
    return res.send({
      success: true,
      titles: existingEntry.titles,
    })
  }

  getStreamingData(filterData).then(async ({ titles }) => {
    await redis.setCachedData({
      key: cacheKey,
      data: { titles },
      ttl: 259200,
    })

    return res.send({
      titles: titles,
    })
  })
})

The process with the Redis caching is similar to that of the local database caching mentioned earlier - local data source is checked and if it exists and is fresh enough, it will be returned to save the extra step of fetching from the other data source (in this case the Redis is checked before moving on to the local database source).

Redis differs from a local database due to the implementation methods of each source. With a Postgres database, a database query is used to retrieve a row, but with Redis, a cache key must be provided. This key needs to be completely unique to the data it’s storing, so one way to do that is to use the data keys themselves. In my situation, filterData is a concatenation of all the filters the user can select on the front end, including streaming service, IMDB rating, genre, and many more. So the cache key might be something like:

netflix:7-10:1990-2020:thriller,romance

That cache key is used to get and set the data in the Redis source:

async function getCachedData(cacheKey) {
  let cachedData = await Promise((resolve, reject) => {
    redis.get(cacheKey, (error, data) => {
      if (error) return reject(error)
      return resolve(data)
    })
  })

  if (!cachedData) return null

  return JSON.parse(cachedData)
}

Although slightly more simple than the local database approach, Redis data is a little less permanent and isn’t really something I’d use for analytics or performing manual queries on in any other setting. It’s super fast though, and works really nicely with Cinepicks because people often search using the same criteria because of how I’ve developed the front end.

For example, when first developing the app, the “release date” range slider would allow the user to set any year between 1900 and 2020. So they could set 1972 - 1999, or 1973 - 1998, all with 1 year increments. This would mean the Redis cache would only be accessed if the same if another user selected these exact dates. To help with this, I have since changed the release date range slider to 10 year increments, meaning the Redis cache is way more likely to be used by other users.

The timeout for all my Redis caching is set to 4 hours, which stops my database, and in turn the external API, from being hit with the same requests in a short time period, but also helps keep the app super fast for those similar requests.

Conclusion

The caching methods in this post have helped me stay within around 20% of my allowed rate limiting set by my external API. The local database allows me to easily see an overview of which titles are being hit, what sort of responses I’m getting back in general, and also perform analysis on all sorts of data points I’m storing along with the ones I’ve shown in this post regarding the API requests.