A concise collection of tips for MongoDB's Mongoose

Creating a new app project on mobile or web is an exciting time while you’re thinking of the architecture and data structures. It’s a chance to build something different and explore new ideas and techniques, but some parts of an application are mostly always the same - database querying. This post is an accumulation of a few years of “gottchas” and snippets to form a kind of cheat sheat for MongoDB’s ORM, Mongoose.

Updated 11th September 2018. Thanks to Peter Cooper for sending in a few typo corrections.

MongoDB vs Mongoose

MongoDB is a noSQL database which uses Object Document Mapping (ODM) to enable flexible data structures which don’t adhere to a strict Schema. This is useful for unpredictable data sets where the attributes of data can change any time.

Say for example you have a database of cars which stores “colour”, “engine size” and “top speed”. Later, you find that you need to add electric cars which require another attribute for “charge time”, and after that you introduce flying cars which require a “fuel type” attribute. MongoDB will be great for this situation because you can add any data to the structure.

Mongoose is an abstraction layer which sits on top of MongoDB and actually requires MongoDB as a dependency. While still being a noSQL database, Mongoose uses Object Relational Mapping (ORM) for it’s data modelling which is one of it’s defining features. This means the data conforms to a Schema which outlines the data attributes, types, default values etc which can be a huge help for most projects.

Aside from the fundemental data modelling mentioned above (which I believe are the biggest/most important differences) MongoDB is reported to be faster than Mongoose for larger scale projects, but Mongoose is thought to make development times generally faster.

One of the reasons for writing this post is that when I do use MongoDB, I will mostly always go with Mongoose for an ORM, but I find the documentation saturated and fragmented to find how to do the simplest tasks which almost all apps will do.

Example Schema

Here’s an example Schema which Mongoose will use to define the data. When something is saved, updated, deleted etc. it will need to conform to this data structure. It can also be changed over time if more data needs to be added.

The following structure outlines basicuser information, as well as an embedded array of objects. Remember - nosql databases are flat lists:

var mongoose = require("mongoose")
var Schema = mongoose.Schema

var usersSchema = new Schema({
  ID: Number,
  firstName: String,
  lastName: String,
  favouriteFlix: [{ showName: String, rating: Int }],
})

var Users = mongoose.model("User", usersSchema)
module.exports = Users

Creating a Schema looks for plural of collection name

One of the gotcha’s when defining and using a Schema is that the model will automatically look for the plural of the collection name. For example:

var Users = mongoose.model('User', usersSchema);

The above will look for the plural of collection (first argument) which in this case would be Users. If you’ve already created the Users collection and use that as the first argument, the call will not find the collection:

// this will not work
var Users = mongoose.model("Users", usersSchema)

You can, however, explicitly define the collection name as a third parameter:

// this will work
var Users = mongoose.model("Users", usersSchema, "Users")

Saving created and updated time by using schema:

A common pattern is to save the current time/date of when something is created, and then when it’s updated thereafter. This can be done in the Schema instead of littering it throughout different queries:

var userSchema = new Schema({
	ID: Number,
	...
	created_at: { type: Date, default: Date.now },
	updated_at: { type: Date, default: Date.now }
});

userSchema.pre('save', next => {
	let now = new Date();
	this.updated_at = now;
	if (!this.created_at) {
		this.created_at = now;
	}
	next();
});

var Users = mongoose.model('User', usersSchema);
module.exports = Users;

Mongoose query structuring

There are different ways to write queries with Mongoose - so much so that it can be confusing. Here are some of the main ways at a basic level:

Callback

The callback route is the more traditional Node async pattern:

User.findOne({ id: 12 }, (err, user) => {
  //user data
})

Callbacks are sometimes seen as the less favorable way of structuring a query because it can lead to multiple callback levels when there’s a need to run other queries based on the returned data:

Users.findOne({ id: 12 }, (err, user) => {
  let name = user.name
  let newName = name + " the cat"
  Users.findOneAndUpdate(
    { id: 12 },
    { name: newName },
    { upsert: true, new: true },
    (err, success) => {
      //user updated
    }
  )
})

If a query has the second parameter passed (the callback parameter as shown above) the query is executed and results passed to the callback (as shown in the above examples).

If a query doesn’t have the second parameter (callback parameter) a Query instance is returned. This query instance can be used as a promise, and as such it returns then() and catch() functions.

Then

The then() returned from a Query object mentioned above is not part of a fully-fledged ES6 Promise, although it is a promise which conforms to the Promises/A+ specification. This means you can use the Query promise by chaining then(), catch() for errors (which I think is nicer than error handling with the callback approach), and even use as part of other Promise functional such as Promise.all(). Here’s an example of a simple promise based query structure:

User.findOne({ id: 12 })
  .then((user) => {
    // success
  })
  .catch((err) => {
    // catch any errors
  })

Using this promise type can also be great for avoiding the callback hell mentioned above when updating data by using previously queried data:

Users.findOne({ id: 12 })
  .then((theUser) => {
    let name = theUser.name
    let newName = name + " the cat"
    // promise returned from findOneAndUpdate() is returned to the next then()
    return Users.findOneAndUpdate(
      { id: 12 },
      { name: newName },
      { upsert: true, new: true }
    )
  })
  .then((updated) => {
    //user updated
  })
  .catch((err) => {
    console.log(err)
  })

The then()’s can be chained to run as many separate queries as you need, and all errors will be caught by the last catch() function which keeps the code squeaky clean.

Exec

As mentioned above, the Query object which is returned from a query without a callback parameter can be used as a promise which is not the official ES6 Promise. However, the exec() function can be used after any query function such as find() to return a fully-fledged ES6 Promise:

Users.findOne({ id: 12 })
  .exec() // returns full ES6 Promise
  .then((theUser) => {
    let name = theUser.name
    let newName = name + " the cat"
    return Users.findOneAndUpdate(
      { id: 12 },
      { name: newName },
      { upsert: true, new: true }
    )
  })
  .then((updated) => {
    //user updated
  })
  .catch((err) => {
    console.log(err)
  })

I’ve used the same example as the previous one here to illustrate that there’s not much of a difference whether you use exec() or not.

Query building

Another feature of the Query object is that it can be used to chain query functions together, for example:

Users.find()
  .where("_id")
  .equals(userID)
  .exec((err, results) => {
    // view results
  })

This is a different way of querying data as opposed to passing an object to the find() function as shown here:

Users.find({ _id: card }).exec((err, results) => {
  // view results
})

The exec() function is used at the end of the query build chain to execute the query. This can be written another way to save the query to a variable and later use the exec() function to execute the query:

let query = Users.find({ _id: card }).where("_id").equals(userID)

//later, process the query
query.exec((err, results) => {
  // view results
})

The query building syntax can also be used alongside the promise functionality from the Query object, which I prefer to use:

Users.find()
  .where("_id")
  .equals(userID)
  .then((results) => {
    // view results
  })
  .catch((err) => {
    // view errors
  })

So that may leave the question, what’s the difference between exec() and then()? As far as I can tell, the exec() returns a full Promise and that’s it. The then() function can execute queries later from the Query object, as well as execute queries and be used in the query builder process.

Mongoose file structure and general usage

The Node ecosystem is an extremely unopinionated as it allows you to write code anywhere in the application - either all in the same file or splitting up into chunks and using require() or import syntax. When an app gets bigger it can be better to separate the concerns of the app into small, managable parts. Here’s how I like to do that with the database file structure:

As shown here, all database related files go in a folder called util. Lines 27-29 in the screen shot also shows the only place the database is initiated in the main entry point file (server.js in my case). The boilerplate setup code for making the database connection is in the mongo.js file inside util, which has the following:

const mongoose = require("mongoose")
const chalk = require("chalk")

module.exports = {
  connectToServer: (callback) => {
    var mongoDB = process.env.MONGODB_URI
    mongoose.Promise = global.Promise
    mongoose.connect(mongoDB)
    mongoose.connection.on(
      "error",
      console.error.bind(
        console,
        "%s MongoDB connection error. Please make sure MongoDB is running.",
        chalk.red("✗")
      )
    )
    return callback()
  },
}

You’ll notice I’m using environment variables to work with the sensitive and instance specific code here (local or remote). I’ve written another post about that if you want to take a look!

The database setup and Schema definitions are all in the same folder and only 3 lines of code initiate the database connection when the server loads, but what about actually makign queries? That’s simple, just include the Schema export in any file which needs it. For example, if you have one file which handles all API requests, you may include both database Schemas:

const express = require("express")
const router = express.Router()
const Users = require("../util/Users")
const Widgets = require("../util/Widgets")

Popular queries

Here are some popular queries which I end up using constantly, so it’s nice to have then in this quick reference quide. I’ve used the callback style of querying here rather than using the returned promise and chaining as it’s a little quicker to write and easy to understand if you’re new as many examples you see may be like this. I would use the promise syntax though when adding in to my code as it works a lot better when the app grows.

Creating an entry

People.create({ userID: 12, name: "Jon Snow" }, (err, success) => {
  if (err) return res.status(400).send(err)
  res.status(200).send(success)
})

Find all entries

Users.find({}, (err, users) => {
  if (err) return res.status(400).send(err)
  res.status(200).send(users)
})

Find entry where

Users.find({ First: { $regex: param, $options: "i" } }, (err, users) => {
  if (err) return res.status(400).send(err)
  res.status(200).send(users)
})

Upate an entry where

return Users.findOne({ id: 12 }, (err, users) => {
  return Users.findOneAndUpdate(
    { id: 12 },
    { name: "Tyrion Lannister" },
    { upsert: true, new: true }
  ).then((res) => {
    if (err) return res.status(400).send(err)
    res.status(200).send(users)
  })
})

Upate an entries embedded array

return Lists.findOne({ id: 12 }, (err, users) => {
  return Lists.findOneAndUpdate(
    { id: 12 },
    {
      $push: {
        favouriteFlix: { showname: "Breaking Bad", notes: "Best show ever" },
      },
    },
    { upsert: true, new: true }
  ).then((res) => {
    if (err) return res.status(400).send(err)
    res.status(200).send(users)
  })
})

That’s all

Thanks for reading. I’ll try and update this post every now and again to add to the list of popular queries. Even now, I find myself learning new ways to save data and implement new functionality. Tweet me if you have any questions or I’ve made any mistakes :).