Use MongoDB References

Introduction

In this lab, you will learn how to use MongoDB references to design data relations and manage relationships between parent and child documents. You will start by creating a simple library management system with books and authors, then explore techniques for linking parent and child documents, updating child references, querying parent-child relationships, and maintaining referential integrity. The lab covers key principles of document references in MongoDB and provides practical examples to help you understand and apply these concepts effectively.

Skills Graph

Design Data Relations

In this step, we'll explore how to design data relations in MongoDB by creating a practical example of linked documents. We'll model a simple library management system with books and authors.

Understanding Document References in MongoDB

MongoDB provides two main ways to create relationships between documents:

1. Embedded Documents
2. Document References

For our library system, we'll use document references to demonstrate how to link related data across collections.

First, let's start the MongoDB shell:

mongosh

Now, let's create our database and collections:

use library_database

db.authors.insertOne({
    _id: ObjectId("author1"),
    name: "Jane Austen",
    nationality: "British",
    birthYear: 1775
})

db.books.insertOne({
    title: "Pride and Prejudice",
    author_id: ObjectId("author1"),
    published: 1813,
    genre: "Classic Literature"
})

Breaking Down the Reference

Let's examine what we've done:

• We created an authors collection with a unique _id
• We created a books collection that references the author using author_id
• The author_id field contains the _id of the corresponding author document

To verify our references, we can query the documents:

db.authors.findOne({ name: "Jane Austen" })
db.books.findOne({ title: "Pride and Prejudice" })

Example Output

{
  _id: ObjectId("author1"),
  name: 'Jane Austen',
  nationality: 'British',
  birthYear: 1775
}

{
  _id: ObjectId(...),
  title: 'Pride and Prejudice',
  author_id: ObjectId("author1"),
  published: 1813,
  genre: 'Classic Literature'
}

Key Principles of Document References

• Use _id to create links between documents
• Keep references simple and consistent
• Choose references when data is large or changes frequently
• Normalize data to reduce duplication

Link Parent Child Docs

In this step, we'll expand our library database by creating more complex relationships between parent and child documents. We'll demonstrate how to link multiple books to an author and manage these relationships effectively.

Adding Multiple Books for an Author

Let's continue with our existing database and add more books for Jane Austen:

db.books.insertMany([
    {
        title: "Sense and Sensibility",
        author_id: ObjectId("author1"),
        published: 1811,
        genre: "Classic Literature"
    },
    {
        title: "Emma",
        author_id: ObjectId("author1"),
        published: 1815,
        genre: "Classic Literature"
    }
])

Querying Related Documents

To find all books by Jane Austen, we'll use the author_id:

db.books.find({ author_id: ObjectId("author1") })

Example Output

[
  {
    _id: ObjectId(...),
    title: 'Pride and Prejudice',
    author_id: ObjectId("author1"),
    published: 1813,
    genre: 'Classic Literature'
  },
  {
    _id: ObjectId(...),
    title: 'Sense and Sensibility',
    author_id: ObjectId("author1"),
    published: 1811,
    genre: 'Classic Literature'
  },
  {
    _id: ObjectId(...),
    title: 'Emma',
    author_id: ObjectId("author1"),
    published: 1815,
    genre: 'Classic Literature'
  }
]

Counting Books by an Author

We can also count the number of books for a specific author:

db.books.countDocuments({ author_id: ObjectId("author1") })

Example Output

3

Advanced Querying with Aggregation

Let's use the aggregation framework to get more detailed information:

db.books.aggregate([
    { $match: { author_id: ObjectId("author1") } },
    { $group: {
        _id: "$author_id",
        totalBooks: { $sum: 1 },
        earliestPublished: { $min: "$published" },
        latestPublished: { $max: "$published" }
    }}
])

Example Output

[
  {
    _id: ObjectId("author1"),
    totalBooks: 3,
    earliestPublished: 1811,
    latestPublished: 1815
  }
]

Update Child References

In this step, we'll learn how to update references between parent and child documents in MongoDB. We'll explore different techniques to modify document references and maintain data integrity.

Adding a New Author and Updating Book References

First, let's add another author to our database:

db.authors.insertOne({
    _id: ObjectId("author2"),
    name: "Charles Dickens",
    nationality: "British",
    birthYear: 1812
})

Updating a Single Book's Author Reference

Let's update a book to change its author reference:

db.books.updateOne(
    { title: "Emma" },
    { $set: { author_id: ObjectId("author2") } }
)

Verifying the Update

Check the updated book's author reference:

db.books.findOne({ title: "Emma" })

Example Output

{
  _id: ObjectId(...),
  title: 'Emma',
  author_id: ObjectId("author2"),
  published: 1815,
  genre: 'Classic Literature'
}

Bulk Update of References

We can also update multiple documents at once:

db.books.updateMany(
    { author_id: ObjectId("author1") },
    { $set: { genre: "Romantic Novel" } }
)

Checking Multiple Document Updates

Verify the genre update:

db.books.find({ author_id: ObjectId("author1") })

Example Output

[
  {
    _id: ObjectId(...),
    title: 'Pride and Prejudice',
    author_id: ObjectId("author1"),
    published: 1813,
    genre: 'Romantic Novel'
  },
  {
    _id: ObjectId(...),
    title: 'Sense and Sensibility',
    author_id: ObjectId("author1"),
    published: 1811,
    genre: 'Romantic Novel'
  }
]

Upsert: Update or Insert

We can use the upsert option to update a document or create it if it doesn't exist:

db.books.updateOne(
    { title: "Oliver Twist" },
    { $set: {
        author_id: ObjectId("author2"),
        published: 1837,
        genre: "Historical Fiction"
    }},
    { upsert: true }
)

Query Parent Children

In this step, we'll explore advanced querying techniques to retrieve related documents across parent and child collections in MongoDB. We'll demonstrate how to effectively find and join related data.

Basic Filtering Queries

Let's start by finding books by a specific author:

db.books.find({ author_id: ObjectId("author1") })

Filtering with Multiple Conditions

Query books with multiple filters:

db.books.find({
    author_id: ObjectId("author1"),
    published: { $gt: 1812 }
})

This query finds books by Jane Austen published after 1812.

Aggregation Pipeline for Complex Queries

Use the aggregation framework to join author and book information:

db.books.aggregate([
    { $lookup: {
        from: "authors",
        localField: "author_id",
        foreignField: "_id",
        as: "author_details"
    }},
    { $match: {
        "author_details.nationality": "British"
    }},
    { $project: {
        title: 1,
        published: 1,
        "author_name": "$author_details.name"
    }}
])

Example Output

[
  {
    _id: ObjectId(...),
    title: 'Pride and Prejudice',
    published: 1813,
    author_name: ['Jane Austen']
  },
  {
    _id: ObjectId(...),
    title: 'Emma',
    published: 1815,
    author_name: ['Charles Dickens']
  }
]

Sorting and Limiting Results

Query and sort books by publication year:

db.books.find()
    .sort({ published: 1 })
    .limit(2)

This retrieves the two earliest published books.

Advanced Filtering with Regular Expressions

Find books with titles containing specific words:

db.books.find({
    title: { $regex: /Sense/, $options: 'i' }
})

The $options: 'i' makes the search case-insensitive.

Counting Related Documents

Count books for each author:

db.books.aggregate([
    { $group: {
        _id: "$author_id",
        book_count: { $sum: 1 }
    }},
    { $lookup: {
        from: "authors",
        localField: "_id",
        foreignField: "_id",
        as: "author_info"
    }},
    { $project: {
        author_name: "$author_info.name",
        book_count: 1
    }}
])

Example Output

[
  {
    _id: ObjectId("author1"),
    author_name: ['Jane Austen'],
    book_count: 2
  },
  {
    _id: ObjectId("author2"),
    author_name: ['Charles Dickens'],
    book_count: 2
  }
]

Maintain References

In this final step, we'll explore strategies for maintaining document references in MongoDB, focusing on data integrity, cleanup, and best practices for managing relationships between collections.

Handling Orphaned References

Sometimes, references can become outdated when parent documents are deleted. Let's demonstrate how to manage this:

Create a Test Author and Book

db.authors.insertOne({
    _id: ObjectId("author3"),
    name: "Mark Twain",
    nationality: "American",
    birthYear: 1835
})

db.books.insertOne({
    title: "The Adventures of Tom Sawyer",
    author_id: ObjectId("author3"),
    published: 1876,
    genre: "Classic Literature"
})

Simulate Reference Cleanup

Delete an author and check for orphaned books:

db.authors.deleteOne({ _id: ObjectId("author3") })

## Find books with non-existent author references
db.books.find({
    author_id: { $nin: db.authors.distinct("_id") }
})

Implementing Reference Management

Create a function to clean up orphaned references:

db.books.deleteMany({
    author_id: { $nin: db.authors.distinct("_id") }
})

Preventing Invalid References

Use validation rules when inserting documents:

db.createCollection("books", {
   validator: {
      $jsonSchema: {
         bsonType: "object",
         required: ["title", "author_id"],
         properties: {
            title: {
               bsonType: "string",
               description: "must be a string and is required"
            },
            author_id: {
               bsonType: "objectId",
               description: "must be a valid author reference"
            }
         }
      }
   }
})

Indexing References for Performance

Create an index on the reference field:

db.books.createIndex({ author_id: 1 })

Check Index Creation

db.books.getIndexes()

Example Output

[
  { v: 2, key: { _id: 1 }, name: '_id_' },
  { v: 2, key: { author_id: 1 }, name: 'author_id_1' }
]

Best Practices for Reference Management

1. Always validate references before inserting
2. Use indexes on reference fields
3. Implement cleanup mechanisms
4. Consider using database transactions for complex operations

Demonstrating Reference Validation

## This will fail due to invalid author_id
db.books.insertOne({
    title: "Invalid Book",
    author_id: ObjectId("invalid_id")
})

Summary

In this lab, we learned how to design data relations in MongoDB using document references. We created a practical example of a library management system, modeling books and authors. We explored the two main ways to create relationships in MongoDB: embedded documents and document references. For our library system, we chose to use document references to link related data across collections. We learned the key principles of document references, such as using _id to create links, keeping references simple and consistent, and choosing references when data is large or changes frequently. We also expanded our library database by creating more complex relationships between parent and child documents.