Comment migrer des volumes Docker vers un autre hôte

Introduction

Docker volumes provide persistent storage for containerized applications, allowing data to remain intact even when containers are stopped or removed. As your infrastructure evolves, you may need to migrate these volumes to different hosts to support scaling, upgrades, or relocation of your services.

This tutorial guides you through the process of migrating Docker volumes between hosts. You will learn how to create volumes, populate them with data, back them up, and restore them on a new host. By the end of this lab, you will have a clear understanding of Docker volume migration techniques to ensure data continuity in your containerized environments.

Skills Graph

Understanding and Creating Docker Volumes

Docker volumes provide persistent storage for your containers. Before migrating volumes, you need to understand what they are and how to create them.

What are Docker Volumes?

Docker volumes are the preferred mechanism for persisting data generated by and used by Docker containers. Unlike data stored in a container's writable layer which is lost when the container is removed, volumes are completely managed by Docker and exist independently from containers.

Some key benefits of Docker volumes include:

• Data persistence across container lifecycles
• Ability to share data between containers
• Better performance than container's filesystem
• Independent management from containers

Creating Your First Docker Volume

Let's start by creating a simple Docker volume:

docker volume create my-data-volume

When you run this command, Docker will create a new volume named my-data-volume. You should see the volume name printed in the terminal:

my-data-volume

You can list all Docker volumes on your system with the following command:

docker volume ls

This will display a table of all volumes:

DRIVER    VOLUME NAME
local     my-data-volume

Inspecting a Docker Volume

To get more details about your volume, use the inspect command:

docker volume inspect my-data-volume

You'll see detailed information about the volume in JSON format:

[
  {
    "CreatedAt": "2023-10-01T12:34:56Z",
    "Driver": "local",
    "Labels": {},
    "Mountpoint": "/var/lib/docker/volumes/my-data-volume/_data",
    "Name": "my-data-volume",
    "Options": {},
    "Scope": "local"
  }
]

Notice the Mountpoint value - this is where Docker stores the volume data on the host system.

Using a Docker Volume with a Container

Now that you have created a volume, let's use it with a container. We'll create a simple container that mounts the volume and adds some data to it:

docker run --name my-container -v my-data-volume:/data ubuntu /bin/bash -c "echo 'Hello from Docker volume' > /data/test.txt"

This command:

• Creates a container named my-container
• Mounts my-data-volume to the /data directory inside the container
• Runs a simple command to write text to a file in that directory

Let's verify the data was written to the volume by creating another container that mounts the same volume:

docker run --rm -v my-data-volume:/data ubuntu cat /data/test.txt

You should see the output:

Hello from Docker volume

This demonstrates that the data persists in the volume and can be accessed by different containers.

Preparing for Volume Migration

Before migrating a Docker volume to another host, we need to prepare our source data and understand the migration process. In this step, we'll add more data to our volume and prepare it for migration.

Adding More Data to the Volume

Let's create a container that will add more data to our volume. This will help demonstrate that all volume data is properly migrated:

docker run --name data-generator -v my-data-volume:/data ubuntu /bin/bash -c "mkdir -p /data/config && echo 'database_url=postgres://user:password@db:5432/mydb' > /data/config/settings.conf && echo 'This is important data' > /data/important.txt"

This command creates directories and files with sample content in our volume.

Let's verify the data structure in our volume:

docker run --rm -v my-data-volume:/data ubuntu ls -la /data

You should see output similar to:

total 12
drwxr-xr-x 3 root root 4096 Oct  1 12:34 .
drwxr-xr-x 1 root root 4096 Oct  1 12:34 ..
drwxr-xr-x 2 root root 4096 Oct  1 12:34 config
-rw-r--r-- 1 root root   21 Oct  1 12:34 important.txt
-rw-r--r-- 1 root root   24 Oct  1 12:34 test.txt

Let's also check the content of one of our files:

docker run --rm -v my-data-volume:/data ubuntu cat /data/important.txt

Output:

This is important data

Understanding Volume Migration Approaches

There are several approaches to migrate Docker volumes:

1. Direct host-to-host copy: Copying the volume data directly between hosts
2. Backup and restore: Creating a backup archive and restoring it on the new host
3. Using Docker plugins: Some storage plugins have built-in replication features

For this lab, we'll use the backup and restore approach, which works in most environments and doesn't require special plugins.

Creating a Backup of the Volume

To create a backup of our volume, we'll use a Docker container to archive the volume contents:

docker run --rm -v my-data-volume:/source -v $(pwd):/backup ubuntu tar cvf /backup/my-data-volume-backup.tar -C /source .

This command:

• Mounts our volume at /source in the container
• Mounts the current directory at /backup in the container
• Creates a tar archive of everything in the /source directory and saves it to /backup/my-data-volume-backup.tar

Verify that the backup file was created:

ls -lh my-data-volume-backup.tar

You should see something like:

-rw-r--r-- 1 labex labex 10K Oct  1 12:34 my-data-volume-backup.tar

Let's examine the contents of the backup file to make sure everything was properly archived:

tar -tf my-data-volume-backup.tar

Output should include:

./
./config/
./config/settings.conf
./important.txt
./test.txt

Now that we have created a backup of our volume, we're ready to simulate migrating it to a new host.

Simulating Volume Migration to a New Host

In a real-world scenario, you would transfer the backup file to a new host using tools like scp, rsync, or a file sharing service. For this lab, we'll simulate a migration by removing our original volume and creating a new one to restore our data.

Simulating a New Host Environment

First, let's clean up our old containers to simulate moving to a new environment:

docker rm -f my-container data-generator

Now, let's remove our original volume to simulate starting fresh on a new host:

docker volume rm my-data-volume

Create a new volume that will represent our volume on the "new host":

docker volume create my-new-host-volume

Restoring the Volume from Backup

Now we'll restore our backup to the new volume:

docker run --rm -v my-new-host-volume:/destination -v $(pwd):/backup ubuntu bash -c "cd /destination && tar xvf /backup/my-data-volume-backup.tar"

This command:

• Creates a temporary container
• Mounts our new volume at /destination
• Mounts the current directory (where our backup is located) at /backup
• Extracts the tar archive to the /destination directory

Verifying the Restored Data

Let's verify that all our data was properly restored to the new volume:

docker run --rm -v my-new-host-volume:/data ubuntu ls -la /data

You should see the same file structure as before:

total 12
drwxr-xr-x 3 root root 4096 Oct  1 12:34 .
drwxr-xr-x 1 root root 4096 Oct  1 12:34 ..
drwxr-xr-x 2 root root 4096 Oct  1 12:34 config
-rw-r--r-- 1 root root   21 Oct  1 12:34 important.txt
-rw-r--r-- 1 root root   24 Oct  1 12:34 test.txt

Check the content of one of our files to ensure the data is intact:

docker run --rm -v my-new-host-volume:/data ubuntu cat /data/important.txt

Output:

This is important data

Also check the configuration file:

docker run --rm -v my-new-host-volume:/data ubuntu cat /data/config/settings.conf

Output:

database_url=postgres://user:password@db:5432/mydb

Using the Restored Volume with a New Container

Now that we've successfully "migrated" our volume, let's use it with a new container:

docker run --name my-new-container -v my-new-host-volume:/app/data -d nginx

This creates a new Nginx container that mounts our restored volume at /app/data.

Let's verify that the container can access the volume data:

docker exec my-new-container ls -la /app/data

You should see the same files as before, confirming that the migration was successful.

Updating Applications to Use the New Volume

In a real-world scenario, after migrating volumes to a new host, you would need to update your Docker Compose files or container run commands to use the new volume names or paths. For example, if you were using Docker Compose, you might update your docker-compose.yml file as follows:

version: "3"
services:
  webapp:
    image: nginx
    volumes:
      - my-new-host-volume:/app/data
volumes:
  my-new-host-volume:
    external: true

This tells Docker to use the externally created volume named my-new-host-volume.

The migration process is now complete! You have successfully:

1. Created a Docker volume and added data to it
2. Backed up the volume data
3. Created a new volume on a simulated "new host"
4. Restored the backup data to the new volume
5. Verified the data integrity
6. Connected a new container to the migrated volume

Best Practices and Advanced Migration Techniques

Now that you understand the basics of Docker volume migration, let's explore some best practices and advanced techniques to make your volume migrations more efficient and robust.

Planning for Volume Migration

When planning for volume migration, consider these factors:

1. Volume Size: Larger volumes take longer to transfer and might require special handling
2. Downtime Requirements: Determine how much downtime your application can tolerate
3. Data Sensitivity: Ensure that sensitive data is properly secured during transfer
4. Network Bandwidth: Consider the available bandwidth between hosts

Automating the Migration Process

For frequent migrations or migrations of multiple volumes, automation can save time and reduce errors. Let's create a simple shell script that automates the backup and restore process:

nano migrate-volume.sh

Add the following content to the script:

#!/bin/bash
## Simple Docker volume migration script

if [ $## -ne 2 ]; then
  echo "Usage: $0 <source_volume> <destination_volume>"
  exit 1
fi

SOURCE_VOLUME=$1
DEST_VOLUME=$2
BACKUP_FILE="/tmp/${SOURCE_VOLUME}-backup.tar"

echo "Creating backup of ${SOURCE_VOLUME}..."
docker run --rm -v ${SOURCE_VOLUME}:/source -v /tmp:/backup ubuntu tar cf /backup/${SOURCE_VOLUME}-backup.tar -C /source .

echo "Creating destination volume ${DEST_VOLUME}..."
docker volume create ${DEST_VOLUME}

echo "Restoring backup to ${DEST_VOLUME}..."
docker run --rm -v ${DEST_VOLUME}:/destination -v /tmp:/backup ubuntu bash -c "cd /destination && tar xf /backup/${SOURCE_VOLUME}-backup.tar"

echo "Migration complete. Verifying volume contents..."
docker run --rm -v ${DEST_VOLUME}:/data ubuntu ls -la /data

echo "Done!"

Save the file (Ctrl+O, then Enter) and exit (Ctrl+X).

Make the script executable:

chmod +x migrate-volume.sh

Now you can migrate volumes with a single command:

./migrate-volume.sh my-new-host-volume another-volume

This will create a backup of my-new-host-volume, create a new volume called another-volume, and restore the backup to it.

Handling Large Volumes

For large volumes, the basic tar approach might be inefficient. Here are some alternatives:

1. Incremental Backups: Only transfer changes since the last backup
2. Compression: Use compression to reduce the size of the backup
3. Direct Copy: Use rsync to directly copy between hosts

Let's implement a compressed backup approach:

docker run --rm -v my-new-host-volume:/source -v $(pwd):/backup ubuntu tar czf /backup/compressed-backup.tar.gz -C /source .

The z option added to tar enables compression, resulting in a smaller backup file. This is especially useful for text-heavy data.

To restore from a compressed backup:

docker volume create compressed-volume
docker run --rm -v compressed-volume:/destination -v $(pwd):/backup ubuntu bash -c "cd /destination && tar xzf /backup/compressed-backup.tar.gz"

Volume Migration in Production Environments

In production environments, you might consider these additional approaches:

1. Storage Driver Replication: Some storage drivers support built-in replication
2. Docker Volume Plugins: Use specialized plugins for your storage system
3. Database Backups: For database volumes, use the database's native backup tools
4. Scheduled Migrations: Use tools like cron to schedule regular backups

Cleaning Up

Before finishing this lab, let's clean up our resources:

docker rm -f my-new-container
docker volume rm my-new-host-volume another-volume compressed-volume
rm my-data-volume-backup.tar compressed-backup.tar.gz migrate-volume.sh

This completes your journey through Docker volume migration. You now have the knowledge and tools to migrate Docker volumes between hosts effectively, ensuring your containerized applications' data remains intact during infrastructure changes.

Summary

In this lab, you learned how to migrate Docker volumes between hosts, which is an essential skill for managing containerized applications. You gained practical experience with:

• Creating and managing Docker volumes
• Adding and accessing data in Docker volumes
• Backing up volume data using the tar command
• Restoring volume data on a new host
• Verifying data integrity after migration
• Creating automation scripts for volume migration
• Handling various migration scenarios including large volumes

These skills enable you to maintain data continuity when moving containers between environments, scaling your infrastructure, or performing system upgrades. Docker volume migration is a fundamental operation that allows you to manage your containerized applications with confidence, knowing that your persistent data can be safely and reliably transferred as needed.