Manage data in Docker
Master data management in Docker: Use volumes and bind mounts to store, persist, and share data across containers.
Published Sep 28, 2024
- The data doesn’t persist when that container no longer exists, and it can be difficult to get the data out of the container if another process needs it.
- A container’s writable layer is tightly coupled to the host machine where the container is running. You can’t easily move the data somewhere else.
- Writing into a container’s writable layer requires a storage driver to manage the filesystem. The storage driver provides a union filesystem, using the Linux kernel. This extra abstraction reduces performance as compared to using data volumes, which write directly to the host filesystem.
Docker has two options for containers to store files on the host machine, so that the files are persisted even after the container stops: volumes, and bind mounts.
Docker also supports containers storing files in-memory on the host machine. Such files are not persisted. If you’re running Docker on Linux,
tmpfs mount is used to store files in the host's system memory. If you're running Docker on Windows, named pipe is used to store files in the host's system memory.No matter which type of mount you choose to use, the data looks the same from within the container. It is exposed as either a directory or an individual file in the container’s filesystem.
An easy way to visualize the difference among volumes, bind mounts, and
tmpfs mounts is to think about where the data lives on the Docker host.
- Volumes are stored in a part of the host filesystem which is managed by Docker (
/var/lib/docker/volumes/on Linux). Non-Docker processes should not modify this part of the filesystem. Volumes are the best way to persist data in Docker. - Bind mounts may be stored anywhere on the host system. They may even be important system files or directories. Non-Docker processes on the Docker host or a Docker container can modify them at any time.
tmpfsmounts are stored in the host system's memory only, and are never written to the host system's filesystem.
Bind mounts and volumes can both be mounted into containers using the
-v or --volume flag, but the syntax for each is slightly different. For tmpfs mounts, you can use the --tmpfs flag. We recommend using the --mount flag for both containers and services, for bind mounts, volumes, or tmpfs mounts, as the syntax is more clear.
Volumes are created and managed by Docker. You can create a volume explicitly using the
docker volume create command, or Docker can create a volume during container or service creation.When you create a volume, it’s stored within a directory on the Docker host. When you mount the volume into a container, this directory is what’s mounted into the container. This is similar to the way that bind mounts work, except that volumes are managed by Docker and are isolated from the core functionality of the host machine.
A given volume can be mounted into multiple containers simultaneously. When no running container is using a volume, the volume is still available to Docker and isn’t removed automatically. You can remove unused volumes using
docker volume prune.When you mount a volume, it may be named or anonymous. Anonymous volumes are given a random name that’s guaranteed to be unique within a given Docker host. Just like named volumes, anonymous volumes persist even if you remove the container that uses them, except if you use the
--rm flag when creating the container, in which case the anonymous volume is destroyed.If you create multiple containers after each other that use anonymous volumes, each container creates its own volume. Anonymous volumes aren’t reused or shared between containers automatically. To share an anonymous volume between two or more containers, you must mount the anonymous volume using the random volume ID.Volumes also support the use of volume drivers, which allow you to store your data on remote hosts or cloud providers, among other possibilities.
Bind mounts have limited functionality compared to volumes. When you use a bind mount, a file or directory on the host machine is mounted into a container. The file or directory is referenced by its full path on the host machine. The file or directory doesn’t need to exist on the Docker host already. It is created on demand if it doesn’t yet exist. Bind mounts are fast, but they rely on the host machine’s filesystem having a specific directory structure available. If you are developing new Docker applications, consider using named volumes instead. You can’t use Docker CLI commands to directly manage bind mounts.
A
tmpfs mount isn't persisted on disk, either on the Docker host or within a container. It can be used by a container during the lifetime of the container, to store non-persistent state or sensitive information. For instance, internally, Swarm services use tmpfs mounts to mount secret into a service's containers.