Docker Bare-Metal – Persistent Volumes with Datrium GraphDriver Plug-In (How-to)

Datrium supports various flavors of container management orchestration platforms, but independent of the orchestration platform, containers can be of persistent or transient nature. Both types may require some level of persistent storage, either to retain data across instances or to temporarily store configuration data.

However, most, if not all, HCI vendors on the market have solutions for containers that require a hypervisor, but often times we see customers wanting to move their container deployment to Linux bare-metal, be it Docker Swarm, Kubernetes, Mesos or Openshift, avoiding hypervisor overhead and associated costs. Ultimately it comes down to architecture design and operational decisions – there’s not right or wrong and we see customers choosing both models.

When it comes to supporting bare-metal deployments, Datrium uniquely presents a single-namespace datastore as a host local mount point across all Linux bare-metal Nodes, upon installation of the Datrium hyperdriver software, and for Docker specifically, Datrium provides a GraphDriver storage plug-in that mount volumes with a .raw extension and may be used in bare-metal production deployments. Here is a good article to learn more about Docker Storage and Graph Drivers.

The Docker Engine volume plugin enables Docker deployments to be integrated with external storage systems such as Datrium or Amazon EBS and allow data in volumes to persist beyond the lifetime of a single Docker host. The Datrium volume plug-in provides seamless integration of Docker containers and Datrium storage.

 

In this article, I describe the steps to deploy Docker with Datrium using Linux bare-metal Nodes.

First, make sure the Datrium Linux Hyperdrive is properly installed on all hosts being configured as compute Node for your Swarm cluster.

The Datrium Linux Hyperdriver installs a systemd script at /usr/lib/systemd/system/datrium.service configured to startup automatically at boot time.

The Datrium Linux Hyperdriver provides support for storing and cloning persistent Docker volumes, and may also be used to store Docker images and containers, but this feature is currently experimental.

Experimental features are included in the standard Docker binaries as of version 1.13.0. To enable experimental features, start the Docker daemon with the –experimental flag or enable the daemon flag in the /etc/docker/daemon.json configuration file:

{
"experimental": true
}

 

After enabling experimental features, copy the configuration file at /opt/datrium/examples/datrium-graphdriver.conf to /etc/systemd/system/docker.service.d/ and restart docker daemon.

# cp /opt/datrium/examples/datrium-graphdriver.conf /etc/systemd/system/docker.service.d/
# sudo systemctl stop docker
# sudo systemctl daemon-reload
# sudo systemctl start docker

 

You can check to see if experimental features are enabled on a running daemon using the following command:

# docker version -f '{{.Server.Experimental}}'
true

 

To verify that Docker is using the Datrium Linux Hyperdriver as a volume driver, look for datrium in the Storage Driver section of docker info. Note that the Datrium Volume plugin will only appear after a docker volume is created with the Datrium plugin.

# docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 1.13.1
Storage Driver: datrium
:
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: datrium local
Network: bridge host ipvlan macvlan null overlay

 

To create a 10 GB persistent volume named MyVolume:

# docker volume create --driver datrium --name MyVolume --opt size=10g

 

Store virtual disks associated with each container are stored in appropriate named sub-directories under /mnt/datrium/Datastore1/. Because the DVX itself handles deduplication, compression, and snapshots, RAW disk format is recommended. If the DVX Datastore is shared between VMware, KVM, and Docker environments, care must be taken to avoid namespace conflicts.

# pwd
/mnt/datrium/Datastore1/docker/volumes/images

 

To list the volumes stored on the DVX:

# docker volume ls --filter driver=datrium
DRIVER              VOLUME NAME
datrium             MyVolume</code>

To mount MyVolume to an ubuntu container:
# docker run -it --rm --volume MyVolume:/mnt/MyVolume ubuntu

 

Volume cloning is implemented as an extension to the create command. To clone MyVolume to MyVolume2:

# docker volume create --driver datrium --name MyVolume2 --opt clone=MyVolume

 

Volume clones are not linked in any way to their parents. After cloning, the volume may be mounted normally.

# docker run -it --rm --volume MyVolume:/mnt/MyVolume --volume MyVolume2:/mnt/MyVolume2

 

Uninstall
To uninstall the docker storage plug-in use the following commands:

# sudo systemctl stop docker
# rm -rf  /etc/systemd/system/docker.service.d/datrium-graphdriver.conf
# sudo systemctl daemon-reload
# sudo systemctl start docker

 

Docker Containers with VMware vSphere

You can configure Docker to access VMware vSphere VMDK Volumes. This includes using VMware vSphere VMDK Volumes as persistent storage for application data with Datrium. The vSphere Cloud Provider allows using vSphere managed storage with Docker and supports Volumes, Persistent volumes, Storage classes, and provisioning volumes.

This service is integrated with Docker Volume Plugin framework. Docker users can now consume vSphere Storage (vSAN, VMFS, NFS, VVol) to stateful containers using Docker.

Docker always provisions new volumes as persistent independent disks to freely attach and detach the volume on any node in the cluster, and for Datrium DVX each persistent volume is an independent VMDK. As a consequence, it is generally not possible to backup volumes that utilize VMware snapshots, and Docker recommends stopping the application utilizing the Persistent Volume, cloning the Persistent Volume, and then restarting the application. However, Datrium can uniquely address this issue with native snapshots that have Zero impact on Persistent Volumes and Applications.

 

Protecting Persistent Volumes

A native self-contained .raw file is created under /mnt/datrium/Datastore1/docker/volumes/images for each persistent volume. Datrium administrators may select individual files to be included into protection groups or the full path may be provided to protect all existing persistent volumes. In my next article I will detail how to use Datrium Protection Groups to easily backup and protect Docker persistent volumes.

 

This article was first published by Andre Leibovici (@andreleibovici) at myvirtualcloud.net

Leave a Reply

Your email address will not be published.