Will database backup if I backup VM?

[dlung]

Backup recommendations suggest pg_dump of Postgresql, suggesting Mayan doesn’t persist updated database file information in the volume. It’s painful to coordinate the pre-backup shell commands required to have the volumes and Portgresql data combined in a single, de-duped, versioned backup.

Will “docker compose down” push cashed database info in the volume so I only need to backup the volumes?

Will running Mayan in a virtual machine allow me to just backup the KVM?

I can’t use Mayan if I can’t backup the critical data properly.

[roberto.rosario]

I’m having trouble understanding the issue. Can you rephrase it?

Backup recommendations suggest pg_dump of PostgreSQL, suggesting Mayan doesn’t persist updated database file information in the volume.

Backing up a database by using pg_dump is not unique to Mayan, it is the recommended method to get a clean and reusable backup of a database. There is no correlation between recommending the use of pg_dump for backups and volume persistence. Those are different topics.

It’s painful to coordinate the pre-backup shell commands required to have the volumes and PostgreSQL data combined in a single, de-duped, versioned backup.

Yes, this is the norm for software, open source and commercial. It is a common systems administrator and/or operations personnel procedure to get working in their organizations.

Will “docker compose down” push cashed database info in the volume so I only need to backup the volumes?

docker compose down just shuts down the stack. The data is not pushed to the volume. It is always in the volume whether the stack is ruining or not. The volume is the storage element in Docker.

Shutting down the stack to backup the database and the documents is the recommended method.

https://docs.mayan-edms.com/chapters/docker/backups.html

If you are asking about just backing up the database binary files, then it is not a good idea. If you change the database image version the backed up binary files will not work. That is an issue with PostgreSQL’s Docker image and has nothing to do with Mayan. They are trying to address it with an automatic version migration tool but it is still not production ready.

Will running Mayan in a virtual machine allow me to just backup the KVM?

Yes, like any other software.

I can’t use Mayan if I can’t backup the critical data properly.

Understandable, but Mayan’s backup process is essentially the same as any other software.

[twp]

As Roberto already made clear, the backup strategy is not specific to Mayan but for all applications. This depends on many factors of your hosting strategy/environment.

As from my experience there are 3 main strategies of backing up applications like Mayan hosted locally in Docker (certainly among many ohers):

1. Raw backup

   • stop application containers (to make sure that no new data is written to DB)
   • run database backup (pg_dump) and copy data files, config, … from volumes to backup folder
   • start containers

2. Docker volume backup (similar to #1)

   • stop all containers (to make sure everything is shutdown cleanly)
   • copy data from all docker volumes to backup folder
   • make sure to also have the docker configuration files (docker-compose.yaml) included in backup
   • start containers

3. VM level backup

   • when using Docker insive a VM, you could run backups on the VM host for the whole VM, i.e.

     • for Hyper-V, you could use VM replication or a fully featured solution like Veeam (already supports deduplication, replication, encryption, maintaining restore points …). This solution does not even need a shutdown of the VM while running a backup.
     • for Proxmox you could use Proxmox Backup server (like Roberto showed on the screenshot)

In my environment I mainly make use of #2 and #3.

For #2 please see parts of the script I am using below:

# -------------------------------------------------------------------
# Please note that the compression step of the tar archive is done
# after the containers are running again to keep the downtime as
# short as possible
# -------------------------------------------------------------------

# config
source_dir="/var/lib/docker/volumes"
backup_dir="/opt/docker_backups"
backup_filename="backup.tar"
remote_user="root"
remote_server="192.168.1.200"
remote_dir="/opt/docker_backups"

remote_target="${remote_user}@${remote_server}"
backup_fullpath="${backup_dir}/${backup_filename}"

# shutdown all containers, tar all volumes and restart
docker stop $(docker ps -q)
tar -cpf "${backup_fullpath}" "${source_dir}"
docker start $(docker ps -a -q)

# compress and push to remote backup server (i.e. NAS)
gzip "${backup_fullpath}"
backup_fullpath="${backup_fullpath}.gz"
scp "${backup_fullpath}" "${remote_target}:${remote_dir}/"

[dlung]

Thanks to both of you so much! I’ve created backups before but never on docker containers. The raw backup is the only one mentioned in the documentation. It is difficult for most backup software on Linux given the “context” of users vs “context” of containers running. The Docker Volume Backup is clean and easy for Borgmatic or Vorta.

Thanks again!