Direct deployment to Docker Compose migration

Thanks to help from @DocCyblade the following procedure was created.

Please try this procedure on a backup copy of your installations (or even better a VM snapshot) and report your results (good or bad) so we can finalize this and add it to the documentation.

Update the direct deployment to the 4.4 series.

Follow the instructions located in: Upgrades — Mayan EDMS 4.4.6 documentation

Purge all file caches.

  • Option 1: Via the user interface. SystemToolsFile caches

Refresh the page until all current size columns display 0.

  • Option 2: Via the command line.
echo "from mayan.apps.file_caching.models import CachePartitionFile;[i.delete() for i in CachePartitionFile.objects.all()]" | sudo --user=mayan MAYAN_MEDIA_ROOT=/opt/mayan-edms/media/ /opt/mayan-edms/bin/mayan-edms.py shell

Create the backup folder

mkdir ~/mayan-direct-backup

Stop the installation

sudo systemctl stop supervisor

Dump the database

IMPORTANT! Make a note of the database credentials.

Use the database credentials from the previous step.

pg_dump -U mayan -h 127.0.0.1 -d mayan -W > ~/mayan-direct-backup/database.sql

Copy the config.yml

cp /opt/mayan-edms/media/config.yml ~/mayan-direct-backup

Copy the SECRET_KEY

cp /opt/mayan-edms/media/SECRET_KEY ~/mayan-direct-backup

Copy the document files

Use rsync to preserve the file attributes and ownership.

rsync -a /opt/mayan-edms/media/document_file_storage /home/ubuntu/mayan-direct-backup/

Stop and disable services

sudo systemctl stop supervisor  # Should be already stopped
sudo systemctl disable supervisor
sudo systemctl stop postgresql
sudo systemctl disable postgresql
sudo systemctl stop redis
sudo systemctl disable redis
# RabbitMQ if installed
sudo systemctl stop rabbitmq
sudo systemctl disable rabbitmq

Install Mayan EDMS via Docker Compose

  • Install Docker.
  • Just download the Mayan Docker Compose files.
  • Do not run docker compose up -d yet!
curl https://gitlab.com/mayan-edms/mayan-edms/-/raw/master/docker/docker-compose.yml -O
curl https://gitlab.com/mayan-edms/mayan-edms/-/raw/master/docker/.env -O

Restore the database

  • Update the new database credentials in the .env file if desired.

  • Start only PostgreSQL in Docker.
docker compose up postgresql -d
  • Restore the database dump. Replace the database name, user as needed.
docker compose exec -T postgresql psql -U mayan -d mayan < ~/mayan-direct-backup/database.sql
  • Inspect the Mayan app volume. Take note of the Mountpoint value.
docker volume inspect mayan_app

Restore the document files

Restore the document_file_storage folder.

sudo rsync -a /home/ubuntu/mayan-direct-backup/document_file_storage /var/lib/docker/volumes/mayan_app/_data/
  • Verify correct ownership of restored files. All files should be owner by mayan:mayan.
docker compose exec -ti app ls /var/lib/mayan -al

Shutdown the entire stack

docker compose down

Clean up volumes

Delete the Redis and RabbiMQ volumes to force a clean setup again.

docker volume rm mayan_rabbitmq mayan_redis

Start the entire stack

docker compose up -d

Initialize the search system

docker compose run frontend run_command search_initialize

Check logs for fatal errors

docker compose logs -f

Migrate custom settings

  • Manually add custom settings from backed up config.yml file to the .env file.
  • Restart the stack after each round of updates.

Sorry for the radio silence, it’s been very busy last few weeks. I can report that I did have errors the first go around on a test server. I think it was error on my part. I plan on trying again next week or so with a cloned copy of the production server. I will report back with my findings.

Hello, I have followed all the steps.
However, I get the following error message when I do the step:

“docker compose up postgresql -d”.

nireins@ubedms2:~$ docker compose up postgresql -d
yaml: unmarshal errors:
line 59: mapping key “<<” already defined at line 58
line 60: mapping key “<<” already defined at line 58
line 60: mapping key “<<” already defined at line 59
line 133: mapping key “image” already defined at line 127
line 138: mapping key “restart” already defined at line 128
line 204: mapping key “<<” already defined at line 203
line 205: mapping key “<<” already defined at line 203
line 205: mapping key “<<” already defined at line 204

What am I doing wrong?

Translated with DeepL

This is a bug with newer versions of Docker Compose.

We are testing how to best support the old and new formats before making a release that workaround the issue.

Version 4.4.6 is out and includes an updated Docker Compose that fixes the issue:

https://docs.mayan-edms.com/releases/4.4.6.html

Hi,

so I converted my direct installation to a Docker based installation yesterday and stumbled over this:

that command threw an error that said service app is not running container #1. This command did start working once I did docker compose up -d, which is a later step.