Direct deployment to Docker Compose migration

Thanks to help from @DocCyblade the following procedure was created.

Please try this procedure on a backup copy of your installations (or even better a VM snapshot) and report your results (good or bad) so we can finalize this and add it to the documentation.

Update the direct deployment to the 4.4 series.

Follow the instructions located in: https://docs.mayan-edms.com/parts/upgrades.html

Purge all file caches.

  • Option 1: Via the user interface. SystemToolsFile caches

Refresh the page until all current size columns display 0.

  • Option 2: Via the command line.
echo "from mayan.apps.file_caching.models import CachePartitionFile;[i.delete() for i in CachePartitionFile.objects.all()]" | sudo --user=mayan MAYAN_MEDIA_ROOT=/opt/mayan-edms/media/ /opt/mayan-edms/bin/mayan-edms.py shell

Create the backup folder

mkdir ~/mayan-direct-backup

Stop the installation

sudo systemctl stop supervisor

Dump the database

IMPORTANT! Make a note of the database credentials.

Use the database credentials from the previous step.

pg_dump -U mayan -h 127.0.0.1 -d mayan -W > ~/mayan-direct-backup/database.sql

Copy the config.yml

cp /opt/mayan-edms/media/config.yml ~/mayan-direct-backup

Copy the SECRET_KEY

cp /opt/mayan-edms/media/SECRET_KEY ~/mayan-direct-backup

Copy the document files

Use rsync to preserve the file attributes and ownership.

rsync -a /opt/mayan-edms/media/document_file_storage /home/ubuntu/mayan-direct-backup/

Stop and disable services

sudo systemctl stop supervisor  # Should be already stopped
sudo systemctl disable supervisor
sudo systemctl stop postgresql
sudo systemctl disable postgresql
sudo systemctl stop redis
sudo systemctl disable redis
# RabbitMQ if installed
sudo systemctl stop rabbitmq
sudo systemctl disable rabbitmq

Install Mayan EDMS via Docker Compose

  • Install Docker.
  • Just download the Mayan Docker Compose files.
  • Do not run docker compose up -d yet!
curl https://gitlab.com/mayan-edms/mayan-edms/-/raw/master/docker/docker-compose.yml -O
curl https://gitlab.com/mayan-edms/mayan-edms/-/raw/master/docker/.env -O

Restore the database

  • Update the new database credentials in the .env file if desired.

  • Start only PostgreSQL in Docker.
docker compose up postgresql -d
  • Restore the database dump. Replace the database name, user as needed.
docker compose exec -T postgresql psql -U mayan -d mayan < ~/mayan-direct-backup/database.sql
  • Inspect the Mayan app volume. Take note of the Mountpoint value.
docker volume inspect mayan_app

Restore the document files

Restore the document_file_storage folder.

sudo rsync -a /home/ubuntu/mayan-direct-backup/document_file_storage /var/lib/docker/volumes/mayan_app/_data/
  • Verify correct ownership of restored files. All files should be owner by mayan:mayan.
docker compose exec -ti app ls /var/lib/mayan -al

Shutdown the entire stack

docker compose down

Clean up volumes

Delete the Redis and RabbiMQ volumes to force a clean setup again.

docker volume rm mayan_rabbitmq mayan_redis

Start the entire stack

docker compose up -d

Initialize the search system

docker compose run frontend run_command search_initialize

Check logs for fatal errors

docker compose logs -f

Migrate custom settings

  • Manually add custom settings from backed up config.yml file to the .env file.
  • Restart the stack after each round of updates.

Sorry for the radio silence, it’s been very busy last few weeks. I can report that I did have errors the first go around on a test server. I think it was error on my part. I plan on trying again next week or so with a cloned copy of the production server. I will report back with my findings.

Hello, I have followed all the steps.
However, I get the following error message when I do the step:

“docker compose up postgresql -d”.

nireins@ubedms2:~$ docker compose up postgresql -d
yaml: unmarshal errors:
line 59: mapping key “<<” already defined at line 58
line 60: mapping key “<<” already defined at line 58
line 60: mapping key “<<” already defined at line 59
line 133: mapping key “image” already defined at line 127
line 138: mapping key “restart” already defined at line 128
line 204: mapping key “<<” already defined at line 203
line 205: mapping key “<<” already defined at line 203
line 205: mapping key “<<” already defined at line 204

What am I doing wrong?

Translated with DeepL

This is a bug with newer versions of Docker Compose.

We are testing how to best support the old and new formats before making a release that workaround the issue.

Version 4.4.6 is out and includes an updated Docker Compose that fixes the issue:

https://docs.mayan-edms.com/releases/4.4.6.html

Hi,

so I converted my direct installation to a Docker based installation yesterday and stumbled over this:

that command threw an error that said service app is not running container #1. This command did start working once I did docker compose up -d, which is a later step.

Hello all,

is this guide now safe?

My installation of MAYAN EDMS is still running as direct deploymend on Ubuntu 22.04.03 as version 4.4.8.

A manual upgrade to 4.5.1 unfortunately fails with the following error messages:

mayan.apps.backends.model_mixins <1877> [ERROR] “get_backend_class() line 50 ImportError while importing backend: mayan.apps.sources.source_backends.SourceBackendWatchFolder; Module “mayan.apps.sources.source_backends” does not define a “SourceBackendWatchFolder” attribute/class”
mayan.apps.backends.model_mixins <1877> [ERROR] “get_backend_class() line 50 ImportError while importing backend: mayan.apps.sources.source_backends.SourceBackendWatchFolder; Module “mayan.apps.sources.source_backends” does not define a “SourceBackendWatchFolder” attribute/class”
mayan.apps.backends.model_mixins <1877> [ERROR] “get_backend_class() line 50 ImportError while importing backend: mayan.apps.sources.source_backends.SourceBackendWatchFolder; Module “mayan.apps.sources.source_backends” does not define a “SourceBackendWatchFolder” attribute/class”
mayan.apps.backends.model_mixins <1877> [ERROR] “get_backend_class() line 50 ImportError while importing backend: mayan.apps.sources.source_backends.SourceBackendWatchFolder; Module “mayan.apps.sources.source_backends” does not define a “SourceBackendWatchFolder” attribute/class”
mayan.apps.backends.model_mixins <1877> [ERROR] “get_backend_class() line 50 ImportError while importing backend: mayan.apps.sources.source_backends.SourceBackendWatchFolder; Module “mayan.apps.sources.source_backends” does not define a “SourceBackendWatchFolder” attribute/class”
mayan.apps.backends.model_mixins <1877> [ERROR] “get_backend_class() line 50 ImportError while importing backend: mayan.apps.sources.source_backends.SourceBackendWatchFolder; Module “mayan.apps.sources.source_backends” does not define a “SourceBackendWatchFolder” attribute/class”
mayan.apps.backends.model_mixins <1877> [ERROR] “get_backend_class() line 50 ImportError while importing backend: mayan.apps.sources.source_backends.SourceBackendWatchFolder; Module “mayan.apps.sources.source_backends” does not define a “SourceBackendWatchFolder” attribute/class”
mayan.apps.backends.model_mixins <1877> [ERROR] “get_backend_class() line 50 ImportError while importing backend: mayan.apps.sources.source_backends.SourceBackendWebForm; Module “mayan.apps.sources.source_backends” does not define a “SourceBackendWebForm” attribute/class”

Am I perhaps doing something wrong?

I use the following commands to perform a manual upgrade:

sudo systemctl stop supervisor
sudo cp /etc/supervisor/conf.d/mayan-edms.conf /etc/supervisor/conf.d/mayan-edms.conf.bck
sudo --user=mayan /opt/mayan-edms/bin/pip install --upgrade pip
sudo --user=mayan curl https://gitlab.com/mayan-edms/mayan-edms/raw/master/removals.txt --output /tmp/removals.txt && sudo --user=mayan /opt/mayan-edms/bin/pip uninstall --requirement /tmp/removals.txt --yes
sudo --user=mayan /opt/mayan-edms/bin/pip install mayan-edms==4.5.1
sudo --user=mayan MAYAN_MEDIA_ROOT=/opt/mayan-edms/media/ /opt/mayan-edms/bin/mayan-edms.py platform_template supervisord | sudo sh -c “cat > /etc/supervisor/conf.d/mayan-edms.conf”
sudo --user=mayan MAYAN_MEDIA_ROOT=/opt/mayan-edms/media/ /opt/mayan-edms/bin/mayan-edms.py common_perform_upgrade
sudo systemctl start supervisor
sudo reboot

Would be great if I can safely migrate to Docker Compose.

Greetings

[EDIT 18.09.2023 07:14 CET] Correction: I just tried again with the already existing instructions and it worked. :slight_smile: MAYAN EDMS is now running as Docker container as version 4.5.1 rejoice.

**After the migration I had to do - as announced - a few settings manually (for example the watch folders) **

Now I hope that with the following versions of MAYAN EDMS the upgrade process will be easier :slight_smile:

1 Like

Hello all,

a new release, a new attempt…

I tried again to migrate from direct deployment 4.4.8 to Docker Compose with version 4.5.4.

After carefully entering all the commands and making all the adjustments, I can reach the web interface, but certain areas look different and I don’t know if this is correct.

There also seems to be something wrong with the watch folders and I don’t know where the error is.

Does anyone want to take a look at this?



Mayan EDMS
(SNIERS EDMS]
why is the project name in brackets now?

The function “Check for updates” has changed?

For the watch folder I see “null backend” - why? I can’t change or delete the entry.

the look of the webinterface can changed in Settings
COMMON_COLLAPSE_LIST_MENU_LIST_FACET
and
COMMON_COLLAPSE_LIST_MENU_OBJECT
this was on old default value

for the problem with the watch folder: this seems to be an old installation, the settings are not migrated, delete the sources and recreate them from scratch

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.