Migrate from direct deployment to docker

Bastian · February 3, 2023, 10:37am

Hi @all

As the direct deployment not being supported any longer it would be great to have a documentation/guide on how to migrate an existing direct deployment installation to a docker based installation.

In the official documentation I found nothing regarding this topic.

Best regards
Bastian

roberto.rosario · February 3, 2023, 10:53am

Hi,

The migration process changes depending on how the direct install varies. It would not be possible to have a single migration process that works for all direct deployments.

A rough procedure would be:

Stop all sources.
Logout all users.
Wait until all background tasks queues are empty (search indexing, document indexing, etc).
Purge all caches.
Stop Mayan, but leave the database running.
Dump the database to SQL (using pg_dump).
Copy the document file storage files.
Install using the Docker Compose process.
Migrate settings from Supervisord to environment variables in the .env file.
Stop the Docker Compose installation leaving only the database container running.
Restore the SQL dump to the running database container.
Copy the document file storage files to the Mayan EDMS app container volume.
Change the ownership of the volume files.
Start up the entire deployment and verify functionality.

If there are community members willing to test and help debug this process in a repeatable and testable manner, it can be added to the documentation.

Bastian · February 3, 2023, 10:56am

thanks for your quick reply.

I have a direct deployment on an ubuntu lxc container on proxmox.
If the procedure works as written by you, I can supply more informations or needed steps.
Then it can be added to the documentation (if others have also tested it successful)

Best regards
Bastian

Bastian · February 3, 2023, 1:52pm

I’m currently trying the migration process.
Would it make sense to migrate to the same version of Mayan (I’m on 4.3.1) can I directly migrate my installation to the latest 4.4.2 version?

Best regards
Bastian

Bastian · February 3, 2023, 4:37pm

I just migrated my deployment to docker and ran into several issues.

First is a missing java runtime when creating previews using libreoffice

I added:

MAYAN_APT_INSTALLS="tesseract-ocr-deu tesseract-ocr-nld openjdk-17-jdk openjdk-17-jdk-headless openjdk-17-jre openjdk-17-jre-headless"

after rebuilding the container with java installed, while generating previews, the log throws the following exceptions:

mayan-app-1         | [2023-02-03 16:27:31,028: ERROR/ForkPoolWorker-2] Error creating document version page cache file named "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-base_image";
mayan-app-1         |
mayan-app-1         |   RAN: /usr/bin/libreoffice --headless --convert-to pdf:writer_pdf_Export /tmp/tmpvtqk63mn --outdir /tmp -env:UserInstallation=file:///tmp/tmpmlzlqcvv/LibreOffice_Conversion
mayan-app-1         |
mayan-app-1         |   STDOUT:
mayan-app-1         |
mayan-app-1         |
mayan-app-1         |   STDERR:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_version_page_model_mixins.py", line 72, in generate_image
mayan-app-1         |     self.cache_partition.get_file(
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/file_caching/model_mixins.py", line 247, in get_file
mayan-app-1         |     return self.files.get(filename=filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
mayan-app-1         |     return getattr(self.get_queryset(), name)(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
mayan-app-1         |     raise self.model.DoesNotExist(
mayan-app-1         | mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist.
mayan-app-1         |
mayan-app-1         | During handling of the above exception, another exception occurred:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_version_page_model_mixins.py", line 214, in get_image
mayan-app-1         |     cache_file = self.cache_partition.get_file(
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/file_caching/model_mixins.py", line 247, in get_file
mayan-app-1         |     return self.files.get(filename=filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
mayan-app-1         |     return getattr(self.get_queryset(), name)(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
mayan-app-1         |     raise self.model.DoesNotExist(
mayan-app-1         | mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist.
mayan-app-1         |
mayan-app-1         | During handling of the above exception, another exception occurred:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_file_page_model_mixins.py", line 63, in generate_image
mayan-app-1         |     self.cache_partition.get_file(
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/file_caching/model_mixins.py", line 247, in get_file
mayan-app-1         |     return self.files.get(filename=filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
mayan-app-1         |     return getattr(self.get_queryset(), name)(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
mayan-app-1         |     raise self.model.DoesNotExist(
mayan-app-1         | mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist.
mayan-app-1         |
mayan-app-1         | During handling of the above exception, another exception occurred:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_file_page_model_mixins.py", line 173, in get_image
mayan-app-1         |     cache_file = self.cache_partition.get_file(filename=cache_filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/file_caching/model_mixins.py", line 247, in get_file
mayan-app-1         |     return self.files.get(filename=filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
mayan-app-1         |     return getattr(self.get_queryset(), name)(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
mayan-app-1         |     raise self.model.DoesNotExist(
mayan-app-1         | mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist.
mayan-app-1         |
mayan-app-1         | During handling of the above exception, another exception occurred:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_file_model_mixins.py", line 260, in get_intermediate_file
mayan-app-1         |     cache_file = self.cache_partition.get_file(filename=cache_filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/file_caching/model_mixins.py", line 247, in get_file
mayan-app-1         |     return self.files.get(filename=filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
mayan-app-1         |     return getattr(self.get_queryset(), name)(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
mayan-app-1         |     raise self.model.DoesNotExist(
mayan-app-1         | mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist.
mayan-app-1         |
mayan-app-1         | During handling of the above exception, another exception occurred:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/converter/classes.py", line 196, in soffice
mayan-app-1         |     self.command_libreoffice(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/sh.py", line 1566, in __call__
mayan-app-1         |     return RunningCommand(cmd, call_args, stdin, stdout, stderr)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/sh.py", line 822, in __init__
mayan-app-1         |     self.wait()
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/sh.py", line 879, in wait
mayan-app-1         |     self.handle_command_exit_code(exit_code)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/sh.py", line 905, in handle_command_exit_code
mayan-app-1         |     raise exc
mayan-app-1         | sh.ErrorReturnCode_1:
mayan-app-1         |
mayan-app-1         |   RAN: /usr/bin/libreoffice --headless --convert-to pdf:writer_pdf_Export /tmp/tmpvtqk63mn --outdir /tmp -env:UserInstallation=file:///tmp/tmpmlzlqcvv/LibreOffice_Conversion
mayan-app-1         |
mayan-app-1         |   STDOUT:
mayan-app-1         |
mayan-app-1         |
mayan-app-1         |   STDERR:
mayan-app-1         |
mayan-app-1         |
mayan-app-1         | During handling of the above exception, another exception occurred:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_version_page_model_mixins.py", line 221, in get_image
mayan-app-1         |     content_object_cache_filename = self.content_object.generate_image(
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_file_page_model_mixins.py", line 70, in generate_image
mayan-app-1         |     image = self.get_image(
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_file_page_model_mixins.py", line 178, in get_image
mayan-app-1         |     with self.document_file.get_intermediate_file() as file_object:
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_file_model_mixins.py", line 291, in get_intermediate_file
mayan-app-1         |     raise exception
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_file_model_mixins.py", line 269, in get_intermediate_file
mayan-app-1         |     with converter.to_pdf() as pdf_file_object:
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/converter/classes.py", line 264, in to_pdf
mayan-app-1         |     return self.soffice()
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/converter/classes.py", line 199, in soffice
mayan-app-1         |     raise OfficeConversionError(exception)
mayan-app-1         | mayan.apps.converter.exceptions.OfficeConversionError:
mayan-app-1         |
mayan-app-1         |   RAN: /usr/bin/libreoffice --headless --convert-to pdf:writer_pdf_Export /tmp/tmpvtqk63mn --outdir /tmp -env:UserInstallation=file:///tmp/tmpmlzlqcvv/LibreOffice_Conversion
mayan-app-1         |
mayan-app-1         |   STDOUT:
mayan-app-1         |
mayan-app-1         |
mayan-app-1         |   STDERR:
mayan-app-1         |
mayan-app-1         | "
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_version_page_model_mixins.py", line 72, in generate_image
mayan-app-1         |     self.cache_partition.get_file(
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/file_caching/model_mixins.py", line 247, in get_file
mayan-app-1         |     return self.files.get(filename=filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
mayan-app-1         |     return getattr(self.get_queryset(), name)(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
mayan-app-1         |     raise self.model.DoesNotExist(
mayan-app-1         | mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist.
mayan-app-1         |
mayan-app-1         | During handling of the above exception, another exception occurred:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_version_page_model_mixins.py", line 214, in get_image
mayan-app-1         |     cache_file = self.cache_partition.get_file(
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/file_caching/model_mixins.py", line 247, in get_file
mayan-app-1         |     return self.files.get(filename=filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
mayan-app-1         |     return getattr(self.get_queryset(), name)(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
mayan-app-1         |     raise self.model.DoesNotExist(
mayan-app-1         | mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist.
mayan-app-1         |
mayan-app-1         | During handling of the above exception, another exception occurred:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_file_page_model_mixins.py", line 63, in generate_image
mayan-app-1         |     self.cache_partition.get_file(
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/file_caching/model_mixins.py", line 247, in get_file
mayan-app-1         |     return self.files.get(filename=filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
mayan-app-1         |     return getattr(self.get_queryset(), name)(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
mayan-app-1         |     raise self.model.DoesNotExist(
mayan-app-1         | mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist.
mayan-app-1         |
mayan-app-1         | During handling of the above exception, another exception occurred:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_file_page_model_mixins.py", line 173, in get_image
mayan-app-1         |     cache_file = self.cache_partition.get_file(filename=cache_filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/file_caching/model_mixins.py", line 247, in get_file
mayan-app-1         |     return self.files.get(filename=filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
mayan-app-1         |     return getattr(self.get_queryset(), name)(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
mayan-app-1         |     raise self.model.DoesNotExist(
mayan-app-1         | mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist.
mayan-app-1         |
mayan-app-1         | During handling of the above exception, another exception occurred:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_file_model_mixins.py", line 260, in get_intermediate_file
mayan-app-1         |     cache_file = self.cache_partition.get_file(filename=cache_filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/file_caching/model_mixins.py", line 247, in get_file
mayan-app-1         |     return self.files.get(filename=filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
mayan-app-1         |     return getattr(self.get_queryset(), name)(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
mayan-app-1         |     raise self.model.DoesNotExist(
mayan-app-1         | mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist.
mayan-app-1         |
mayan-app-1         | During handling of the above exception, another exception occurred:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/converter/classes.py", line 196, in soffice
mayan-app-1         |     self.command_libreoffice(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/sh.py", line 1566, in __call__
mayan-app-1         |     return RunningCommand(cmd, call_args, stdin, stdout, stderr)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/sh.py", line 822, in __init__
mayan-app-1         |     self.wait()
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/sh.py", line 879, in wait
mayan-app-1         |     self.handle_command_exit_code(exit_code)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/sh.py", line 905, in handle_command_exit_code
mayan-app-1         |     raise exc
mayan-app-1         | sh.ErrorReturnCode_1:
mayan-app-1         |
mayan-app-1         |   RAN: /usr/bin/libreoffice --headless --convert-to pdf:writer_pdf_Export /tmp/tmpqfuyjqen --outdir /tmp -env:UserInstallation=file:///tmp/tmpu5pd9h7r/LibreOffice_Conversion
mayan-app-1         |
mayan-app-1         |   STDOUT:
mayan-app-1         |
mayan-app-1         |
mayan-app-1         |   STDERR:
mayan-app-1         |
mayan-app-1         |
mayan-app-1         | During handling of the above exception, another exception occurred:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_version_page_model_mixins.py", line 221, in get_image
mayan-app-1         |     content_object_cache_filename = self.content_object.generate_image(
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_file_page_model_mixins.py", line 70, in generate_image
mayan-app-1         |     image = self.get_image(
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_file_page_model_mixins.py", line 178, in get_image
mayan-app-1         |     with self.document_file.get_intermediate_file() as file_object:
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_file_model_mixins.py", line 291, in get_intermediate_file
mayan-app-1         |     raise exception
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_file_model_mixins.py", line 269, in get_intermediate_file
mayan-app-1         |     with converter.to_pdf() as pdf_file_object:
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/converter/classes.py", line 264, in to_pdf
mayan-app-1         |     return self.soffice()
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/converter/classes.py", line 199, in soffice
mayan-app-1         |     raise OfficeConversionError(exception)
mayan-app-1         | mayan.apps.converter.exceptions.OfficeConversionError:
mayan-app-1         |
mayan-app-1         |   RAN: /usr/bin/libreoffice --headless --convert-to pdf:writer_pdf_Export /tmp/tmpqfuyjqen --outdir /tmp -env:UserInstallation=file:///tmp/tmpu5pd9h7r/LibreOffice_Conversion
mayan-app-1         |
mayan-app-1         |   STDOUT:
mayan-app-1         |
mayan-app-1         |
mayan-app-1         |   STDERR:
mayan-app-1         |

any help is appreciated

Best regards
Bastian

roberto.rosario · February 4, 2023, 10:51am

What is the error message? The Docker image includes all dependencies to allow LibreOffice to run.

I added:
openjdk-17-jdk openjdk-17-jdk-headless openjdk-17-jre openjdk-17-jre-headless

The Mayan and Docker image are created and tested in each release with a specific set of dependencies. When debugging an issue keep the original dependencies to avoid adding additional artifacts.

mayan-app-1         | mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist.

Left over invalid references. This means one of these step did not complete correctly before the migration:

Stop all sources.
Logout all users.
Wait until all background tasks queues are empty (search indexing, document indexing, etc).
Purge all caches.

Bastian · February 5, 2023, 7:16am

Hi and thanks for your reply.

Stop all sources.

what exactly do you mean by stopping all sources?
I found sources in the Settings menu.
There’s only the “Default” one for the Webform. Do I need to disable this?

Logout all users.

my admin user was the only one in the system for clearing the caches.
Shall I close this session before exporting the database? Or would it be best to restart the system once before exporting the database?

Wait until all background tasks queues are empty (search indexing, document indexing, etc).

is there a way to check this in the UI when the workers/indexers are idle?

Purge all caches.

I purged the file caches using the system → tools → File Caches view
in there I deleted all caches listed before exporting the data.
Are there any other caches to clear?

Thanks for your support!

Best regards
Bastian

Bastian · February 6, 2023, 7:33am

I started the container again without the java runtime and get this errors:

mayan-app-1         | [2023-02-06 07:32:09,663: ERROR/ForkPoolWorker-4] Error creating document version page cache file named "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-base_image";
mayan-app-1         |
mayan-app-1         |   RAN: /usr/bin/libreoffice --headless --convert-to pdf:writer_pdf_Export /tmp/tmp4lb3_ja_ --outdir /tmp -env:UserInstallation=file:///tmp/tmp_o5sfira/LibreOffice_Conversion
mayan-app-1         |
mayan-app-1         |   STDOUT:
mayan-app-1         |
mayan-app-1         |
mayan-app-1         |   STDERR:
mayan-app-1         | javaldx: Could not find a Java Runtime Environment!
mayan-app-1         | Please ensure that a JVM and the package libreoffice-java-common
mayan-app-1         | is installed.
mayan-app-1         | If it is already installed then try removing ~/.config/libreoffice/4/user/config/javasettings_Linux_*.xml
mayan-app-1         | Warning: failed to read path from javaldx
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_version_page_model_mixins.py", line 72, in generate_image
mayan-app-1         |     self.cache_partition.get_file(
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/file_caching/model_mixins.py", line 247, in get_file
mayan-app-1         |     return self.files.get(filename=filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
mayan-app-1         |     return getattr(self.get_queryset(), name)(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
mayan-app-1         |     raise self.model.DoesNotExist(
mayan-app-1         | mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist.
mayan-app-1         |
mayan-app-1         | During handling of the above exception, another exception occurred:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_version_page_model_mixins.py", line 214, in get_image
mayan-app-1         |     cache_file = self.cache_partition.get_file(
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/file_caching/model_mixins.py", line 247, in get_file
mayan-app-1         |     return self.files.get(filename=filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
mayan-app-1         |     return getattr(self.get_queryset(), name)(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
mayan-app-1         |     raise self.model.DoesNotExist(
mayan-app-1         | mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist.
mayan-app-1         |
mayan-app-1         | During handling of the above exception, another exception occurred:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_file_page_model_mixins.py", line 63, in generate_image
mayan-app-1         |     self.cache_partition.get_file(
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/file_caching/model_mixins.py", line 247, in get_file
mayan-app-1         |     return self.files.get(filename=filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
mayan-app-1         |     return getattr(self.get_queryset(), name)(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
mayan-app-1         |     raise self.model.DoesNotExist(
mayan-app-1         | mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist.
mayan-app-1         |
mayan-app-1         | During handling of the above exception, another exception occurred:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_file_page_model_mixins.py", line 173, in get_image
mayan-app-1         |     cache_file = self.cache_partition.get_file(filename=cache_filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/file_caching/model_mixins.py", line 247, in get_file
mayan-app-1         |     return self.files.get(filename=filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
mayan-app-1         |     return getattr(self.get_queryset(), name)(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
mayan-app-1         |     raise self.model.DoesNotExist(
mayan-app-1         | mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist.
mayan-app-1         |
mayan-app-1         | During handling of the above exception, another exception occurred:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/documents/models/document_file_model_mixins.py", line 260, in get_intermediate_file
mayan-app-1         |     cache_file = self.cache_partition.get_file(filename=cache_filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/file_caching/model_mixins.py", line 247, in get_file
mayan-app-1         |     return self.files.get(filename=filename)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
mayan-app-1         |     return getattr(self.get_queryset(), name)(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
mayan-app-1         |     raise self.model.DoesNotExist(
mayan-app-1         | mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist.
mayan-app-1         |
mayan-app-1         | During handling of the above exception, another exception occurred:
mayan-app-1         |
mayan-app-1         | Traceback (most recent call last):
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/mayan/apps/converter/classes.py", line 196, in soffice
mayan-app-1         |     self.command_libreoffice(*args, **kwargs)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/sh.py", line 1566, in __call__
mayan-app-1         |     return RunningCommand(cmd, call_args, stdin, stdout, stderr)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/sh.py", line 822, in __init__
mayan-app-1         |     self.wait()
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/sh.py", line 879, in wait
mayan-app-1         |     self.handle_command_exit_code(exit_code)
mayan-app-1         |   File "/opt/mayan-edms/lib/python3.9/site-packages/sh.py", line 905, in handle_command_exit_code
mayan-app-1         |     raise exc
mayan-app-1         | sh.ErrorReturnCode_1:
mayan-app-1         |
mayan-app-1         |   RAN: /usr/bin/libreoffice --headless --convert-to pdf:writer_pdf_Export /tmp/tmp4lb3_ja_ --outdir /tmp -env:UserInstallation=file:///tmp/tmp_o5sfira/LibreOffice_Conversion
mayan-app-1         |
mayan-app-1         |   STDOUT:
mayan-app-1         |
mayan-app-1         |
mayan-app-1         |   STDERR:
mayan-app-1         | javaldx: Could not find a Java Runtime Environment!
mayan-app-1         | Please ensure that a JVM and the package libreoffice-java-common
mayan-app-1         | is installed.
mayan-app-1         | If it is already installed then try removing ~/.config/libreoffice/4/user/config/javasettings_Linux_*.xml
mayan-app-1         | Warning: failed to read path from javaldx
mayan-app-1         |

roberto.rosario · February 7, 2023, 4:18pm

Will need to setup a lab to try to replicate this because it does not make sense that the JVM is not found when it is already included in the Docker image.

The other thing that does not make sense are the invalid cache references you are seeing if the cache was purged before the migration.

Bastian · February 8, 2023, 7:08am

Thanks for your feedback.

I have deleted all the file caches showing up in the setting described above.
Then stopped the supervisor, made the databse backup and pushed the files to the new server.

Please let me know if I can supply any infos or data that would help the process of finding the issue.

Best regards
Bastian

Bastian · February 20, 2023, 8:23am

Did you already have the chance to look into this issue?
Is there another way e.g. from the command line to ensure that the caches are purged?

woec · February 28, 2023, 9:51am

Hi,
I tried to switch from direct deployment to docker (version 4.34).
Basicly it seems to work, but I do not get preview images of most of the documents.
There is no error message.
How to rebuild the previews?
Regards,
Johann

Blockquote

woec · February 28, 2023, 2:17pm

completion: after emptying the caches in the docker installation again, the previews are ok now.

Bastian · June 19, 2023, 10:55am

I finally managed to migrate my existing direct deployment version to docker.

I migrated to the same minor version (v4.3.x) and made an update afterwards to the latest v4 release (s4) without any issues.

It did not work to migrate directly to the same version 4.3.1. I don’t know why. It seems as if there were some issues inside the app container when migrating the data. I got some permission issues which were gone after migrating to the last 4.3 release.

Best regards
Bastian