[SOLVED] IMAP Source and Metadata types for Document Type Timeout

When things don't work as they should.
Post Reply
sna-cmarko
Posts: 10
Joined: Mon Dec 30, 2019 10:56 pm
Contact:

[SOLVED] IMAP Source and Metadata types for Document Type Timeout

Post by sna-cmarko »

Hello,

First, thank you so much for creating Mayan EDMS. I'm migrating from the last supported version of Knowledge Tree and it is almost a 1:1 move which simplifies the transition process!

I started testing this product with the docker 3.2.11 version. Everything was working and great. I had email sources working for different folders and was able to assign metadata types to document types. I did a test upgrade run from that version to 3.3.2. I didn't notice the timeout during that period, but I was focused on that data being there, so I could have missed the problem.

I recently setup a new box (Cent7) using the docker 3.3.6 version. I set up some metadata types. I tried to set one metadata type as "Required" for a document type and then set the rest as "Optional". When I hit save, I see the circle spinning in the top left corner and after 30-60 seconds I get a "Server Communication Error". I've checked the logs that I can find and am not coming up with anything to point me in the right direction. It only appears to time out if I make one of the document types required.

I also get a similar message when I run "Check Now" for the IMAP sources I have setup. They worked in the original version, but I see that there have been extra fields added since then, so it's possible that I am just missing the extra information. It appears that the messages are not being moved over like they were before, and again, I can't find an error log that would point me towards the culprit. The network is fine.
Error message after timeout
Error message after timeout
Screen Shot 2019-12-30 at 6.12.16 PM.png (145.51 KiB) Viewed 2059 times
Thanks in advance for any input!
User avatar
rssfed23
Moderator
Moderator
Posts: 213
Joined: Mon Oct 14, 2019 1:18 pm
Location: United Kingdom
Contact:

Re: IMAP Source and Metadata types for Document Type Timeout

Post by rssfed23 »

Hello.

Couple of things: you mention 3.3.2. Are you using RabbitMQ as the message broker? - There was a known issue fixed in 3.3.4 that caused timeout issues whenever a message needed to be sent/task run (https://gitlab.com/mayan-edms/mayan-edms/issues/699). I would strongly suggest giving 3.3.6 a try as there were some other upgrade improvements that happened between 3.3.2 and 3.3.6.

Additionally, did you change any of your environment variables when doing the upgrade? Some values have changed now. One that might be relevant here is the change of MAYAN_BROKER_URL to MAYAN_CELERY_BROKER_URL. If you're still using MAYAN_BROKER_URL then tasks again won't run and there'll be connection errors in the logs.

If you've not checked them out already I do encourage everyone to read the release notes (https://docs.mayan-edms.com/releases/3.3.html) - if you start with 3.3.0 and then read all the subsequent ones to 3.3.6 then you can make sure any settings that need to be updated are changed before running the upgrade. The upgrade to 3.3 isn't a standard "docker pull" exercise due to the changes in environment variables/supervisor configuration required. Settings inside your config.yml will be transitioned automatically but environment variables won't be.

Additionally, as you're using docker if you're using the standard Redis configuration all you need to do is ensure Mayan is pointed to the correct database you're using today and volumes for the media and document storage and use a fresh docker-compose configuration without fear of data loss (ultimately as long as the database and document_storage folders are okay it's usually always possible to recover an installation with no document loss. Mayan is very forgiving in that sense :) )
If you look at https://gitlab.com/mayan-edms/mayan-edm ... ompose.yml you'll see the most up to date default compose and be able to see if there's any changes in there from the version you're using.

If you've not updated your compose file at all as part of your upgrade then it's possible there's no redis running as this was split out into a seperate container that has to be deployed manually in 3.3 (in 3.2.11 redis was bundled alongside Mayan but now you need to deploy it as a separate container). Not having redis up could also cause the issues you're seeing.

It's also possible you're hitting one of the bugs fixed between 3.3.0 and 3.3.6, so if you could give 3.3.6 a try after reading through the release notes and report back we can do some more diagnostics then (please also let us know exactly what upgrade steps you followed and changes you made so we can figure out how best to go forward :) ).

Rob
Please bear with us during the current global situation. The team all have families and local communities to look after as well as the community here. Responses may be delayed during this time, but rest assured we will get to your query eventually.
sna-cmarko
Posts: 10
Joined: Mon Dec 30, 2019 10:56 pm
Contact:

Re: IMAP Source and Metadata types for Document Type Timeout

Post by sna-cmarko »

Hi Rob,

Thanks for getting back to me. I mentioned in my original post that I setup a brand new CentOS 7 box and a clean install of the latest version 3.3.6 which is where I was experiencing these issues. I didn't migrate any data. I setup the Redis box and the Postgres as instructed by the installation docs and made sure they were running.

Caitlyn
---
Caitlyn Marko
Sine Nomine Associates
www.sinenomine.net
User avatar
rssfed23
Moderator
Moderator
Posts: 213
Joined: Mon Oct 14, 2019 1:18 pm
Location: United Kingdom
Contact:

Re: IMAP Source and Metadata types for Document Type Timeout

Post by rssfed23 »

Hi Caitlyn.

Apologies that I missed that!

We really need to try and get some logs so we can look into it. Are you using docker-compose or just a docker run to start Mayan?
If docker compose, run docker-compose down then docker-compose up and logs should be printed out to the screen for us to capture.
Docker logs -f containername will do it if using standard docker.

What we're looking for is either errors in the startup of mayan or output from the task queues when you click the UI.

What also would be handy is if you ssh'd into the running container (docker exec -ti docker_app bash) where docker_app is the name of the mayan container and then run "env" and paste here what's shown.
The command you're using to start mayan (or your docker-compose file) would also be helpful. I've just done a fresh docker-compose deployment on 3.3.7 so this has to relate to an environment or configuration value somewhere.

It's also possible the documentation you were following was for 3.3.6 but the container deployed was 3.3.7 as it was released around the same time as your post and some changes were made to the container setup (a password for redis was added to the redis container). You don't need to delete the environment but if you could try following it again for 3.3.7 (https://docs.mayan-edms.com/chapters/do ... imple.html) just to be on the safe side and see what logs we get that would be helpful as we can't investigate much with the detail in the thread so far :).

Rob
Please bear with us during the current global situation. The team all have families and local communities to look after as well as the community here. Responses may be delayed during this time, but rest assured we will get to your query eventually.
sna-cmarko
Posts: 10
Joined: Mon Dec 30, 2019 10:56 pm
Contact:

Re: IMAP Source and Metadata types for Document Type Timeout

Post by sna-cmarko »

Looks like I might have missed the updates. I think I caught 3.3.6 when it was first released.

Code: Select all

8b8248b88bf0        mayanedms/mayanedms:latest   2 weeks ago         Up 2 weeks          0.0.0.0:80->8000/tcp     mayan-edms
This is the first error I see in the logs that happened during start up.

Code: Select all

[2019-12-19 16:20:29,437: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 111] Connection refused.
Trying again in 2.00 seconds...

[2019-12-19 16:20:31,452: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 111] Connection refused.
Trying again in 4.00 seconds...
I tried to set a required metadata type on a document type to get a recent entry in the logs. I noticed that the database is set to SQLite, but I should be using Postgres(again I followed the simple Docker install and just kept with a local Postgres install.) Could that be causing the timeout issue?

Code: Select all

[2020-01-02 19:54:30 +0000] [32] [CRITICAL] WORKER TIMEOUT (pid:1914)
[2020-01-02 19:54:30 +0000] [1914] [INFO] Worker exiting (pid: 1914)
[2020-01-02 19:54:30 +0000] [1934] [INFO] Booting worker with pid: 1934
/opt/mayan-edms/lib/python3.7/site-packages/mayan/apps/common/apps.py:93: DatabaseWarning: Your database backend is set to use SQLite. SQLite should only be used for development and testing, not for production.
  message=force_text(MESSAGE_SQLITE_WARNING)
I'm going to go ahead with the upgrade to 3.3.7 by following the docs. I'll post new logs once I have them.
---
Caitlyn Marko
Sine Nomine Associates
www.sinenomine.net
User avatar
rssfed23
Moderator
Moderator
Posts: 213
Joined: Mon Oct 14, 2019 1:18 pm
Location: United Kingdom
Contact:

Re: IMAP Source and Metadata types for Document Type Timeout

Post by rssfed23 »

Thanks for the logs! - we know the root cause now. Mayan is starting but it isn't connecting to your redis instance or your postgres database instance

Both of those are caused by mayan not having the correct environment variables set (or not being set at all) for redis (which is why it's trying to talk to rabbitmq instead) or the database.

When running Mayan (example below) it's important to pass in the -e variables for MAYAN_CELERY_BROKER_URL and the MAYAN_DATABASES value (after doing the docker run for postgres and redis from the documentation step 5 + 6 https://docs.mayan-edms.com/chapters/do ... imple.html).

If you run into issues again please paste the commands you're using to start Mayan (and check the link to the docs above is the same one your using as there are still some old cached links on the forum that may send you to an older version of the docs with incorrect commands).

docker run \
-d \
--name mayan-edms \
--restart=always \
-p 80:8000 \
-e MAYAN_DATABASES="{'default':{'ENGINE':'django.db.backends.postgresql','NAME':'mayan','PASSWORD':'mayanuserpass','USER':'mayan','HOST':'172.17.0.1'}}" \
-e MAYAN_CELERY_BROKER_URL="redis://:mayanredispassword@172.17.0.1:6379/0" \
-e MAYAN_CELERY_RESULT_BACKEND="redis://:mayanredispassword@172.17.0.1:6379/1" \
-v /docker-volumes/mayan-edms/media:/var/lib/mayan \
mayanedms/mayanedms:3.3.7
Please bear with us during the current global situation. The team all have families and local communities to look after as well as the community here. Responses may be delayed during this time, but rest assured we will get to your query eventually.
sna-cmarko
Posts: 10
Joined: Mon Dec 30, 2019 10:56 pm
Contact:

Re: IMAP Source and Metadata types for Document Type Timeout

Post by sna-cmarko »

Hi Rob,

I pulled down 3.3.7 and walked through the installation. It appears though that Postgres is having a hard time connecting? Log below:

Code: Select all

docker logs -f mayan-edms
mayan: starting entrypoint.sh
mayan: update_uid_gid()
mayan: os_package_installs()
mayan: pip_installs()
mayan: performupgrade()
JSONField implementation is: <class 'django.db.models.fields.TextField'>
CommandError: Error during pre_upgrade signal: could not connect to server: No route to host
	Is the server running on host "172.17.0.1" and accepting
	TCP/IP connections on port 5432?
, <class 'django.db.utils.OperationalError'>
mayan: starting entrypoint.sh
mayan: update_uid_gid()
mayan: os_package_installs()
mayan: pip_installs()
mayan: performupgrade()
JSONField implementation is: <class 'django.db.models.fields.TextField'>
CommandError: Error during pre_upgrade signal: could not connect to server: No route to host
	Is the server running on host "172.17.0.1" and accepting
	TCP/IP connections on port 5432?
, <class 'django.db.utils.OperationalError'>
Caitlyn
---
Caitlyn Marko
Sine Nomine Associates
www.sinenomine.net
sna-cmarko
Posts: 10
Joined: Mon Dec 30, 2019 10:56 pm
Contact:

Re: IMAP Source and Metadata types for Document Type Timeout

Post by sna-cmarko »

I believe that I found my answer.

Thanks so much, Rob for your help. I'll considered this solved and start a different forum post if the original errors reappear!

Caitlyn
---
Caitlyn Marko
Sine Nomine Associates
www.sinenomine.net
User avatar
rssfed23
Moderator
Moderator
Posts: 213
Joined: Mon Oct 14, 2019 1:18 pm
Location: United Kingdom
Contact:

Re: IMAP Source and Metadata types for Document Type Timeout

Post by rssfed23 »

Glad things worked out in the end! - Don't hesitate to reach out again if you run into any issues, questions or help with a production deployment :)
Please bear with us during the current global situation. The team all have families and local communities to look after as well as the community here. Responses may be delayed during this time, but rest assured we will get to your query eventually.
Post Reply