Automatic document import from watch folder sources falls (sometimes)

[DrRSatzteil]

I run Mayan in docker and have a strange behaviour since a couple of months now (also in previous mayan versions)

I find a lot of errors like these in the logs (retrieved through docker-compose logs):

Code: Select all

mayan.apps.sources.tasks <391> [ERROR] "task_check_interval_source() line 47 Error processing source id: 8; "
app_1               | mayan.apps.sources.tasks <390> [ERROR] "task_check_interval_source() line 47 Error processing source id: 6; "
app_1               | [2021-02-09 13:07:35,799: ERROR/ForkPoolWorker-1] Error processing source id: 6; 
app_1               | [2021-02-09 13:07:35,798: ERROR/ForkPoolWorker-2] Error processing source id: 8;

However, most of the time the imports work fine anyway. Sometimes however the importer actually stops working and I need to restart the container to make it work again (even though still producing these errors). Unfortunately these errors don't give any hint on what is going wrong here and there exists no log file in /var/lib/Mayan/error.log.

Any hints on how to debug this issue?

[DrRSatzteil]

Oh I just found that on the first occasion of this error there is some more in the logs...


However it seems to be the same problem as I already described here: viewtopic.php?f=9&t=4027&p=7153#p7153

I thought there was a different problem but it does not necessarily seem to be the case.

It's still strange that:
1. There is no entry in the error.log
and
2. I see the stack trace only once (when the error occurs the first time after a container restart)

Anyway I would say this is a duplicate. Sorry for posting...

[rosarior]

Thanks for the report. We have tried replicating the issue without success. The error seems to occur in the Whoosh driver but could be itself caused by another source. We added more debug information to the error in hopes of obtaining more details: https://gitlab.com/mayan-edms/mayan-edm ... a7df046000

This might get ported to the 3.5 series too once we finish and release version 4.0.

[sacul]

Just wanted to chime in that I have the same errors in my logs, and I'm happy to help debug if that's helpful!

Code: Select all

mayan.apps.sources.tasks <56> [ERROR] "task_check_interval_source() line 47 Error
[2021-02-14 15:44:26,686: ERROR/ForkPoolWorker-3] Error processing source id: 23;
mayan.apps.sources.tasks <57> [ERROR] "task_check_interval_source() line 47 Error
[2021-02-14 15:44:26,740: ERROR/ForkPoolWorker-4] Error processing source id: 30;
mayan.apps.sources.tasks <56> [ERROR] "task_check_interval_source() line 47 Error
[2021-02-14 15:49:26,669: ERROR/ForkPoolWorker-3] Error processing source id: 23;
mayan.apps.sources.tasks <57> [ERROR] "task_check_interval_source() line 47 Error
[2021-02-14 15:49:26,686: ERROR/ForkPoolWorker-4] Error processing source id: 26;
mayan.apps.sources.tasks <56> [ERROR] "task_check_interval_source() line 47 Error
[2021-02-14 15:49:26,709: ERROR/ForkPoolWorker-3] Error processing source id: 29;
mayan.apps.sources.tasks <56> [ERROR] "task_check_interval_source() line 47 Error
[2021-02-14 15:49:26,748: ERROR/ForkPoolWorker-3] Error processing source id: 32;
mayan.apps.sources.tasks <57> [ERROR] "task_check_interval_source() line 47 Error
[2021-02-14 15:54:26,707: ERROR/ForkPoolWorker-4] Error processing source id: 26;
mayan.apps.sources.tasks <55> [ERROR] "task_check_interval_source() line 47 Error
[2021-02-14 15:59:26,681: ERROR/ForkPoolWorker-2] Error processing source id: 24;
mayan.apps.sources.tasks <54> [ERROR] "task_check_interval_source() line 47 Error
[2021-02-14 15:59:26,703: ERROR/ForkPoolWorker-1] Error processing source id: 25;

[DrRSatzteil]

I'm also willing to post some more debug information but would require some guidance regarding what parameters I would need to change to get some more verbose logging.

In case this could be somehow related to a special way my scanner creates PDF documents I could also send you a scanned document for testing. However as stated in my previous report of this issue the error also occurs on an empty folder.

Another special thing about my setup that might be somehow related: The watch folder I am using is a WebDav mount. However the syslog does not show anything suspicious.