Enable logging and finding log files in Mayan EDMS

When things don't work as they should.
Post Reply
abockhold
Posts: 2
Joined: Thu May 21, 2020 9:00 am

Enable logging and finding log files in Mayan EDMS

Post by abockhold »

Hello everybody,

I came across Mayan EDMS an have just installed a copy ("direct installation" on Debian 10) to find out whether this might be the product to solve my problems. I got a copy of the book and do find the information on and the structure of the app quite promising.

Then I wanted to activate the LDAP integration but something does not work. To troubleshoot this I had a look around for log files but did not find any. After that I came across the option MAYAN_COMMON_PRODUCTION_ERROR_LOGGING and set it to true in mayan.conf. This gave me a file error.log in the media-folder but it stays empty.

So I wonder
- how do I enable logging,
- how can I set the log level and
- where does the corresponding logging takes place (which logfile is used)?

Your help is much appreciated, thanks in advance!

Best regards!
abockhold
Posts: 2
Joined: Thu May 21, 2020 9:00 am

Re: Enable logging and finding log files in Mayan EDMS

Post by abockhold »

Hm, anybody here with an idea which could help me?

I dug around some more: in config_backup.yml in the media-folder I find a copy of the currently used configuration of the running server. This helps in that I can see that changing for example "MAYAN_ALLOWED_HOSTS" in /etc/supervisor/conf.d/mayan.conf works and after a restart of the server is reflected in /srv/mayan-edms/media/config_backup.yml (I can see the change). But adding "MAYAN_SETTINGS_MODULE=mayan_settings.ldap_connection_settings," in the section "[supervisord] environment=" as described in viewtopic.php?t=1606 to activate LDAP does not work; it does not show up in /srv/mayan-edms/media/config_backup.yml. How is this?

Besides I found /var/log/supervisor. There all the files are empty, except supervisord.log, which shows after a "systemctl restart supervisor" the following:

Code: Select all

2020-05-25 19:31:12,482 INFO waiting for mayan-gunicorn, mayan-worker-slow, mayan-worker-medium, mayan-worker-fast, mayan-celery-beat to die
2020-05-25 19:31:12,510 WARN received SIGTERM indicating exit request
2020-05-25 19:31:15,676 INFO waiting for mayan-gunicorn, mayan-worker-slow, mayan-worker-medium, mayan-worker-fast, mayan-celery-beat to die
2020-05-25 19:31:18,684 INFO waiting for mayan-gunicorn, mayan-worker-slow, mayan-worker-medium, mayan-worker-fast, mayan-celery-beat to die
2020-05-25 19:31:21,691 INFO waiting for mayan-gunicorn, mayan-worker-slow, mayan-worker-medium, mayan-worker-fast, mayan-celery-beat to die
2020-05-25 19:31:22,693 WARN killing 'mayan-gunicorn' (30172) with SIGKILL
2020-05-25 19:31:22,750 INFO stopped: mayan-gunicorn (terminated by SIGKILL)
2020-05-25 19:31:23,772 WARN killing 'mayan-celery-beat' (30171) with SIGKILL
2020-05-25 19:31:24,021 INFO stopped: mayan-celery-beat (terminated by SIGKILL)
2020-05-25 19:31:25,047 WARN killing 'mayan-worker-fast' (30170) with SIGKILL
2020-05-25 19:31:25,082 INFO waiting for mayan-worker-slow, mayan-worker-medium, mayan-worker-fast to die
2020-05-25 19:31:25,154 INFO stopped: mayan-worker-fast (terminated by SIGKILL)
2020-05-25 19:31:26,177 WARN killing 'mayan-worker-medium' (30169) with SIGKILL
2020-05-25 19:31:26,343 INFO stopped: mayan-worker-medium (terminated by SIGKILL)
2020-05-25 19:31:27,377 WARN killing 'mayan-worker-slow' (30168) with SIGKILL
2020-05-25 19:31:27,444 INFO stopped: mayan-worker-slow (terminated by SIGKILL)
2020-05-25 19:31:37,597 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
2020-05-25 19:31:37,597 INFO Included extra file "/etc/supervisor/conf.d/mayan.conf" during parsing
2020-05-25 19:31:37,772 INFO RPC interface 'supervisor' initialized
2020-05-25 19:31:37,772 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2020-05-25 19:31:37,772 INFO supervisord started with pid 12936
2020-05-25 19:31:38,788 INFO spawned: 'mayan-worker-slow' with pid 12939
2020-05-25 19:31:38,825 INFO spawned: 'mayan-worker-medium' with pid 12940
2020-05-25 19:31:38,954 INFO spawned: 'mayan-worker-fast' with pid 12941
2020-05-25 19:31:39,025 INFO spawned: 'mayan-celery-beat' with pid 12942
2020-05-25 19:31:39,470 INFO spawned: 'mayan-gunicorn' with pid 12943
2020-05-25 19:31:40,737 INFO success: mayan-gunicorn entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-05-25 19:31:49,212 INFO success: mayan-worker-slow entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2020-05-25 19:31:49,212 INFO success: mayan-worker-medium entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2020-05-25 19:31:49,212 INFO success: mayan-worker-fast entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2020-05-25 19:31:49,213 INFO success: mayan-celery-beat entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
Adding the above mentioned does not change anything here, only syntactically malforming the /etc/supervisor/conf.d/mayan.conf blocks the server from starting up (as expected).

Any more ideas how to debug this? Thanks!
ancientskyscraper
Posts: 9
Joined: Fri May 29, 2020 9:26 pm

Re: Enable logging and finding log files in Mayan EDMS

Post by ancientskyscraper »

I'd be interested in learning more about how to enable debug logging as well, especially from within a Docker container. I got as far as adding `DEBUG = True` to my `settings/local.py` file, and switching the `COMMON_PRODUCTION_ERROR_LOGGING` setting to `true` but I cannot figure out where the logs are actually kept.

I was going to try to use the `mayan-edms.py runserver` command and watch logging on the console, but I'm trying to troubleshoot mirroring (in another post) and I'm using a Docker container. Even if I could figure out how to get an HTTP request into the development server in the Docker container (which I'm sure is more than possible), I'm not quite sure how to troubleshoot mirroring on a development server.

Any guidance on how to obtain debug-level logs of a running, production-environment Mayan instance would be most helpful!

Sorry for the n00b questions!
michelw
Posts: 4
Joined: Mon Jun 01, 2020 9:34 am

Re: Enable logging and finding log files in Mayan EDMS

Post by michelw »

Those two settings should enable logging to error.log i discovered, and this file is written to the media directory.
ancientskyscraper
Posts: 9
Joined: Fri May 29, 2020 9:26 pm

Re: Enable logging and finding log files in Mayan EDMS

Post by ancientskyscraper »

Great news! I discovered that error logging works when I tried to access a staging folder that I forgot to create:

Code: Select all

/var/lib/mayan# tail error.log
2020-06-07 15:40:46,223 mayan.apps.sources.models.staging_folder_sources <307> [ERROR] "get_files() line 93 Unable get list of staging files from source: Staging; [Errno 2] No such file or directory: '/staging'"
 . . .
Through Mayan's web GUI, I have COMMON_AUTO_LOGGING set to 'true', COMMON_PRODUCTION_ERROR_LOGGING also set to 'true' and COMMON_PRODUCTION_ERROR_LOG_PATH set to '/var/lib/mayan/error.log' (not sure if all three settings are required to get this error log message).

Creating a known error by trying to access that missing folder allowed me to verify that errors were getting logged to that file (I previously thought it just wasn't working).

The next step is to figure out how to set the logging level from ERROR to DEBUG so that I can try to see what is happening with my mirroring problem (that doesn't seem to generate ERROR messages). I'm continuing my deep dive into Django :-)
ancientskyscraper
Posts: 9
Joined: Fri May 29, 2020 9:26 pm

Re: Enable logging and finding log files in Mayan EDMS

Post by ancientskyscraper »

I was able to set the logging level to debug by editing the config.yml file within the media root directory (/var/lib/mayan/config.yml on my installation):

Code: Select all

DEBUG: true
ALLOWED_HOSTS:
- '*'
APPEARANCE_MAXIMUM_TITLE_LENGTH: 120
. . . 
(I only added the first line, but put a couple more in the quote above for context.)

I restarted Mayan to get it to take the new setting. I'm using docker, so I ran 'sudo docker restart mayan_app_1' to do this.

Now, I'm getting all sorts of DEBUG and INFO messages written to /var/lib/mayan/error.log and i can use that to debug my mirroring problem.

Just figured I'd post this in case it's helpful to other folks.
Post Reply