Page 1 of 2

LDAP integration problems

Posted: Fri May 01, 2020 2:53 pm
by ebotzki
I'm having trouble getting LDAP integration to work.

I am on 3.4.6 and followed this guide for setting LDAP up it is a direct install.

viewtopic.php?f=15&t=1606

I cant find any logs as to why its not working. As far as i can tell everything is set up right and not failing but I cant even find logs about if the bind is successful let alone if my maps are right.


Here is my ldap_connection_settings.py

Code: Select all

import ldap

from django_auth_ldap.config import (
    LDAPSearch, LDAPSearchUnion, NestedActiveDirectoryGroupType
)

from mayan.settings.production import *  # NOQA

# Makes sure this works in Active Directory
ldap.set_option(ldap.OPT_REFERRALS, False)

# Turn of debug output, turn this off when everything is working as expected
ldap.set_option(ldap.OPT_DEBUG_LEVEL, 4095)

# Default: True
AUTH_LDAP_ALWAYS_UPDATE_USER = TRUE
SECRET_KEY = '*r2&%aa=5a10v4g)48h%wehi5l4yb4ygh((qgk*7fmo8i2($d_'


AUTH_LDAP_START_TLS = False

LDAP_ADMIN_DN = 'CN=REDACTED,OU=REDACTED,OU=REDACTED,DC=REDACTED,DC=local'
LDAP_BASE_DN = 'OU=REDACTED,DC=REDACTED,DC=REDACTED'
LDAP_PASSWORD = 'REDACTED'
LDAP_USER_AUTO_CREATION = 'False'
LDAP_URL = 'ldap://REDACTED:389/'

AUTH_LDAP_BIND_DN = LDAP_ADMIN_DN
AUTH_LDAP_BIND_PASSWORD = LDAP_PASSWORD
AUTH_LDAP_SERVER_URI = LDAP_URL

# Simple search
AUTH_LDAP_USER_SEARCH = LDAPSearch(
   "ou='OU=REDACTED,DC=REDACTED,DC=local', ldap.SCOPE_SUBTREE, '(uid=%(user)s')

# User attributes to map from LDAP to Mayan's user model.
AUTH_LDAP_USER_ATTR_MAP = {
    'user': 'uid',
    'first_name': 'cn',
    'last_name': 'sn',
    'email': 'mail'
}

AUTHENTICATION_BACKENDS = (
    'django_auth_ldap.backend.LDAPBackend',
)




Here is my mayan.conf

Code: Select all

[supervisord]
environment=
    PYTHONPATH="/opt/mayan-edms/media/mayan_settings",
    DJANGO_SETTINGS_MODULE=mayan.settings.production,
    MAYAN_MEDIA_ROOT="/opt/mayan-edms/media",
    MAYAN_ALLOWED_HOSTS="['*']",
    MAYAN_CELERY_RESULT_BACKEND="redis://:REDACTED@127.0.0.1:6379/1",
    MAYAN_CELERY_BROKER_URL="redis://:REDACTED@127.0.0.1:6379/0",
    MAYAN_DATABASES="{default: {ENGINE: django.db.backends.postgresql, HOST: 127.0.0.1, NAME: mayan, PASSWORD: REDACTED, USER: mayan}}",
    MAYAN_PIP_INSTALLS="python-ldap django_auth_ldap",
    MAYAN_APT_INSTALLS="libsasl2-dev python3-dev libldap2-dev libssl-dev libgle3 build-essential autoconf libtool pkg-config gcc",
    MAYAN_SETTINGS_MODULE=mayan_settings.ldap_connection_settings,


[program:mayan-gunicorn]
autorestart = true
autostart = true
command = /opt/mayan-edms/bin/gunicorn -w 2 mayan.wsgi --max-requests 500 --max-requests-jitter 50 --worker-class sync --bind 0.0.0.0:8000 --timeout 120
user = mayan


[program:mayan-worker-fast]
autorestart = true
autostart = true
command = nice -n 1 /opt/mayan-edms/bin/celery worker -A mayan -Ofair -l ERROR -Q document_states_fast,converter,sources_fast -n mayan-worker-fast.%%h --concurre$
killasgroup = true
numprocs = 1
priority = 998
startsecs = 10
stopwaitsecs = 1
user = mayan

[program:mayan-worker-medium]
autorestart = true
autostart = true
command = nice -n 18 /opt/mayan-edms/bin/celery worker -A mayan -Ofair -l ERROR -Q default,checkouts_periodic,indexing,signatures,documents_periodic,uploads,docu$
killasgroup = true
numprocs = 1
priority = 998
startsecs = 10
stopwaitsecs = 1
user = mayan

[program:mayan-worker-medium]
autorestart = true
autostart = true
command = nice -n 18 /opt/mayan-edms/bin/celery worker -A mayan -Ofair -l ERROR -Q default,checkouts_periodic,indexing,signatures,documents_periodic,uploads,docu$
killasgroup = true
numprocs = 1
priority = 998
startsecs = 10
stopwaitsecs = 1
user = mayan

[program:mayan-worker-slow]
autorestart = true
autostart = true
command = nice -n 19 /opt/mayan-edms/bin/celery worker -A mayan -Ofair -l ERROR -Q statistics,tools,common_periodic,parsing,document_states,mailing,ocr -n mayan-$
killasgroup = true
numprocs = 1
priority = 998
startsecs = 10
stopwaitsecs = 1
user = mayan


[program:mayan-celery-beat]
autorestart = true
autostart = true
command = nice -n 1 /opt/mayan-edms/bin/celery beat -A mayan --pidfile= -l ERROR
killasgroup = true
numprocs = 1
priority = 998
startsecs = 10
stopwaitsecs = 1
user = mayan

The logs just say that everything started and is running.

If anyone can please help it would be greatly appreciated.

Re: LDAP integration problems

Posted: Tue May 05, 2020 6:36 pm
by sna-cmarko
Hi,

I had similar issues when I was setting up LDAP using the Docker install method. Did you create a super user in Mayan that matches a user in your LDAP before including the LDAP settings? I suppose that step might be a little different between Docker and Direct, but it made the difference between connecting and actually being able to login to the system.

Also, it's good to note that from my experience, a user will not show up in the "User" list once LDAP is working until he/she logs in for the first time. (Makes assigning groups and roles in bulk a pain.)

Best,

Caitlyn

Re: LDAP integration problems

Posted: Tue May 05, 2020 7:02 pm
by ebotzki
Hi,

I just tried that and it still did not seem to work.

I noticed somewhere that this line in the mayan.conf needed changed

Code: Select all

 DJANGO_SETTINGS_MODULE=mayan.settings.production,
but when i did it broke on a reboot.

i changed it to

Code: Select all

 DJANGO_SETTINGS_MODULE=DJANGO_SETTINGS_MODULE=mayan.media.mayan_settings.ldap_connection_settings,

I originally tried a docker install and still have it but i kept getting ldap not found and chased that around for a while.

I have no clue if that was because GCCcould not install which seemed to be because the source list did not have them. I could not figure out how to add new repositories when it kept rebooting because it failed.

Which ever version I can get more help on is the one I will go with.

Thank you for your help.

Erich

Re: LDAP integration problems

Posted: Wed May 06, 2020 5:44 pm
by ebotzki
So I got it to finaly read the settings file but im getting this output in the gunicorn logs. So im getting closer but im not sure what to do.

Code: Select all

[2020-05-06 17:33:02 +0000] [1458] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/opt/mayan-edms/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/opt/mayan-edms/lib/python3.6/site-packages/gunicorn/workers/base.py", line 119, in init_process
    self.load_wsgi()
  File "/opt/mayan-edms/lib/python3.6/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/opt/mayan-edms/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/opt/mayan-edms/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
    return self.load_wsgiapp()
  File "/opt/mayan-edms/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/opt/mayan-edms/lib/python3.6/site-packages/gunicorn/util.py", line 358, in import_app
    mod = importlib.import_module(module)
  File "/opt/mayan-edms/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/mayan-edms/lib/python3.6/site-packages/mayan/wsgi.py", line 15, in <module>
    application = get_wsgi_application()
  File "/opt/mayan-edms/lib/python3.6/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application
    django.setup(set_prefix=False)
  File "/opt/mayan-edms/lib/python3.6/site-packages/django/__init__.py", line 19, in setup
    configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
  File "/opt/mayan-edms/lib/python3.6/site-packages/django/conf/__init__.py", line 79, in __getattr__
    self._setup(name)
  File "/opt/mayan-edms/lib/python3.6/site-packages/django/conf/__init__.py", line 66, in _setup
    self._wrapped = Settings(settings_module)
  File "/opt/mayan-edms/lib/python3.6/site-packages/django/conf/__init__.py", line 157, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/opt/mayan-edms/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'mayan.media'
The last error is a problem with the django_settings_module path but im not sure what it should be.


This is what I changed the mayan.conf settings after following the instructions here for the symlink with a slight modification for python 3.6. viewtopic.php?f=10&t=825

Code: Select all


    DJANGO_SETTINGS_MODULE='mayan.media.mayan_settings.ldap_connection_settings',


Re: LDAP integration problems

Posted: Wed May 06, 2020 6:20 pm
by ebotzki
So i got it to load the settings files i wanted which is now working I will share the config settings in a bit. But where are the auth logs and the bind logs. I know ldap is now being used for authentication because i cant use the main admin account anymore but i also can not login with an ldap account. I have a feeling i have something wrong with my ldap settings but im not sure what part now.

Re: LDAP integration problems

Posted: Wed May 06, 2020 8:54 pm
by ebotzki
I got it to work now. Is there a way to give permissions based on ldap groups the guide here says there is but i cant find any more info than that.

viewtopic.php?f=15&t=1606

Re: LDAP integration problems

Posted: Thu May 07, 2020 11:38 am
by sna-cmarko
Great! I am so glad. I am sorry that my reply didn't post yesterday.

I was able to setup groups and roles in Mayan, but I haven't found a way to mass add/import users to the groups based on their LDAP groups. I was hoping to find a way to import users without needing to have each of them login once and then add them to their appropriate groups.

I will say, the new impersonate feature is pretty useful while updating permissions for various group/role combinations. Before I had to have a separate test account, or have a user do a screen share with me while I got the role permissions just right.

Best,

Caitlyn

Re: LDAP integration problems

Posted: Thu May 07, 2020 3:30 pm
by ebotzki
So i think i got it to give permissions based on LDAP but now my superuser has normal permissions. How do i allow it to let me login with the original admin credentials.

Once i figure this out I will write up some how to's for exactly what I did.

Re: LDAP integration problems

Posted: Thu May 07, 2020 4:16 pm
by ebotzki
So I got back in and ldap is pulling groups it looks like based on the log entries but I'm still not sure how to give permissions based on groups from ldap. With out that it makes ldap useless. All i want is to give all new logins the same set of default permissions. I'm fine with editing the admins if needed.

Erich

Re: LDAP integration problems

Posted: Mon May 11, 2020 3:19 pm
by ebotzki
So I got it all to work and these are my final config files. as well as one other command That i did that I do not know if it worked or not.

/opt/mayan-edms/media/mayan_settings/ldap_connection_settings.py

Code: Select all


from __future__ import absolute_import

import ldap

from django_auth_ldap.config import (
   GroupOfNamesType, LDAPSearch, LDAPSearchUnion, NestedActiveDirectoryGroupType
)

from mayan.settings.production import *  # NOQA

# Makes sure this works in Active Directory
ldap.set_option(ldap.OPT_REFERRALS, False)

# Turn of debug output, turn this off when everything is working as expected
ldap.set_option(ldap.OPT_DEBUG_LEVEL, 1)

# Default: True
AUTH_LDAP_ALWAYS_UPDATE_USER = True
SECRET_KEY = '*REDACTED'

AUTH_LDAP_START_TLS = False

LDAP_ADMIN_DN = 'CN=REDACTED,OU=REDACTED,OU=REDACTED,DC=REDACTED,DC=REDACTED'
LDAP_BASE_DN = 'DC=REDACTED,DC=REDACTED'
LDAP_PASSWORD = 'REDACTED'
LDAP_USER_AUTO_CREATION = 'False'
LDAP_URL = 'ldap://REDACTED:389/'
AUTH_LDAP_BIND_DN = LDAP_ADMIN_DN
AUTH_LDAP_BIND_PASSWORD = LDAP_PASSWORD
AUTH_LDAP_SERVER_URI = LDAP_URL

# If you need to search in more than one place for a user, you can use
# LDAPSearchUnion. This takes multiple LDAPSearch objects and returns the
# union of the results. The precedence of the underlying searches is
# unspecified.
# https://django-auth-ldap.readthedocs.io/en/latest/authentication.html
AUTH_LDAP_USER_SEARCH = LDAPSearchUnion(
     LDAPSearch(
         'OU=REDACTED,OU=REDACTED,DC=REDACTED,DC=REDACTED', ldap.SCOPE_SUBTREE,
         'samaccountname=%(user)s'
     ),
     LDAPSearch(
        'OU=REDACTED,OU=REDACTED,DC=REDACTED,DC=REDACTED',
# User attributes to map from LDAP to Mayan's user model.
AUTH_LDAP_USER_ATTR_MAP = {
    'user': 'samAccountName',
    'first_name': 'cn',
    'last_name': 'sn',
    'email': 'mail'
}


# Simple group search
AUTH_LDAP_GROUP_SEARCH = LDAPSearch('OU=REDACTED,OU=REDACTED,DC=REDACTED,DC=REDACTED', ldap.SCOPE_SUBTREE, '(objectClass=top)')
AUTH_LDAP_GROUP_TYPE = GroupOfNamesType(name_attr="CN")

AUTH_LDAP_CACHE_GROUPS = True
AUTH_LDAP_FIND_GROUP_PERMS = True
#AUTH_LDAP_GROUP_TYPE = NestedActiveDirectoryGroupType()
AUTH_LDAP_MIRROR_GROUPS = True

# To minimize traffic to the LDAP server, LDAPBackend can make use of
# Django’s cache framework to keep a copy of a user’s LDAP group memberships.
# To enable this feature, set AUTH_LDAP_CACHE_TIMEOUT, which determines
# the timeout of cache entries in seconds.
AUTH_LDAP_GROUP_CACHE_TIMEOUT = 3600


AUTHENTICATION_BACKENDS = (
    'django_auth_ldap.backend.LDAPBackend','ldap_connection_settings.EmailOrUsernameModelBackend','django.contrib.auth.backends.ModelBackend'
)

class EmailOrUsernameModelBackend(object):
    """
    This is a ModelBacked that allows authentication with either a username or an email address.

    """
    def authenticate(self, username=None, password=None):
        if '@' in username:
            kwargs = {'email': username}
        else:
            kwargs = {'username': username}
        try:
            user = get_user_model().objects.get(**kwargs)
            if user.check_password(password):
                return user
        except User.DoesNotExist:
            return None

    def get_user(self, username):
        try:
            return get_user_model().objects.get(pk=username)
        except get_user_model().DoesNotExist:
            return None
/etc/supervisor/conf.d/mayan.conf

Code: Select all


[supervisord]
environment=
    PYTHONPATH="/opt/mayan-edms/media/mayan_settings",
    DJANGO_SETTINGS_MODULE='ldap_connection_settings',
    MAYAN_MEDIA_ROOT="/opt/mayan-edms/media",
    MAYAN_ALLOWED_HOSTS="['*']",
    MAYAN_CELERY_RESULT_BACKEND="redis://:REDACTED@127.0.0.1:6379/1",
    MAYAN_CELERY_BROKER_URL="redis://:REDACTED@127.0.0.1:6379/0",
    MAYAN_DATABASES="{default: {ENGINE: django.db.backends.postgresql, HOST: 127.0.0.1, NAME: REDACTED, PASSWORD: REDACTED, USER: REDACTED}}",
    MAYAN_PIP_INSTALLS="python-ldap django_auth_ldap pyldap ldap3",
    MAYAN_APT_INSTALLS="libsasl2-dev python3-dev libldap2-dev libssl-dev libgle3 build-essential autoconf libtool pkg-config gcc",
    MAYAN_SETTINGS_MODULE='mayan_settings.ldap_connection_settings',


[program:mayan-gunicorn]
autorestart = true
autostart = true
command = /opt/mayan-edms/bin/gunicorn -w 2 mayan.wsgi --max-requests 500 --max-requests-jitter 50 --worker-class sync --bind 0.0.0.0:8000 --timeout 120
user = mayan


[program:mayan-worker-fast]
autorestart = true
autostart = true
command = nice -n 1 /opt/mayan-edms/bin/celery worker -A mayan -Ofair -l ERROR -Q document_states_fast,converter,sources_fast -n mayan-worker-fast.%%h --concurrency=1
killasgroup = true
umprocs = 1
priority = 998
startsecs = 10
stopwaitsecs = 1
user = mayan

[program:mayan-worker-medium]
autorestart = true
autostart = true
command = nice -n 18 /opt/mayan-edms/bin/celery worker -A mayan -Ofair -l ERROR -Q default,checkouts_periodic,indexing,signatures,documents_periodic,uploads,documents,file_m$
killasgroup = true
numprocs = 1
priority = 998
startsecs = 10
stopwaitsecs = 1
user = mayan

[program:mayan-worker-slow]
autorestart = true
autostart = true
command = nice -n 19 /opt/mayan-edms/bin/celery worker -A mayan -Ofair -l ERROR -Q statistics,tools,common_periodic,parsing,document_states,mailing,ocr -n mayan-worker-slow.$
killasgroup = true
numprocs = 1
priority = 998
startsecs = 10
stopwaitsecs = 1
user = mayan


[program:mayan-celery-beat]
autorestart = true
autostart = true
command = nice -n 1 /opt/mayan-edms/bin/celery beat -A mayan --pidfile= -l ERROR
killasgroup = true
numprocs = 1
priority = 998
startsecs = 10
stopwaitsecs = 1
user = mayan



i used this command i don't know if its needed.

Code: Select all

 ln -s /opt/mayan-edms/media/ /opt/mayan-edms/lib/python3.6/site-packages/media


For groups if you create a group inside of mayan with the same name spelled exactly as an AD group it will populate with the ad group users when they log in. This inside of the ldap settings causes them to auto populate. I would be careful on a not new install as it seems to wipe out some pre exisitng settings for users when doing this. On thing to note is that domain users and other groups like that are not going to be added. It makes things a little bit difficult for me but only because I do not have access to our AD to make changes to make a new everyone group.

Code: Select all

# Simple group search
AUTH_LDAP_GROUP_SEARCH = LDAPSearch('OU=REDACTED,OU=REDACTED,DC=REDACTED,DC=REDACTED', ldap.SCOPE_SUBTREE, '(objectClass=top)')
AUTH_LDAP_GROUP_TYPE = GroupOfNamesType(name_attr="CN")

AUTH_LDAP_CACHE_GROUPS = True
AUTH_LDAP_FIND_GROUP_PERMS = True
#AUTH_LDAP_GROUP_TYPE = NestedActiveDirectoryGroupType()
AUTH_LDAP_MIRROR_GROUPS = True

adding

Code: Select all

'django.contrib.auth.backends.ModelBackend'


to this section allows login of the original admin account which is helpful for initial setup.

Code: Select all

AUTHENTICATION_BACKENDS = (
  
)

Make sure that this is all included or parts will not work

Code: Select all

import ldap

from django_auth_ldap.config import (
   GroupOfNamesType, LDAPSearch, LDAPSearchUnion, NestedActiveDirectoryGroupType
)
all logs for ldap will be found under the last bit is random but all the logs you want are in that log file just use tab after/var/log/supervisor/mayan-gunicorn-stderr to get it ! took me a while to find that.

Code: Select all

/var/log/supervisor/mayan-gunicorn-stderr---supervisor-RANDOM STRING
So far the people using it love it and once i figured out where the logs where it was super easy to get everything working. If you have any questions i will try to help out where I can. I am a security analyst by nature so this was not really my wheel house.

also if you want it secure install ngnix like a normal program and these are the 2 files you will need. also remove default from sites-enabled



/etc/nginx/sites-enabled/mayan

Code: Select all


# redirect http to https
server {
    listen 80 default_server;
    server_name _;
    return 302 https://$host$request_uri;
}

# reverse proxy for Mayan-EDMS
server {
    listen 0.0.0.0:443 ssl;
    access_log /var/log/nginx/mayan.log;
    error_log /var/log/nginx/mayan_error.log;
    include /etc/nginx/include/ssl;

    client_max_body_size 500M;  # Increase to upload bigger documents

    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass_header Set-Cookie;

        proxy_pass http://127.0.0.1:8000/;
        proxy_redirect off;
        proxy_read_timeout 30s; # Increase if your doc uploads take more than 30 sec
    }
}


/etc/nginx/include/ssl

note this is for a very secure HTTPS installation it is not very backwards compatible friendly adjust ssl_protocols and ssl_ciphers accordingly.

Code: Select all

ssl_certificate      /etc/ssl/private/REDACTED.crt;
ssl_certificate_key  /etc/ssl/private/REDACTED.key;
ssl_session_timeout  5m;
ssl_session_cache shared:SSL:50m;

ssl_protocols TLSv1.2 TLSv1.3;

# ciphers added by conf/turnkey.d/zz-ssl-ciphers script
ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384;


ssl_prefer_server_ciphers   on;

ssl_dhparam /etc/ssl/private/dhparams.pem;
add_header X-Content-Type-Options nosniff;
server_tokens off;


If you need help creating the certs and keys and dhparms let me know I am more comfortable with those than Mayan to be honest but i left if out because it needed more step by step instructions.