Sub-app API URLs?

Questions, comments, discussions. Over time certain topics might be moved to their own category.
Post Reply
hercovandyk
Posts: 8
Joined: Tue Nov 13, 2018 7:28 am

Sub-app API URLs?

Post by hercovandyk » Thu Nov 15, 2018 6:17 am

Hi!

I am trying to integrate a one of my own django apps into Mayan.

The app structure is as follows:

main_app
------- app1
------- app2
------- app3

In the apps.py of the 3 applications I have MayanAppConfig's defined. The challenge however, is that these apps are referenced with main_app.app1, main_app.app2 etc.

class App1(MayanAppConfig):
___has_rest_api = True
___name = 'main_app.app1'
___label = 'app1'


All 3 apps have api urls. But whenever I enable the has_rest_api = True, Mayan is unable to find app1.urls. It should be main_app.app1.urls

Should I define this someplace else? It seems Mayan does not support api urls in sub-apps?

I have a big framework that is built in Django that also has many sub-apps. I would like to integrate this into Mayan as well.

User avatar
rosarior
Posts: 159
Joined: Tue Aug 21, 2018 3:28 am

Re: Sub-app API URLs?

Post by rosarior » Thu Nov 15, 2018 6:54 am

I'm not sure I follow.

I'm not sure Django supports sub-apps (an app made up of apps). I would suggest making main_app a module with __init__.py and extend the Python PATH in settings/base.py to cover this new module level.

If main_app is indeed an app and not just a module to group sub-apps, then the way the .urls module is found need to be updated in https://gitlab.com/mayan-edms/mayan-edm ... pps.py#L64

and in:

https://gitlab.com/mayan-edms/mayan-edm ... pps.py#L33

These are hardcoded to look up a urls.py module and a urls.api_urls per app.

An error in way the api_urls import could be causing the rest of the urls.py from being updated. URLs are only read during initialization. Import errors in Python are very hard to diagnose. Being a dynamic language, an import cause the imported code to be executed. Errors and the tracelog usually shows the error as originating somewhere else. This is why we show this message here in the MayanAppConfig (https://gitlab.com/mayan-edms/mayan-edm ... pps.py#L74) since the tracelog following the message will not always point to the real location of the import error.

This is the location where Mayan expands the PythonPath used for imports (https://gitlab.com/mayan-edms/mayan-edm ... se.py#L238)

Other reference that might help: https://stackoverflow.com/questions/298 ... django-app

hercovandyk
Posts: 8
Joined: Tue Nov 13, 2018 7:28 am

Re: Sub-app API URLs?

Post by hercovandyk » Thu Nov 15, 2018 7:07 am

Thanks a lot rosarior!

I will have a look at those links and revise my code.

I will report back here if I have more questions or with my solution.

KevinPawsey
Posts: 50
Joined: Wed Aug 22, 2018 2:52 pm

Re: Sub-app API URLs?

Post by KevinPawsey » Thu Nov 15, 2018 2:22 pm

I am going to try and install Analyzer in my Mayan, but I am a little confused as to where to even put apps! lol

I know it says in the MayanRoot/apps but is that the config root, or under /opt/mayan-edms (inside the container?) ... but I don't see an apps folder under /opt/mayan-edms, nor do I see one in /var/lib/mayan.

Any help would be appreciated, as I think that this is going to be my best method of doing my indexing ;)

Thanks in advance.


Kevin

hercovandyk
Posts: 8
Joined: Tue Nov 13, 2018 7:28 am

Re: Sub-app API URLs?

Post by hercovandyk » Thu Nov 15, 2018 2:26 pm

KevinPawsey wrote:
Thu Nov 15, 2018 2:22 pm
I am going to try and install Analyzer in my Mayan, but I am a little confused as to where to even put apps! lol

I know it says in the MayanRoot/apps but is that the config root, or under /opt/mayan-edms (inside the container?) ... but I don't see an apps folder under /opt/mayan-edms, nor do I see one in /var/lib/mayan.

Any help would be appreciated, as I think that this is going to be my best method of doing my indexing ;)

Thanks in advance.


Kevin
You should see one inside your mayan django application. Have a look under site-packages in your virtual env, mayan should be there.

@rosarior,

I am having a hard time getting my collected static files to serve with Mayan. I have resolved my URL issue, but am stuck with the css and js files now.

When I run collectstatic, I find that my app's static files are in the correct places as I expect them to be. But for some reason, they are unavailable when called from the front end.

KevinPawsey
Posts: 50
Joined: Wed Aug 22, 2018 2:52 pm

Re: Sub-app API URLs?

Post by KevinPawsey » Thu Nov 15, 2018 2:28 pm

hercovandyk wrote:
Thu Nov 15, 2018 2:26 pm
You should see one inside your mayan django application. Have a look under site-packages in your virtual env, mayan should be there.
Thanks for that... I will take a look and report back... not sure if the folder structure would be the same in a docker image... will let you know.

EDIT: I have just had a look... and I have a few different paths that I presume would fit that description...

Code: Select all

./lib/python2.7/site-packages/mayan/apps
./lib/python2.7/site-packages/django/apps
./lib/python2.7/site-packages/celery/apps
I am assuming that it would be the .../mayan/apps folder?

if I run a docker version, is there going to be an easy way for me to install this without it breaking when I update Mayan in future? I know that I could map a volume to .../apps/document_analyzer > external volume ... trouble is, if I upgrade in the future, will the "document_analyzer" folder still be there (will it be created if it is in the Docker Volumes mapping?).

For now I will install it and see if it gets me around the indexing issue... then I will cross the persistence issue later if it does :D

Thanks again


Kevin
Last edited by KevinPawsey on Thu Nov 15, 2018 2:36 pm, edited 1 time in total.

hercovandyk
Posts: 8
Joined: Tue Nov 13, 2018 7:28 am

Re: Sub-app API URLs?

Post by hercovandyk » Thu Nov 15, 2018 2:32 pm

KevinPawsey wrote:
Thu Nov 15, 2018 2:28 pm
hercovandyk wrote:
Thu Nov 15, 2018 2:26 pm
You should see one inside your mayan django application. Have a look under site-packages in your virtual env, mayan should be there.
Thanks for that... I will take a look and report back... not sure if the folder structure would be the same in a docker image... will let you know.

Thanks again


Kevin
Oh, you have a good point. Let me know how it goes. I've never used the docker image, so I'm not sure at all.

KevinPawsey
Posts: 50
Joined: Wed Aug 22, 2018 2:52 pm

Re: Sub-app API URLs?

Post by KevinPawsey » Thu Nov 15, 2018 2:40 pm

Looks like our two messages cross each other :D ^^^
KevinPawsey wrote:
Thu Nov 15, 2018 2:28 pm
EDIT: I have just had a look... and I have a few different paths that I presume would fit that description...
EDIT: good news... in case anyone wants to try this at home (or at work!!)... if you map a volume to a subfolder in the container... it does 'appear' in there.

What I have done is:
  • on the external volume I created a folder called apps inside the Mayan-EDMS folder (next to the config folder)
  • I then changed to the apps folder and did

    Code: Select all

    git clone https://gitlab.com/mayan-edms/document_analyzer.git
    this created the "document_analyzer" folder
  • I then went to the docker settings (I use OpenMediaVault), and I mapped the volume as follows:

    Code: Select all

    /sharedfolders/AppData/MayanEDMS/apps/document_analyzer > /opt/mayan-edms/lib/python2.7/site-packages/mayan/apps/document_analyzer
    I believe that the alternative method is to add the switch

    Code: Select all

    -v /sharedfolders/AppData/MayanEDMS/apps/document_analyzer:/opt/mayan-edms/lib/python2.7/site-packages/mayan/apps/document_analyzer
    I did not create the document_analyzer folder in the container
  • then on restart, the folder is there as a sub-folder
... next all I need to do is configure it so that it works... easier said than done I think ;)

KevinPawsey
Posts: 50
Joined: Wed Aug 22, 2018 2:52 pm

Re: Sub-app API URLs?

Post by KevinPawsey » Thu Nov 15, 2018 5:47 pm

KevinPawsey wrote:
Thu Nov 15, 2018 2:40 pm
.. easier said than done I think
indeed it is easier said than done... :shock:

I have mounted the volume in:

Code: Select all

/opt/mayan-edms/lib/python2.7/site-packages/mayan/apps
I have checked the folder permissions:

Code: Select all

drwxr-xr-x 5 mayan mayan 4096 Nov  1 08:26 django_gpg
drwxr-sr-x 6 mayan mayan   16 Nov 15 16:43 document_analyzer
drwxr-xr-x 5 mayan mayan 4096 Nov  1 08:26 document_comments
note: it is a mounted folder from the docker host... I guess that is the reason why the permissions are slightly different. In effect, the permission is right for the user (mayan), so it shouldn't be an issue.

I have also added the following to the /var/lib/mayan/settings/local.py

Code: Select all

INSTALLED_APPS += (
    'document_analyzer',
)
but when I start Mayan it doesn't look like it is seeing the document_analyzer folder :(

when I try and run the migration I get the following:

Code: Select all

root@mayan-edms:/opt/mayan-edms/bin# ./mayan-edms.py migrate
Operations to perform:
  Apply all migrations: acls, actstream, admin, auth, authtoken, autoadmin, cabinets, checkouts, common, contenttypes, converter, django_gpg, djcelery, document_comments, document_indexing, document_parsing, document_signatures, document_states, documents, dynamic_search, events, linking, lock_manager, mailer, mayan_statistics, metadata, motd, ocr, permissions, sessions, sites, sources, tags, user_management
Running migrations:
  No migrations to apply.
between djcelery and document_comments is where document_analyzer is supposed to be... but it isn't being picked up.

Is there anything else that I need to do to get it to be picked up (I have restarted the server... several times), or have I changed the wrong local.py file? Are there any logs that I can check to see what is going on?

Appreciate your help :D

Post Reply