Page 1 of 1

Newbie question: New source drivers pluggable?

Posted: Wed Jun 23, 2021 9:34 am
by erny03
Sorry, I'm a newbie, I'm still evaluating Mayan.... :

Are new source drivers pluggable? I'm considering to add Dynamic WebTwain (I did this integration already in another django app.) My question is more oriented to: is there a way to define sources in third party libraries so that I can configure Mayan (through settings) to use it as new source without modifying the "sources" app?

(I'm still reading the source code and the docs, but it seems to me that the sources apps of v4.0.7 does not consider this case:

Code: Select all

    ...
    def ready(self):
        super().ready()

        DocumentCreateWizardStep.load_modules()

        DocumentType = apps.get_model(
            app_label='documents', model_name='DocumentType'
        )

        IMAPEmail = self.get_model(model_name='IMAPEmail')
        POP3Email = self.get_model(model_name='POP3Email')
        Source = self.get_model(model_name='Source')
        SaneScanner = self.get_model(model_name='SaneScanner')
        StagingFolderSource = self.get_model(model_name='StagingFolderSource')
        WatchFolderSource = self.get_model(model_name='WatchFolderSource')
        WebFormSource = self.get_model(model_name='WebFormSource')

        error_log = ErrorLog(app_config=self)
        error_log.register_model(model=IMAPEmail)
        error_log.register_model(model=POP3Email)
        error_log.register_model(model=SaneScanner)
        error_log.register_model(model=WatchFolderSource)
        ...



If not, could this be on the roadmap for a future version? We would / could have some cases of getting documents of external apps / sources (using a search / preview page).

Thanks in advance.

Re: Newbie question: New source drivers pluggable?

Posted: Wed Jun 23, 2021 4:12 pm
by rosarior
Hi,

The sources app does not allow external registration of new sources. This was something we were aiming to fix for version 4.0.

The branch for this task is almost ready but there were some last minute details that prevented it from being merged. We are re targeting this for version 4.1.

The working branches are these: https://gitlab.com/mayan-edms/mayan-edm ... series_4_0 and https://gitlab.com/mayan-edms/mayan-edm ... p_refactor

The refactor allows the sources app to work using source backends or drivers, and allows apps other the source app itself to also register new ones.

Here is the list of issues depending on or relating to this work: https://gitlab.com/mayan-edms/mayan-edm ... 3A+Sources

Best regards.
2021-06-23_12-15.png
2021-06-23_12-15.png (81.58 KiB) Viewed 353 times
2021-06-23_12-16.png
2021-06-23_12-16.png (114.06 KiB) Viewed 353 times

Re: Newbie question: New source drivers pluggable?

Posted: Thu Jun 24, 2021 3:51 pm
by erny03
Thanks for your quick response. I'll take a look at it. ;)
BR

Re: Newbie question: New source drivers pluggable?

Posted: Wed Aug 25, 2021 3:24 am
by rosarior
This branch has been merged and is part of the series/4.1 branch which will become version 4.1 soon.

Each app can define its own sources in the source_backends.py modules which is loaded at startup.

The model was simplified. We removed the model inheritance and made it into a single flat Source model. This model stores the Python path to the class that defines the source backend. Each backend defines it own dynamic fields for configuration which are stored as JSON data in the base Source model. This simplifies the Source model and allows the flexibility of defining sources outside of the sources app. Since each source backend is just a class, you have the new benefit that you can also subclass existing sources to make your own custom ones with minimal code.
2021-08-24_23-20.png
2021-08-24_23-20.png (107.56 KiB) Viewed 128 times
2021-08-24_23-19.png
2021-08-24_23-19.png (10.07 KiB) Viewed 128 times
2021-08-24_23-18.png
2021-08-24_23-18.png (21.43 KiB) Viewed 128 times