Page 1 of 1

Advice on setting up a Mayan instance

Posted: Thu Feb 06, 2020 4:16 pm
by aaribaud
Hello all,

I need to set up an Mayan instance which will serve both as a production-ready PoC and a development platform,and I am wondering which approach between direct deployment and docker install would better fit the bill. My constraints are:
  1. It will run in a Debian Buster VM with unattended upgrades
  2. I can install or uninstall whatever packages I want/need.
  3. All of the Mayan instance state (database, storage, etc) must be under a single directory so that it can be easily grown through e.g. LVM, moved around, etc.
  4. Performance tuning must be feasible
  5. Keeping things up to date is a strong requirement
  6. I must be able to experiment with custom interfaces
  7. I will not alter the database schema or the API
  8. I should be able to easily submit PRs corresponding to local trivial or minor fixes
  9. I am fine with non-ideal development setup (e.g. accessing the source code only through SSH, working through a container)
What do you people think would be the best approach for this? I suspect a direct install rather than a docker one, but beyond that I don't really know what I should worry about.

Thanks for your feedback!

Re: Advice on setting up a Mayan instance

Posted: Fri Feb 07, 2020 5:26 pm
by rssfed23
So far the majority of enterprise customers deploy using Direct Install methods, and that's the way our consultants tend to do things.
There are improvements happening on the Docker side (mainly around Kubernetes) in the background, but without knowing more about the project in terms of size it can be hard to say if that would apply in the future.

But in terms of what you've put there there's a few flags. I won't go into all of the potential issues here (after all that's why we offer enterprise support and consulting), but in general I really wouldn't recommend unattended updates.
You never know when it's going to reboot the machine and take Mayan down. That could be for a few minutes, but if Mayan or Postgres don't survive that reboot then you can run into unnecessary downtime that can be avoided by having a maintenance window and patching the OS on a schedule. It's fine for a POC, but definitely not for production (and then of course the POC wouldn't match production).

Also, I wouldn't recommend keeping the database and document storage in the same directory. Separating out the DB from Mayan can help in terms of scaling. It also reduces risk if something bad were to happen. It doesn't allow for proper growth of Mayan over time we've seen that a few times before. Deploying things the "easy way" can cause problems further down the line, and that's before we get into potential issues with contention/IO.

You mention being able to submit PRs. Of course anyone is welcome to submit a PR! - That's the beauty of open source! But the project is under no obligation to accept them. For fixes like you suggest that's generally fine, but for larger ones that have an impact on users we won't accept anything that doesn't align with our strategic direction or causes a user impact to everyone else. Depending on the PR (if its larger or you're submitting frequent ones), you may have to sign the contributor agreement. For anything but minor fixes, you'll also have to ensure any code follows the development standards and rules in place.

We're only able to provide some general guidance here though. Mayan EDMS offers a variety of different support plans, and also a plethora of consulting options as well. For something that's running in production at an organisation (especially something as important as Document Management or archival), it's always recommended to have support and someone to call if things go wrong. We can also provide more specific guidance on the proposed architecture based on the specific project, as depending on the number of documents and size of the userbase there's some issues with some of the items you posted. Always happy to have that discussion if desired :)

Re: Advice on setting up a Mayan instance

Posted: Mon Feb 10, 2020 9:49 am
by aaribaud
Thank you for the very instructive feedback!

Regarding separating the DB and data, that's duly noted -- thanks!

Regarding PRs, I may not have been explicit enough, my bad; I was not thinking about being allowed to PR,I was thinking about how I should set up my Mayan instance to make it easy to turn local changes into PRs. Basically, the official direct install procedure uses pip to install Mayan, so I was wondering if it was possible to install it in a local Git clone instead. That would make it much easier to manage local changes and PR those for which it makes sense; OTOH, could using a git repo instead of pip have negative consequences e.g on the install or upgrade process?

Just a remark regarding unattended upgrades: I've configured it so that if reboot is needed this is just requested by e-mail to me, as I want to always schedule reboots myself for a time when it does not affect users and when I can handle any complication properly. :)

Regarding support -- that's on the table for after this experimental phase.

Again: thanks for your answers!

Re: Advice on setting up a Mayan instance

Posted: Wed Feb 12, 2020 10:16 am
by rssfed23
Ahh I understand what you mean now in terms of local changes.

So in theory there's nothing stopping you from cloning the releases/all or master branch, creating a virtualenv and then installing all the dependencies into that folder. That's how we create local developer environments.
But we stopped supporting using direct git clone installs in 3.3.2: https://docs.mayan-edms.com/releases/3.3.2.html

So in essence, you're welcome to create a custom installation any way you like, but there'd be limited support for it.

I recommend that if you're looking to create more of a development environment, that you create 2 Mayan instances a test and a prod. The prod installed using Pip or docker and the test one for your local changes. You can then easily submit PRs for those local changes from the test instance and usually we do releases around every 1.5-2 weeks so it wouldn't be long before they actually made their way into the released code.

Re: Advice on setting up a Mayan instance

Posted: Fri Feb 21, 2020 12:48 pm
by aaribaud
Thanks for the advice. I'll go for the separate production (direct install) and dev setups.