Mayan on Kubernetes - Experiences?

Reverse proxies, SSL termination, web servers, helm charts, integrations, batch files, anything having to do with installing Mayan.
Post Reply
User avatar
rssfed23
Moderator
Moderator
Posts: 213
Joined: Mon Oct 14, 2019 1:18 pm
Location: United Kingdom
Contact:

Mayan on Kubernetes - Experiences?

Post by rssfed23 »

Hey Mayan users!

One of the items I'm going to be working on soon in the background is Kubernetes deployments for Mayan (documentation, fixes, helm charts, etc). To be clear and set expectations this isn't something on the roadmap but a community contribution that I think would be nice to have.
I've seen a couple of people mention K8s in the past on here and gitlab issues so I'm putting wondering if you're able to share your experiences with us.

If you're in a position to share deployment yaml or any tooling you've used to deploy Mayan to k8s then please do share (if you aren't able to publicly then feel free to log a confidential issue on gitlab)

Hearing your experiences of using Mayan on k8s (what works well, what doesn't work, etc) will greatly speed up the process.

My initial plan is to produce deployment spec yamls and then hopefully helm charts. A longer term view would be a native Mayan K8s Operator but that is definitely way down the line.

I'm aware of the locking issues with NFS and the preparestatic generation (and the fact it runs on every startup) but any other feedback or experiences will be truly welcome to help shape things.

A couple of other initial questions:
- If producing a helm chart, should our chart contain all services required (Mayan, Redis, RabbitMQ, Flower, Postgres, etc) which can be enabled/disable through the values.yaml or should it be only Mayan with documentation on how to use the upstream charts for those other services?
- Are you using operators for postgres, redis & rabbitmq? - This is related to the first question in a way because I wouldn't want to go through the work of creating deployments of those services or documentation for them if it's commonplace that people are using the operator for that technology already

Many thanks!
Rob
Please bear with us during the current global situation. The team all have families and local communities to look after as well as the community here. Responses may be delayed during this time, but rest assured we will get to your query eventually.
jcollie
Posts: 2
Joined: Sun Mar 01, 2020 3:53 am

Re: Mayan on Kubernetes - Experiences?

Post by jcollie »

I've been running Mayan in my Kubernetes cluster for a couple of weeks now and it's working well so far (although I wish that I had more resources to dedicate to it). This is just a small home/personal deployment so YMMV. Right now I just have everything running in a single pod although I'd like to break the pod up into several pods. One main Mayan/gunicorn pod serving up the web interface and then a number of pods running the workers. Then using autoscalers it should be possible to dynamically add workers as needed.

I've put sanitized versions of my yaml files up here:

https://gist.github.com/jcollie/bb27ae1 ... 1f19b4124e

Based on my experience so far I'd start out by only deploying Mayan and Redis in the Helm chart. Let the user install other things like RabbitMQ or PostgreSQL. Then maybe build up to optionally deploying those other parts.
User avatar
rssfed23
Moderator
Moderator
Posts: 213
Joined: Mon Oct 14, 2019 1:18 pm
Location: United Kingdom
Contact:

Re: Mayan on Kubernetes - Experiences?

Post by rssfed23 »

Jcollie, I'm no longer working personally on this topic at all, but your contribution there is much appreciated! - It is indeed quite close to what I would have gone for myself, and you're right, one deployment for gunicorn (I did get it working with Passenger also) + one deployment for each worker type (fast/medium/slow) is the most ideal setup.
Ultimately, I'd go as far as to say one deployment for each worker queue would be most ideal for an enterprise grade deployment. We did have such a thing architected out and drawn to fit, but as of yet not implemented.

One of the larger problems to solve is storage. I can see in your provided yaml you're using hostpath storage, which of course wouldn't be suitable for most. The biggest issue I see is a lot of people expect these days for us to support dynamic/StorageClass, but that's impossible to do if a user wants to delete the deployment and then redeploy pointing to an existing setup (well without modifying the helm chart during install time anyway). My initial recommendation to the team was to use manual PV creation with documentation on how to do it, and we can worry about StorageClasses later on.

As far as I'm concerned, the Mayan team should only really be concerned about deploying Mayan. Redis, RabbitMQ (a must I would say for enterprise k8s support) and PostgreSQL should all be provisioned and managed via operators. I did get a POC working with the RedisFailover operator and one of the Postgres operators as well and the whole setup worked really quite well! So thanks for sharing your Yaml, I'm sure someone here will find it very useful, in particular because you're using all the right k8s patterns with ConfigMaps, Ingresses and Secrets! - Great job :)
Please bear with us during the current global situation. The team all have families and local communities to look after as well as the community here. Responses may be delayed during this time, but rest assured we will get to your query eventually.
abc
Posts: 1
Joined: Fri May 22, 2020 7:12 am

Re: Mayan on Kubernetes - Experiences?

Post by abc »

Hey Mayan users,
I want to deploy Mayan on a kubernetes cluster but I am pretty new to this technologie. I have a lot of trouble with the container PostgreSQL :D . If you have yaml that can help me, or other newbie to deploy mayan on a kubernetes cluster on the simple way that would be awsome.

Thank and happy end of quarantine ;)
Post Reply