Error upgrading to 3.3/3.3.1

When things don't work as they should.
Post Reply
pleblancq
Posts: 9
Joined: Sat Oct 26, 2019 2:09 pm

Error upgrading to 3.3/3.3.1

Post by pleblancq » Wed Dec 04, 2019 6:42 pm

Hi rosario, here is the log after upgrading to 3.3 or 3.3.1, same error:

Traceback (most recent call last):
File "/data/mayan-edms/.virtualenv/bin/mayan-edms.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/data/mayan-edms/.virtualenv/lib/python3.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
utility.execute()
File "/data/mayan-edms/.virtualenv/lib/python3.7/site-packages/django/core/management/__init__.py", line 338, in execute
django.setup()
File "/data/mayan-edms/.virtualenv/lib/python3.7/site-packages/django/__init__.py", line 27, in setup
apps.populate(settings.INSTALLED_APPS)
File "/data/mayan-edms/.virtualenv/lib/python3.7/site-packages/django/apps/registry.py", line 85, in populate
app_config = AppConfig.create(entry)
File "/data/mayan-edms/.virtualenv/lib/python3.7/site-packages/django/apps/config.py", line 120, in create
mod = import_module(mod_path)
File "/data/mayan-edms/.virtualenv/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/data/mayan-edms/.virtualenv/lib/python3.7/site-packages/mayan/apps/sources/apps.py", line 17, in <module>
from .classes import StagingFile
File "/data/mayan-edms/.virtualenv/lib/python3.7/site-packages/mayan/apps/sources/classes.py", line 19, in <module>
from .storages import storage_staging_file_image_cache
File "/data/mayan-edms/.virtualenv/lib/python3.7/site-packages/mayan/apps/sources/storages.py", line 12, in <module>
)(**setting_staging_file_image_cache_storage_arguments.value)
TypeError: type object argument after ** must be a mapping, not str

riopangeran
Posts: 14
Joined: Thu Sep 12, 2019 8:46 am

Re: Error upgrading to 3.3/3.3.1

Post by riopangeran » Wed Dec 04, 2019 11:36 pm

I got quite similar error too while upgrading to newest version. But now i have it running.
What i did just update it via clean installation and point the storage location to where it belong.
There is one thing i missed and takes me long time to address the issue, it's the path of redis' result backend configuration, after latest backslash, i changed it 0 (maybe it's because i use rabbitmq for the broker, idk, cmiiw)

I am also waiting the correct response too from others and also from rosario.

Note: 3.3 layout is good, thanks @rosario

User avatar
rosarior
Posts: 457
Joined: Tue Aug 21, 2018 3:28 am

Re: Error upgrading to 3.3/3.3.1

Post by rosarior » Thu Dec 05, 2019 7:12 am

@pleblancq

Version 3.3.2 is out and includes the fixed setting migration for the source app that is shown in the error. Give it a try.

@riopangeran thanks for the feedback. The upgrade issue is cause by a change in format of some config.yml file entries. In the past the upgrade process was like the upgrade of any other program with a config or ini file. Manual update by users. We felt we could improve on that and developed a setting migration solution. We missed a detail in the sources app change migration which is the cause of the issue but is now fixed. This feature will help users continue doing upgrade without having to worry about future config file format changes.

For the setup, use RabbitMQ as the broker to support heavier loads, but always use Redis for the results backend. Here is my PyCon talk explaining the reason behind this configuration: "Django tasty salad DOs and DONT using celery" (https://www.youtube.com/watch?v=0UJTG5QU7Ss)

pleblancq
Posts: 9
Joined: Sat Oct 26, 2019 2:09 pm

Re: Error upgrading to 3.3/3.3.1

Post by pleblancq » Thu Dec 05, 2019 2:13 pm

Thanks Rosario,

I updated and it worked.

Last question: Now generating the thumbnails is much slower than before, the way they are generated has changed?

Thanks

User avatar
rosarior
Posts: 457
Joined: Tue Aug 21, 2018 3:28 am

Re: Error upgrading to 3.3/3.3.1

Post by rosarior » Thu Dec 05, 2019 11:42 pm

Thanks for the feedback.

Yes, the image preview now has a layers system to support redactions and the file caching system keeps track of disk usage of the images. Both of these might cause some extra delays. Try increasing the concurrency of the fast worker. It defaults to 1 as a safe value, but can be increased gradually or removed to let Celery auto scale the worker based on CPU core count. If you modify the supervisor file don't forget to stop the service with systemctl. Using supervisorctl CLI utility requires extra steps that are not entirely intuitive.

pleblancq
Posts: 9
Joined: Sat Oct 26, 2019 2:09 pm

Re: Error upgrading to 3.3/3.3.1

Post by pleblancq » Fri Dec 06, 2019 1:44 pm

I added more concurrency and its now faster.

I prefer the python package over the Docker image (anyway, I'm the guy running it natively on OSX).

More controls.

Thanks.

Post Reply