Watch folder source raised: KeyError ('shared_uploaded_file_id')

Using Mayan EDMS Bugs or errors
1

Hi,

Just in the process of upgrading to 4.5. Thanks!

I noticed that I needed to recreate my watch folder; it had null backend. Ok. So I recreated it in the form. However, the watch still doesn’t work. I get the following error:

[2023-09-13 17:45:17,198: ERROR/ForkPoolWorker-7] Task mayan.apps.sources.tasks.task_source_backend_action_background_task[708ebd37-c0b5-40be-986f-250807fa1227] raised unexpected: KeyError('shared_uploaded_file_id')
Traceback (most recent call last):
  File "/opt/mayan-edms/lib/python3.11/site-packages/celery/app/trace.py", line 477, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/opt/mayan-edms/lib/python3.11/site-packages/celery/app/trace.py", line 760, in __protected_call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/mayan-edms/lib/python3.11/site-packages/mayan/apps/sources/tasks.py", line 34, in task_source_backend_action_background_task
    action.background_task(interface_load_kwargs=action_interface_kwargs)
  File "/opt/mayan-edms/lib/python3.11/site-packages/mayan/apps/sources/source_backend_actions/base.py", line 122, in background_task
    self._background_task(**task_kwargs)
  File "/opt/mayan-edms/lib/python3.11/site-packages/mayan/apps/sources/source_backend_actions/mixins/document_mixins.py", line 156, in _background_task
    document_task_kwargs['shared_uploaded_file_id'] = server_upload_entry[
                                                      ^^^^^^^^^^^^^^^^^^^^
KeyError: 'shared_uploaded_file_id'

Hope someone can help!

2

I had this also on build v4.5.1-2-gb96e222e319_Fri Sep 15 10:32:19 2023 -0400

I created watch folders then make test. My docker logs

2023-09-18 04:34:34 [2023-09-18 09:34:34,698: ERROR/ForkPoolWorker-8] Task mayan.apps.sources.tasks.task_source_backend_action_background_task[28912030-be7a-4329-8e8a-0b026fc35af3] raised unexpected: KeyError('shared_uploaded_file_id')
2023-09-18 04:34:34 Traceback (most recent call last):
2023-09-18 04:34:34   File "/opt/mayan-edms/lib/python3.11/site-packages/celery/app/trace.py", line 477, in trace_task
2023-09-18 04:34:34     R = retval = fun(*args, **kwargs)
2023-09-18 04:34:34                  ^^^^^^^^^^^^^^^^^^^^
2023-09-18 04:34:34   File "/opt/mayan-edms/lib/python3.11/site-packages/celery/app/trace.py", line 760, in __protected_call__
2023-09-18 04:34:34     return self.run(*args, **kwargs)
2023-09-18 04:34:34            ^^^^^^^^^^^^^^^^^^^^^^^^^
2023-09-18 04:34:34   File "/opt/mayan-edms/lib/python3.11/site-packages/mayan/apps/sources/tasks.py", line 34, in task_source_backend_action_background_task
2023-09-18 04:34:34     action.background_task(interface_load_kwargs=action_interface_kwargs)
2023-09-18 04:34:34   File "/opt/mayan-edms/lib/python3.11/site-packages/mayan/apps/sources/source_backend_actions/base.py", line 122, in background_task
2023-09-18 04:34:34     self._background_task(**task_kwargs)
2023-09-18 04:34:34   File "/opt/mayan-edms/lib/python3.11/site-packages/mayan/apps/sources/source_backend_actions/mixins/document_mixins.py", line 156, in _background_task
2023-09-18 04:34:34     document_task_kwargs['shared_uploaded_file_id'] = server_upload_entry[
2023-09-18 04:34:34                                                       ^^^^^^^^^^^^^^^^^^^^
2023-09-18 04:34:34 KeyError: 'shared_uploaded_file_id'
3

I have the same error on a direct install. Migration from Version 4.4.8 to also v4.5.1-2-gb96e222e319_Fri Sep 15 10:32:19 2023 -0400

mayan.apps.source_compressed.source_backend_actions.mixins <765051> [DEBUG] “_background_task() line 64 ^[[36mNot expanding; Exception: NoMIMETypeMatch^[[0m”
[2023-09-18 14:37:50,339: DEBUG/ForkPoolWorker-4] ^[[36mNot expanding; Exception: NoMIMETypeMatch^[[0m
[2023-09-18 14:37:50,341: ERROR/ForkPoolWorker-4] Task mayan.apps.sources.tasks.task_source_backend_action_background_task[18278699-3d5f-4553-b508-e2a77634f462] raised unexpected: KeyError(‘shared_uploaded_file_id’)
Traceback (most recent call last):
File “/opt/mayan-edms/lib/python3.11/site-packages/celery/app/trace.py”, line 477, in trace_task
R = retval = fun(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^
File “/opt/mayan-edms/lib/python3.11/site-packages/celery/app/trace.py”, line 760, in protected_call
return self.run(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File “/opt/mayan-edms/lib/python3.11/site-packages/mayan/apps/sources/tasks.py”, line 34, in task_source_backend_action_background_task
action.background_task(interface_load_kwargs=action_interface_kwargs)
File “/opt/mayan-edms/lib/python3.11/site-packages/mayan/apps/sources/source_backend_actions/base.py”, line 122, in background_task
self._background_task(**task_kwargs)
File “/opt/mayan-edms/lib/python3.11/site-packages/mayan/apps/sources/source_backend_actions/mixins/document_mixins.py”, line 156, in _background_task
document_task_kwargs[‘shared_uploaded_file_id’] = server_upload_entry[
^^^^^^^^^^^^^^^^^^^^
KeyError: ‘shared_uploaded_file_id’

4

Fix has been merged: 83c92e770cdbcb1a47ee3c9341e39f356956fac4

Waiting for another fix in the queue before the release of version 4.5.2.

The issue is triggered when the watch folder uncompressed logic is set to “Always” and it tries to process and non compressed file.

A temporary workaround is to set the 'Uncompress` option to ‘Never’ and not use compressed files or keep it to ‘Always’ but not include non compressed files in the watch folder.

2 Likes
5

Thanks for the input!

Now running 4.5.2, and the Watch folder issue I’m now facing (with 'Uncompress` option to ‘Never’) is that the automatically trigger is seemingly not working at all: I have to hit Test to trigger the watch. Then (as far as I can see), at most 1 file is imported, and it’s never deleted (yes, checked for permissions issues)

Hope that helps. Thanks!