Mayan 4.2.5 - Inability to generate cache images / File name too long

When things don't work as they should.
Post Reply
jeremy.warren
Posts: 1
Joined: Fri Jun 17, 2022 2:13 pm

Mayan 4.2.5 - Inability to generate cache images / File name too long

Post by jeremy.warren »

Hello everyone,

I've run into an unsual problem here with my installation of mayan (I'm running a directly installed version on Ubuntu 20.04 + Postgresql on XFS). Whenever I try and run parsing on a document, the parsing goes through successfully, but the preview thumbnail generation for the file cache fails.
I usually get logs which look like this:

Code: Select all

Task mayan.apps.converter.tasks.task_content_object_image_generate[ae466090-bfb1-4486-a978-937e55b6b11f] raised unexpected: DataError('value too long for type character varying(255)\n') 
Traceback (most recent call last):                                                                                                                                                         
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/models/query.py", line 581, in get_or_create                                                                                 
    return self.get(**kwargs), False                                                                                                                                                       
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get                                                                                           
    raise self.model.DoesNotExist(                                                                                                                                                        
mayan.apps.file_caching.models.CachePartition.DoesNotExist: CachePartition matching query does not exist.                                                                                  
                                                                                                                                                                                           
During handling of the above exception, another exception occurred:                                                                                                                        
                                                                                                                                                                                           
Traceback (most recent call last):                                                                                                                                                         
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute                                                                                     
    return self.cursor.execute(sql, params)                                                                                                                                                
psycopg2.errors.StringDataRightTruncation: value too long for type character varying(255)

The above exception was the direct cause of the following exception:                                                                                                                       
                                                                                                                                                                                           
Traceback (most recent call last):                                                                                                                                                         
  File "/opt/mayan-edms/lib/python3.8/site-packages/celery/app/trace.py", line 450, in trace_task                                                                                          
    R = retval = fun(*args, **kwargs)                                                                                                                                                      
  File "/opt/mayan-edms/lib/python3.8/site-packages/celery/app/trace.py", line 731, in __protected_call__                                                                                  
    return self.run(*args, **kwargs)                                                                                                                                                       
  File "/opt/mayan-edms/lib/python3.8/site-packages/mayan/apps/converter/tasks.py", line 46, in task_content_object_image_generate                                                         
    return obj.generate_image(                                                                                                                                                             
  File "/opt/mayan-edms/lib/python3.8/site-packages/mayan/apps/documents/models/document_version_page_models.py", line 145, in generate_image                                              
    self.cache_partition.get_file(                                                                                                                                                         
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/utils/functional.py", line 48, in __get__                                                                                       
    res = instance.__dict__[self.name] = self.func(instance)                                                                                                                               
  File "/opt/mayan-edms/lib/python3.8/site-packages/mayan/apps/documents/models/document_version_page_models.py", line 77, in cache_partition                                              
    partition, created = self.document_version.cache.partitions.get_or_create(                                                                                                             
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py", line 683, in get_or_create                                                            
    return super(RelatedManager, self.db_manager(db)).get_or_create(**kwargs)                                                                                                              
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method                                                                               
    return getattr(self.get_queryset(), name)(*args, **kwargs)                                                                                                                             
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/models/query.py", line 588, in get_or_create                                                                                 
    return self.create(**params), True                                                                                                                                                     
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/models/query.py", line 453, in create                                                                                        
    obj.save(force_insert=True, using=self.db)                                                                                                                                             
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/models/base.py", line 739, in save                                                                                           
    self.save_base(using=using, force_insert=force_insert,                                                                                                                                 
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/models/base.py", line 776, in save_base                                                                                      
    updated = self._save_table(                                                                                                                                                            
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/models/base.py", line 881, in _save_table                                                                                    
    results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)                                                                                                     
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/models/base.py", line 919, in _do_insert                                                                                     
    return manager._insert(                                                                                                                                                                
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method                                                                               
    return getattr(self.get_queryset(), name)(*args, **kwargs)                                                                                                                             
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/models/query.py", line 1270, in _insert                                                                                      
    return query.get_compiler(using=using).execute_sql(returning_fields)                                                                                                                   
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1416, in execute_sql                                                                           
    cursor.execute(sql, params)                                                                                                                                                            
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/backends/utils.py", line 66, in execute                                                                                      
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)                                                                                                    
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers                                                                       
    return executor(sql, params, many, context)                                                                                                                                            
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute                                                                                     
    return self.cursor.execute(sql, params)                                                                                                                                                
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__   
      raise dj_exc_value.with_traceback(traceback) from exc_value                                                                                                                            
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute                                                                                     
    return self.cursor.execute(sql, params)                                                                                                                                                
django.db.utils.DataError: value too long for type character varying(255) 
In order to better understand what was going on, I changed the field width on 'name' (which was at 255, hence the error above) in file_caching_cachepartition to 512, just to see what happened. Upon rerunning the parsing, I still didn't get any thumbnail generation. Instead, the process failed with the following error:

Code: Select all

                                                
2022-06-17 14:16:46,586: ERROR/ForkPoolWorker-1] Task mayan.apps.converter.tasks.task_content_object_image_generate[aa2f4ff6-82f1-4ba9-b48b-5a950410efa8] raised unexpected: OSError(36, ''File name too long')                                                                         
Traceback (most recent call last):                                                                                                                                                         
  File "/opt/mayan-edms/lib/python3.8/site-packages/mayan/apps/documents/models/document_version_page_models.py", line 145, in generate_image                                              
    self.cache_partition.get_file(                                                                                                                                                         
  File "/opt/mayan-edms/lib/python3.8/site-packages/mayan/apps/file_caching/models.py", line 288, in get_file                                                                              
    return self.files.get(filename=filename)                                                                                                                                               
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method                                                                               
    return getattr(self.get_queryset(), name)(*args, **kwargs)                                                                                                                             
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get                                                                                           
    raise self.model.DoesNotExist(                                                                                                                                                         
mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist.                                                                          
                                                                                                                                                                                           
During handling of the above exception, another exception occurred:                                                                                                                        
                                                                                                                                                                                           
Traceback (most recent call last):                                                                                                                                                         
  File "/opt/mayan-edms/lib/python3.8/site-packages/mayan/apps/documents/models/document_version_page_models.py", line 283, in get_image                                                   
    cache_file = self.cache_partition.get_file(filename=cache_filename)                                                                                                                    
  File "/opt/mayan-edms/lib/python3.8/site-packages/mayan/apps/file_caching/models.py", line 288, in get_file                                                                              
    return self.files.get(filename=filename)                                                                                                                                               
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method                                                                               
    return getattr(self.get_queryset(), name)(*args, **kwargs)                                                                                                                             
  File "/opt/mayan-edms/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get                                                                                           
    raise self.model.DoesNotExist(                                                                                                                                                         
mayan.apps.file_caching.models.CachePartitionFile.DoesNotExist: CachePartitionFile matching query does not exist. 
OSError: [Errno 36] File name too long: '/opt/mayan-edms/media/document_file_page_image_cache/04249ad3-8e02-4f74-94bc-f59732234686     
                                                                                                                                                                  -1512-85241-base_image'                                                    
Note the huge number of spaces in the filename. If I pull a record out of the table, it looks like this:

Code: Select all

04249ad3-8e02-4f74-94bc-f59732234686                                                                                                                                                                                                                           -5340-50588
For the life of me, I can't trace where this is coming from. It looks the the data in the filename is ok, just that exactly 219 blank spaces are getting injected into the file name before the two final parts. Does anyone have any ideas?
Post Reply