Upgrade error 4.5.8 to 4.6.2

twp · March 26, 2024, 9:10pm

Hi,

after pulling the latest docker-compose and env files (and of course after adjusting values) I am not able to run the new version 4.6. It seems that some migration script fails and tries to restart which ends up in a loop of log error messages. The core message seems to be this one:

postgresql-1  | 2024-03-26 20:47:19.672 UTC [53] ERROR:  could not create unique index "file_metadata_filemetada_document_file_driver_ent_223f9656_uniq"
postgresql-1  | 2024-03-26 20:47:19.672 UTC [53] DETAIL:  Key (document_file_driver_entry_id, internal_name)=(5607, filename_1) is duplicated.
postgresql-1  | 2024-03-26 20:47:19.672 UTC [53] STATEMENT:  ALTER TABLE "file_metadata_filemetadataentry" ADD CONSTRAINT "file_metadata_filemetada_document_file_driver_ent_223f9656_uniq" UNIQUE ("document_file_driver_entry_id", "internal_name")

same for the app-service:

app-1         | Traceback (most recent call last):
app-1         |   File "/opt/mayan-edms/lib/python3.11/site-packages/django/db/backends/utils.py", line 87, in _execute
app-1         |     return self.cursor.execute(sql)
app-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^
app-1         |   File "/opt/mayan-edms/lib/python3.11/site-packages/psycopg/cursor.py", line 737, in execute
app-1         |     raise ex.with_traceback(None)
app-1         | psycopg.errors.UniqueViolation: could not create unique index "file_metadata_filemetada_document_file_driver_ent_223f9656_uniq"
app-1         | DETAIL:  Key (document_file_driver_entry_id, internal_name)=(5607, filename_1) is duplicated.
app-1         |
app-1         | The above exception was the direct cause of the following exception:
app-1         |
app-1         | Traceback (most recent call last):
app-1         |   File "/opt/mayan-edms/lib/python3.11/site-packages/mayan/apps/common/management/base.py", line 104, in do_perform_upgrade
app-1         |     signal_pre_upgrade.send(sender=self)

and this one:

postgresql-1  | 2024-03-26 20:47:18.384 UTC [53] STATEMENT:  SELECT "smart_settings_updatedsetting"."id", "smart_settings_updatedsetting"."global_name", "smart_settings_updatedsetting"."value_new", "smart_settings_updatedsetting"."value_old" FROM "smart_settings_updatedsetting" WHERE "smart_settings_updatedsetting"."global_name" = 'VIEWS_PAGINATE_BY' LIMIT 21
postgresql-1  | 2024-03-26 20:47:18.384 UTC [53] ERROR:  relation "smart_settings_updatedsetting" does not exist at character 187
postgresql-1  | 2024-03-26 20:47:18.384 UTC [53] STATEMENT:  SELECT "smart_settings_updatedsetting"."id", "smart_settings_updatedsetting"."global_name", "smart_settings_updatedsetting"."value_new", "smart_settings_updatedsetting"."value_old" FROM "smart_settings_updatedsetting" WHERE "smart_settings_updatedsetting"."global_name" = 'VIEWS_PAGING_ARGUMENT' LIMIT 21
postgresql-1  | 2024-03-26 20:47:18.385 UTC [53] ERROR:  relation "smart_settings_updatedsetting" does not exist at character 187
postgresql-1  | 2024-03-26 20:47:18.385 UTC [53] STATEMENT:  SELECT "smart_settings_updatedsetting"."id", "smart_settings_updatedsetting"."global_name", "smart_settings_updatedsetting"."value_new", "smart_settings_updatedsetting"."value_old" FROM "smart_settings_updatedsetting" WHERE "smart_settings_updatedsetting"."global_name" = 'VIEWS_SHOW_DROPZONE_SUBMIT_BUTTON' LIMIT 21
postgresql-1  | 2024-03-26 20:47:18.386 UTC [53] ERROR:  relation "smart_settings_updatedsetting" does not exist at character 187
postgresql-1  | 2024-03-26 20:47:18.386 UTC [53] STATEMENT:  SELECT "smart_settings_updatedsetting"."id", "smart_settings_updatedsetting"."global_name", "smart_settings_updatedsetting"."value_new", "smart_settings_updatedsetting"."value_old" FROM "smart_settings_updatedsetting" WHERE "smart_settings_updatedsetting"."global_name" = 'WORKFLOWS_GRAPHVIZ_DOT_PATH' LIMIT 21
postgresql-1  | 2024-03-26 20:47:18.387 UTC [53] ERROR:  relation "smart_settings_updatedsetting" does not exist at character 187
postgresql-1  | 2024-03-26 20:47:18.387 UTC [53] STATEMENT:  SELECT "smart_settings_updatedsetting"."id", "smart_settings_updatedsetting"."global_name", "smart_settings_updatedsetting"."value_new", "smart_settings_updatedsetting"."value_old" FROM "smart_settings_updatedsetting" WHERE "smart_settings_updatedsetting"."global_name" = 'WORKFLOWS_IMAGE_CACHE_MAXIMUM_SIZE' LIMIT 21
postgresql-1  | 2024-03-26 20:47:18.387 UTC [53] ERROR:  relation "smart_settings_updatedsetting" does not exist at character 187
postgresql-1  | 2024-03-26 20:47:18.387 UTC [53] STATEMENT:  SELECT "smart_settings_updatedsetting"."id", "smart_settings_updatedsetting"."global_name", "smart_settings_updatedsetting"."value_new", "smart_settings_updatedsetting"."value_old" FROM "smart_settings_updatedsetting" WHERE "smart_settings_updatedsetting"."global_name" = 'WORKFLOWS_IMAGE_CACHE_STORAGE_BACKEND' LIMIT 21
postgresql-1  | 2024-03-26 20:47:18.388 UTC [53] ERROR:  relation "smart_settings_updatedsetting" does not exist at character 187
postgresql-1  | 2024-03-26 20:47:18.388 UTC [53] STATEMENT:  SELECT "smart_settings_updatedsetting"."id", "smart_settings_updatedsetting"."global_name", "smart_settings_updatedsetting"."value_new", "smart_settings_updatedsetting"."value_old" FROM "smart_settings_updatedsetting" WHERE "smart_settings_updatedsetting"."global_name" = 'WORKFLOWS_IMAGE_CACHE_STORAGE_BACKEND_ARGUMENTS' LIMIT 21
postgresql-1  | 2024-03-26 20:47:18.389 UTC [53] ERROR:  relation "smart_settings_updatedsetting" does not exist at character 187
postgresql-1  | 2024-03-26 20:47:18.389 UTC [53] STATEMENT:  SELECT "smart_settings_updatedsetting"."id", "smart_settings_updatedsetting"."global_name", "smart_settings_updatedsetting"."value_new", "smart_settings_updatedsetting"."value_old" FROM "smart_settings_updatedsetting" WHERE "smart_settings_updatedsetting"."global_name" = 'WORKFLOWS_WORKFLOW_STATE_ESCALATION_CHECK_INTERVAL' LIMIT 21

Any idea? Thanks

UPDATE:

After querying that error message in Postgres, I found a pair of record which seems to be responsible for the constraint creation fail:

select * 
from   file_metadata_filemetadataentry
where  document_file_driver_entry_id = 5607 
       and internal_name='filename_1'

id	key	value	document_file_driver_entry_id	internal_name
132635	FileName	VENREG_181011.pdf	5607	filename_1
132667	Filename	Ven_Req_de_04.pdf	5607	filename_1

Is there a chance to change a value here to fix the error?

UPDATE 2:

After deleting one of those entries from the database, it now runs smoothly.
FIXED

roberto.rosario · March 26, 2024, 10:09pm

Hi,

Seems that somehow the EXIF driver generated two entries for the same time of metadata. Not impossible but unexpected. We’ll add a de-duplication migration before enforcing the field to be unique.

Thanks for the report and all the debug information!

roberto.rosario · March 29, 2024, 9:03am

Migration 0011 of the file metadata app already performs duplication. Seems the algorithm needs to be reinforced to catch yours and other instances of your use case.