Unable to upgrade from 4.4 to 4.5

I am following the docker compose upgrade instructions. however I am unable to successfully complete the upgrade.

I am able to recover the old database to the new one and then I try running all of the containers together however the mayan container does not output any logs after outputting ‘running migrations’. is there anything i can do? do you offer assistance in upgrading?

Running migrations:
Applying credentials.0001_initial… OK
Applying credentials.0002_auto_20201215_0523… OK
Applying credentials.0003_auto_20210207_0823… OK
Applying credentials.0004_auto_20210207_0823… OK
Applying credentials.0005_auto_20210207_0840… OK
Applying django_celery_beat.0016_alter_crontabschedule_timezone… OK
Applying document_indexing.0029_alter_indexinstancenode_value… OK
Applying document_states.0029_auto_20230505_1716… OK
Applying document_states.0030_auto_20230505_1724… OK
Applying documents.0081_alter_documentfile_size… OK
Applying documents.0082_populate_document_file_size… OK
Applying documents.0083_auto_20230325_0542… OK
Applying documents.0084_alter_documenttype_document_stub_pruning_enabled… OK
Applying documents.0085_document_file_latest… OK
(after this nothing at all !)

complete mayan container and postgres logs are below.

mayan container logs are below

[legosiv]$ docker logs ce2570ea3520
mayan: starting entrypoint.sh
Connection attempt #1 to: port postgresql:5432; Connected.
Connection attempt #1 to: port rabbitmq:5672; [Errno 111] Connection refused
Connection attempt #2 to: port rabbitmq:5672; [Errno 111] Connection refused
Connection attempt #3 to: port rabbitmq:5672; [Errno 111] Connection refused
Connection attempt #4 to: port rabbitmq:5672; [Errno 111] Connection refused
Connection attempt #5 to: port rabbitmq:5672; [Errno 111] Connection refused
Connection attempt #6 to: port rabbitmq:5672; [Errno 111] Connection refused
Connection attempt #7 to: port rabbitmq:5672; Connected.
Connection attempt #1 to: port redis:6379; Connected.
mayan: update_uid_gid()
usermod: no changes
mayan: os_package_installs()
mayan: pip_installs()
Font directories:
        /root/.local/share/fonts
        /usr/local/share/fonts
        /usr/share/fonts
        /root/.fonts
        /usr/share/fonts/X11
        /usr/share/fonts/cMap
        /usr/share/fonts/cmap
        /usr/share/fonts/opentype
        /usr/share/fonts/truetype
        /usr/share/fonts/type1
        /usr/share/fonts/X11/Type1
        /usr/share/fonts/X11/encodings
        /usr/share/fonts/X11/util
        /usr/share/fonts/cmap/adobe-cns1
        /usr/share/fonts/cmap/adobe-gb1
        /usr/share/fonts/cmap/adobe-japan1
        /usr/share/fonts/cmap/adobe-japan2
        /usr/share/fonts/cmap/adobe-korea1
        /usr/share/fonts/opentype/urw-base35
        /usr/share/fonts/truetype/arphic
        /usr/share/fonts/truetype/libreoffice
        /usr/share/fonts/truetype/unfonts-core
        /usr/share/fonts/type1/urw-base35
        /usr/share/fonts/X11/encodings/large
/root/.local/share/fonts: skipping, no such directory
/usr/local/share/fonts: skipping, existing cache is valid: 0 fonts, 0 dirs
/usr/share/fonts: skipping, existing cache is valid: 0 fonts, 6 dirs
/usr/share/fonts/X11: skipping, existing cache is valid: 0 fonts, 3 dirs
/usr/share/fonts/X11/Type1: skipping, existing cache is valid: 35 fonts, 0 dirs
/usr/share/fonts/X11/encodings: skipping, existing cache is valid: 0 fonts, 1 dirs
/usr/share/fonts/X11/encodings/large: skipping, existing cache is valid: 0 fonts, 0 dirs
/usr/share/fonts/X11/util: skipping, existing cache is valid: 0 fonts, 0 dirs
/usr/share/fonts/cMap: skipping, existing cache is valid: 0 fonts, 0 dirs
/usr/share/fonts/cmap: skipping, existing cache is valid: 0 fonts, 5 dirs
/usr/share/fonts/cmap/adobe-cns1: skipping, existing cache is valid: 0 fonts, 0 dirs
/usr/share/fonts/cmap/adobe-gb1: skipping, existing cache is valid: 0 fonts, 0 dirs
/usr/share/fonts/cmap/adobe-japan1: skipping, existing cache is valid: 0 fonts, 0 dirs
/usr/share/fonts/cmap/adobe-japan2: skipping, existing cache is valid: 0 fonts, 0 dirs
/usr/share/fonts/cmap/adobe-korea1: skipping, existing cache is valid: 0 fonts, 0 dirs
/usr/share/fonts/opentype: skipping, existing cache is valid: 0 fonts, 1 dirs
/usr/share/fonts/opentype/urw-base35: skipping, existing cache is valid: 35 fonts, 0 dirs
/usr/share/fonts/truetype: skipping, existing cache is valid: 0 fonts, 3 dirs
/usr/share/fonts/truetype/arphic: skipping, existing cache is valid: 8 fonts, 0 dirs
/usr/share/fonts/truetype/libreoffice: skipping, existing cache is valid: 1 fonts, 0 dirs
/usr/share/fonts/truetype/unfonts-core: skipping, existing cache is valid: 12 fonts, 0 dirs
/usr/share/fonts/type1: skipping, existing cache is valid: 0 fonts, 1 dirs
/usr/share/fonts/type1/urw-base35: skipping, existing cache is valid: 35 fonts, 0 dirs
/root/.fonts: skipping, no such directory
/usr/share/fonts/X11: skipping, looped directory detected
/usr/share/fonts/cMap: skipping, looped directory detected
/usr/share/fonts/cmap: skipping, looped directory detected
/usr/share/fonts/opentype: skipping, looped directory detected
/usr/share/fonts/truetype: skipping, looped directory detected
/usr/share/fonts/type1: skipping, looped directory detected
/usr/share/fonts/X11/Type1: skipping, looped directory detected
/usr/share/fonts/X11/encodings: skipping, looped directory detected
/usr/share/fonts/X11/util: skipping, looped directory detected
/usr/share/fonts/cmap/adobe-cns1: skipping, looped directory detected
/usr/share/fonts/cmap/adobe-gb1: skipping, looped directory detected
/usr/share/fonts/cmap/adobe-japan1: skipping, looped directory detected
/usr/share/fonts/cmap/adobe-japan2: skipping, looped directory detected
/usr/share/fonts/cmap/adobe-korea1: skipping, looped directory detected
/usr/share/fonts/opentype/urw-base35: skipping, looped directory detected
/usr/share/fonts/truetype/arphic: skipping, looped directory detected
/usr/share/fonts/truetype/libreoffice: skipping, looped directory detected
/usr/share/fonts/truetype/unfonts-core: skipping, looped directory detected
/usr/share/fonts/type1/urw-base35: skipping, looped directory detected
/usr/share/fonts/X11/encodings/large: skipping, looped directory detected
/var/cache/fontconfig: cleaning cache directory
/root/.cache/fontconfig: not cleaning non-existent cache directory
/root/.fontconfig: not cleaning non-existent cache directory
/usr/bin/fc-cache: succeeded
mayan: perform_upgrade()
Operations to perform:
  Apply all migrations: acls, actstream, admin, announcements, appearance, auth, authentication_otp, authtoken, autoadmin, cabinets, checkouts, common, contenttypes, converter, credentials, django_celery_beat, django_gpg, document_comments, document_downloads, document_exports, document_indexing, document_parsing, document_signatures, document_states, documents, dynamic_search, events, file_caching, file_metadata, linking, locales, lock_manager, logging, mailer, mayan_statistics, messaging, metadata, motd, ocr, permissions, quotas, sessions, signature_captures, sites, source_emails, source_periodic, source_sane_scanners, source_staging_folders, source_watch_folders, source_web_forms, sources, storage, tags, user_management, web_links
Running migrations:
  Applying credentials.0001_initial... OK
  Applying credentials.0002_auto_20201215_0523... OK
  Applying credentials.0003_auto_20210207_0823... OK
  Applying credentials.0004_auto_20210207_0823... OK
  Applying credentials.0005_auto_20210207_0840... OK
  Applying django_celery_beat.0016_alter_crontabschedule_timezone... OK
  Applying document_indexing.0029_alter_indexinstancenode_value... OK
  Applying document_states.0029_auto_20230505_1716... OK
  Applying document_states.0030_auto_20230505_1724... OK
  Applying documents.0081_alter_documentfile_size... OK
  Applying documents.0082_populate_document_file_size... OK
  Applying documents.0083_auto_20230325_0542... OK
  Applying documents.0084_alter_documenttype_document_stub_pruning_enabled... OK
  Applying documents.0085_document_file_latest... OK

postgres logs are below

[legosiv]$ docker logs 50d32de33e76
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... UTC
creating configuration files ... ok
running bootstrap script ... ok
sh: locale: not found
2023-11-21 18:23:53.490 UTC [30] WARNING:  no usable system locales were found
performing post-bootstrap initialization ... ok
syncing data to disk ... ok


Success. You can now start the database server using:

    pg_ctl -D /var/lib/postgresql/data -l logfile start

initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
waiting for server to start....2023-11-21 18:23:54.151 UTC [36] LOG:  starting PostgreSQL 13.10 on x86_64-pc-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r4) 12.2.1 20220924, 64-bit
2023-11-21 18:23:54.153 UTC [36] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-11-21 18:23:54.156 UTC [37] LOG:  database system was shut down at 2023-11-21 18:23:53 UTC
2023-11-21 18:23:54.159 UTC [36] LOG:  database system is ready to accept connections
 done
server started
CREATE DATABASE


/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*

waiting for server to shut down....2023-11-21 18:23:54.301 UTC [36] LOG:  received fast shutdown request
2023-11-21 18:23:54.302 UTC [36] LOG:  aborting any active transactions
2023-11-21 18:23:54.303 UTC [36] LOG:  background worker "logical replication launcher" (PID 43) exited with exit code 1
2023-11-21 18:23:54.304 UTC [38] LOG:  shutting down
2023-11-21 18:23:54.317 UTC [36] LOG:  database system is shut down
 done
server stopped

PostgreSQL init process complete; ready for start up.

2023-11-21 18:23:54.433 UTC [1] LOG:  starting PostgreSQL 13.10 on x86_64-pc-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r4) 12.2.1 20220924, 64-bit
2023-11-21 18:23:54.434 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2023-11-21 18:23:54.434 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2023-11-21 18:23:54.436 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-11-21 18:23:54.439 UTC [51] LOG:  database system was shut down at 2023-11-21 18:23:54 UTC
2023-11-21 18:23:54.442 UTC [1] LOG:  database system is ready to accept connections
2023-11-21 18:24:21.288 UTC [64] ERROR:  role "mayan" already exists
2023-11-21 18:24:21.288 UTC [64] STATEMENT:  CREATE ROLE mayan;
2023-11-21 18:24:21.302 UTC [65] ERROR:  database "mayan" already exists
2023-11-21 18:24:21.302 UTC [65] STATEMENT:  CREATE DATABASE mayan WITH TEMPLATE = template0 ENCODING = 'UTF8' LOCALE = 'en_US.utf8';
2023-11-21 18:25:00.682 UTC [52] LOG:  checkpoints are occurring too frequently (18 seconds apart)
2023-11-21 18:25:00.682 UTC [52] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-11-21 18:26:07.085 UTC [125] ERROR:  canceling autovacuum task
2023-11-21 18:26:07.085 UTC [125] CONTEXT:  while scanning block 10218 of relation "public.file_caching_cachepartition"
        automatic vacuum of table "mayan.public.file_caching_cachepartition"
2023-11-21 18:26:08.822 UTC [125] ERROR:  canceling autovacuum task
2023-11-21 18:26:08.822 UTC [125] CONTEXT:  while scanning block 4149 of relation "public.file_metadata_filemetadataentry"
        automatic vacuum of table "mayan.public.file_metadata_filemetadataentry"
[cabinet_admin@cabinet DhiraaguCabinet]$ docker logs 50d32de33e76
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... UTC
creating configuration files ... ok
running bootstrap script ... ok
sh: locale: not found
2023-11-21 18:23:53.490 UTC [30] WARNING:  no usable system locales were found
performing post-bootstrap initialization ... ok
syncing data to disk ... ok


Success. You can now start the database server using:

    pg_ctl -D /var/lib/postgresql/data -l logfile start

initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
waiting for server to start....2023-11-21 18:23:54.151 UTC [36] LOG:  starting PostgreSQL 13.10 on x86_64-pc-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r4) 12.2.1 20220924, 64-bit
2023-11-21 18:23:54.153 UTC [36] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-11-21 18:23:54.156 UTC [37] LOG:  database system was shut down at 2023-11-21 18:23:53 UTC
2023-11-21 18:23:54.159 UTC [36] LOG:  database system is ready to accept connections
 done
server started
CREATE DATABASE


/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*

waiting for server to shut down....2023-11-21 18:23:54.301 UTC [36] LOG:  received fast shutdown request
2023-11-21 18:23:54.302 UTC [36] LOG:  aborting any active transactions
2023-11-21 18:23:54.303 UTC [36] LOG:  background worker "logical replication launcher" (PID 43) exited with exit code 1
2023-11-21 18:23:54.304 UTC [38] LOG:  shutting down
2023-11-21 18:23:54.317 UTC [36] LOG:  database system is shut down
 done
server stopped

PostgreSQL init process complete; ready for start up.

2023-11-21 18:23:54.433 UTC [1] LOG:  starting PostgreSQL 13.10 on x86_64-pc-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r4) 12.2.1 20220924, 64-bit
2023-11-21 18:23:54.434 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2023-11-21 18:23:54.434 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2023-11-21 18:23:54.436 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-11-21 18:23:54.439 UTC [51] LOG:  database system was shut down at 2023-11-21 18:23:54 UTC
2023-11-21 18:23:54.442 UTC [1] LOG:  database system is ready to accept connections
2023-11-21 18:24:21.288 UTC [64] ERROR:  role "mayan" already exists
2023-11-21 18:24:21.288 UTC [64] STATEMENT:  CREATE ROLE mayan;
2023-11-21 18:24:21.302 UTC [65] ERROR:  database "mayan" already exists
2023-11-21 18:24:21.302 UTC [65] STATEMENT:  CREATE DATABASE mayan WITH TEMPLATE = template0 ENCODING = 'UTF8' LOCALE = 'en_US.utf8';
2023-11-21 18:25:00.682 UTC [52] LOG:  checkpoints are occurring too frequently (18 seconds apart)
2023-11-21 18:25:00.682 UTC [52] HINT:  Consider increasing the configuration parameter "max_wal_size".
2023-11-21 18:26:07.085 UTC [125] ERROR:  canceling autovacuum task
2023-11-21 18:26:07.085 UTC [125] CONTEXT:  while scanning block 10218 of relation "public.file_caching_cachepartition"
        automatic vacuum of table "mayan.public.file_caching_cachepartition"
2023-11-21 18:26:08.822 UTC [125] ERROR:  canceling autovacuum task
2023-11-21 18:26:08.822 UTC [125] CONTEXT:  while scanning block 4149 of relation "public.file_metadata_filemetadataentry"
        automatic vacuum of table "mayan.public.file_metadata_filemetadataentry"

One more thing i noticed was that the command to load data from sql backup is missing a "

below is the command from the upgrade docs

cat mayan.sql | docker-compose --file docker-compose.yml --project-name mayan exec -T postgresql /bin/bash -c ‘psql --dbname=“$POSTGRES_DB --username=”$POSTGRES_USER"’

shouldn’t -dbname="$POSTGRES_DB have a closing " ?

like -dbname=“$POSTGRES_DB”

Migration documents.0086 is not a schema migration but a data migration. It will access the document storage and extract the size of the physical files.

How you are storing your documents will have a direct effect on the speed of this migration.

If you’ve made changes to customize your document repository (like keeping document stubs around before it was an official feature in version 4.5) this could also cause the migration to slow down, get stuck, or fail.

if I get rid of the document stubs will that help? my next plan is to change the stubs expiration to one day, wait till the stubs are gone and then try the upgrade. all I did was change the expiration to a year.

I checked the migration again and I made a mistake. I was thinking of migration documents.0080, that’s the one that populates the physical document file size as a database field.

Migration documents.0086 populates the latest file as a database field:

document.file_latest = document.files.order_by('timestamp').only('id').last()

Since it is querying for the last created file of a document, and since document stubs don’t have any files, there could be an issue.

We’ll add a test to see how this migration behaves with document stubs and add corrective code if there are any artifacts.

If changes are needed we’ll release patch version 4.5.8. If there are none I’ll reply back on this topic to confirm.

my next plan is to change the stubs expiration to one day, wait till the stubs are gone and then try the upgrade

Sounds like a good idea to try out. However, keep your backup even on success until your installation upgrades with document stubs so you don’t lose any archival data.