Version 4.9

Announcements Releases
1

Version 4.9

Released: January 20, 2025

Changes

Appearance

Added support for two new navigation elements. The first is dropdown element that allows faster pagination. The dropdown shows a range of pages based on the current page. This elemenet is controlled by the settings APPEARANCE_PAGINATION_DROPDOWN_RANGE and APPEARANCE_PAGINATION_DROPDOWN_ENABLE.

The second element is an text field that allows jumping to any user specified page. This element is controlled by the new setting APPEARANCE_PAGINATION_INPUT_ENABLE.

The layout of the pager is now user configurable. The new settings added are: APPEARANCE_ELIDED_PAGER_ON_EACH_SIDE and APPEARANCE_ELIDED_PAGER_ON_ENDS.

Minor caching and speedups changes were made to the AJAX content code. These are small optimizations but since this code affects all content updates over time user experience is improved.

There were many other small optimization and modernization changes to the JavaScript code. These result in minor speed improvements and lower browser resource usage.

Celery

We have re-balanced the Celery workers’ maximum tasks per child configuration, based on a comprehensive analysis of performance data from a default installation under various workload types. By averaging the most effective results, we’ve optimized this setting to strike a balance between minimizing memory leaks and reducing disconnections to RabbitMQ. The new values ensure that the system operates efficiently while maintaining reliability and robustness even without specialized fine tuning.

Dependencies

We have updated the system's version check functionality to leverage the latest version of the PyPI (Python Package Index) API. This ensures that version checks are performed using the most current and efficient methods.

Docker

Support was added for passing a custom consumer timeout value to RabbitMQ via the new environment variable MAYAN_RABBITMQ_CONSUMER_TIMEOUT, which defaults to 1800000 milliseconds or 30 minutes.

Support was also added for passing custom server Erlang arguments to RabbitMQ via the new environment variable MAYAN_RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS. This variable defaults to settings the consumer timeout via MAYAN_RABBITMQ_CONSUMER_TIMEOUT.

Added a Docker Compose service to mount a cabinet.

Removed the example insecure Traefik dashboard password. This password was not enabled and was only added as an example.

Support setting the supervisor autorestart flag via the environment. This adds the environment variable MAYAN_SUPERVISOR_AUTORESTART. This variable defaults to true for the Supervisod template, and defaults to false for the Docker Supervisord template.

Documentation

Added a new documentation app. The documentation code and the Sphinx dependencies were moved to it.

The documentation file that outlines the features is now built dynamically from the enabled apps.

Fixed old inter document documentation references.

Updated the file .gitignore to not include final app documentations builds.

Documents

Adjusted the document version list and document file list permission requirements. The permission grant can now be more granular.

Converter errors that occur during document page preview generation are now logged on the corresponding source document file object.

We have expanded the functionality of the template sandbox to include support for document files and document versions. This enhancement enables users to test templates with these other types of content.

The intervals of tasks task_document_type_document_stubs_delete, task_document_type_document_trash_periods_check,
task_document_type_trashed_document_delete_periods_check are now configurable via the new settings DOCUMENTS_TRASH_PERIOD_CHECK_TASK_INTERVAL, DOCUMENTS_STUBS_DELETE_TASK_INTERVAL, and
DOCUMENTS_TRASHED_DOCUMENT_DELETE_PERIODS_CHECK_TASK_INTERVAL.

The default periodic intervals for the tasks that check the document trashing, trashed document deletion, and document stub pruning tasks were raised to 30 minutes.

Support was added for searching documents by language.

Document comments

Added help texts to the document comment model.

The document comments field is now available for linking.

Document linking

Fieldsets were added to the conditions form.

The select2-templating widget class is now used for the foreign document data field making field exploration and usage much easier.

The select2 class is now used for the inclusion and operator fields to improve user experience.

Events

Event ensure event namespaces are checked to be unique.

Event namespaces instances are now singletons. They are cached in memory and reused.

The calculated event types are now loaded at startup to pre-cache them. This adds a small delay at startup but results in faster event commits once the system is running. These changes also allowed removing event type refresh code in views and in the API eliminating multiple blocks of
code. The event type cache system and the event views are now fully idempotent.

A new feature was added to prune old events. This feature is controlled by a new backend system and adds the settings EVENTS_PRUNE_BACKEND, EVENTS_PRUNE_BACKEND_ARGUMENTS and EVENTS_PRUNE_TASK_INTERVAL (defaults to 30 days).

The default event pruning backends included are:
EventLogPruneBackendLatest, EventLogPruneBackendLatestPerObject, EventLogPruneBackendLatestPerObjectEventType, EventLogPruneBackendOlderThanDays.

File metadata

Added a PDF file metadata driver using PyPDF. This driver specializes in extracting metadata embedded in PDF files.

Added error logging to the file metadata drivers. This will cache errors during file metadata processing such as when the Ollama LLM server is offline or unreachable.

Icons

Tweaked the icon spacing and switched to use the font awesome fixed width icons style.

Moved all icon related code and assets to a new icons app.

MIME types

Removed the MIME type detection backend based on Python Magic.

OCR

We have enhanced the system's integration with the Tesseract OCR (Optical Character Recognition) engine by improving the method for passing custom arguments to the backend. This update provides greater flexibility and control when configuring the OCR processing.

Search

Support disabling the search app and the search system completely. Disabling the search app will also disable search indexing processing.

Also added support for disabling individual search model fields. This is controlled via the new setting named SEARCH_MODEL_FIELD_DISABLE.

Added two new information views to the tools menu, one view shows search model details while the other presents individual search field information.

Added the description, search field class label, model field class label to the search field API serializer. This exports more search field information via the API and matches the information presented in the new tool menu views.

Fixed the regression that limited the Whoosh search results.

Settings

Since Mayan EDMS is used in many business critical deployments and based on real life scenarios, we have enhanced the system’s ability to recover from configuration-related issues by introducing support for ignoring formatting errors, corrupted configuration files, or invalid
environment variable settings. This change enables the system to start even in scenarios where such errors occur. Please note that this feature does not apply to bootstrap settings, which are critical low-level configurations required for the base system to initialize successfully. This change added the setting SETTINGS_IGNORE_ERRORS which defaults to True.

Fixed the management command settings_show internal interface usage.

Added a column to show if a setting had errors during loading and reverted to the default.

Updated the Setting and SettingNamespace classes to be singletons. Calling the creation of a new setting in a namespace or a namespace in a cluster with a duplicated identifier will no longer raise and error. Instead the previously created instance will be returned.

Split the setting classes into separate modules.

Sources

The document file source metadata is now available for linking in the document linking app.

The file upload success icon now persist to help identify successful upload when submitting a large number of files.

Storage

Added support for opening PDF as archives. This allows extracting PDF attachments.

Added a sharded directory filesystem storage backend.

Task manager

Added a task type information view.

Improved the worker, queue, and task type list views to include more information, memory size formatting, and view inter-linking.

Templating

This release add many new tags and filters. The following filters were added:

  • json_load filter.
  • dictionary_flatten filter.
  • object_flatten filter.
  • New math filters implemented locally. This change deprecates the
    math filters provided by the django-mathfilters libray which will be
    removed in the next minor version.
  • dictionary_get filter. This deprecates the existing filter dict_get.
    Both operate exactly the same differing only by name.
  • Tag version of filter date_parse to allow passing arguments. This
    deprecates the filter version and will be removed in a future minor
    version.
  • Filter date_parse_iso to parse ISO 8601 dates.
  • range tag.

The math filters are prefixed with math_. The full list of math filters
is now: math_add, math_absolute, math_divide, math_exponentiate,
math_floor_divide, math_modulo, math_multiply, math_square_root,
math_substract.

Fixed the flatten_map not using the separator argument.

Ensured the select2 fields also apply the markup template to the selection as well as the result.

Improved the .select2-templating entry template to not assume all entries have descriptions.

Generalized the document sandbox to work with other models.

Added the template sandbox API endpoint.

Added a new model property information tool view.

Support height change persistence for the template string, preview and result fields. This change also add a reset button to the height change persistence elements.

Added a button to copy template code to the clipboard.

Convert the .select2-templating elements code to execute only on when new content loads.

User management

Removed overridden user model meta ordering value.

Added help texts to the user first name and last name fields.

Views

Added support for remembering the use choice to ignore confirmation dialogs. Applies to action confirmations, single object deletion, multi object deletion.

Improved the columns of the user view modes by replacing the app config representation with the app config verbose name.

Workflows

Added a flag to workflow template states to identify them as the final state of the workflow. The final state is identified in the state list of the workflow template as well as marked with a darker color in the preview.

Changed the behavior of the workflow instance creation. It is no longer possible to launch workflows that do not have one state set as initial.

Log an error when attempting to launch a workflow without an initial state.

Fixed the context of the workflow launch view when only one document is selected.

Updated the logic to calculate the current state of a workflow instance. This new method is several times faster than the previous implementation.

Added model property workflow.\< workflow internal name \>.state_active. This deprecates workflow.\< workflow internal name \>.get_current_state.

Made the document workflow instance active state column sortable.

Made the document workflow instance completion column sortable.

Added support to ignore completed workflows. Documents in these workflows will not show up in the workflow main menu. They are also excluded from escalation checking and transition trigger processing. This is another change that not only improves user experience but leads to big performance gain in deployments with large number of running workflows.

Added the ignore_completed column to workflow templates.

Added the auto_launch column to workflow templates.

Optimized the valid transition calculation by converting it into a Django query filter and using the exists() method.

Fixed workflow model completion model property typo.

Added help texts to the workflow instance fields: workflow, document, state active.

Added help text to workflow state actions informing that they will be execute in alphabetical order.

Other and development

A large number of changes were made to the internals and under the hood.

The following makefile target were renamed:

  • generate-setup to python-setup-generate
  • increase-version to version-increase
  • generate-requirements to requirements-generate

All instances of the prefix self.TestModel were renamed to self._TestModel to prevent possible collisions with other test scaffolding objects and for uniformity with other test objects being
made semi private a similar way.

Reorganized the base test case mixins. Ensure ID randomizer has higher priority followed by the event test mixin.

Workflow tests are now tagged at the mixin level. Metadata tests are now tagged at the mixin level.

Renamed links and icons to follow the great to lesser nomenclature.

Ensured random.seed is called only once per test suit execution.

Changed how the navigation classes detected querysets. This allows the navigation code to work with non model objects that have a .model attribute.

Moved the SilenceLoggerTestCaseMixin mixin to the logging app.

Renamed the mixin SilenceLoggerTestCaseMixin to TestCaseMixinSilenceLogger.

Moved the ClientMethodsTestCaseMixin, DownloadTestCaseMixin, TestServerTestCaseMixin, TestViewTestCaseMixin mixins to the views app.

Moved the ConnectionsCheckTestCaseMixin, ModelTestCaseMixin, TestModelTestCaseMixin, RandomPrimaryKeyModelMonkeyPatchMixin to the databases app.

Moved the DescriptorLeakCheckTestCaseMixin, OpenFileCheckTestCaseMixin, TempfileCheckTestCasekMixin to the storage app.

Moved the psutil dependency to the storage app.

Moved nested field label and help text extraction from the navigation app to the databases app and made it general purpose.

Added label_for_field_recursive and help_text_for_field_recursive utility functions.

Override Django’s makemigrations command to support ignoring third party apps.

Renamed ViewMixinOwnerPlusFilteredQueryset.optional_object_permission to object_optional_permission for consistency.

Centralized the AJAX content update logic.

Renamed the AJAX content events for clarity.

Cached the AJAX content element selector.

Trigger the AJAX content updated event only when the content is ready for manipulation.

Removed the unused JavaScript formBeforeSerializeCallbacks.

Removed hard-coded id_list from views and JavaScript.

URL query string are now composed using the URI API.

Replaced string concatenation with template literals.

Document indexing app updates:

  • Split test mixins.
  • Improve test mixin class inheritance. Reduces imports and test scaffolding.

Tag event tests at the mixin level.

Removed the dropzone widget hard coded icons. There are no immediate visual changes. This makes is possible to change the icons of the widget without changing template code.

Moved the document file parsing event commit outside of the document file parsing cleanup transaction loop.

Explicitly cleanup test only event types.

Support deleting event types and event type namespaces.

Ensure even types are unique per event type namespace.

Support unregistering model event types.

Improved test mixin class inheritance. Reduces imports and test scaffolding.

Removed the hard-coded CD/CI and Docker package names.

Move the GitLab CI platform template into its own app named platform_gitlab.

Reduce the SSH connections used by the GitLab CI deploy stage.

Support separate GitLab CI registry push and deploy credentials.

Reduce the demo stage SSH connections.

Reduce the demo stage downtime.

Split templating app template tag into separate modules.

Split the user management app test and test mixin modules.

Moved the is_url_query_positive utility from the search to the views app.

Formalize the pattern used to get all the request GET and POST data.

Unified the label of empty dropdown options.

Use the best possible model label when composing the search form fieldset labels.

Updated ObjectLinkWidget to work with objects other than models.

Split the workflow template state action test module.

Moved workflow model business logic code to the models mixins module.

Moved transition trigger code from the handler to the manager module.

Known issues

The output message warning about missing migration for the auth app is incorrect:

` Your models in app(s): 'auth' have changes that are not yet reflected in a migration, and so won't be applied. Run 'manage.py makemigrations' to make new migrations, and then [re-run]{.pre} 'manage.py migrate' to apply them. `

This is due to help text changes that Mayan EDMS adds to Django models at runtime. These changes do not alter the database schema are a just to workaround the help text that Django itself is missing from the user model.