Error while indexing over metadata values

Using Mayan EDMS Bugs or errors
1

First one big thank for this software. But now I discovered a first bug/discrepancy while using it.

There is a discrepancy with the field sizes of metadata values and values of indexes.
The following line numbers belong to the code at the tag 4.4.2.
It can be seen in apps/metadata/models.py at line 181ff, where the Value of DocumentMetadata is declared as a TextField (which I would prefer) and in apps/document_indexing/models/index_instance_models.py at line 61 where it is a models.CharField with max_length 128.
So if there is a value which is longer than these 128 characters in the metadatafield value, then the indexing process is aborting with a stacktrace (and the further IndexInstanceNodes for this index are not created)

I would prefer to be both available as TextFields, so that we can have values longer than 128 characters in IndexNodes. Or at least the indexing process should have the shortened value in the IndexInstanceNode, so it continue to build the full index, instead of aborting.

2

See the bug reports about the subject:

You can manually change this in the db as I did though.

3

Hi,

This is not a bug but a design necessity. Historically databases had issue indexing text fields (SQL varchar) and this is why char field (SQL char) is used on models that require speed when queried. Would have to delve deeper to see if this is still the case in 2023.

Document indexes are created and updated many times and speed is very important. Unless there is evidence that changing the field to Text won’t cause major issues it is unlikely to be changed.

Increased the field from 128 to 255 characters to allow bigger values but kept it a Char until testing can be done to ensure changing to Text won’t cause major problems or severe regressions.