Indexing Error

When things don't work as they should.
Post Reply
briboe
Posts: 14
Joined: Tue Aug 20, 2019 1:26 pm

Indexing Error

Post by briboe »

I was working on some new indexes and while rebuilding them received the following error:

[2019-09-10 04:11:52,006: ERROR/MainProcess] Task mayan.apps.document_indexing.tasks.task_rebuild_index[7e18a99c-4fd0-4560-9633-651e8b7805a0] raised unexpected: MultipleObjectsReturned('get() returned more than one IndexInstanceNode -- it returned 2!',)
Traceback (most recent call last):
File "/opt/mayan-edms/local/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
R = retval = fun(*args, **kwargs)
File "/opt/mayan-edms/local/lib/python2.7/site-packages/celery/app/trace.py", line 438, in __protected_call__
return self.run(*args, **kwargs)
File "/opt/mayan-edms/local/lib/python2.7/site-packages/mayan/apps/document_indexing/tasks.py", line 68, in task_rebuild_index
index.rebuild()
File "/opt/mayan-edms/local/lib/python2.7/site-packages/mayan/apps/document_indexing/models.py", line 155, in rebuild
self.index_document(document=document)
File "/opt/mayan-edms/local/lib/python2.7/site-packages/mayan/apps/document_indexing/models.py", line 121, in index_document
for index_instance_node in self.instance_root.get_leafnodes():
File "/opt/mayan-edms/local/lib/python2.7/site-packages/mayan/apps/document_indexing/models.py", line 128, in instance_root
return self.template_root.get_instance_root_node()
File "/opt/mayan-edms/local/lib/python2.7/site-packages/mayan/apps/document_indexing/models.py", line 260, in get_instance_root_node
index_instance_root_node, create = self.index_instance_nodes.get_or_create(parent=None)
File "/opt/mayan-edms/local/lib/python2.7/site-packages/django/db/models/fields/related_descriptors.py", line 659, in get_or_create
return super(RelatedManager, self.db_manager(db)).get_or_create(**kwargs)
File "/opt/mayan-edms/local/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/opt/mayan-edms/local/lib/python2.7/site-packages/django/db/models/query.py", line 464, in get_or_create
return self.get(**lookup), False
File "/opt/mayan-edms/local/lib/python2.7/site-packages/django/db/models/query.py", line 384, in get
(self.model._meta.object_name, num)
Exception: get() returned more than one IndexInstanceNode -- it returned 2!

Now I get a blank screen when I try to view my indexes.

Tried deleting the child node I was working on but didn't help.
Not sure what to do next.
User avatar
rosarior
Developer
Developer
Posts: 624
Joined: Tue Aug 21, 2018 3:28 am
Location: Puerto Rico
Contact:

Re: Indexing Error

Post by rosarior »

Thanks for the report.

Reading the debug I can see understand the reason for the error. A query that can only return a single value, the runtime instance of an index template node, returned two value, which is impossible. I'm guessing an index rebuild or update was executing in the background while the index template was being edited. This resulted in two runtime instances of the same index template node. Is your Mayan instance running on two machines. For example to load balance?

This one will be challenging to fix. A way to detect that an index template is being edited must be found and used to block updates to the index from happening.

In the meantime we will discuss the option of adding a tool button that fully resets the index and deletes all runtime instances. This will make the rebuild take longer but will result in an index runtime instance that's guaranteed to be clean and consistent.
briboe
Posts: 14
Joined: Tue Aug 20, 2019 1:26 pm

Re: Indexing Error

Post by briboe »

What you are saying sounds like what happened. I was playing around with indexes trying to learn the syntax so it is quite possible I was editing and rebuilding at the same time.

Only running on one machine so not load balanced. Just a test implementation.

I have disabled the index it seems to be happening on and everything else seems to be OK. Will it be a problem if I just delete the index and re-create with the same name or do I need to have a different name/slug?

Appreciate your prompt reply and the good explanation.

Really enjoying working with Mayan.
vanconge
Posts: 3
Joined: Thu Dec 17, 2020 2:50 pm

Re: Indexing Error

Post by vanconge »

I am sorry to bump an old thread, but this is the only Google search result so thought it would be useful to get some further info here.

I am encountering the same issue with indexing, and suspect it's because I've killed the container that Mayan is running on during an index operation, or something similar that has caused indexing issues.

Is there a way to clear this error and start with a fresh index? Something that can be deleted manually in the database through pgadmin for example?

Thanks!
VPC
Posts: 1
Joined: Mon Sep 14, 2020 2:49 pm

Re: Indexing Error

Post by VPC »

This just happened to me as well.

I was moving my docker based installation from one docker host to another. I backed up my docker volumes and restored them to the new docker host. Everything came up with Mayan as expected on the new docker host except indexing. In my case

returned more than one IndexInstanceNode -- it returned 4

In order to fix this I had to delete all of the indexes from the new Mayan and start over. Luckily I still had the old installation to refer back to. Going forwards I will document my index setups in case this happens again.
vanconge
Posts: 3
Joined: Thu Dec 17, 2020 2:50 pm

Re: Indexing Error

Post by vanconge »

Yes, this is what I did as well. Just deleted and recreated.

Would be great if there was an option in the UI to recreate the index in the database without having to delete and recreate manually.
User avatar
rosarior
Developer
Developer
Posts: 624
Joined: Tue Aug 21, 2018 3:28 am
Location: Puerto Rico
Contact:

Re: Indexing Error

Post by rosarior »

Index instances can be deleted from the Tools menus.

Once deleted, they can be cleanly recreated from scratch.
2021-01-05_04-06_1.png
2021-01-05_04-06_1.png (15.24 KiB) Viewed 248 times
2021-01-05_04-06.png
2021-01-05_04-06.png (109.17 KiB) Viewed 248 times
vanconge
Posts: 3
Joined: Thu Dec 17, 2020 2:50 pm

Re: Indexing Error

Post by vanconge »

Thank you for this, I used this option to recover from that situation, but it would be good to automate it as well (i.e. delete and recreate the indexes from the UI by having a "fix broken indexes" option or something that did this for you).
Post Reply