Searching for multiple tags does not return any results

When things don't work as they should.
Post Reply
evilphish
Posts: 1
Joined: Sun Sep 20, 2020 10:47 am

Searching for multiple tags does not return any results

Post by evilphish »

Greetings,

I am running Mayan 3.4.17 on Debian GNU/Linux and currently experience issues when searching for multiple tags at once.

The situation is as follows:
- 6 documents in the system
- all documents are tagged with the tag Invoice
- 2 documents are (in addition) tagged with Heating
- 2 documents are (in addition) tagged with Water
- 1 document is (in addition) tagged with Renovation

I observe the following:
- Searching for invoice returns all 6 documents (correct)
- Searching for heating returns the 2 documents that have heating as a tag (correct)
- Searching for invoice heating returns 0 documents (incorrect, should be 2)
- Searching for invoice water returns 0 documents (incorrect, should be 2)
- Searching for invoice renovation returns 0 documents (incorrect, should be 1)
- Searching for water OR heating returns 4 documents (correct)
- Searching for invoice OR water returns 6 documents (correct)

Of course all other AND searching regarding two tags that exist on the same document (heating and invoice, water and invoice) always return zero documents. All other searches involving multiple keywords work as expected (i.e. searching metadata, file contents, etc). The problem seems to be only with tags.

Any ideas? Am I doing something wrong here? :)

cheers
Alex
User avatar
rosarior
Developer
Developer
Posts: 651
Joined: Tue Aug 21, 2018 3:28 am
Location: Puerto Rico
Contact:

Re: Searching for multiple tags does not return any results

Post by rosarior »

Thanks for the report!

We'll take a look to find the cause of the behavior.
tom24
Posts: 1
Joined: Sun Mar 01, 2020 6:31 pm

Re: Searching for multiple tags does not return any results

Post by tom24 »

Hi,

are there news in this problematic behavior? I have the same issue since a long time, but actually it would be more difficult to use mayan. In the past we search for one tag and sort the result document to document, but now we have so many docs that this workflow comes to the end.
I habe enabled the woosh search engine, without success.

Thank you for your great work and this great and super useful EDMS.
Thomas
User avatar
michael
Developer
Developer
Posts: 89
Joined: Sun Apr 19, 2020 6:21 am

Re: Searching for multiple tags does not return any results

Post by michael »

Howdy,

We have come to understand this behavior and it is not a bug but a misinterpretation of how search works.

When searching for tags, these are indexed by their label as text strings and not as individual objects. When searching for invoice heating, the search app will look for documents with labels that have the words (or terms) invoice AND heating. This happens because the search app is a text search engine and not an object search engine.

To improve on this, we added support for search scopes. This allows the search app to also work as a query app. Search scopes allow for complex search algebra that mimics SQL. Each search scope is equivalent to an entire search form. Scopes are reduced by pairs using operators (AND, OR, NOT) and their results become runtime scopes, which themselves are reduced against other user defined or runtime scopes. You can think of scope as mathematical parenthesis. To keep backwards compatibility, when no scopes are specified, the search syntax will emulate single scope named "0".

In the case of tags, the search app is doing the following search operation.

Code: Select all

Scope 0 = (Tag label ~ "invoice" AND Tab label ~ "heating")
However to search for each individual term isolated (to simulate different tag objects), the search query would be:

Code: Select all

Scope 3 (Result) = Scope 1(Tag label ~ "invoice") AND Scope 2(Tab label ~ "heating")
What this does is perform two separate searches, which are treated as sets, and then perform an AND operation (or intersection) and provide that result.

Search scopes are very powerful but also hard to create an interface, so we decided to add the backend support now and roll out the user interface changes in the next version.

The search scopes feature is available via the app URL or the API and works regardless of the underlying search backend (Django or Whoosh) because it works in a different layer of the search stack.

The above example would translate to the following URL:

Code: Select all

/search/results/documents.DocumentSearchResult/?__1_tags__label=invoice&__2_tags_label=heating&__operator_0_1=AND_3&__result=3
  • Code: Select all

    /search/results/documents.DocumentSearchResult/
    URL of the search app with the model you want to search (the Document proxy named DocumentSearchResult in this case).
  • Code: Select all

    __1_tags__label=invoice
    Defines the scope named "1" which is a text query for the string invoice in the tags labels.
  • Code: Select all

    __2_tags_label=heating
    Defines the scope named "2" which is another text query for the tag label string invoice.
  • Code: Select all

    __operator_1_2=AND_3
    Sets the mathematical operation that will be used to reduce scopes "1" together with "2" and store the result into a new scope named "3".
  • Code: Select all

    __result=3
    This tells the search app which scope you want returned as the result. This allows getting different results in order to allow testing your query without having to rewrite it all every time.
We are testing several new user interfaces to interact with search scopes. One is an improvement to the existing search form but we have other interfaces to create the search scope but not actually execute it. There are more improvements and additions in the pipeline for the search scope and search app, but I'm getting ahead of myself.

I'm personally working on this subproject and it very exciting because it turn Mayan into something we have not seen before: A perfectly extensive, document search and query engine :)
Post Reply