search: enable AND operator and cross_field

[ntarocco]

Original issue:

• Book cannot be found: https://catalogue.library.cern/literature/0s8wv-bxz77
• Query: https://catalogue.library.cern/search?q=quantum%20field%20theory%20schwartz&l=grid&order=asc&p=1&s=15&sort=bestmatch
• NEEDS mapping: update mappings inveniosoftware/invenio-app-ils#1221

[sakshamarora1]

The original example mentioned in the inline comment wasn't the complete case. I updated with the correct explanation according to my testing and observations.

Essentially, we can not use cross_fields with AND operator as the operator forces both the terms to be in the same field which means it ignores the cross_fields type.

If we remove the AND operator, it works but will increase the number of results that would loosely match due to the default OR.

The best way I was able to find was to inject "AND" keyword between each word, this seems to solve the problem completely but it is not an elegant solution.

[zzacharo]

Here some tests in devtools of opensearch (from @ntarocco ):

# 0s8wv-bxz77
GET /documents/_search
{
  "profile": true,
  "query": {
    "query_string": {
      "query": "quantum AND schwartz",
      "fields": ["title", "authors.full_name"]
    }
  }
}

GET /documents/_search
{
  "profile": false,
  "query": {
    "query_string": {
      "query": "quantum schwartz",
      "type": "cross_fields",
      "default_operator": "AND"
    }
  }
}

GET /documents/_search
{
  "profile": true,
  "query": {
    "query_string": {
      "query": "quantum field theory schwartz",
      "type": "cross_fields",
      "default_operator": "AND",
      "fields": ["title", "authors.full_name"]
    }
  }
}

GET /documents/_search
{
  "profile": false,
  "query": {
    "bool": {
      "must": [
        {"query_string": {
      "query": "field schwartz",
      "type": "cross_fields",
      "default_operator": "AND",
      "fields": ["title", "*"]
    }},
    {"term": {
      "pid": {
        "value": "0s8wv-bxz77"
      }
    }}
    
      ]
    }
    
  }
}

[zzacharo]

Working query

GET /documents/_search 
{
  "profile": true,
  "query": {
    "query_string": {
      "query": "quantum schwartz",
      "type": "cross_fields",
      "default_operator": "AND",
    "analyzer": "standard"
    }
  }
}

Findings

Cross field queries: https://www.elastic.co/guide/en/elasticsearch/guide/current/_cross_fields_queries.html

For the cross_fields query type to work optimally, all fields should have the same analyzer. Fields that share an analyzer are grouped together as blended fields.
If you include fields with a different analysis chain, they will be added to the query in the same way as for best_fields. For instance, if we added the title field to the preceding query (assuming it uses a different analyzer), the explanation would be as follows:
(+title:peter +title:smith)

It seems that fields title and authors.full_name use a different analyzer. Forcing the analyzer on the query level as above seems to make the cross_fields type work as expected.