[jira] [Created] (LUCENE-9657) Unified Highlighter throws too_complex_to_determinize_exception with >288 filter terms

["Isaac Doub (Jira)" <ji...@apache.org>]

Isaac Doub created LUCENE-9657:
----------------------------------

             Summary: Unified Highlighter throws too_complex_to_determinize_exception with >288 filter terms
                 Key: LUCENE-9657
                 URL: https://issues.apache.org/jira/browse/LUCENE-9657
             Project: Lucene - Core
          Issue Type: Bug
    Affects Versions: 8.6.2
            Reporter: Isaac Doub


There seems to be a problem with the Unified Highlighter in Lucene 8.6.2 that is affecting ElasticSearch 7.9.1. If a search is performed with >288 filter terms using the unified highlighter it throws a too_complex_to_determinize_exception error, but if you switch to the plain highlighter it works fine. Alternatively, if you filter on a "copy_to" field instead of the indexed field, it also works.
 
This throws the error
{code:java}
 {
    "highlight": {
        "type": "unified",
        "fields": {
            "title": {
                "require_field_match": false
            }
        }
    },
    "query": {
        "bool": {
            "must": [{
                "query_string": {
                    "query": "*"
                }
            }],
            "filter": [{
                "bool": {
                    "must": [{
                        "terms": {
                            "id": [ ">288 terms here" ]
                        }
                    }]
                }
            }]
        }
    }
}{code}
 
 
But this works fine
{code:java}
 {
    "highlight": {
        "type": "plain",
        "fields": {
            "title": {
                "require_field_match": false
            }
        }
    },
    "query": {
        "bool": {
            "must": [{
                "query_string": {
                    "query": "*"
                }
            }],
            "filter": [{
                "bool": {
                    "must": [{
                        "terms": {
                            "id": [ ">288 terms here" ]
                        }
                    }]
                }
            }]
        }
    }
}{code}
 
 
Or if I adjust the search to use the copy_to field it works as well (note "id" is now "_id")
{code:java}
 {
    "highlight": {
        "type": "unified",
        "fields": {
            "title": {
                "require_field_match": false
            }
        }
    },
    "query": {
        "bool": {
            "must": [{
                "query_string": {
                    "query": "*"
                }
            }],
            "filter": [{
                "bool": {
                    "must": [{
                        "terms": {
                            "_id": [ ">288 terms here" ]
                        }
                    }]
                }
            }]
        }
    }
}{code}
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org