Migration to Lucene 6.5 - Filters to Queries

[Rilpa Jain <ri...@gmail.com>]

> Hi,
>  
> We plan to migrate from lucene 5.5 to 6.5. We have been using DocValuesTermsFilter extensively which was deprecated in Lucene 5.5 and removed in Lucene 6.0.
> The Javadoc specifies to use DocValuesTermsQuery and BoolenaClause.Occur.Filter instead. However, as per our local tests, the time taken to search documents has increased with this change.
>  
> Below is one of the scenarios in our application -
> We do a search within a search.
>  
> (Before migration to Lucene 5.5)
> The first search is on a text field with discrete values. (There is no pattern to the value of this text field. Here the terms[] ranges from 1 to 200k in size.)  – We use DocValuesTermsFilter and pass it is as Filter parameter to search method.
> The second search is on result of step 1- This could be either a TermQuery or NumericRangeQuery, evaluated to query and added as query parameter to search method.
>  
> (After migration to Lucene 6.5)
> The first search is on a text field with discrete values. (There is no pattern to the value of this text field. Here the terms[] ranges from 1 to 200k in size.)  – We use DocValuesTermsQuery and add it to BooleanQuery with Occur.Filter.
> The second search is on result of step 1- This could be either a TermQuery or NumericRangeQuery added to BooleanQuery with Occur.MUST.
> The booleanQuery is build and passed to search method.
>  
> This query execution after migration takes 5x-10x times more as compared to using DocValuesTermsFilter.
>  
> Is there a better class to generate query in our scenario than the one used above? Or is there anything that I am missing?
> Any insights would help! Thanks.