[jira] [Comment Edited] (LUCENE-10509) Performance degraded after upgrade from 8.8.2 to 8.9.0

["Rajesh T (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-10509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17525652#comment-17525652 ] 

Rajesh T edited comment on LUCENE-10509 at 4/21/22 11:59 AM:
-------------------------------------------------------------

[~jpountz] I have made the change in our custom plugin as you suggested. There was no performance improvement even after the change. Please have a look at the below code which is running slower with lucene 8.9.0. For 1M documents, below code is executed 290 times/minute with 8.8.2 version, and 140 times/minutes with 8.9.0 version.

Time taken to execute one iteration (with lucene 8.8.2): 3.45 ms

Time taken to execute one iteration (with lucene 8.9.0): 7.14 ms
{code:java}
SortedSetDocValues values = ((ValuesSource.Bytes.WithOrdinals.FieldData) valuesSource).ordinalsValues(leafReaderContext);
BitSet ordSet = new BitSet();
int doc = values.nextDoc();
while (doc != DocIdSetIterator.NO_MORE_DOCS) {
    while (true) {
        long ord = values.nextOrd();
        if (ord == SortedSetDocValues.NO_MORE_ORDS) {
            break;
        }
        ordSet.set((int) ord);
    }
    doc = values.nextDoc();
}
ordSet.stream().forEach(x -> {
    try {
        byte[] bytes = values.lookupOrd(x).bytes;
    } catch (IOException e) {
        e.printStackTrace();
    }
});{code}


was (Author: JIRAUSER287787):
[~jpountz] I have made the change in our custom plugin as you suggested. There was no performance improvement even after the change. Please have a look at the below code which is running slower with lucene 8.9.0. For 1M documents, below code is executed 290 times/minute with 8.8.2 version, and 140 times/minutes with 8.9.0 version. 

 
{code:java}
SortedSetDocValues values = ((ValuesSource.Bytes.WithOrdinals.FieldData) valuesSource).ordinalsValues(leafReaderContext);
BitSet ordSet = new BitSet();
int doc = values.nextDoc();
while (doc != DocIdSetIterator.NO_MORE_DOCS) {
    while (true) {
        long ord = values.nextOrd();
        if (ord == SortedSetDocValues.NO_MORE_ORDS) {
            break;
        }
        ordSet.set((int) ord);
    }
    doc = values.nextDoc();
}
ordSet.stream().forEach(x -> {
    try {
        byte[] bytes = values.lookupOrd(x).bytes;
    } catch (IOException e) {
        e.printStackTrace();
    }
});
 {code}
 

 

> Performance degraded after upgrade from 8.8.2 to 8.9.0
> ------------------------------------------------------
>
>                 Key: LUCENE-10509
>                 URL: https://issues.apache.org/jira/browse/LUCENE-10509
>             Project: Lucene - Core
>          Issue Type: Bug
>    Affects Versions: 8.8.2
>            Reporter: Rajesh T
>            Priority: Minor
>
> We are planning to upgrade from elasticsearch 7.7.1 to opensearch 1.1.0 (with lucene version 8.9.0). We have noticed that the performance of opensearch 1.1.0 is worse than elasticsearch 7.7.1 due to lucene 8.9.0 version. Whereas performance of opensearch 1.0.1 (with lucene version 8.8.2) is almost same as elasticsearch 7.7.1 version.
> We have tested following scenarios and observed the slowness is caused by lucene 8.9.0 version. The performance is degraded by 50% for cardinality aggregations.
> Elasticsearch 7.7.1 version (with lucene-core 8.5.1) : Fast
> OpenSearch 1.0.1 version (with lucene-core 8.8.2) : Fast
> OpenSearch 1.1.0 version (with lucene-core 8.9.0) : Slow
> OpenSearch 1.1.0 version (with lucene-core 8.8.2) : Fast
> This is the snippet of openseach code which is running slow with lucene 8.9.0
> {code:java}
> QueryBuilder qb = QueryBuilders.boolQuery().mustNot(QueryBuilders.termQuery("__id.keyword", randomValue));
> CardinalityAggregationBuilder agg = AggregationBuilders
>         .cardinality("somename")
>         .field("__id.keyword");
> return client.prepareSearch(index).setQuery(qb).addAggregation(agg);
>  {code}
> Please let us know if this is something that can be fixed.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org