You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "LuXugang (via GitHub)" <gi...@apache.org> on 2023/06/14 12:42:23 UTC
[GitHub] [lucene] LuXugang opened a new issue, #12370: Improve comparator when search sort field does not exist
LuXugang opened a new issue, #12370:
URL: https://github.com/apache/lucene/issues/12370
### Description
```java
public void test111() throws IOException{
Directory dir = newDirectory();
IndexWriterConfig iwc = new IndexWriterConfig(new MockAnalyzer(random()));
RandomIndexWriter indexWriter = new RandomIndexWriter(random(), dir, iwc);
Document doc;
Random random = new Random();
int count = 0;
while (count++ < 100000){
doc = new Document();
doc.add(new SortedSetDocValuesField("sortedSet", new BytesRef("a")));
doc.add(new StringField("name", String.valueOf(random.nextInt(1000000)), StringField.Store.YES));
indexWriter.addDocument(doc);
}
indexWriter.commit();
IndexReader reader = indexWriter.getReader();
IndexSearcher searcher = newSearcher(reader);
assert reader.maxDoc() == 100000;
Query query = new MatchAllDocsQuery();
Sort sort = new Sort(new SortedSetSortField("field no exist ", false));
TopDocs noSearchField = searcher.search(query, 2000);
assert noSearchField.totalHits.value == 2001;
TopDocs hasSearchField = searcher.search(query, 2000, sort);
// if the search sort field is not exist, should early terminate after Top 2000 collected?
assert hasSearchField.totalHits.value == 100000;
indexWriter.close();
reader.close();
dir.close();
}
```
If search sort field does not exist, should we early terminate collection after TopN collected?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Improve comparator when search sort field does not exist [lucene]
Posted by "serhiy-bzhezytskyy (via GitHub)" <gi...@apache.org>.
serhiy-bzhezytskyy commented on issue #12370:
URL: https://github.com/apache/lucene/issues/12370#issuecomment-5035665408
I put up a PR for this: #16411.
Root cause: when the sort field is missing from the whole index, every doc ties on the missing value, so with no tie-breaker the remaining docs are non-competitive once the top-N queue is full — but `TermOrdValComparator` didn't recognize that and scanned every document. The PR mirrors the existing `singleSort` competitive-iterator recognition to the `minOrd` path (gated on `bottomValue == null`) so it early-terminates. Results are unchanged; it's purely an optimization, with a regression test in `TestSortOptimization`.
Happy to adjust the approach if there's a preferred way to handle it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Improve comparator when search sort field does not exist [lucene]
Posted by "romseygeek (via GitHub)" <gi...@apache.org>.
romseygeek closed issue #12370: Improve comparator when search sort field does not exist
URL: https://github.com/apache/lucene/issues/12370
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org