[incubator-pinot] 01/01: Use docBase from leaf context for multi-segment search

[si...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

siddteotia pushed a commit to branch txt-hotfix
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git

commit 3e1326fbc009e597d8dc7b17aca86bc4370bcf0d
Author: Siddharth Teotia <st...@steotia-mn1.linkedin.biz>
AuthorDate: Mon Oct 26 09:39:37 2020 -0700

    Use docBase from leaf context for multi-segment search
---
 .../pinot/core/segment/index/readers/text/LuceneDocIdCollector.java | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/text/LuceneDocIdCollector.java b/pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/text/LuceneDocIdCollector.java
index b24667b..723dd00 100644
--- a/pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/text/LuceneDocIdCollector.java
+++ b/pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/text/LuceneDocIdCollector.java
@@ -67,7 +67,11 @@ public class LuceneDocIdCollector implements Collector {
 
       @Override
       public void collect(int doc) throws IOException {
-        _docIds.add(_docIdTranslator.getPinotDocId(doc));
+        // even though we merge lucene sub-indexes, there could still be cases
+        // that can lead to multiple sub-indexes. For search on multiple
+        // sub-indexes, we need to use compute the absolute lucene docID across
+        // sub-indexes because that's how the lookup table in docIdTranslator is built
+        _docIds.add(_docIdTranslator.getPinotDocId(context.docBase + doc));
       }
     };
   }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org