You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by GitBox <gi...@apache.org> on 2021/09/27 22:48:21 UTC
[GitHub] [lucene] dnhatn opened a new pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
dnhatn opened a new pull request #324:
URL: https://github.com/apache/lucene/pull/324
The competitive iterator can wrongly skip a document that is advanced but not collected in the previous scoreRange.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] jpountz commented on a change in pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
jpountz commented on a change in pull request #324:
URL: https://github.com/apache/lucene/pull/324#discussion_r717275045
##########
File path: lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java
##########
@@ -292,7 +293,11 @@ public long cost() {
@Override
public int advance(int target) throws IOException {
- return docID = competitiveIterator.advance(target);
+ if (target <= startDocID) {
Review comment:
Can you leave a comment about why we need to do this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] dnhatn merged pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
dnhatn merged pull request #324:
URL: https://github.com/apache/lucene/pull/324
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] dnhatn commented on pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
dnhatn commented on pull request #324:
URL: https://github.com/apache/lucene/pull/324#issuecomment-929485273
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] dnhatn merged pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
dnhatn merged pull request #324:
URL: https://github.com/apache/lucene/pull/324
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] dnhatn commented on a change in pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
dnhatn commented on a change in pull request #324:
URL: https://github.com/apache/lucene/pull/324#discussion_r717864552
##########
File path: lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java
##########
@@ -271,30 +271,7 @@ public void visit(int docID, byte[] packedValue) {
@Override
public DocIdSetIterator competitiveIterator() {
- if (enableSkipping == false) return null;
- return new DocIdSetIterator() {
- private int docID = -1;
-
- @Override
- public int nextDoc() throws IOException {
- return advance(docID + 1);
- }
-
- @Override
- public int docID() {
- return docID;
- }
-
- @Override
- public long cost() {
- return competitiveIterator.cost();
- }
-
- @Override
- public int advance(int target) throws IOException {
- return docID = competitiveIterator.advance(target);
- }
- };
+ return enableSkipping ? competitiveIterator : null;
Review comment:
Good catch. I've reverted this change. Thanks Mayya!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mayya-sharipova commented on pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
mayya-sharipova commented on pull request #324:
URL: https://github.com/apache/lucene/pull/324#issuecomment-929275183
@dnhatn Thanks for spotting this bug. I was wondering if instead of `NumericComparator`, we should make this change in `DefaultBulkScorer::score(LeafCollector collector, Bits acceptDocs, int min, int max)`. Two reasons for this:
- This bug is originated from DefaultBulkScorer, we should not advance `filteredIterator.advance` if `collectorIterator` is already on `minDoc`
- this bug affects not only `NumericComparator` but also `DocComparator`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mayya-sharipova commented on a change in pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
mayya-sharipova commented on a change in pull request #324:
URL: https://github.com/apache/lucene/pull/324#discussion_r717623319
##########
File path: lucene/core/src/test/org/apache/lucene/search/TestSortOptimization.java
##########
@@ -730,4 +732,52 @@ public void testRandomLong() throws IOException {
reader.close();
dir.close();
}
+
+ private static class ChunkedBulkScorer extends BulkScorer {
+ private final BulkScorer in;
+
+ ChunkedBulkScorer(BulkScorer in) {
Review comment:
Great test addition!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mayya-sharipova commented on a change in pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
mayya-sharipova commented on a change in pull request #324:
URL: https://github.com/apache/lucene/pull/324#discussion_r717833807
##########
File path: lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java
##########
@@ -271,30 +271,7 @@ public void visit(int docID, byte[] packedValue) {
@Override
public DocIdSetIterator competitiveIterator() {
- if (enableSkipping == false) return null;
- return new DocIdSetIterator() {
- private int docID = -1;
-
- @Override
- public int nextDoc() throws IOException {
- return advance(docID + 1);
- }
-
- @Override
- public int docID() {
- return docID;
- }
-
- @Override
- public long cost() {
- return competitiveIterator.cost();
- }
-
- @Override
- public int advance(int target) throws IOException {
- return docID = competitiveIterator.advance(target);
- }
- };
+ return enableSkipping ? competitiveIterator : null;
Review comment:
In `DefaultBulkScorer:score`, we get:
```java
DocIdSetIterator competitiveIterator = collector.competitiveIterator();
```
If we use your current change, we will always get the same object, and even if `competitiveIterator` inside `NumericComparator` gets updated we will get the same object, while we want to get a new updated competitive iterator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mayya-sharipova commented on pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
mayya-sharipova commented on pull request #324:
URL: https://github.com/apache/lucene/pull/324#issuecomment-929275183
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mayya-sharipova commented on a change in pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
mayya-sharipova commented on a change in pull request #324:
URL: https://github.com/apache/lucene/pull/324#discussion_r717623319
##########
File path: lucene/core/src/test/org/apache/lucene/search/TestSortOptimization.java
##########
@@ -730,4 +732,52 @@ public void testRandomLong() throws IOException {
reader.close();
dir.close();
}
+
+ private static class ChunkedBulkScorer extends BulkScorer {
+ private final BulkScorer in;
+
+ ChunkedBulkScorer(BulkScorer in) {
Review comment:
Great test addition!
##########
File path: lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java
##########
@@ -271,30 +271,7 @@ public void visit(int docID, byte[] packedValue) {
@Override
public DocIdSetIterator competitiveIterator() {
- if (enableSkipping == false) return null;
- return new DocIdSetIterator() {
- private int docID = -1;
-
- @Override
- public int nextDoc() throws IOException {
- return advance(docID + 1);
- }
-
- @Override
- public int docID() {
- return docID;
- }
-
- @Override
- public long cost() {
- return competitiveIterator.cost();
- }
-
- @Override
- public int advance(int target) throws IOException {
- return docID = competitiveIterator.advance(target);
- }
- };
+ return enableSkipping ? competitiveIterator : null;
Review comment:
@dnhatn Thanks so much for working on this. This is a neat approach, but I don't think in the current form it will work, because `competitiveIterator` variable gets updated.
##########
File path: lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java
##########
@@ -271,30 +271,7 @@ public void visit(int docID, byte[] packedValue) {
@Override
public DocIdSetIterator competitiveIterator() {
- if (enableSkipping == false) return null;
- return new DocIdSetIterator() {
- private int docID = -1;
-
- @Override
- public int nextDoc() throws IOException {
- return advance(docID + 1);
- }
-
- @Override
- public int docID() {
- return docID;
- }
-
- @Override
- public long cost() {
- return competitiveIterator.cost();
- }
-
- @Override
- public int advance(int target) throws IOException {
- return docID = competitiveIterator.advance(target);
- }
- };
+ return enableSkipping ? competitiveIterator : null;
Review comment:
In `DefaultBulkScorer:score`, we get:
```java
DocIdSetIterator competitiveIterator = collector.competitiveIterator();
```
If we use your current change, we will always get the same object, and if `competitiveIterator` gets updated we will get the same object, while we want to get a new updated competitive iterator.
##########
File path: lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java
##########
@@ -271,30 +271,7 @@ public void visit(int docID, byte[] packedValue) {
@Override
public DocIdSetIterator competitiveIterator() {
- if (enableSkipping == false) return null;
- return new DocIdSetIterator() {
- private int docID = -1;
-
- @Override
- public int nextDoc() throws IOException {
- return advance(docID + 1);
- }
-
- @Override
- public int docID() {
- return docID;
- }
-
- @Override
- public long cost() {
- return competitiveIterator.cost();
- }
-
- @Override
- public int advance(int target) throws IOException {
- return docID = competitiveIterator.advance(target);
- }
- };
+ return enableSkipping ? competitiveIterator : null;
Review comment:
In `DefaultBulkScorer:score`, we get:
```java
DocIdSetIterator competitiveIterator = collector.competitiveIterator();
```
If we use your current change, we will always get the same object, and even if `competitiveIterator` gets updated we will get the same object, while we want to get a new updated competitive iterator.
##########
File path: lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java
##########
@@ -271,30 +271,7 @@ public void visit(int docID, byte[] packedValue) {
@Override
public DocIdSetIterator competitiveIterator() {
- if (enableSkipping == false) return null;
- return new DocIdSetIterator() {
- private int docID = -1;
-
- @Override
- public int nextDoc() throws IOException {
- return advance(docID + 1);
- }
-
- @Override
- public int docID() {
- return docID;
- }
-
- @Override
- public long cost() {
- return competitiveIterator.cost();
- }
-
- @Override
- public int advance(int target) throws IOException {
- return docID = competitiveIterator.advance(target);
- }
- };
+ return enableSkipping ? competitiveIterator : null;
Review comment:
In `DefaultBulkScorer:score`, we get:
```java
DocIdSetIterator competitiveIterator = collector.competitiveIterator();
```
If we use your current change, we will always get the same object, and even if `competitiveIterator` inside `NumericComparator` gets updated we will get the same object, while we want to get a new updated competitive iterator.
##########
File path: lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java
##########
@@ -271,30 +271,7 @@ public void visit(int docID, byte[] packedValue) {
@Override
public DocIdSetIterator competitiveIterator() {
- if (enableSkipping == false) return null;
- return new DocIdSetIterator() {
- private int docID = -1;
-
- @Override
- public int nextDoc() throws IOException {
- return advance(docID + 1);
- }
-
- @Override
- public int docID() {
- return docID;
- }
-
- @Override
- public long cost() {
- return competitiveIterator.cost();
- }
-
- @Override
- public int advance(int target) throws IOException {
- return docID = competitiveIterator.advance(target);
- }
- };
+ return enableSkipping ? competitiveIterator : null;
Review comment:
I think we can keep here the same code as before just modify the starting `docID` as:
```java
public DocIdSetIterator competitiveIterator() {
if (enableSkipping == false) return null;
return new DocIdSetIterator() {
private int docID = competitiveIterator.docID();
```
##########
File path: lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java
##########
@@ -271,30 +271,7 @@ public void visit(int docID, byte[] packedValue) {
@Override
public DocIdSetIterator competitiveIterator() {
- if (enableSkipping == false) return null;
- return new DocIdSetIterator() {
- private int docID = -1;
-
- @Override
- public int nextDoc() throws IOException {
- return advance(docID + 1);
- }
-
- @Override
- public int docID() {
- return docID;
- }
-
- @Override
- public long cost() {
- return competitiveIterator.cost();
- }
-
- @Override
- public int advance(int target) throws IOException {
- return docID = competitiveIterator.advance(target);
- }
- };
+ return enableSkipping ? competitiveIterator : null;
Review comment:
I think we can keep here the same code as before just modify the starting `docID` as:
```java
public DocIdSetIterator competitiveIterator() {
if (enableSkipping == false) return null;
return new DocIdSetIterator() {
private int docID = competitiveIterator.docID();
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] dnhatn commented on pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
dnhatn commented on pull request #324:
URL: https://github.com/apache/lucene/pull/324#issuecomment-929557401
Thanks Mayya for reviewing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] dnhatn commented on pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
dnhatn commented on pull request #324:
URL: https://github.com/apache/lucene/pull/324#issuecomment-929485273
@jpountz @mayya-sharipova Thanks for reviews. I think this is ready again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mayya-sharipova commented on pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
mayya-sharipova commented on pull request #324:
URL: https://github.com/apache/lucene/pull/324#issuecomment-929369296
Sorry, I've realized it may not be easy to change things in DefaultBulkScorer::score. I am ok with the current approach as well, but we also need to add these changes in `DocComparator`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mayya-sharipova commented on a change in pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
mayya-sharipova commented on a change in pull request #324:
URL: https://github.com/apache/lucene/pull/324#discussion_r717832090
##########
File path: lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java
##########
@@ -271,30 +271,7 @@ public void visit(int docID, byte[] packedValue) {
@Override
public DocIdSetIterator competitiveIterator() {
- if (enableSkipping == false) return null;
- return new DocIdSetIterator() {
- private int docID = -1;
-
- @Override
- public int nextDoc() throws IOException {
- return advance(docID + 1);
- }
-
- @Override
- public int docID() {
- return docID;
- }
-
- @Override
- public long cost() {
- return competitiveIterator.cost();
- }
-
- @Override
- public int advance(int target) throws IOException {
- return docID = competitiveIterator.advance(target);
- }
- };
+ return enableSkipping ? competitiveIterator : null;
Review comment:
@dnhatn Thanks so much for working on this. This is a neat approach, but I don't think in the current form it will work, because `competitiveIterator` variable gets updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mayya-sharipova commented on a change in pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
mayya-sharipova commented on a change in pull request #324:
URL: https://github.com/apache/lucene/pull/324#discussion_r717839238
##########
File path: lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java
##########
@@ -271,30 +271,7 @@ public void visit(int docID, byte[] packedValue) {
@Override
public DocIdSetIterator competitiveIterator() {
- if (enableSkipping == false) return null;
- return new DocIdSetIterator() {
- private int docID = -1;
-
- @Override
- public int nextDoc() throws IOException {
- return advance(docID + 1);
- }
-
- @Override
- public int docID() {
- return docID;
- }
-
- @Override
- public long cost() {
- return competitiveIterator.cost();
- }
-
- @Override
- public int advance(int target) throws IOException {
- return docID = competitiveIterator.advance(target);
- }
- };
+ return enableSkipping ? competitiveIterator : null;
Review comment:
I think we can keep here the same code as before just modify the starting `docID` as:
```java
public DocIdSetIterator competitiveIterator() {
if (enableSkipping == false) return null;
return new DocIdSetIterator() {
private int docID = competitiveIterator.docID();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mayya-sharipova commented on a change in pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
mayya-sharipova commented on a change in pull request #324:
URL: https://github.com/apache/lucene/pull/324#discussion_r717833807
##########
File path: lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java
##########
@@ -271,30 +271,7 @@ public void visit(int docID, byte[] packedValue) {
@Override
public DocIdSetIterator competitiveIterator() {
- if (enableSkipping == false) return null;
- return new DocIdSetIterator() {
- private int docID = -1;
-
- @Override
- public int nextDoc() throws IOException {
- return advance(docID + 1);
- }
-
- @Override
- public int docID() {
- return docID;
- }
-
- @Override
- public long cost() {
- return competitiveIterator.cost();
- }
-
- @Override
- public int advance(int target) throws IOException {
- return docID = competitiveIterator.advance(target);
- }
- };
+ return enableSkipping ? competitiveIterator : null;
Review comment:
In `DefaultBulkScorer:score`, we get:
```java
DocIdSetIterator competitiveIterator = collector.competitiveIterator();
```
If we use your current change, we will always get the same object, and if `competitiveIterator` gets updated we will get the same object, while we want to get a new updated competitive iterator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mayya-sharipova commented on a change in pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
mayya-sharipova commented on a change in pull request #324:
URL: https://github.com/apache/lucene/pull/324#discussion_r717839238
##########
File path: lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java
##########
@@ -271,30 +271,7 @@ public void visit(int docID, byte[] packedValue) {
@Override
public DocIdSetIterator competitiveIterator() {
- if (enableSkipping == false) return null;
- return new DocIdSetIterator() {
- private int docID = -1;
-
- @Override
- public int nextDoc() throws IOException {
- return advance(docID + 1);
- }
-
- @Override
- public int docID() {
- return docID;
- }
-
- @Override
- public long cost() {
- return competitiveIterator.cost();
- }
-
- @Override
- public int advance(int target) throws IOException {
- return docID = competitiveIterator.advance(target);
- }
- };
+ return enableSkipping ? competitiveIterator : null;
Review comment:
I think we can keep here the same code as before just modify the starting `docID` as:
```java
public DocIdSetIterator competitiveIterator() {
if (enableSkipping == false) return null;
return new DocIdSetIterator() {
private int docID = competitiveIterator.docID();
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mayya-sharipova commented on a change in pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
mayya-sharipova commented on a change in pull request #324:
URL: https://github.com/apache/lucene/pull/324#discussion_r717833807
##########
File path: lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java
##########
@@ -271,30 +271,7 @@ public void visit(int docID, byte[] packedValue) {
@Override
public DocIdSetIterator competitiveIterator() {
- if (enableSkipping == false) return null;
- return new DocIdSetIterator() {
- private int docID = -1;
-
- @Override
- public int nextDoc() throws IOException {
- return advance(docID + 1);
- }
-
- @Override
- public int docID() {
- return docID;
- }
-
- @Override
- public long cost() {
- return competitiveIterator.cost();
- }
-
- @Override
- public int advance(int target) throws IOException {
- return docID = competitiveIterator.advance(target);
- }
- };
+ return enableSkipping ? competitiveIterator : null;
Review comment:
In `DefaultBulkScorer:score`, we get:
```java
DocIdSetIterator competitiveIterator = collector.competitiveIterator();
```
If we use your current change, we will always get the same object, and even if `competitiveIterator` gets updated we will get the same object, while we want to get a new updated competitive iterator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] dnhatn commented on a change in pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
dnhatn commented on a change in pull request #324:
URL: https://github.com/apache/lucene/pull/324#discussion_r717864552
##########
File path: lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java
##########
@@ -271,30 +271,7 @@ public void visit(int docID, byte[] packedValue) {
@Override
public DocIdSetIterator competitiveIterator() {
- if (enableSkipping == false) return null;
- return new DocIdSetIterator() {
- private int docID = -1;
-
- @Override
- public int nextDoc() throws IOException {
- return advance(docID + 1);
- }
-
- @Override
- public int docID() {
- return docID;
- }
-
- @Override
- public long cost() {
- return competitiveIterator.cost();
- }
-
- @Override
- public int advance(int target) throws IOException {
- return docID = competitiveIterator.advance(target);
- }
- };
+ return enableSkipping ? competitiveIterator : null;
Review comment:
Good catch. I've reverted this change. Thanks Mayya!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] jpountz commented on a change in pull request #324: LUCENE-10126: Fix competitiveIterator wrongly skip documents
Posted by GitBox <gi...@apache.org>.
jpountz commented on a change in pull request #324:
URL: https://github.com/apache/lucene/pull/324#discussion_r717275045
##########
File path: lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java
##########
@@ -292,7 +293,11 @@ public long cost() {
@Override
public int advance(int target) throws IOException {
- return docID = competitiveIterator.advance(target);
+ if (target <= startDocID) {
Review comment:
Can you leave a comment about why we need to do this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org