You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by so...@apache.org on 2024/09/28 21:46:19 UTC
(lucene) branch main updated (a38443bc6a5 -> 0a8604d908c)
This is an automated email from the ASF dual-hosted git repository.
sokolov pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/lucene.git
from a38443bc6a5 Add back-compat indices for 9.12.0
new f21ec8414c1 SlowCompositeCodecReaderWrapper KnnVectorValues handles binary search over array with repeated values
new c8eeacb39af fix safety check in KnnVectorsWriter
new 0a8604d908c Merge remote-tracking branch 'origin/main' into main
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../src/java/org/apache/lucene/codecs/KnnVectorsWriter.java | 10 ++++++----
.../apache/lucene/index/SlowCompositeCodecReaderWrapper.java | 9 +++++++--
2 files changed, 13 insertions(+), 6 deletions(-)
(lucene) 03/03: Merge remote-tracking branch 'origin/main' into main
Posted by so...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
sokolov pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/lucene.git
commit 0a8604d908c7657187ed6d6926484ef7406d9794
Merge: c8eeacb39af a38443bc6a5
Author: Michael Sokolov <so...@amazon.com>
AuthorDate: Sat Sep 28 17:45:05 2024 -0400
Merge remote-tracking branch 'origin/main' into main
.../apache/lucene/backward_index/index.9.12.0-cfs.zip | Bin 0 -> 17386 bytes
.../lucene/backward_index/index.9.12.0-nocfs.zip | Bin 0 -> 17362 bytes
.../apache/lucene/backward_index/int8_hnsw.9.12.0.zip | Bin 0 -> 4932 bytes
.../org/apache/lucene/backward_index/sorted.9.12.0.zip | Bin 0 -> 137013 bytes
.../test/org/apache/lucene/backward_index/versions.txt | 1 +
5 files changed, 1 insertion(+)
(lucene) 01/03: SlowCompositeCodecReaderWrapper KnnVectorValues handles binary search over array with repeated values
Posted by so...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
sokolov pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/lucene.git
commit f21ec8414c1a8e6c08aad2783827a6f6848a3ea6
Author: Michael Sokolov <so...@amazon.com>
AuthorDate: Sat Sep 28 17:43:01 2024 -0400
SlowCompositeCodecReaderWrapper KnnVectorValues handles binary search over array with repeated values
---
.../org/apache/lucene/index/SlowCompositeCodecReaderWrapper.java | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/lucene/core/src/java/org/apache/lucene/index/SlowCompositeCodecReaderWrapper.java b/lucene/core/src/java/org/apache/lucene/index/SlowCompositeCodecReaderWrapper.java
index ec2423310d9..69d557d270a 100644
--- a/lucene/core/src/java/org/apache/lucene/index/SlowCompositeCodecReaderWrapper.java
+++ b/lucene/core/src/java/org/apache/lucene/index/SlowCompositeCodecReaderWrapper.java
@@ -898,9 +898,9 @@ final class SlowCompositeCodecReaderWrapper extends CodecReader {
public float[] vectorValue(int ord) throws IOException {
assert ord >= 0 && ord < size;
// We need to implement fully random-access API here in order to support callers like
- // SortingCodecReader that
- // rely on it.
+ // SortingCodecReader that rely on it.
lastSubIndex = findSub(ord, lastSubIndex, starts);
+ assert subs[lastSubIndex].sub != null;
return ((FloatVectorValues) subs[lastSubIndex].sub)
.vectorValue(ord - subs[lastSubIndex].ordStart);
}
@@ -1002,6 +1002,11 @@ final class SlowCompositeCodecReaderWrapper extends CodecReader {
// subtract one since binarySearch returns an *insertion point*
return -2 - pos;
} else {
+ while (pos < starts.length - 1 && starts[pos + 1] == ord) {
+ // Arrays.binarySearch can return any of a sequence of repeated value
+ // but we always want the last one
+ ++pos;
+ }
return pos;
}
}
(lucene) 02/03: fix safety check in KnnVectorsWriter
Posted by so...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
sokolov pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/lucene.git
commit c8eeacb39afeec02e00b812152e95f88a46739a3
Author: Michael Sokolov <so...@amazon.com>
AuthorDate: Sat Sep 28 17:44:33 2024 -0400
fix safety check in KnnVectorsWriter
---
.../src/java/org/apache/lucene/codecs/KnnVectorsWriter.java | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsWriter.java b/lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsWriter.java
index cbf68dd8c20..50af32a7e16 100644
--- a/lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsWriter.java
+++ b/lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsWriter.java
@@ -336,6 +336,7 @@ public abstract class KnnVectorsWriter implements Accountable, Closeable {
index = NO_MORE_DOCS;
} else {
docId = current.mappedDocID;
+ ++lastOrd;
++index;
}
return docId;
@@ -355,11 +356,12 @@ public abstract class KnnVectorsWriter implements Accountable, Closeable {
@Override
public float[] vectorValue(int ord) throws IOException {
- if (ord != lastOrd + 1) {
+ if (ord != lastOrd) {
throw new IllegalStateException(
- "only supports forward iteration: ord=" + ord + ", lastOrd=" + lastOrd);
- } else {
- lastOrd = ord;
+ "only supports forward iteration with a single iterator: ord="
+ + ord
+ + ", lastOrd="
+ + lastOrd);
}
return current.values.vectorValue(current.index());
}