You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "ChrisHegarty (via GitHub)" <gi...@apache.org> on 2025/04/01 10:59:08 UTC
[PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
ChrisHegarty opened a new pull request, #14426:
URL: https://github.com/apache/lucene/pull/14426
This PR adds support to `KnnVectorsReader` in order to determine the off-heap memory requirements.
Somewhat analogous to the existing `Accountable` interface - which computes the actual in-heap memory usage, this new interface `OffHeapAccountable` determines the off-heap memory requirements - not usage, but requirements.
The motivation here is to give better insight into the size of off-heap memory that will be needed, so that deployments can be better scaled so that vector search workloads fit in memory, in order to provide best execution performance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "navneet1v (via GitHub)" <gi...@apache.org>.
navneet1v commented on PR #14426:
URL: https://github.com/apache/lucene/pull/14426#issuecomment-2804604505
> > @ChrisHegarty this PR provides the info on how much off heap space is needed but this doesn't provide info on how much is loaded into memory correct? and do we have any plans to expose the current usage info?
>
> Hi @navneet1v. You are correct, this is informational only. I do plan to follow up with the actual usage, which may evolve this particular API.
thanks that will be super useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "benwtrent (via GitHub)" <gi...@apache.org>.
benwtrent commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2023094888
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene102/Lucene102BinaryQuantizedVectorsReader.java:
##########
@@ -257,6 +257,15 @@ public long ramBytesUsed() {
return size;
}
+ @Override
+ public long offHeapBytes() {
+ long bytes = 0L;
+ for (var field : fields.values()) {
+ bytes += field.vectorDataLength();
+ }
+ return bytes;
+ }
+
Review Comment:
@ChrisHegarty yeah, I am not sure :(. I think this is a nice first step, but the goal here is to signal the off-heap size so that the user can determine if they have "enough" off-heap for speed.
But, then they need to know how to actually calculate what "enough" is for their typical search case for that format.
Not ideal, but maybe its enough for a step 1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2022995174
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene102/Lucene102BinaryQuantizedVectorsReader.java:
##########
@@ -257,6 +257,15 @@ public long ramBytesUsed() {
return size;
}
+ @Override
+ public long offHeapBytes() {
+ long bytes = 0L;
+ for (var field : fields.values()) {
+ bytes += field.vectorDataLength();
+ }
+ return bytes;
+ }
+
Review Comment:
I don't deny that this is not ideal, but it's not a new model, it already exists - `Accountable`. One should always consider whether or not children should be counted, and if so how. Lemme see if/how named accountable could help here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2029105175
##########
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##########
@@ -130,4 +134,56 @@ public KnnVectorsReader getMergeInstance() {
* <p>The default implementation is empty
*/
public void finishMerge() throws IOException {}
+
+ /** A string representing the off-heap category for quantized vectors. */
+ public static final String QUANTIZED = "QUANTIZED";
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "benwtrent (via GitHub)" <gi...@apache.org>.
benwtrent commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2027504697
##########
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##########
@@ -130,4 +134,56 @@ public KnnVectorsReader getMergeInstance() {
* <p>The default implementation is empty
*/
public void finishMerge() throws IOException {}
+
+ /** A string representing the off-heap category for quantized vectors. */
+ public static final String QUANTIZED = "QUANTIZED";
Review Comment:
> I wonder if we should rather reflect the underlying format here. Something like flat_vector_float, flat_vector_byte, flat_vector_bbq?
IMO, this is then transforming to just the codec format name as I would expect different formats to have different characteristics, even between versions....
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty merged PR #14426:
URL: https://github.com/apache/lucene/pull/14426
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on PR #14426:
URL: https://github.com/apache/lucene/pull/14426#issuecomment-2804247850
> @ChrisHegarty this PR provides the info on how much off heap space is needed but this doesn't provide info on how much is loaded into memory correct? and do we have any plans to expose the current usage info?
Hi @navneet1v. You are correct, this is informational only. I do plan to follow up with the actual usage, which may evolve this particular API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "jimczi (via GitHub)" <gi...@apache.org>.
jimczi commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2027537161
##########
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##########
@@ -130,4 +134,56 @@ public KnnVectorsReader getMergeInstance() {
* <p>The default implementation is empty
*/
public void finishMerge() throws IOException {}
+
+ /** A string representing the off-heap category for quantized vectors. */
+ public static final String QUANTIZED = "QUANTIZED";
Review Comment:
Yep, then maybe using the file name extension is more consistent? That would be `VEC`, `VEX`, `VEQ` and `VEB`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "benwtrent (via GitHub)" <gi...@apache.org>.
benwtrent commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2042590532
##########
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##########
@@ -130,4 +134,48 @@ public KnnVectorsReader getMergeInstance() {
* <p>The default implementation is empty
*/
public void finishMerge() throws IOException {}
+
+ /**
+ * Returns the desired size of off-heap memory the given field. This size can be used to help
+ * determine the memory requirements for optimal search performance, which can be greatly affected
+ * by page faults when not enough memory is available.
+ *
+ * <p>For reporting purposes, the size of the off-heap index structures is broken down by their
+ * file extension, which provides a logical categorization of their purpose, e.g. the {@code
+ * Lucene99HnswVectorsFormat} stores the HNSW graph neighbours lists in a file with the "vex"
+ * extension.
+ *
+ * <p>The long value is the size in bytes of the off-heap space needed if the associated index
+ * structure were to be fully loaded in memory. While somewhat analogous to {@link
+ * Accountable#ramBytesUsed()} (which reports actual on-heap memory usage), the sizes reported by
+ * this method are not actual usage but rather the amount of available memory needed to fully load
+ * the index into memory, rather than an actual RAM usage requirement.
+ *
+ * <p>To determine the total desired off-heap memory size for the given field:
+ *
+ * <pre>{@code
+ * getOffHeapByteSize(field).values().stream().mapToLong(Long::longValue).sum();
+ * }</pre>
+ *
+ * @param fieldInfo the fieldInfo
+ * @return a map of the desired off-heap memory requirements by category
+ * @lucene.experimental
+ */
+ public abstract Map<String, Long> getOffHeapByteSize(FieldInfo fieldInfo);
Review Comment:
Doesn't this need to be backwards compatible? Here we are adding a new required base function to be overridden by all extenders of the `KnnVectorsReader`.
Marking it as experimental is good. But maybe it supplies a default implementation that simply returns `null` or `Map.of()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2043960727
##########
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##########
@@ -130,4 +134,48 @@ public KnnVectorsReader getMergeInstance() {
* <p>The default implementation is empty
*/
public void finishMerge() throws IOException {}
+
+ /**
+ * Returns the desired size of off-heap memory the given field. This size can be used to help
+ * determine the memory requirements for optimal search performance, which can be greatly affected
+ * by page faults when not enough memory is available.
+ *
+ * <p>For reporting purposes, the size of the off-heap index structures is broken down by their
+ * file extension, which provides a logical categorization of their purpose, e.g. the {@code
+ * Lucene99HnswVectorsFormat} stores the HNSW graph neighbours lists in a file with the "vex"
+ * extension.
+ *
+ * <p>The long value is the size in bytes of the off-heap space needed if the associated index
+ * structure were to be fully loaded in memory. While somewhat analogous to {@link
+ * Accountable#ramBytesUsed()} (which reports actual on-heap memory usage), the sizes reported by
+ * this method are not actual usage but rather the amount of available memory needed to fully load
+ * the index into memory, rather than an actual RAM usage requirement.
+ *
+ * <p>To determine the total desired off-heap memory size for the given field:
+ *
+ * <pre>{@code
+ * getOffHeapByteSize(field).values().stream().mapToLong(Long::longValue).sum();
+ * }</pre>
+ *
+ * @param fieldInfo the fieldInfo
+ * @return a map of the desired off-heap memory requirements by category
+ * @lucene.experimental
+ */
+ public abstract Map<String, Long> getOffHeapByteSize(FieldInfo fieldInfo);
Review Comment:
D'oh! yeah. Added a default implementation returning an empty map. See [4507034](https://github.com/apache/lucene/pull/14426/commits/4507034ab41c7c0ac8c01b7e501876a29262d04c)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2022920202
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene102/Lucene102BinaryQuantizedVectorsReader.java:
##########
@@ -257,6 +257,15 @@ public long ramBytesUsed() {
return size;
}
+ @Override
+ public long offHeapBytes() {
+ long bytes = 0L;
+ for (var field : fields.values()) {
+ bytes += field.vectorDataLength();
+ }
+ return bytes;
+ }
+
Review Comment:
Right, that is what `getChildOffHeapResources` tries to address - by allowing to separate graph, quantized, and raw vectors. But I do accept that this may not be all that obvious to the user. Actually I need to fix a missing `getChildOffHeapResources` from Lucene102... for the raw vectors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2028456102
##########
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##########
@@ -130,4 +134,56 @@ public KnnVectorsReader getMergeInstance() {
* <p>The default implementation is empty
*/
public void finishMerge() throws IOException {}
+
+ /** A string representing the off-heap category for quantized vectors. */
+ public static final String QUANTIZED = "QUANTIZED";
Review Comment:
~The file extension is quite cute, but discloses more of the implementation details to the caller which seems unnecessary.~
~I deliberately chose to represent the categories as a `String` (rather than say an enum), since this offers more flexibility to the implementation while standardising on the three main categories.~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "benwtrent (via GitHub)" <gi...@apache.org>.
benwtrent commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2022890436
##########
lucene/core/src/java/org/apache/lucene/util/OffHeapAccountable.java:
##########
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.lucene.util;
+
+import java.util.Collection;
+import java.util.Collections;
+
+/**
+ * An object whose off-heap memory requirements can be computed.
+ *
+ * @lucene.internal
+ */
+public interface OffHeapAccountable {
Review Comment:
I wonder if we want to call this "OffHeapAccountable".
I am not sure, "OnHeap" has a "hard limit" on heap usage to prevent OOMs.
This "OffHeap" is a "soft limit" that relates to performance, not the system running.
So, its not a "requirements" type of thing.
Maybe I am just stuck on the phrase "resource requirements". As I also see this being useful for other field types, besides HNSW knn, where holding everything in off-heap memory is critical.
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene102/Lucene102BinaryQuantizedVectorsReader.java:
##########
@@ -257,6 +257,15 @@ public long ramBytesUsed() {
return size;
}
+ @Override
+ public long offHeapBytes() {
+ long bytes = 0L;
+ for (var field : fields.values()) {
+ bytes += field.vectorDataLength();
+ }
+ return bytes;
+ }
+
Review Comment:
Here is where we need to be careful with the API. Technically binary quantized vectors keeps the raw vectors (same with other quantized techniques).
Technically, the add to the overall off-heap size, though they may not need to be loaded into off-heap memory for search to be fast.
For example, if somebody is using `byte` type vectors, they aren't actually quantized, but its searchable. Meaning, they would all need to be in off-heap memory to be fast.
##########
lucene/core/src/java/org/apache/lucene/util/OffHeapAccountable.java:
##########
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.lucene.util;
+
+import java.util.Collection;
+import java.util.Collections;
+
+/**
+ * An object whose off-heap memory requirements can be computed.
+ *
+ * @lucene.internal
Review Comment:
Is this really an "internal" API? Maybe I misunderstand its use, but it seems designed for folks to make decisions or provide feedback directly using this API.
So, maybe we call it "experimental" if unsure of its usage/stability right now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "jimczi (via GitHub)" <gi...@apache.org>.
jimczi commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2027392059
##########
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##########
@@ -130,4 +134,56 @@ public KnnVectorsReader getMergeInstance() {
* <p>The default implementation is empty
*/
public void finishMerge() throws IOException {}
+
+ /** A string representing the off-heap category for quantized vectors. */
+ public static final String QUANTIZED = "QUANTIZED";
Review Comment:
nit: I wonder if we should rather reflect the underlying format here. Something like flat_vector_float, flat_vector_byte, flat_vector_bbq?
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene102/Lucene102BinaryQuantizedVectorsReader.java:
##########
@@ -257,6 +259,19 @@ public long ramBytesUsed() {
return size;
}
+ @Override
+ public Map<String, Long> getOffHeapByteSize(FieldInfo fieldInfo) {
+ Objects.requireNonNull(fieldInfo);
+ var raw = rawVectorsReader.getOffHeapByteSize(fieldInfo);
+ var fieldEntry = fields.get(fieldInfo.name);
+ if (fieldEntry == null) {
+ assert fieldInfo.getVectorEncoding() == VectorEncoding.BYTE;
Review Comment:
This is not possible, this format doesn't accept raw vector in the byte format.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "mayya-sharipova (via GitHub)" <gi...@apache.org>.
mayya-sharipova commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2027061812
##########
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##########
@@ -130,4 +134,56 @@ public KnnVectorsReader getMergeInstance() {
* <p>The default implementation is empty
*/
public void finishMerge() throws IOException {}
+
+ /** A string representing the off-heap category for quantized vectors. */
+ public static final String QUANTIZED = "QUANTIZED";
+
+ /** A string representing the off-heap category for the HNSW graph. */
+ public static final String HNSW_GRAPH = "HNSW_GRAPH";
+
+ /** A string representing the off-heap category for raw vectors. */
+ public static final String RAW = "RAW";
+
+ /**
+ * Returns the desired size of off-heap memory the given field. This size can be used to help
+ * determine the memory requirements for optimal search performance, which can be greatly affected
+ * by page faults when not enough memory is available.
+ *
+ * <p>For reporting purposes, the backing off-heap index structures are broken into three
+ * categories: 1. {@link #RAW}, 2. {@link #HNSW_GRAPH}, and 3. {@link #QUANTIZED}. The returned
+ * map will have zero or one entry for each of these categories.
+ *
+ * <p>The long value is the size in bytes of the off-heap space needed if the associated index
+ * structure were to be fully loaded in memory. While somewhat analogous to {@link
+ * Accountable#ramBytesUsed()} (which reports actual on-heap memory usage), the metrics reported
+ * by this method are not actual usage but rather the amount of available memory needed to fully
+ * load the index into memory, rather than an actual RAM usage requirement.
+ *
+ * <p>To determine the total desired off-heap memory size for the given field:
+ *
+ * <pre>{@code
+ * getOffHeapByteSize(field).values().stream().mapToLong(Long::longValue).sum();
+ * }</pre>
+ *
+ * @param fieldInfo the fieldInfo
+ * @return a map of the desired off-heap memory requirements by category
+ * @lucene.experimental
+ */
+ public abstract Map<String, Long> getOffHeapByteSize(FieldInfo fieldInfo);
Review Comment:
Very nice API! +1
But I am also thinking about other use cases: where we need to know the total size across all vector fields and we don't know/don't remember field names. Do you think it is worth to have this API: may be that returns a map of maps?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "tteofili (via GitHub)" <gi...@apache.org>.
tteofili commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2022684543
##########
lucene/backward-codecs/src/java/org/apache/lucene/backward_codecs/lucene90/Lucene90HnswVectorsReader.java:
##########
@@ -306,6 +306,16 @@ private HnswGraph getGraphValues(FieldEntry entry) throws IOException {
return new OffHeapHnswGraph(entry, bytesSlice);
}
+ @Override
+ public long offHeapBytes() {
+ long bytes = 0L;
+ for (var field : fields.values()) {
+ var f = field.value;
+ bytes += f.vectorDataLength() + f.indexDataLength();
+ }
+ return bytes;
Review Comment:
what about an `util` static method that can be applied here and for other `*VectorsReader` without having to re-implement the same code block multiple times?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2028417528
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene102/Lucene102BinaryQuantizedVectorsReader.java:
##########
@@ -257,6 +259,19 @@ public long ramBytesUsed() {
return size;
}
+ @Override
+ public Map<String, Long> getOffHeapByteSize(FieldInfo fieldInfo) {
+ Objects.requireNonNull(fieldInfo);
+ var raw = rawVectorsReader.getOffHeapByteSize(fieldInfo);
+ var fieldEntry = fields.get(fieldInfo.name);
+ if (fieldEntry == null) {
+ assert fieldInfo.getVectorEncoding() == VectorEncoding.BYTE;
Review Comment:
I did think that too, however both this format and the ScalarQuantizedFormat can be used directly with bytes - in which case there is no auto-quantization, just a byte vector store.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2022923407
##########
lucene/core/src/java/org/apache/lucene/util/OffHeapAccountable.java:
##########
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.lucene.util;
+
+import java.util.Collection;
+import java.util.Collections;
+
+/**
+ * An object whose off-heap memory requirements can be computed.
+ *
+ * @lucene.internal
+ */
+public interface OffHeapAccountable {
Review Comment:
Yeah, "requirements" is not the best word here. It's just a "size".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2022995174
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene102/Lucene102BinaryQuantizedVectorsReader.java:
##########
@@ -257,6 +257,15 @@ public long ramBytesUsed() {
return size;
}
+ @Override
+ public long offHeapBytes() {
+ long bytes = 0L;
+ for (var field : fields.values()) {
+ bytes += field.vectorDataLength();
+ }
+ return bytes;
+ }
+
Review Comment:
I don't deny that this is not ideal, but it not a new model, it already exists - `Accountable`. One should always consider whether or not children should be counted, and if so how.
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene102/Lucene102BinaryQuantizedVectorsReader.java:
##########
@@ -257,6 +257,15 @@ public long ramBytesUsed() {
return size;
}
+ @Override
+ public long offHeapBytes() {
+ long bytes = 0L;
+ for (var field : fields.values()) {
+ bytes += field.vectorDataLength();
+ }
+ return bytes;
+ }
+
Review Comment:
I don't deny that this is not ideal, but it's not a new model, it already exists - `Accountable`. One should always consider whether or not children should be counted, and if so how.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on PR #14426:
URL: https://github.com/apache/lucene/pull/14426#issuecomment-2773007104
Given the feedback so far, I've pivot this quite a bit to now include per-field metrics. To support this I removed the previously proposed OffHeapAccountable interface and put the accessor on KnnVectorsReader - where other per-field accessors are. I used FieldInfo as the lookup since that it needed to determine if the field is either Byte of Float32, which is required to know in the "richer" codec implementations which provide quantization of floats and passthrough for bytes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2027333343
##########
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##########
@@ -130,4 +134,56 @@ public KnnVectorsReader getMergeInstance() {
* <p>The default implementation is empty
*/
public void finishMerge() throws IOException {}
+
+ /** A string representing the off-heap category for quantized vectors. */
+ public static final String QUANTIZED = "QUANTIZED";
+
+ /** A string representing the off-heap category for the HNSW graph. */
+ public static final String HNSW_GRAPH = "HNSW_GRAPH";
+
+ /** A string representing the off-heap category for raw vectors. */
+ public static final String RAW = "RAW";
+
+ /**
+ * Returns the desired size of off-heap memory the given field. This size can be used to help
+ * determine the memory requirements for optimal search performance, which can be greatly affected
+ * by page faults when not enough memory is available.
+ *
+ * <p>For reporting purposes, the backing off-heap index structures are broken into three
+ * categories: 1. {@link #RAW}, 2. {@link #HNSW_GRAPH}, and 3. {@link #QUANTIZED}. The returned
+ * map will have zero or one entry for each of these categories.
+ *
+ * <p>The long value is the size in bytes of the off-heap space needed if the associated index
+ * structure were to be fully loaded in memory. While somewhat analogous to {@link
+ * Accountable#ramBytesUsed()} (which reports actual on-heap memory usage), the metrics reported
+ * by this method are not actual usage but rather the amount of available memory needed to fully
+ * load the index into memory, rather than an actual RAM usage requirement.
+ *
+ * <p>To determine the total desired off-heap memory size for the given field:
+ *
+ * <pre>{@code
+ * getOffHeapByteSize(field).values().stream().mapToLong(Long::longValue).sum();
+ * }</pre>
+ *
+ * @param fieldInfo the fieldInfo
+ * @return a map of the desired off-heap memory requirements by category
+ * @lucene.experimental
+ */
+ public abstract Map<String, Long> getOffHeapByteSize(FieldInfo fieldInfo);
Review Comment:
@mayya-sharipova the expected usage is that the caller would get all the fieldInfos from the reader ( `LeafReader::getFieldInfos` ), and then iterate over them checking for vector info. Which should be straightforward to do without an additional API point.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2028541365
##########
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##########
@@ -130,4 +134,56 @@ public KnnVectorsReader getMergeInstance() {
* <p>The default implementation is empty
*/
public void finishMerge() throws IOException {}
+
+ /** A string representing the off-heap category for quantized vectors. */
+ public static final String QUANTIZED = "QUANTIZED";
Review Comment:
I chatted with Jim offline; the file extension relates nicely with the how preLoading currently works, and makes that connection more concrete, which is a very nice property. It also reduces the surface of the API change, allowing to remove the constant strings. I'll update the use the file extension.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2028456102
##########
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##########
@@ -130,4 +134,56 @@ public KnnVectorsReader getMergeInstance() {
* <p>The default implementation is empty
*/
public void finishMerge() throws IOException {}
+
+ /** A string representing the off-heap category for quantized vectors. */
+ public static final String QUANTIZED = "QUANTIZED";
Review Comment:
The file extension is quite cute, but discloses more of the implementation details to the caller which seems unnecessary.
I deliberately chose to represent the categories as a `String` (rather than say an enum), since this offers more flexibility to the implementation while standardising on the three main categories.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2027373074
##########
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##########
@@ -130,4 +134,56 @@ public KnnVectorsReader getMergeInstance() {
* <p>The default implementation is empty
*/
public void finishMerge() throws IOException {}
+
+ /** A string representing the off-heap category for quantized vectors. */
+ public static final String QUANTIZED = "QUANTIZED";
+
+ /** A string representing the off-heap category for the HNSW graph. */
+ public static final String HNSW_GRAPH = "HNSW_GRAPH";
+
+ /** A string representing the off-heap category for raw vectors. */
+ public static final String RAW = "RAW";
+
+ /**
+ * Returns the desired size of off-heap memory the given field. This size can be used to help
+ * determine the memory requirements for optimal search performance, which can be greatly affected
+ * by page faults when not enough memory is available.
+ *
+ * <p>For reporting purposes, the backing off-heap index structures are broken into three
+ * categories: 1. {@link #RAW}, 2. {@link #HNSW_GRAPH}, and 3. {@link #QUANTIZED}. The returned
+ * map will have zero or one entry for each of these categories.
+ *
+ * <p>The long value is the size in bytes of the off-heap space needed if the associated index
+ * structure were to be fully loaded in memory. While somewhat analogous to {@link
+ * Accountable#ramBytesUsed()} (which reports actual on-heap memory usage), the metrics reported
+ * by this method are not actual usage but rather the amount of available memory needed to fully
+ * load the index into memory, rather than an actual RAM usage requirement.
+ *
+ * <p>To determine the total desired off-heap memory size for the given field:
+ *
+ * <pre>{@code
+ * getOffHeapByteSize(field).values().stream().mapToLong(Long::longValue).sum();
+ * }</pre>
+ *
+ * @param fieldInfo the fieldInfo
+ * @return a map of the desired off-heap memory requirements by category
+ * @lucene.experimental
+ */
+ public abstract Map<String, Long> getOffHeapByteSize(FieldInfo fieldInfo);
Review Comment:
While it's convenient to reuse Maps, an alternative here is to encapsulate the categorised stats in a container, which we could enrich later with information about whether the index is actually resident in memory.
```
public abstract OffHeapStats getOffHeapByteSize(FieldInfo fieldInfo);
record OffHeapStats(String field, long raw, long hnswGraph, long quantized) { }
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "navneet1v (via GitHub)" <gi...@apache.org>.
navneet1v commented on PR #14426:
URL: https://github.com/apache/lucene/pull/14426#issuecomment-2802385473
@ChrisHegarty this PR provides the info on how much off heap space is needed but this doesn't provide info on how much is loaded into memory correct? and do we have any plans to expose the current usage info?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2022693959
##########
lucene/backward-codecs/src/java/org/apache/lucene/backward_codecs/lucene90/Lucene90HnswVectorsReader.java:
##########
@@ -306,6 +306,16 @@ private HnswGraph getGraphValues(FieldEntry entry) throws IOException {
return new OffHeapHnswGraph(entry, bytesSlice);
}
+ @Override
+ public long offHeapBytes() {
+ long bytes = 0L;
+ for (var field : fields.values()) {
+ var f = field.value;
+ bytes += f.vectorDataLength() + f.indexDataLength();
+ }
+ return bytes;
Review Comment:
That a good point, and one that I did think off too, but I wasn't able to come up with something simple as all these fields are their own individual inner classes, and any refactoring I experimented with just seemed to add complexity. :-(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "benwtrent (via GitHub)" <gi...@apache.org>.
benwtrent commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2022982193
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene102/Lucene102BinaryQuantizedVectorsReader.java:
##########
@@ -257,6 +257,15 @@ public long ramBytesUsed() {
return size;
}
+ @Override
+ public long offHeapBytes() {
+ long bytes = 0L;
+ for (var field : fields.values()) {
+ bytes += field.vectorDataLength();
+ }
+ return bytes;
+ }
+
Review Comment:
So, the user needs to know "when to sum them" and when "not to sum them" without any information as to what the child resources actually provide?
I guess they can do an `instanceof` and determine what they want to do per codec name :(
seems brittle. I am not sure of a better way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Add support for determining off-heap memory requirements for KnnVectorsReader [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on code in PR #14426:
URL: https://github.com/apache/lucene/pull/14426#discussion_r2023074747
##########
lucene/core/src/java/org/apache/lucene/util/OffHeapAccountable.java:
##########
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.lucene.util;
+
+import java.util.Collection;
+import java.util.Collections;
+
+/**
+ * An object whose off-heap memory requirements can be computed.
+ *
+ * @lucene.internal
+ */
+public interface OffHeapAccountable {
Review Comment:
I reworded things a bit and added a bit of javadoc to make things clearer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org