You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "jpountz (via GitHub)" <gi...@apache.org> on 2024/08/06 14:47:37 UTC
[PR] Optimize decoding blocks of postings using the vector API. [lucene]
jpountz opened a new pull request, #13636:
URL: https://github.com/apache/lucene/pull/13636
Our postings use a layout that helps take advantage of Java's auto-vectorization to be reasonably fast to decode. But we can make it a bit faster by using explicit vectorization on MemorySegment:
- vectorizing directly from the MemorySegment instead of first copying data into a long[],
- decoding more longs than requested instead of forcing the last longs to be handled via scalar instructions.
This approach only works when the `Directory` uses `MemorySegmentIndexInput` under the hood, ie. `MMapDirectory` on JDK 21+. The `ForUtilBenchmark` micro benchmark reports the following results:
```
Before
Benchmark (bpv) Mode Cnt Score Error Units
ForUtilBenchmark.decode 5 thrpt 15 36.244 ± 0.742 ops/us
ForUtilBenchmark.decode 6 thrpt 15 35.406 ± 0.170 ops/us
ForUtilBenchmark.decode 7 thrpt 15 33.396 ± 0.291 ops/us
ForUtilBenchmark.decode 8 thrpt 15 41.064 ± 2.269 ops/us
ForUtilBenchmark.decode 9 thrpt 15 30.288 ± 0.172 ops/us
ForUtilBenchmark.decode 10 thrpt 15 31.203 ± 0.791 ops/us
ForUtilBenchmark.decodeAndPrefixSum 5 thrpt 15 19.421 ± 0.272 ops/us
ForUtilBenchmark.decodeAndPrefixSum 6 thrpt 15 18.932 ± 0.356 ops/us
ForUtilBenchmark.decodeAndPrefixSum 7 thrpt 15 16.824 ± 1.080 ops/us
ForUtilBenchmark.decodeAndPrefixSum 8 thrpt 15 21.085 ± 0.316 ops/us
ForUtilBenchmark.decodeAndPrefixSum 9 thrpt 15 15.874 ± 2.085 ops/us
ForUtilBenchmark.decodeAndPrefixSum 10 thrpt 15 17.827 ± 0.210 ops/us
After
Benchmark (bpv) Mode Cnt Score Error Units
ForUtilBenchmark.decode 5 thrpt 15 40.774 ± 1.170 ops/us
ForUtilBenchmark.decode 6 thrpt 15 44.392 ± 0.748 ops/us
ForUtilBenchmark.decode 7 thrpt 15 43.050 ± 0.586 ops/us
ForUtilBenchmark.decode 8 thrpt 15 49.773 ± 0.376 ops/us
ForUtilBenchmark.decode 9 thrpt 15 36.264 ± 0.434 ops/us
ForUtilBenchmark.decode 10 thrpt 15 38.403 ± 1.388 ops/us
ForUtilBenchmark.decodeAndPrefixSum 5 thrpt 15 19.362 ± 0.573 ops/us
ForUtilBenchmark.decodeAndPrefixSum 6 thrpt 15 18.402 ± 3.128 ops/us
ForUtilBenchmark.decodeAndPrefixSum 7 thrpt 15 19.518 ± 0.680 ops/us
ForUtilBenchmark.decodeAndPrefixSum 8 thrpt 15 21.388 ± 0.228 ops/us
ForUtilBenchmark.decodeAndPrefixSum 9 thrpt 15 18.126 ± 0.625 ops/us
ForUtilBenchmark.decodeAndPrefixSum 10 thrpt 15 19.161 ± 0.379 ops/us
```
And `luceneutil` on `wikibigall` reports the following (only look at queries with low p-values):
```
TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value
IntNRQ 132.60 (23.4%) 129.72 (24.1%) -2.2% ( -40% - 59%) 0.772
CountTerm 9419.14 (3.5%) 9377.10 (3.1%) -0.4% ( -6% - 6%) 0.669
Respell 55.22 (1.2%) 55.24 (1.5%) 0.0% ( -2% - 2%) 0.931
Fuzzy2 71.83 (1.1%) 71.87 (1.1%) 0.1% ( -2% - 2%) 0.871
TermDTSort 360.85 (6.8%) 361.40 (5.6%) 0.2% ( -11% - 13%) 0.939
HighTermMonthSort 3491.38 (1.4%) 3499.56 (2.4%) 0.2% ( -3% - 4%) 0.702
Fuzzy1 89.72 (1.1%) 90.06 (1.2%) 0.4% ( -1% - 2%) 0.285
MedSloppyPhrase 7.70 (3.6%) 7.73 (6.6%) 0.4% ( -9% - 10%) 0.791
PKLookup 288.99 (1.7%) 290.39 (1.5%) 0.5% ( -2% - 3%) 0.332
HighTermDayOfYearSort 829.90 (3.4%) 836.92 (3.4%) 0.8% ( -5% - 7%) 0.436
HighPhrase 66.40 (2.8%) 66.97 (5.1%) 0.9% ( -6% - 9%) 0.508
Prefix3 185.20 (3.1%) 186.81 (2.8%) 0.9% ( -4% - 7%) 0.352
HighTermTitleSort 162.00 (4.1%) 163.43 (5.0%) 0.9% ( -7% - 10%) 0.543
HighSloppyPhrase 6.91 (3.0%) 6.98 (6.6%) 1.0% ( -8% - 10%) 0.521
Wildcard 53.89 (3.9%) 54.64 (3.5%) 1.4% ( -5% - 9%) 0.237
OrHighRare 252.56 (9.3%) 257.10 (9.6%) 1.8% ( -15% - 22%) 0.548
MedIntervalsOrdered 3.33 (4.7%) 3.40 (5.6%) 1.9% ( -8% - 12%) 0.259
HighSpanNear 4.60 (1.5%) 4.69 (1.4%) 1.9% ( -1% - 4%) 0.000
OrNotHighHigh 221.99 (3.7%) 226.47 (5.4%) 2.0% ( -6% - 11%) 0.164
OrHighLow 812.48 (2.0%) 828.98 (2.3%) 2.0% ( -2% - 6%) 0.003
HighIntervalsOrdered 1.36 (5.3%) 1.39 (6.4%) 2.0% ( -9% - 14%) 0.272
LowIntervalsOrdered 4.38 (4.1%) 4.47 (4.5%) 2.1% ( -6% - 11%) 0.130
OrHighNotHigh 207.32 (4.1%) 211.83 (5.6%) 2.2% ( -7% - 12%) 0.159
OrNotHighMed 334.67 (3.7%) 342.02 (5.8%) 2.2% ( -7% - 12%) 0.152
MedSpanNear 14.99 (1.8%) 15.32 (1.4%) 2.2% ( -1% - 5%) 0.000
MedPhrase 14.61 (3.2%) 14.97 (4.6%) 2.5% ( -5% - 10%) 0.052
LowPhrase 71.73 (3.3%) 73.49 (4.4%) 2.5% ( -5% - 10%) 0.046
AndHighHigh 70.54 (2.0%) 72.44 (1.3%) 2.7% ( 0% - 6%) 0.000
OrHighHigh 67.00 (1.7%) 68.80 (1.6%) 2.7% ( 0% - 6%) 0.000
OrHighMed 191.06 (1.9%) 196.57 (1.8%) 2.9% ( 0% - 6%) 0.000
LowSloppyPhrase 24.50 (2.9%) 25.27 (4.1%) 3.2% ( -3% - 10%) 0.005
AndHighMed 151.19 (2.2%) 156.00 (1.4%) 3.2% ( 0% - 6%) 0.000
CountPhrase 3.18 (9.0%) 3.29 (9.1%) 3.3% ( -13% - 23%) 0.244
Or2Terms2StopWords 160.27 (4.4%) 165.68 (1.5%) 3.4% ( -2% - 9%) 0.001
And2Terms2StopWords 157.18 (2.8%) 162.48 (1.4%) 3.4% ( 0% - 7%) 0.000
OrHighNotMed 301.42 (4.1%) 311.77 (6.1%) 3.4% ( -6% - 14%) 0.038
LowSpanNear 9.83 (1.5%) 10.17 (1.4%) 3.5% ( 0% - 6%) 0.000
CountAndHighHigh 46.76 (1.3%) 48.48 (2.3%) 3.7% ( 0% - 7%) 0.000
HighTermTitleBDVSort 11.40 (5.6%) 11.82 (8.1%) 3.7% ( -9% - 18%) 0.092
OrHighNotLow 342.09 (4.6%) 355.83 (7.0%) 4.0% ( -7% - 16%) 0.033
And3Terms 165.26 (3.0%) 171.96 (1.6%) 4.1% ( 0% - 8%) 0.000
Or3Terms 165.25 (4.5%) 171.96 (1.5%) 4.1% ( -1% - 10%) 0.000
AndHighLow 993.19 (2.8%) 1034.27 (2.8%) 4.1% ( -1% - 10%) 0.000
OrNotHighLow 989.09 (3.1%) 1030.58 (3.5%) 4.2% ( -2% - 11%) 0.000
AndStopWords 29.62 (4.1%) 30.97 (1.9%) 4.6% ( -1% - 11%) 0.000
OrStopWords 32.89 (7.0%) 34.41 (2.6%) 4.6% ( -4% - 15%) 0.006
LowTerm 978.60 (3.0%) 1023.81 (6.1%) 4.6% ( -4% - 14%) 0.002
CountAndHighMed 140.50 (1.6%) 147.18 (2.6%) 4.8% ( 0% - 9%) 0.000
CountOrHighHigh 57.94 (15.1%) 60.85 (16.7%) 5.0% ( -23% - 43%) 0.319
CountOrHighMed 113.79 (11.2%) 120.21 (13.2%) 5.6% ( -16% - 33%) 0.145
HighTerm 363.62 (5.0%) 384.13 (8.2%) 5.6% ( -7% - 19%) 0.009
MedTerm 546.01 (4.2%) 580.61 (7.9%) 6.3% ( -5% - 19%) 0.002
```
### Description
<!--
If this is your first contribution to Lucene, please make sure you have reviewed the contribution guide.
https://github.com/apache/lucene/blob/main/CONTRIBUTING.md
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1709899394
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java:
##########
@@ -16,20 +16,56 @@
*/
package org.apache.lucene.internal.vectorization;
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.util.Locale;
import java.util.logging.Logger;
import jdk.incubator.vector.FloatVector;
+import jdk.incubator.vector.VectorShape;
+import jdk.incubator.vector.VectorSpecies;
import org.apache.lucene.codecs.hnsw.FlatVectorsScorer;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.MemorySegmentAccessInput;
import org.apache.lucene.util.Constants;
import org.apache.lucene.util.SuppressForbidden;
/** A vectorization provider that leverages the Panama Vector API. */
final class PanamaVectorizationProvider extends VectorizationProvider {
+ /** Preferred with in bits for vectors. */
+ static final int PREFERRED_VECTOR_BITSIZE;
+
+ static final VectorSpecies<Long> PRERERRED_LONG_SPECIES;
+ static final VectorSpecies<Integer> PRERERRED_INT_SPECIES;
+
+ /** Whether integer vectors can be trusted to actually be fast. */
+ static final boolean HAS_FAST_INTEGER_VECTORS;
+
private final VectorUtilSupport vectorUtilSupport;
+ static {
+ // default to platform supported bitsize
+ int vectorBitSize = VectorShape.preferredShape().vectorBitSize();
Review Comment:
I was also afraid that the other new constants may call the bad code, as this problem occurs in one of the most internal classes that are called from almost everywhere. So my idea was to make sure the constructor is the first one who loads any class from vector incubator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1713051966
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ private static final VectorSpecies<Long> LONG_SPECIES =
+ PanamaVectorConstants.PRERERRED_LONG_SPECIES;
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void readLongs(int count, long[] b) throws IOException {
+ if (count < LONG_SPECIES.length()) {
+ in.readLongs(b, 0, count);
+ } else {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int loopBound = LONG_SPECIES.loopBound(count - 1);
+ for (int i = 0;
+ i < loopBound;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector.fromMemorySegment(LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN)
Review Comment:
We can check the readLongs performance in a separate issue.
Was this about shorter sized readLongs when performance was better? If not, I have no explanation. 🥹
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on PR #13636:
URL: https://github.com/apache/lucene/pull/13636#issuecomment-2277393866
I cleaned up the constants a bit more, there are still some duplicated in VectorUtilSupport, but I'd change this in the followup PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1709961906
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java:
##########
@@ -16,20 +16,56 @@
*/
package org.apache.lucene.internal.vectorization;
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.util.Locale;
import java.util.logging.Logger;
import jdk.incubator.vector.FloatVector;
+import jdk.incubator.vector.VectorShape;
+import jdk.incubator.vector.VectorSpecies;
import org.apache.lucene.codecs.hnsw.FlatVectorsScorer;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.MemorySegmentAccessInput;
import org.apache.lucene.util.Constants;
import org.apache.lucene.util.SuppressForbidden;
/** A vectorization provider that leverages the Panama Vector API. */
final class PanamaVectorizationProvider extends VectorizationProvider {
+ /** Preferred with in bits for vectors. */
+ static final int PREFERRED_VECTOR_BITSIZE;
+
+ static final VectorSpecies<Long> PRERERRED_LONG_SPECIES;
+ static final VectorSpecies<Integer> PRERERRED_INT_SPECIES;
+
+ /** Whether integer vectors can be trusted to actually be fast. */
+ static final boolean HAS_FAST_INTEGER_VECTORS;
+
private final VectorUtilSupport vectorUtilSupport;
+ static {
+ // default to platform supported bitsize
+ int vectorBitSize = VectorShape.preferredShape().vectorBitSize();
Review Comment:
Done. I like your comment added to the provider class!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1709880508
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java:
##########
@@ -16,20 +16,56 @@
*/
package org.apache.lucene.internal.vectorization;
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.util.Locale;
import java.util.logging.Logger;
import jdk.incubator.vector.FloatVector;
+import jdk.incubator.vector.VectorShape;
+import jdk.incubator.vector.VectorSpecies;
import org.apache.lucene.codecs.hnsw.FlatVectorsScorer;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.MemorySegmentAccessInput;
import org.apache.lucene.util.Constants;
import org.apache.lucene.util.SuppressForbidden;
/** A vectorization provider that leverages the Panama Vector API. */
final class PanamaVectorizationProvider extends VectorizationProvider {
+ /** Preferred with in bits for vectors. */
+ static final int PREFERRED_VECTOR_BITSIZE;
+
+ static final VectorSpecies<Long> PRERERRED_LONG_SPECIES;
+ static final VectorSpecies<Integer> PRERERRED_INT_SPECIES;
+
+ /** Whether integer vectors can be trusted to actually be fast. */
+ static final boolean HAS_FAST_INTEGER_VECTORS;
+
private final VectorUtilSupport vectorUtilSupport;
+ static {
+ // default to platform supported bitsize
+ int vectorBitSize = VectorShape.preferredShape().vectorBitSize();
Review Comment:
I understand this, I was worried that someone could make a small change that would reference PanamaVectorConstants before the check without realizing what the implications are. Thanks for helping, feel free to push to my branch, I believe that you have access to it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1712624281
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ private static final VectorSpecies<Long> LONG_SPECIES =
+ PanamaVectorConstants.PRERERRED_LONG_SPECIES;
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void readLongs(int count, long[] b) throws IOException {
+ if (count < LONG_SPECIES.length()) {
+ in.readLongs(b, 0, count);
+ } else {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int loopBound = LONG_SPECIES.loopBound(count - 1);
+ for (int i = 0;
+ i < loopBound;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector.fromMemorySegment(LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN)
Review Comment:
Hi, just a guess: `copyMemory()` has some overhead for very short arrays.
The good old ByteBuffers (MappedByteBuffer) had some optimization on ByteBuffer.getLongs() and other methods to copy a few items with a for-loop instead of copyMemory, see https://github.com/openjdk/jdk/blob/358d77dafbe0e35d5b20340fccddc0fb8f3db82a/src/java.base/share/classes/java/nio/X-Buffer.java.template#L954
With MemorySegment theres no such optimization anymore. We discussed this in early times of MemorySegment usage in Lucene, and Maurizio (from OpenJDK) at that time suggested to try out to use a simple loop instead of copyMemory for very small arays `<= Bits.JNI_COPY_TO_ARRAY_THRESHOLD`. The value is 6 array entries: https://github.com/openjdk/jdk/blob/358d77dafbe0e35d5b20340fccddc0fb8f3db82a/src/java.base/share/classes/java/nio/Bits.java#L232-L236 (I did this, but with Mike's benchmarks I did not see a large difference, only < stddev).
It could be that the above code is using a loop and not copyMemory behind scenes and therefor has a small speedup for long arrays < 6 items.
We could, separately from that issue implement some "short array scalar optimization" in `MemorySegmentIndexInput#getBytes/getLongs/getFloats/....` like in `XBuffer` classes.
I'd revert this additional complexity here and better check for optimizing `MemorySegmentIndexInput`. When doing this is would speedup other usages in Lucene, too! It is quite easy to add a simple if/else here: https://github.com/apache/lucene/blob/5aa1aa98ea2cdb37d7d8d48657703a5eb7fbaefa/lucene/core/src/java21/org/apache/lucene/store/MemorySegmentIndexInput.java#L222-L233
Basically as quick try add a (`if (length <= 6) { super.readLongs(....); return; }` here and measure again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1707445042
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import java.util.Optional;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ static Optional<PostingDecodingUtil> wrap(IndexInput in, MemorySegment memorySegment) {
+ if (64 % LONG_SPECIES.length() != 0) {
+ // Required to meet PostingDecodingUtil's contract that we do not write entries past index 64
+ // for any `count` in 0..64.
+ return Optional.empty();
+ }
+ return Optional.of(new MemorySegmentPostingDecodingUtil(in, memorySegment));
+ }
+
+ private static final VectorSpecies<Long> LONG_SPECIES = VectorSpecies.ofPreferred(long.class);
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ private MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int i;
+ // Note: this loop may apply to more than `count` entries due to the width of the preferred
+ // species. But doing this is faster than handling the remainder with scalar code.
+ for (i = 0;
+ i < count;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector vector =
+ LongVector.fromMemorySegment(
+ LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN);
+ vector
+ .lanewise(VectorOperators.LSHR, bShift)
+ .lanewise(VectorOperators.AND, bMask)
+ .intoArray(b, i);
+ vector.lanewise(VectorOperators.AND, cMask).intoArray(c, i);
Review Comment:
I think the check should just be in the factory method. If there are no fast integer vectors it returns the default impl.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "ChrisHegarty (via GitHub)" <gi...@apache.org>.
ChrisHegarty commented on PR #13636:
URL: https://github.com/apache/lucene/pull/13636#issuecomment-2288426886
Apologies for not interacting on this PR before, but I was away. Thanks for fixing the type of the length parameter in MemorySegmentIndexInput. I'll spend some time studying the code in more detail, but I like how this turned out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1709359720
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene912/PostingIndexInput.java:
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.codecs.lucene912;
+
+import java.io.IOException;
+import org.apache.lucene.internal.vectorization.PostingDecodingUtil;
+import org.apache.lucene.internal.vectorization.VectorizationProvider;
+import org.apache.lucene.store.IndexInput;
+
+/**
+ * Wrapper around an {@link IndexInput} and a {@link ForUtil} that optionally optimizes decoding
+ * using vectorization.
+ */
+public final class PostingIndexInput {
Review Comment:
Yeah I noticed. Agreed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1708936620
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import java.util.Optional;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ static Optional<PostingDecodingUtil> wrap(IndexInput in, MemorySegment memorySegment) {
+ if (64 % LONG_SPECIES.length() != 0) {
+ // Required to meet PostingDecodingUtil's contract that we do not write entries past index 64
+ // for any `count` in 0..64.
+ return Optional.empty();
+ }
+ return Optional.of(new MemorySegmentPostingDecodingUtil(in, memorySegment));
+ }
+
+ private static final VectorSpecies<Long> LONG_SPECIES = VectorSpecies.ofPreferred(long.class);
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ private MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int i;
+ // Note: this loop may apply to more than `count` entries due to the width of the preferred
+ // species. But doing this is faster than handling the remainder with scalar code.
+ for (i = 0;
+ i < count;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector vector =
+ LongVector.fromMemorySegment(
+ LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN);
+ vector
+ .lanewise(VectorOperators.LSHR, bShift)
+ .lanewise(VectorOperators.AND, bMask)
+ .intoArray(b, i);
+ vector.lanewise(VectorOperators.AND, cMask).intoArray(c, i);
Review Comment:
I removed the padding by processing some values twice within the range. Benchmarks suggest comparable numbers to the ones that I was getting previously:
```
Benchmark (bpv) Mode Cnt Score Error Units
PostingIndexInputBenchmark.decode 5 thrpt 15 41.278 ± 1.275 ops/us
PostingIndexInputBenchmark.decode 6 thrpt 15 43.173 ± 0.534 ops/us
PostingIndexInputBenchmark.decode 7 thrpt 15 42.270 ± 1.204 ops/us
PostingIndexInputBenchmark.decode 8 thrpt 15 47.889 ± 2.962 ops/us
PostingIndexInputBenchmark.decode 9 thrpt 15 35.049 ± 0.609 ops/us
PostingIndexInputBenchmark.decode 10 thrpt 15 38.714 ± 0.367 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 5 thrpt 15 19.071 ± 1.412 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 6 thrpt 15 20.228 ± 0.467 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 7 thrpt 15 20.718 ± 0.624 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 8 thrpt 15 21.001 ± 0.164 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 9 thrpt 15 18.273 ± 1.209 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 10 thrpt 15 17.666 ± 0.393 ops/us
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1706278927
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import java.util.Optional;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ static Optional<PostingDecodingUtil> wrap(IndexInput in, MemorySegment memorySegment) {
+ if (64 % LONG_SPECIES.length() != 0) {
+ // Required to meet PostingDecodingUtil's contract that we do not write entries past index 64
+ // for any `count` in 0..64.
+ return Optional.empty();
+ }
+ return Optional.of(new MemorySegmentPostingDecodingUtil(in, memorySegment));
+ }
+
+ private static final VectorSpecies<Long> LONG_SPECIES = VectorSpecies.ofPreferred(long.class);
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ private MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int i;
+ // Note: this loop may apply to more than `count` entries due to the width of the preferred
+ // species. But doing this is faster than handling the remainder with scalar code.
+ for (i = 0;
+ i < count;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector vector =
+ LongVector.fromMemorySegment(
+ LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN);
+ vector
+ .lanewise(VectorOperators.LSHR, bShift)
+ .lanewise(VectorOperators.AND, bMask)
+ .intoArray(b, i);
+ vector.lanewise(VectorOperators.AND, cMask).intoArray(c, i);
Review Comment:
i would also be for the solution to move this check in with the other sanity checks, and just don't use vector api at all for such configurations (currently we allow floating point, it works with the SSE), so nobody has to worry about adding checks to their vector code for every new function. The vector api is documented as being setup for AVX2+/Neon+ anyway. I just want to avoid the possibility of 30x slowdowns which are a big trap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1710505009
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ private static final VectorSpecies<Long> LONG_SPECIES =
+ PanamaVectorConstants.PRERERRED_LONG_SPECIES;
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void readLongs(int count, long[] b) throws IOException {
+ if (count < LONG_SPECIES.length()) {
+ in.readLongs(b, 0, count);
+ } else {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int loopBound = LONG_SPECIES.loopBound(count - 1);
+ for (int i = 0;
+ i < loopBound;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector.fromMemorySegment(LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN)
+ .intoArray(b, i);
+ }
+
+ // Handle the tail by reading a vector that is aligned with with `count` on the right side.
+ int i = count - LONG_SPECIES.length();
+ offset = endOffset - LONG_SPECIES.length() * Long.BYTES;
+ LongVector.fromMemorySegment(LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN)
+ .intoArray(b, i);
+
+ in.seek(endOffset);
+ }
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ if (count < LONG_SPECIES.length()) {
+ // Not enough data to vectorize without going out-of-bounds. In practice, this branch is never
+ // used if the bit width is 256, and is used for 2 and 3 bits per value if the bit width is
+ // 512.
+ in.readLongs(c, 0, count);
+ for (int i = 0; i < count; ++i) {
+ b[i] = (c[i] >>> bShift) & bMask;
+ c[i] &= cMask;
+ }
+ } else {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int loopBound = LONG_SPECIES.loopBound(count - 1);
Review Comment:
why the `loopBound(N - 1)` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1710505391
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import java.util.Optional;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ static Optional<PostingDecodingUtil> wrap(IndexInput in, MemorySegment memorySegment) {
+ if (64 % LONG_SPECIES.length() != 0) {
+ // Required to meet PostingDecodingUtil's contract that we do not write entries past index 64
+ // for any `count` in 0..64.
+ return Optional.empty();
+ }
+ return Optional.of(new MemorySegmentPostingDecodingUtil(in, memorySegment));
+ }
+
+ private static final VectorSpecies<Long> LONG_SPECIES = VectorSpecies.ofPreferred(long.class);
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ private MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int i;
+ // Note: this loop may apply to more than `count` entries due to the width of the preferred
+ // species. But doing this is faster than handling the remainder with scalar code.
+ for (i = 0;
+ i < count;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector vector =
+ LongVector.fromMemorySegment(
+ LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN);
+ vector
+ .lanewise(VectorOperators.LSHR, bShift)
+ .lanewise(VectorOperators.AND, bMask)
+ .intoArray(b, i);
+ vector.lanewise(VectorOperators.AND, cMask).intoArray(c, i);
Review Comment:
thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1711262142
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ private static final VectorSpecies<Long> LONG_SPECIES =
+ PanamaVectorConstants.PRERERRED_LONG_SPECIES;
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void readLongs(int count, long[] b) throws IOException {
+ if (count < LONG_SPECIES.length()) {
+ in.readLongs(b, 0, count);
+ } else {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int loopBound = LONG_SPECIES.loopBound(count - 1);
+ for (int i = 0;
+ i < loopBound;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector.fromMemorySegment(LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN)
Review Comment:
Why is this better/faster than a simple copyMemory as `in.readLongs()` is doing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1709866139
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java:
##########
@@ -16,20 +16,56 @@
*/
package org.apache.lucene.internal.vectorization;
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.util.Locale;
import java.util.logging.Logger;
import jdk.incubator.vector.FloatVector;
+import jdk.incubator.vector.VectorShape;
+import jdk.incubator.vector.VectorSpecies;
import org.apache.lucene.codecs.hnsw.FlatVectorsScorer;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.MemorySegmentAccessInput;
import org.apache.lucene.util.Constants;
import org.apache.lucene.util.SuppressForbidden;
/** A vectorization provider that leverages the Panama Vector API. */
final class PanamaVectorizationProvider extends VectorizationProvider {
+ /** Preferred with in bits for vectors. */
+ static final int PREFERRED_VECTOR_BITSIZE;
+
+ static final VectorSpecies<Long> PRERERRED_LONG_SPECIES;
+ static final VectorSpecies<Integer> PRERERRED_INT_SPECIES;
+
+ /** Whether integer vectors can be trusted to actually be fast. */
+ static final boolean HAS_FAST_INTEGER_VECTORS;
+
private final VectorUtilSupport vectorUtilSupport;
+ static {
+ // default to platform supported bitsize
+ int vectorBitSize = VectorShape.preferredShape().vectorBitSize();
Review Comment:
Nah, if you use them in ctor after the call to our check all is fine. Actually I'd add an additional `Lookup#ensureClassInitialized(PanamaVectorConstants.class)` into the try/catch block.
If you like I can commit to this branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #13636:
URL: https://github.com/apache/lucene/pull/13636#issuecomment-2276947096
> This class in the Panama codebase is huge and hard to read, so splitting into byte/float implementations looks like a good cleanup. No public API is involved.
Cool idea, thanks for looking into it, definitely want to remove this trap, especially since most of lucene is integers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz merged PR #13636:
URL: https://github.com/apache/lucene/pull/13636
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "gsmiller (via GitHub)" <gi...@apache.org>.
gsmiller commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1714509772
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene912/ForUtil.java:
##########
@@ -300,13 +301,14 @@ int numBytes(int bitsPerValue) {
return bitsPerValue << (BLOCK_SIZE_LOG2 - 3);
}
- private static void decodeSlow(int bitsPerValue, DataInput in, long[] tmp, long[] longs)
+ private static void decodeSlow(
+ int bitsPerValue, IndexInput in, PostingDecodingUtil pdu, long[] tmp, long[] longs)
Review Comment:
nit: looks like `in` is never actually referenced
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene912/ForUtil.java:
##########
@@ -300,13 +301,14 @@ int numBytes(int bitsPerValue) {
return bitsPerValue << (BLOCK_SIZE_LOG2 - 3);
}
- private static void decodeSlow(int bitsPerValue, DataInput in, long[] tmp, long[] longs)
+ private static void decodeSlow(
+ int bitsPerValue, IndexInput in, PostingDecodingUtil pdu, long[] tmp, long[] longs)
Review Comment:
Also, I think this is generally true of all the decode* methods?
Hmm... ah, I see. There are a couple rare cases where you need `in` but not `pdu` and the others are vice versa. And I assume it's easier for code generation to make the argument list consistent. OK... :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #13636:
URL: https://github.com/apache/lucene/pull/13636#issuecomment-2271546834
Oh, I had forgotten about this other PR! Thanks Uwe, I'll look into it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #13636:
URL: https://github.com/apache/lucene/pull/13636#issuecomment-2272145780
Thanks @uschindler, it was helpful. I refactored the PR a bit based on your recommendation. It's now ready for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1707463821
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java:
##########
@@ -79,4 +102,18 @@ public VectorUtilSupport getVectorUtilSupport() {
public FlatVectorsScorer getLucene99FlatVectorsScorer() {
return Lucene99MemorySegmentFlatVectorsScorer.INSTANCE;
}
+
+ @Override
+ public PostingDecodingUtil getPostingDecodingUtil(IndexInput input) throws IOException {
Review Comment:
I would do the fast integer vectors check here and return default if not supported.
I'd rename this method to have "new" in its name, like said before. The returned instance is a new one with some kind of state.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1707498886
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import java.util.Optional;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ static Optional<PostingDecodingUtil> wrap(IndexInput in, MemorySegment memorySegment) {
+ if (64 % LONG_SPECIES.length() != 0) {
+ // Required to meet PostingDecodingUtil's contract that we do not write entries past index 64
+ // for any `count` in 0..64.
+ return Optional.empty();
+ }
+ return Optional.of(new MemorySegmentPostingDecodingUtil(in, memorySegment));
+ }
+
+ private static final VectorSpecies<Long> LONG_SPECIES = VectorSpecies.ofPreferred(long.class);
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ private MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int i;
+ // Note: this loop may apply to more than `count` entries due to the width of the preferred
+ // species. But doing this is faster than handling the remainder with scalar code.
+ for (i = 0;
+ i < count;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector vector =
+ LongVector.fromMemorySegment(
+ LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN);
+ vector
+ .lanewise(VectorOperators.LSHR, bShift)
+ .lanewise(VectorOperators.AND, bMask)
+ .intoArray(b, i);
+ vector.lanewise(VectorOperators.AND, cMask).intoArray(c, i);
Review Comment:
Sorry, i responded to the wrong thread. I propose:
* make this function enabled for SVE/AVX512-only
* dont pad index files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1713050918
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ private static final VectorSpecies<Long> LONG_SPECIES =
+ PanamaVectorConstants.PRERERRED_LONG_SPECIES;
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void readLongs(int count, long[] b) throws IOException {
+ if (count < LONG_SPECIES.length()) {
+ in.readLongs(b, 0, count);
+ } else {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int loopBound = LONG_SPECIES.loopBound(count - 1);
+ for (int i = 0;
+ i < loopBound;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector.fromMemorySegment(LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN)
Review Comment:
I reverted the optimization, I don't mind it too much as I'd rather speed up decoding for the slower numbers of bits per value than faster bits per value and 8 is already the fastest. I am running benchmarks again and I'm getting inconsistent results with what I was getting before, using IndexInput#readLongs now seems on par with what I was getting with the optimized readLongs. I am unsure if I misran the benchmark before, or if something else changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #13636:
URL: https://github.com/apache/lucene/pull/13636#issuecomment-2291136659
A few queries got a small-ish speedup, e.g.
- [AndMedOrHighHigh](https://home.apache.org/~mikemccand/lucenebench/AndMedOrHighHigh.html) +2.5%
- [AndHighHigh](https://home.apache.org/~mikemccand/lucenebench/AndHighHigh.html) +1.5%
- [CountAndHighMed](https://home.apache.org/~mikemccand/lucenebench/CountAndHighMed.html) +1.5%
- [AndHighMed](https://home.apache.org/~mikemccand/lucenebench/AndHighMed.html) +1%
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1712554332
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ private static final VectorSpecies<Long> LONG_SPECIES =
+ PanamaVectorConstants.PRERERRED_LONG_SPECIES;
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void readLongs(int count, long[] b) throws IOException {
+ if (count < LONG_SPECIES.length()) {
+ in.readLongs(b, 0, count);
+ } else {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int loopBound = LONG_SPECIES.loopBound(count - 1);
+ for (int i = 0;
+ i < loopBound;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector.fromMemorySegment(LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN)
Review Comment:
This method is only used when decoding 8 or 16 bits per value. I am not sure why, but the micro benchmark sees a small but consistent speedup by copying memory this way rather than using readLongs. If you don't like it, I can remove it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1711262142
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ private static final VectorSpecies<Long> LONG_SPECIES =
+ PanamaVectorConstants.PRERERRED_LONG_SPECIES;
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void readLongs(int count, long[] b) throws IOException {
+ if (count < LONG_SPECIES.length()) {
+ in.readLongs(b, 0, count);
+ } else {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int loopBound = LONG_SPECIES.loopBound(count - 1);
+ for (int i = 0;
+ i < loopBound;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector.fromMemorySegment(LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN)
Review Comment:
Why is this better/faster than a simple copyMemory as `in.readLongs()` is doing? At the end it is copied to an on-heap array anyways.
I was expecting the whole PR to do the splitLongs directly on the MemorySegment and not on on-heap arrays.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on PR #13636:
URL: https://github.com/apache/lucene/pull/13636#issuecomment-2271539092
Hi,
your setup needs to be a bit different:
- Don't invent new providers, use the existing one, so add a new factory method to the generic VectorizationProvider. Here you return a new implementation instance (like the existing one called VectorUtilSupport). If the implementation is a singleton, call the factory method `getVectorUtilSupport()`, if it creates a new one (with state, like ForUtil) call method `newForUtil()`
- In src/main/java, you let the factory return the default instance
- In src/main/javaXX you let it return the optimal instance
I have another Draft PR: #12412
You can copy everything from there - yes I spent some work already on that!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1709965505
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java:
##########
@@ -37,15 +44,23 @@ private static <T> T doPrivileged(PrivilegedAction<T> action) {
return AccessController.doPrivileged(action);
}
+ private final VectorUtilSupport vectorUtilSupport;
+
PanamaVectorizationProvider() {
// hack to work around for JDK-8309727:
try {
doPrivileged(
- () ->
- FloatVector.fromArray(
- FloatVector.SPECIES_PREFERRED,
- new float[FloatVector.SPECIES_PREFERRED.length()],
- 0));
+ () -> {
+ try {
+ MethodHandles.lookup().ensureInitialized(PanamaVectorConstants.class);
Review Comment:
Backport note: In Java 11 this method does not exist, replace by: `Class.forName(PanamaVectorConstants.class.getName())`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on PR #13636:
URL: https://github.com/apache/lucene/pull/13636#issuecomment-2275507708
> To help with @rmuir's fast integer vectors issues: Once this is merged, I plan to split the VectorUtilSupport into ByteVectorUtilSupport and FloatVectorUtilSupport and use 2 getters in VectorizationProvider.
This class in the Panama codebase is huge and hard to read, so splitting into byte/float implementations looks like a good cleanup. No public API is involved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1706272955
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import java.util.Optional;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ static Optional<PostingDecodingUtil> wrap(IndexInput in, MemorySegment memorySegment) {
+ if (64 % LONG_SPECIES.length() != 0) {
+ // Required to meet PostingDecodingUtil's contract that we do not write entries past index 64
+ // for any `count` in 0..64.
+ return Optional.empty();
+ }
+ return Optional.of(new MemorySegmentPostingDecodingUtil(in, memorySegment));
+ }
+
+ private static final VectorSpecies<Long> LONG_SPECIES = VectorSpecies.ofPreferred(long.class);
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ private MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int i;
+ // Note: this loop may apply to more than `count` entries due to the width of the preferred
+ // species. But doing this is faster than handling the remainder with scalar code.
Review Comment:
ok, i see it digging more. this is a bit strange to pad postings files with kilobytes of data just for the purpose: I'm a little worried it might hide all bugs (e.g. the infamous "Read EOF").
have you experimented with indexInRange (VectorMask)? Maybe try it on AVX-512 or ARM SVE that supports it in hardware and see if it works just as well without this padding. If so, we could consider keeping it clean and only using this function for those cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1707373761
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import java.util.Optional;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ static Optional<PostingDecodingUtil> wrap(IndexInput in, MemorySegment memorySegment) {
+ if (64 % LONG_SPECIES.length() != 0) {
+ // Required to meet PostingDecodingUtil's contract that we do not write entries past index 64
+ // for any `count` in 0..64.
+ return Optional.empty();
+ }
+ return Optional.of(new MemorySegmentPostingDecodingUtil(in, memorySegment));
+ }
+
+ private static final VectorSpecies<Long> LONG_SPECIES = VectorSpecies.ofPreferred(long.class);
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ private MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int i;
+ // Note: this loop may apply to more than `count` entries due to the width of the preferred
+ // species. But doing this is faster than handling the remainder with scalar code.
Review Comment:
I'm proposing looking at vector masks on ARM SVE and AVX-512 only. With loop matching what is in the Vector API JEP for masks.
We know it can't work for your machine without AVX-512, we just wouldnt use this function unless SVE or AVX-512 are present with the predicate instructions in hardware.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1707377294
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import java.util.Optional;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ static Optional<PostingDecodingUtil> wrap(IndexInput in, MemorySegment memorySegment) {
+ if (64 % LONG_SPECIES.length() != 0) {
+ // Required to meet PostingDecodingUtil's contract that we do not write entries past index 64
+ // for any `count` in 0..64.
+ return Optional.empty();
+ }
+ return Optional.of(new MemorySegmentPostingDecodingUtil(in, memorySegment));
+ }
+
+ private static final VectorSpecies<Long> LONG_SPECIES = VectorSpecies.ofPreferred(long.class);
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ private MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int i;
+ // Note: this loop may apply to more than `count` entries due to the width of the preferred
+ // species. But doing this is faster than handling the remainder with scalar code.
+ for (i = 0;
+ i < count;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector vector =
+ LongVector.fromMemorySegment(
+ LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN);
+ vector
+ .lanewise(VectorOperators.LSHR, bShift)
+ .lanewise(VectorOperators.AND, bMask)
+ .intoArray(b, i);
+ vector.lanewise(VectorOperators.AND, cMask).intoArray(c, i);
Review Comment:
in pseudocode:
```
// no padding to index files
HAS_HARDWARE_MASKS = HAS_SVE || HAS_AVX512;
if (HAS_HARDWARE_MASKS) {
return explicitVectorized();
} else {
return generic(); /* autovectorized */
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1711141270
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ private static final VectorSpecies<Long> LONG_SPECIES =
+ PanamaVectorConstants.PRERERRED_LONG_SPECIES;
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void readLongs(int count, long[] b) throws IOException {
+ if (count < LONG_SPECIES.length()) {
+ in.readLongs(b, 0, count);
+ } else {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int loopBound = LONG_SPECIES.loopBound(count - 1);
+ for (int i = 0;
+ i < loopBound;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector.fromMemorySegment(LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN)
+ .intoArray(b, i);
+ }
+
+ // Handle the tail by reading a vector that is aligned with with `count` on the right side.
+ int i = count - LONG_SPECIES.length();
+ offset = endOffset - LONG_SPECIES.length() * Long.BYTES;
+ LongVector.fromMemorySegment(LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN)
+ .intoArray(b, i);
+
+ in.seek(endOffset);
+ }
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ if (count < LONG_SPECIES.length()) {
+ // Not enough data to vectorize without going out-of-bounds. In practice, this branch is never
+ // used if the bit width is 256, and is used for 2 and 3 bits per value if the bit width is
+ // 512.
+ in.readLongs(c, 0, count);
+ for (int i = 0; i < count; ++i) {
+ b[i] = (c[i] >>> bShift) & bMask;
+ c[i] &= cMask;
+ }
+ } else {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int loopBound = LONG_SPECIES.loopBound(count - 1);
Review Comment:
This is to make sure that the loop doesn't handle the rightmost vector. We could do `loopBound(N)` but then we'd need to protect the code after the loop with `if (loopBound < N)` to avoid doing redundant work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1709378126
##########
lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultPostingDecodingUtil.java:
##########
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import org.apache.lucene.store.IndexInput;
+
+final class DefaultPostingDecodingUtil extends PostingDecodingUtil {
+
+ protected final IndexInput in;
+
+ public DefaultPostingDecodingUtil(IndexInput in) {
+ this.in = in;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ assert count <= 64;
+ in.readLongs(c, 0, count);
+ // The below loop is auto-vectorized
+ for (int i = 0; i < count; ++i) {
+ b[i] = (c[i] >>> bShift) & bMask;
+ c[i] &= cMask;
Review Comment:
And numbers when the vector module is enabled for reference:
```
Benchmark (bpv) Mode Cnt Score Error Units
PostingIndexInputBenchmark.decode 5 thrpt 15 40.887 ± 0.565 ops/us
PostingIndexInputBenchmark.decode 6 thrpt 15 44.043 ± 0.633 ops/us
PostingIndexInputBenchmark.decode 7 thrpt 15 42.804 ± 0.813 ops/us
PostingIndexInputBenchmark.decode 8 thrpt 15 51.670 ± 1.368 ops/us
PostingIndexInputBenchmark.decode 9 thrpt 15 36.076 ± 0.177 ops/us
PostingIndexInputBenchmark.decode 10 thrpt 15 38.462 ± 0.379 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 5 thrpt 15 19.973 ± 0.230 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 6 thrpt 15 17.797 ± 3.315 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 7 thrpt 15 20.274 ± 0.664 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 8 thrpt 15 21.163 ± 0.604 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 9 thrpt 15 18.460 ± 0.493 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 10 thrpt 15 18.622 ± 0.941 ops/us
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #13636:
URL: https://github.com/apache/lucene/pull/13636#issuecomment-2271488324
@uschindler @ChrisHegarty I could use a bit of help with this change regarding code organization and how to make it properly work out of the box (without enabling the vector module and preview features). Feel free to give suggestions or push changes directly if that works better for you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1709339018
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene912/PostingIndexInput.java:
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.codecs.lucene912;
+
+import java.io.IOException;
+import org.apache.lucene.internal.vectorization.PostingDecodingUtil;
+import org.apache.lucene.internal.vectorization.VectorizationProvider;
+import org.apache.lucene.store.IndexInput;
+
+/**
+ * Wrapper around an {@link IndexInput} and a {@link ForUtil} that optionally optimizes decoding
+ * using vectorization.
+ */
+public final class PostingIndexInput {
Review Comment:
This is what I did at first, but I also wanted to be able to write a micro benchmark and the vectorization package is not exposed, so I introduced this intermediate abstraction. If you have other ideas, I'm open to suggestions. The micro benchmark did not suggest noticeable overhead, but I haven't tried with luceneutil.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1709403728
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java:
##########
@@ -16,20 +16,56 @@
*/
package org.apache.lucene.internal.vectorization;
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.util.Locale;
import java.util.logging.Logger;
import jdk.incubator.vector.FloatVector;
+import jdk.incubator.vector.VectorShape;
+import jdk.incubator.vector.VectorSpecies;
import org.apache.lucene.codecs.hnsw.FlatVectorsScorer;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.MemorySegmentAccessInput;
import org.apache.lucene.util.Constants;
import org.apache.lucene.util.SuppressForbidden;
/** A vectorization provider that leverages the Panama Vector API. */
final class PanamaVectorizationProvider extends VectorizationProvider {
+ /** Preferred with in bits for vectors. */
+ static final int PREFERRED_VECTOR_BITSIZE;
+
+ static final VectorSpecies<Long> PRERERRED_LONG_SPECIES;
+ static final VectorSpecies<Integer> PRERERRED_INT_SPECIES;
+
+ /** Whether integer vectors can be trusted to actually be fast. */
+ static final boolean HAS_FAST_INTEGER_VECTORS;
+
private final VectorUtilSupport vectorUtilSupport;
+ static {
+ // default to platform supported bitsize
+ int vectorBitSize = VectorShape.preferredShape().vectorBitSize();
Review Comment:
I applied your suggestion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1709520659
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java:
##########
@@ -16,20 +16,56 @@
*/
package org.apache.lucene.internal.vectorization;
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.util.Locale;
import java.util.logging.Logger;
import jdk.incubator.vector.FloatVector;
+import jdk.incubator.vector.VectorShape;
+import jdk.incubator.vector.VectorSpecies;
import org.apache.lucene.codecs.hnsw.FlatVectorsScorer;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.MemorySegmentAccessInput;
import org.apache.lucene.util.Constants;
import org.apache.lucene.util.SuppressForbidden;
/** A vectorization provider that leverages the Panama Vector API. */
final class PanamaVectorizationProvider extends VectorizationProvider {
+ /** Preferred with in bits for vectors. */
+ static final int PREFERRED_VECTOR_BITSIZE;
+
+ static final VectorSpecies<Long> PRERERRED_LONG_SPECIES;
+ static final VectorSpecies<Integer> PRERERRED_INT_SPECIES;
+
+ /** Whether integer vectors can be trusted to actually be fast. */
+ static final boolean HAS_FAST_INTEGER_VECTORS;
+
private final VectorUtilSupport vectorUtilSupport;
+ static {
+ // default to platform supported bitsize
+ int vectorBitSize = VectorShape.preferredShape().vectorBitSize();
Review Comment:
Looks fine, but I would move ALL constants and static initializers from here to the other class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1712624281
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ private static final VectorSpecies<Long> LONG_SPECIES =
+ PanamaVectorConstants.PRERERRED_LONG_SPECIES;
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void readLongs(int count, long[] b) throws IOException {
+ if (count < LONG_SPECIES.length()) {
+ in.readLongs(b, 0, count);
+ } else {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int loopBound = LONG_SPECIES.loopBound(count - 1);
+ for (int i = 0;
+ i < loopBound;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector.fromMemorySegment(LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN)
Review Comment:
Hi, just a guess: `copyMemory()` has some overhead for very short arrays.
The good old ByteBuffers (MappedByteBuffer) had some optimization on ByteBuffer.getLongs() and other methods to copy a few items with a for-loop instead of copyMemory, see https://github.com/openjdk/jdk/blob/358d77dafbe0e35d5b20340fccddc0fb8f3db82a/src/java.base/share/classes/java/nio/X-Buffer.java.template#L954
With MemorySegment theres no such optimization anymore. We discussed this in early times of MemorySegment usage in Lucene, and Maurizio (from OpenJDK) at that time suggested to try out to use a simple loop instead of copyMemory for very small arays `<= Bits.JNI_COPY_TO_ARRAY_THRESHOLD`. The value is 6 array entries: https://github.com/openjdk/jdk/blob/358d77dafbe0e35d5b20340fccddc0fb8f3db82a/src/java.base/share/classes/java/nio/Bits.java#L232-L236 (I did this, but with Mike's benchmarks I did not see a large difference, only < stddev).
It could be that the above code is using a loop and not copyMemory behind scenes and therefor has a small speedup for long arrays < 6 items.
We could, separately from that issue implement some "short array scalar optimization" in `MemorySegmentIndexInput#getBytes/getLongs/getFloats/....` like in `XBuffer` classes.
I'd revert this additional complexity here and better check for optimizing `MemorySegmentIndexInput`. When doing this is would speedup other usages in Lucene, too! It is quite easy to add a simple if/else here: https://github.com/apache/lucene/blob/5aa1aa98ea2cdb37d7d8d48657703a5eb7fbaefa/lucene/core/src/java21/org/apache/lucene/store/MemorySegmentIndexInput.java#L222-L233
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1712624281
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ private static final VectorSpecies<Long> LONG_SPECIES =
+ PanamaVectorConstants.PRERERRED_LONG_SPECIES;
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void readLongs(int count, long[] b) throws IOException {
+ if (count < LONG_SPECIES.length()) {
+ in.readLongs(b, 0, count);
+ } else {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int loopBound = LONG_SPECIES.loopBound(count - 1);
+ for (int i = 0;
+ i < loopBound;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector.fromMemorySegment(LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN)
Review Comment:
Hi, just a guess: `copyMemory()` has some overhead for very short arrays.
The good old ByteBuffers (MappedByteBuffer) had some optimization on ByteBuffer.getLongs() and other methods to copy a few items with a for-loop instead of copyMemory, see https://github.com/openjdk/jdk/blob/358d77dafbe0e35d5b20340fccddc0fb8f3db82a/src/java.base/share/classes/java/nio/X-Buffer.java.template#L954
With MemorySegment theres no such optimization anymore. We discussed this in early times of MemorySegmentUsage, and Murizio at that time suggested to use a simple loop instead of copyMemory for very small arays `<= Bits.JNI_COPY_TO_ARRAY_THRESHOLD`. The value is 6 array entries: https://github.com/openjdk/jdk/blob/358d77dafbe0e35d5b20340fccddc0fb8f3db82a/src/java.base/share/classes/java/nio/Bits.java#L232-L236
It could be that the above code is using a loop and not copyMemory behind scenes and therefor has a small speedup for long arrays < 6 items.
We could, separately from that issue implement some "short array scalar optimization" in `MemorySegmentIndexInput#getBytes/getLongs/getFloats/....` like in `XBuffer` classes.
I'd revert this additional complexity here and better check for optimizing `MemorySegmentIndexInput`. When doing this is would speedup other usages in Lucene, too! It is quite easy to add a simple if/else here: https://github.com/apache/lucene/blob/5aa1aa98ea2cdb37d7d8d48657703a5eb7fbaefa/lucene/core/src/java21/org/apache/lucene/store/MemorySegmentIndexInput.java#L222-L233
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #13636:
URL: https://github.com/apache/lucene/pull/13636#issuecomment-2284765651
@uschindler @rmuir Any objections to merging this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on PR #13636:
URL: https://github.com/apache/lucene/pull/13636#issuecomment-2271580963
> how to make it properly work out of the box (without enabling the vector module and preview features)
The vector module must always be enabled, but that's also the case for `VectorUtil` class :-) But the VectorizationProvider code (don't forget to add your class to the caller whitelist!) will take care to decide if it needs to be enabled or not and logs warnings. It also disables vectorization in invalid configurations like people using Graal or OpenJ9.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1706268616
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import java.util.Optional;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ static Optional<PostingDecodingUtil> wrap(IndexInput in, MemorySegment memorySegment) {
+ if (64 % LONG_SPECIES.length() != 0) {
+ // Required to meet PostingDecodingUtil's contract that we do not write entries past index 64
+ // for any `count` in 0..64.
+ return Optional.empty();
+ }
+ return Optional.of(new MemorySegmentPostingDecodingUtil(in, memorySegment));
+ }
+
+ private static final VectorSpecies<Long> LONG_SPECIES = VectorSpecies.ofPreferred(long.class);
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ private MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int i;
+ // Note: this loop may apply to more than `count` entries due to the width of the preferred
+ // species. But doing this is faster than handling the remainder with scalar code.
Review Comment:
how is this safe to do? what if cpu has 2048-bit wide vectors (theoretically possible for arm sve)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1707370205
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import java.util.Optional;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ static Optional<PostingDecodingUtil> wrap(IndexInput in, MemorySegment memorySegment) {
+ if (64 % LONG_SPECIES.length() != 0) {
+ // Required to meet PostingDecodingUtil's contract that we do not write entries past index 64
+ // for any `count` in 0..64.
+ return Optional.empty();
+ }
+ return Optional.of(new MemorySegmentPostingDecodingUtil(in, memorySegment));
+ }
+
+ private static final VectorSpecies<Long> LONG_SPECIES = VectorSpecies.ofPreferred(long.class);
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ private MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int i;
+ // Note: this loop may apply to more than `count` entries due to the width of the preferred
+ // species. But doing this is faster than handling the remainder with scalar code.
+ for (i = 0;
+ i < count;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector vector =
+ LongVector.fromMemorySegment(
+ LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN);
+ vector
+ .lanewise(VectorOperators.LSHR, bShift)
+ .lanewise(VectorOperators.AND, bMask)
+ .intoArray(b, i);
+ vector.lanewise(VectorOperators.AND, cMask).intoArray(c, i);
Review Comment:
thanks, we may still want to move the `HAS_FAST_INTEGER_VECTORS` to "Uwe code" and not do any vector api at all in that case.
it was first implemented this way, because we were trying to accelerate floating point, and integer stuff was added after that. But most of lucene uses integers, so it may be the exception, not the rule. So I still think it is a trap, if someone adds a new PR with new vectorization (like this one), they have to always remember to check this boolean for every new function, or risk a 30x slowdown. Doesn't seem like the right tradeoff.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "gsmiller (via GitHub)" <gi...@apache.org>.
gsmiller commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1707451406
##########
lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultPostingDecodingUtil.java:
##########
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import org.apache.lucene.store.IndexInput;
+
+final class DefaultPostingDecodingUtil extends PostingDecodingUtil {
+
+ protected final IndexInput in;
+
+ public DefaultPostingDecodingUtil(IndexInput in) {
+ this.in = in;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ assert count <= 64;
+ in.readLongs(c, 0, count);
+ // The below loop is auto-vectorized
+ for (int i = 0; i < count; ++i) {
+ b[i] = (c[i] >>> bShift) & bMask;
+ c[i] &= cMask;
Review Comment:
If I'm understanding things correctly, we only need to apply this masking for some of the BPV values but not all. For example, we don't need this masking step for bpv4 but we do need it for bpv12. Our current code is selective about when it does this but it looks like here we're going be always doing it, which creates some wasted overhead. I wonder if there's an opportunity to do this selectively? Or maybe it doesn't actually matter in practice? (Same comment applies of course to the vectorized version as well)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1709362066
##########
lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultPostingDecodingUtil.java:
##########
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import org.apache.lucene.store.IndexInput;
+
+final class DefaultPostingDecodingUtil extends PostingDecodingUtil {
+
+ protected final IndexInput in;
+
+ public DefaultPostingDecodingUtil(IndexInput in) {
+ this.in = in;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ assert count <= 64;
+ in.readLongs(c, 0, count);
+ // The below loop is auto-vectorized
+ for (int i = 0; i < count; ++i) {
+ b[i] = (c[i] >>> bShift) & bMask;
+ c[i] &= cMask;
Review Comment:
I tried a different approach that consists of only specializing 8 and 16. Otherwise, we keep applying the mask all, but then take advantage of the fact that this mask has been applied when decoding tail values using scalar code. This seems to yield a small performance improvement when the vector module is disabled and make performance similar to what it was before. This effectively generalizes the current optimization we have to all numbers of bits per value, so I removed the specialization.
```
Benchmark (bpv) Mode Cnt Score Error Units
PostingIndexInputBenchmark.decode 5 thrpt 15 34.857 ± 0.678 ops/us
PostingIndexInputBenchmark.decode 6 thrpt 15 33.752 ± 1.060 ops/us
PostingIndexInputBenchmark.decode 7 thrpt 15 28.966 ± 4.743 ops/us
PostingIndexInputBenchmark.decode 8 thrpt 15 47.009 ± 0.518 ops/us
PostingIndexInputBenchmark.decode 9 thrpt 15 28.328 ± 0.978 ops/us
PostingIndexInputBenchmark.decode 10 thrpt 15 32.007 ± 1.173 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 5 thrpt 15 18.161 ± 0.244 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 6 thrpt 15 17.470 ± 1.149 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 7 thrpt 15 16.907 ± 1.788 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 8 thrpt 15 20.259 ± 1.487 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 9 thrpt 15 14.214 ± 0.521 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 10 thrpt 15 15.618 ± 1.410 ops/us
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1709095157
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java:
##########
@@ -16,20 +16,56 @@
*/
package org.apache.lucene.internal.vectorization;
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.util.Locale;
import java.util.logging.Logger;
import jdk.incubator.vector.FloatVector;
+import jdk.incubator.vector.VectorShape;
+import jdk.incubator.vector.VectorSpecies;
import org.apache.lucene.codecs.hnsw.FlatVectorsScorer;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.MemorySegmentAccessInput;
import org.apache.lucene.util.Constants;
import org.apache.lucene.util.SuppressForbidden;
/** A vectorization provider that leverages the Panama Vector API. */
final class PanamaVectorizationProvider extends VectorizationProvider {
+ /** Preferred with in bits for vectors. */
+ static final int PREFERRED_VECTOR_BITSIZE;
+
+ static final VectorSpecies<Long> PRERERRED_LONG_SPECIES;
+ static final VectorSpecies<Integer> PRERERRED_INT_SPECIES;
+
+ /** Whether integer vectors can be trusted to actually be fast. */
+ static final boolean HAS_FAST_INTEGER_VECTORS;
+
private final VectorUtilSupport vectorUtilSupport;
+ static {
+ // default to platform supported bitsize
+ int vectorBitSize = VectorShape.preferredShape().vectorBitSize();
Review Comment:
In 9.x we have to be a bit of careful here, because the ctor of this class checks some incompatibility with security manager first. If this static code runs first we may hit the Java 20 bug! https://bugs.openjdk.org/browse/JDK-8309727
We could decide on backporting if we drop support for the buggy jdk or possible surround this with a try/catch.
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java:
##########
@@ -79,4 +113,15 @@ public VectorUtilSupport getVectorUtilSupport() {
public FlatVectorsScorer getLucene99FlatVectorsScorer() {
return Lucene99MemorySegmentFlatVectorsScorer.INSTANCE;
}
+
+ @Override
+ public PostingDecodingUtil newPostingDecodingUtil(IndexInput input) throws IOException {
+ if (HAS_FAST_INTEGER_VECTORS && input instanceof MemorySegmentAccessInput msai) {
+ MemorySegment ms = msai.segmentSliceOrNull(0, input.length());
Review Comment:
Do we really need to have a view on the full length?
This may disable the wohle optimization for index files > 16 GiB (because we mmap them it chunks of 16 GiB).
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene912/PostingIndexInput.java:
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.codecs.lucene912;
+
+import java.io.IOException;
+import org.apache.lucene.internal.vectorization.PostingDecodingUtil;
+import org.apache.lucene.internal.vectorization.VectorizationProvider;
+import org.apache.lucene.store.IndexInput;
+
+/**
+ * Wrapper around an {@link IndexInput} and a {@link ForUtil} that optionally optimizes decoding
+ * using vectorization.
+ */
+public final class PostingIndexInput {
Review Comment:
Do we really need that wrapper class? It does not hurt, but adds complexity. We could just get the `newPostingDecodingUtil(in)` in `Lucene912PostingsReader`.
On the other hand this abstraction looks fine. If it does not change performance I am fine with it.
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java:
##########
@@ -79,4 +113,15 @@ public VectorUtilSupport getVectorUtilSupport() {
public FlatVectorsScorer getLucene99FlatVectorsScorer() {
return Lucene99MemorySegmentFlatVectorsScorer.INSTANCE;
}
+
+ @Override
+ public PostingDecodingUtil newPostingDecodingUtil(IndexInput input) throws IOException {
+ if (HAS_FAST_INTEGER_VECTORS && input instanceof MemorySegmentAccessInput msai) {
+ MemorySegment ms = msai.segmentSliceOrNull(0, input.length());
Review Comment:
But this would only affect people with fully forcemerged indexes, by default our index files (more correct: segment sizes) are <= 5 GiB.
We don't need any tests for this extra case, because it just falls back to (slower) default decoding thats used for everything else than MMapDir, and thats tested enough!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1709336210
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java:
##########
@@ -79,4 +113,15 @@ public VectorUtilSupport getVectorUtilSupport() {
public FlatVectorsScorer getLucene99FlatVectorsScorer() {
return Lucene99MemorySegmentFlatVectorsScorer.INSTANCE;
}
+
+ @Override
+ public PostingDecodingUtil newPostingDecodingUtil(IndexInput input) throws IOException {
+ if (HAS_FAST_INTEGER_VECTORS && input instanceof MemorySegmentAccessInput msai) {
+ MemorySegment ms = msai.segmentSliceOrNull(0, input.length());
Review Comment:
Right. I leaned this way to avoid paying the overhead per decoded block. If segments > 16GB are not compound segments, then they would also need the doc (or pos, or pay) file to exceed 16GB on its own, which is a pretty big segment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1708959969
##########
lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultPostingDecodingUtil.java:
##########
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import org.apache.lucene.store.IndexInput;
+
+final class DefaultPostingDecodingUtil extends PostingDecodingUtil {
+
+ protected final IndexInput in;
+
+ public DefaultPostingDecodingUtil(IndexInput in) {
+ this.in = in;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ assert count <= 64;
+ in.readLongs(c, 0, count);
+ // The below loop is auto-vectorized
+ for (int i = 0; i < count; ++i) {
+ b[i] = (c[i] >>> bShift) & bMask;
+ c[i] &= cMask;
Review Comment:
I just ran the benchmark without enabling the vector module so that results are comparable with the current implementation in `main`:
```
Benchmark (bpv) Mode Cnt Score Error Units
PostingIndexInputBenchmark.decode 5 thrpt 15 33.918 ± 0.422 ops/us
PostingIndexInputBenchmark.decode 6 thrpt 15 33.804 ± 0.525 ops/us
PostingIndexInputBenchmark.decode 7 thrpt 15 32.172 ± 0.653 ops/us
PostingIndexInputBenchmark.decode 8 thrpt 15 36.298 ± 0.781 ops/us
PostingIndexInputBenchmark.decode 9 thrpt 15 27.755 ± 0.504 ops/us
PostingIndexInputBenchmark.decode 10 thrpt 15 29.801 ± 0.598 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 5 thrpt 15 17.911 ± 0.373 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 6 thrpt 15 17.775 ± 0.997 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 7 thrpt 15 15.688 ± 1.106 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 8 thrpt 15 17.817 ± 1.342 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 9 thrpt 15 15.794 ± 0.461 ops/us
PostingIndexInputBenchmark.decodeAndPrefixSum 10 thrpt 15 15.291 ± 1.781 ops/us
```
There seems to be a performance hit in all cases, with a higher performance hit for bits per value that do not need to apply the mask (5, 8, 9 and 10). So it looks like you are right, I will look into it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1708943154
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import java.util.Optional;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ static Optional<PostingDecodingUtil> wrap(IndexInput in, MemorySegment memorySegment) {
+ if (64 % LONG_SPECIES.length() != 0) {
+ // Required to meet PostingDecodingUtil's contract that we do not write entries past index 64
+ // for any `count` in 0..64.
+ return Optional.empty();
+ }
+ return Optional.of(new MemorySegmentPostingDecodingUtil(in, memorySegment));
+ }
+
+ private static final VectorSpecies<Long> LONG_SPECIES = VectorSpecies.ofPreferred(long.class);
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ private MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int i;
+ // Note: this loop may apply to more than `count` entries due to the width of the preferred
+ // species. But doing this is faster than handling the remainder with scalar code.
Review Comment:
I followed a slightly different approach that looks fast regardless of whether the host supports AVX-512 or SVE.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1708960612
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java:
##########
@@ -79,4 +102,18 @@ public VectorUtilSupport getVectorUtilSupport() {
public FlatVectorsScorer getLucene99FlatVectorsScorer() {
return Lucene99MemorySegmentFlatVectorsScorer.INSTANCE;
}
+
+ @Override
+ public PostingDecodingUtil getPostingDecodingUtil(IndexInput input) throws IOException {
Review Comment:
Sorry I had missed your previous comment. This makes total sense, I renamed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "gsmiller (via GitHub)" <gi...@apache.org>.
gsmiller commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1711646318
##########
lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultPostingDecodingUtil.java:
##########
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import org.apache.lucene.store.IndexInput;
+
+final class DefaultPostingDecodingUtil extends PostingDecodingUtil {
+
+ protected final IndexInput in;
+
+ public DefaultPostingDecodingUtil(IndexInput in) {
+ this.in = in;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ assert count <= 64;
+ in.readLongs(c, 0, count);
+ // The below loop is auto-vectorized
+ for (int i = 0; i < count; ++i) {
+ b[i] = (c[i] >>> bShift) & bMask;
+ c[i] &= cMask;
Review Comment:
Thanks @jpountz I think the version you landed on makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1709340084
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java:
##########
@@ -16,20 +16,56 @@
*/
package org.apache.lucene.internal.vectorization;
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.util.Locale;
import java.util.logging.Logger;
import jdk.incubator.vector.FloatVector;
+import jdk.incubator.vector.VectorShape;
+import jdk.incubator.vector.VectorSpecies;
import org.apache.lucene.codecs.hnsw.FlatVectorsScorer;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.MemorySegmentAccessInput;
import org.apache.lucene.util.Constants;
import org.apache.lucene.util.SuppressForbidden;
/** A vectorization provider that leverages the Panama Vector API. */
final class PanamaVectorizationProvider extends VectorizationProvider {
+ /** Preferred with in bits for vectors. */
+ static final int PREFERRED_VECTOR_BITSIZE;
+
+ static final VectorSpecies<Long> PRERERRED_LONG_SPECIES;
+ static final VectorSpecies<Integer> PRERERRED_INT_SPECIES;
+
+ /** Whether integer vectors can be trusted to actually be fast. */
+ static final boolean HAS_FAST_INTEGER_VECTORS;
+
private final VectorUtilSupport vectorUtilSupport;
+ static {
+ // default to platform supported bitsize
+ int vectorBitSize = VectorShape.preferredShape().vectorBitSize();
Review Comment:
Woops, thanks for pointing out, I'll refactor to not make this initialized statically.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1709705545
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java:
##########
@@ -16,20 +16,56 @@
*/
package org.apache.lucene.internal.vectorization;
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.util.Locale;
import java.util.logging.Logger;
import jdk.incubator.vector.FloatVector;
+import jdk.incubator.vector.VectorShape;
+import jdk.incubator.vector.VectorSpecies;
import org.apache.lucene.codecs.hnsw.FlatVectorsScorer;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.MemorySegmentAccessInput;
import org.apache.lucene.util.Constants;
import org.apache.lucene.util.SuppressForbidden;
/** A vectorization provider that leverages the Panama Vector API. */
final class PanamaVectorizationProvider extends VectorizationProvider {
+ /** Preferred with in bits for vectors. */
+ static final int PREFERRED_VECTOR_BITSIZE;
+
+ static final VectorSpecies<Long> PRERERRED_LONG_SPECIES;
+ static final VectorSpecies<Integer> PRERERRED_INT_SPECIES;
+
+ /** Whether integer vectors can be trusted to actually be fast. */
+ static final boolean HAS_FAST_INTEGER_VECTORS;
+
private final VectorUtilSupport vectorUtilSupport;
+ static {
+ // default to platform supported bitsize
+ int vectorBitSize = VectorShape.preferredShape().vectorBitSize();
Review Comment:
I hesitated moving these two constants since they are referenced in the constructor, I was worried that an apparently harmless refactoring could suddenly make PanamaVectorConstants load before PanamaVectorizationProvider.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1709749808
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene912/ForUtil.java:
##########
@@ -630,55 +637,55 @@ private static void decode1(DataInput in, long[] tmp, long[] longs) throws IOExc
shiftLongs(tmp, 2, longs, 14, 0, MASK8_1);
}
- private static void decode2(DataInput in, long[] tmp, long[] longs) throws IOException {
- in.readLongs(tmp, 0, 4);
- shiftLongs(tmp, 4, longs, 0, 6, MASK8_2);
+ private static void decode2(PostingDecodingUtil pdu, long[] tmp, long[] longs)
+ throws IOException {
+ pdu.splitLongs(4, longs, 6, MASK8_2, tmp, MASK8_6);
shiftLongs(tmp, 4, longs, 4, 4, MASK8_2);
shiftLongs(tmp, 4, longs, 8, 2, MASK8_2);
shiftLongs(tmp, 4, longs, 12, 0, MASK8_2);
}
- private static void decode3(DataInput in, long[] tmp, long[] longs) throws IOException {
- in.readLongs(tmp, 0, 6);
- shiftLongs(tmp, 6, longs, 0, 5, MASK8_3);
+ private static void decode3(PostingDecodingUtil pdu, long[] tmp, long[] longs)
+ throws IOException {
+ pdu.splitLongs(6, longs, 5, MASK8_3, tmp, MASK8_5);
shiftLongs(tmp, 6, longs, 6, 2, MASK8_3);
+ maskLongs(tmp, 6, MASK8_2);
for (int iter = 0, tmpIdx = 0, longsIdx = 12; iter < 2; ++iter, tmpIdx += 3, longsIdx += 2) {
- long l0 = (tmp[tmpIdx + 0] & MASK8_2) << 1;
+ long l0 = tmp[tmpIdx + 0] << 1;
l0 |= (tmp[tmpIdx + 1] >>> 1) & MASK8_1;
longs[longsIdx + 0] = l0;
long l1 = (tmp[tmpIdx + 1] & MASK8_1) << 2;
- l1 |= (tmp[tmpIdx + 2] & MASK8_2) << 0;
+ l1 |= tmp[tmpIdx + 2] << 0;
longs[longsIdx + 1] = l1;
}
}
- private static void decode4(DataInput in, long[] tmp, long[] longs) throws IOException {
- in.readLongs(tmp, 0, 8);
- shiftLongs(tmp, 8, longs, 0, 4, MASK8_4);
+ private static void decode4(PostingDecodingUtil pdu, long[] tmp, long[] longs)
+ throws IOException {
+ pdu.splitLongs(8, longs, 4, MASK8_4, tmp, MASK8_4);
shiftLongs(tmp, 8, longs, 8, 0, MASK8_4);
Review Comment:
Note to self: if `splitLongs` took an offset for the second array, this could be just `pdu.splitLongs(8, longs, 4, MASK8_4, longs, 8, MASK8_4);`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #13636:
URL: https://github.com/apache/lucene/pull/13636#issuecomment-2272150091
For what it's worth, this PR is quite different from https://github.com/apache/lucene/pull/12412 in that it does not rewrite `ForUtil.java` completely, only the bits where we read some longs from the directory, apply some shifts and masks and store results in 2 arrays.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1706267079
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import java.util.Optional;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ static Optional<PostingDecodingUtil> wrap(IndexInput in, MemorySegment memorySegment) {
+ if (64 % LONG_SPECIES.length() != 0) {
+ // Required to meet PostingDecodingUtil's contract that we do not write entries past index 64
+ // for any `count` in 0..64.
+ return Optional.empty();
+ }
+ return Optional.of(new MemorySegmentPostingDecodingUtil(in, memorySegment));
+ }
+
+ private static final VectorSpecies<Long> LONG_SPECIES = VectorSpecies.ofPreferred(long.class);
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ private MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int i;
+ // Note: this loop may apply to more than `count` entries due to the width of the preferred
+ // species. But doing this is faster than handling the remainder with scalar code.
+ for (i = 0;
+ i < count;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector vector =
+ LongVector.fromMemorySegment(
+ LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN);
+ vector
+ .lanewise(VectorOperators.LSHR, bShift)
+ .lanewise(VectorOperators.AND, bMask)
+ .intoArray(b, i);
+ vector.lanewise(VectorOperators.AND, cMask).intoArray(c, i);
Review Comment:
Where are safety checks to avoid this when integer vectors are not supported? For integer vectors, we always need such checks to avoid massive performance traps where things get 30x slower. Check benchmark with `-XX:UseAVX=0`/-XX:UseAVX=1 to make sure the safety works
Current check is here: https://github.com/apache/lucene/blob/main/lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java#L81-L83
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1707325377
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import java.util.Optional;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ static Optional<PostingDecodingUtil> wrap(IndexInput in, MemorySegment memorySegment) {
+ if (64 % LONG_SPECIES.length() != 0) {
+ // Required to meet PostingDecodingUtil's contract that we do not write entries past index 64
+ // for any `count` in 0..64.
+ return Optional.empty();
+ }
+ return Optional.of(new MemorySegmentPostingDecodingUtil(in, memorySegment));
+ }
+
+ private static final VectorSpecies<Long> LONG_SPECIES = VectorSpecies.ofPreferred(long.class);
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ private MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int i;
+ // Note: this loop may apply to more than `count` entries due to the width of the preferred
+ // species. But doing this is faster than handling the remainder with scalar code.
+ for (i = 0;
+ i < count;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector vector =
+ LongVector.fromMemorySegment(
+ LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN);
+ vector
+ .lanewise(VectorOperators.LSHR, bShift)
+ .lanewise(VectorOperators.AND, bMask)
+ .intoArray(b, i);
+ vector.lanewise(VectorOperators.AND, cMask).intoArray(c, i);
Review Comment:
Thanks for the pointer, I moved some of the existing logit to PamamaVectorizationProvider to be able to reuse it and I disable the explicitly vectorized impl for decoding postings when `HAS_FAST_INTEGER_VECTORS` is `false`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1707332904
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import java.util.Optional;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ static Optional<PostingDecodingUtil> wrap(IndexInput in, MemorySegment memorySegment) {
+ if (64 % LONG_SPECIES.length() != 0) {
+ // Required to meet PostingDecodingUtil's contract that we do not write entries past index 64
+ // for any `count` in 0..64.
+ return Optional.empty();
+ }
+ return Optional.of(new MemorySegmentPostingDecodingUtil(in, memorySegment));
+ }
+
+ private static final VectorSpecies<Long> LONG_SPECIES = VectorSpecies.ofPreferred(long.class);
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ private MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int i;
+ // Note: this loop may apply to more than `count` entries due to the width of the preferred
+ // species. But doing this is faster than handling the remainder with scalar code.
+ for (i = 0;
+ i < count;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector vector =
+ LongVector.fromMemorySegment(
+ LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN);
+ vector
+ .lanewise(VectorOperators.LSHR, bShift)
+ .lanewise(VectorOperators.AND, bMask)
+ .intoArray(b, i);
+ vector.lanewise(VectorOperators.AND, cMask).intoArray(c, i);
Review Comment:
I also tried to run the benchmark with AVX disabled. Decoding runs at ~25 ops/us. My understanding is that it's a a bit slower than the auto-vectorized impl because this auto-vectorized impl also takes advantage of AVX instructions. But it's not a 30x slowdown.
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import java.util.Optional;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ static Optional<PostingDecodingUtil> wrap(IndexInput in, MemorySegment memorySegment) {
+ if (64 % LONG_SPECIES.length() != 0) {
+ // Required to meet PostingDecodingUtil's contract that we do not write entries past index 64
+ // for any `count` in 0..64.
+ return Optional.empty();
+ }
+ return Optional.of(new MemorySegmentPostingDecodingUtil(in, memorySegment));
+ }
+
+ private static final VectorSpecies<Long> LONG_SPECIES = VectorSpecies.ofPreferred(long.class);
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ private MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int i;
+ // Note: this loop may apply to more than `count` entries due to the width of the preferred
+ // species. But doing this is faster than handling the remainder with scalar code.
+ for (i = 0;
+ i < count;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector vector =
+ LongVector.fromMemorySegment(
+ LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN);
+ vector
+ .lanewise(VectorOperators.LSHR, bShift)
+ .lanewise(VectorOperators.AND, bMask)
+ .intoArray(b, i);
+ vector.lanewise(VectorOperators.AND, cMask).intoArray(c, i);
Review Comment:
Thanks for the pointer, I moved some of the existing logic to PamamaVectorizationProvider to be able to reuse it and I disable the explicitly vectorized impl for decoding postings when `HAS_FAST_INTEGER_VECTORS` is `false`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1707341860
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import java.util.Optional;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ static Optional<PostingDecodingUtil> wrap(IndexInput in, MemorySegment memorySegment) {
+ if (64 % LONG_SPECIES.length() != 0) {
+ // Required to meet PostingDecodingUtil's contract that we do not write entries past index 64
+ // for any `count` in 0..64.
+ return Optional.empty();
+ }
+ return Optional.of(new MemorySegmentPostingDecodingUtil(in, memorySegment));
+ }
+
+ private static final VectorSpecies<Long> LONG_SPECIES = VectorSpecies.ofPreferred(long.class);
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ private MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int i;
+ // Note: this loop may apply to more than `count` entries due to the width of the preferred
+ // species. But doing this is faster than handling the remainder with scalar code.
Review Comment:
I played with vector masks, but it's even slower than the auto-vectorized impl on my machine (which doesn't have AVX-512 support). I tried to reduce the amount of padding instead, and improved tests to likely trigger reads past EOF if an impl would read more than allowed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1707448409
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import java.util.Optional;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ static Optional<PostingDecodingUtil> wrap(IndexInput in, MemorySegment memorySegment) {
+ if (64 % LONG_SPECIES.length() != 0) {
+ // Required to meet PostingDecodingUtil's contract that we do not write entries past index 64
+ // for any `count` in 0..64.
+ return Optional.empty();
+ }
+ return Optional.of(new MemorySegmentPostingDecodingUtil(in, memorySegment));
+ }
+
+ private static final VectorSpecies<Long> LONG_SPECIES = VectorSpecies.ofPreferred(long.class);
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ private MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int i;
+ // Note: this loop may apply to more than `count` entries due to the width of the preferred
+ // species. But doing this is faster than handling the remainder with scalar code.
+ for (i = 0;
+ i < count;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector vector =
+ LongVector.fromMemorySegment(
+ LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN);
+ vector
+ .lanewise(VectorOperators.LSHR, bShift)
+ .lanewise(VectorOperators.AND, bMask)
+ .intoArray(b, i);
+ vector.lanewise(VectorOperators.AND, cMask).intoArray(c, i);
Review Comment:
No need to disable all. Let's keep VectorUtil the only exception.
I tend to split VectorUtilSupport into two Interfaces, ByteVectorUtilSupport and Float ectorUtilSupport. Then we gave 2 factory methods and only one check per implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "gsmiller (via GitHub)" <gi...@apache.org>.
gsmiller commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1707451406
##########
lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultPostingDecodingUtil.java:
##########
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import org.apache.lucene.store.IndexInput;
+
+final class DefaultPostingDecodingUtil extends PostingDecodingUtil {
+
+ protected final IndexInput in;
+
+ public DefaultPostingDecodingUtil(IndexInput in) {
+ this.in = in;
+ }
+
+ @Override
+ public void splitLongs(int count, long[] b, int bShift, long bMask, long[] c, long cMask)
+ throws IOException {
+ assert count <= 64;
+ in.readLongs(c, 0, count);
+ // The below loop is auto-vectorized
+ for (int i = 0; i < count; ++i) {
+ b[i] = (c[i] >>> bShift) & bMask;
+ c[i] &= cMask;
Review Comment:
If I'm understanding things correctly, we only need to apply this masking for some of the BPV values but not all. For example, we don't need this masking step for bpv4 but we do need it for bpv12. Our current code is selective about when it does this but it looks like here we're going be always doing it, which creates some wasted overhead. I wonder if there's an opportunity to do this selectively? Or maybe it doesn't actually matter in practice?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on PR #13636:
URL: https://github.com/apache/lucene/pull/13636#issuecomment-2275500767
As discussed before: I'd move all static constants to a package-private class, so it is initialized a bit delayed and not in class loading of VectorizationProvider.
To help with @rmuir's fast integer vectors issues: Once this is merged, I plan to split the VectorUtilSupport into ByteVectorUtilSupport and FloatVectorUtilSupport and use 2 getters in VectorizationProvider. By that all the if/else logic is moved to the VectorizationProvider class and we don't need if/else logic in the implementation. Like in Adrien's actual code, the getter for the ByteVectorUtilSupport instance just returns the default one when integer vectors are slow. The checks for this could then all also be outside of static initializers, as they are no longer on hot paths.
This can be done separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1714276855
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java:
##########
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.internal.vectorization;
+
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
+import java.nio.ByteOrder;
+import jdk.incubator.vector.LongVector;
+import jdk.incubator.vector.VectorOperators;
+import jdk.incubator.vector.VectorSpecies;
+import org.apache.lucene.store.IndexInput;
+
+final class MemorySegmentPostingDecodingUtil extends PostingDecodingUtil {
+
+ private static final VectorSpecies<Long> LONG_SPECIES =
+ PanamaVectorConstants.PRERERRED_LONG_SPECIES;
+
+ private final IndexInput in;
+ private final MemorySegment memorySegment;
+
+ MemorySegmentPostingDecodingUtil(IndexInput in, MemorySegment memorySegment) {
+ this.in = in;
+ this.memorySegment = memorySegment;
+ }
+
+ @Override
+ public void readLongs(int count, long[] b) throws IOException {
+ if (count < LONG_SPECIES.length()) {
+ in.readLongs(b, 0, count);
+ } else {
+ long offset = in.getFilePointer();
+ long endOffset = offset + count * Long.BYTES;
+ int loopBound = LONG_SPECIES.loopBound(count - 1);
+ for (int i = 0;
+ i < loopBound;
+ i += LONG_SPECIES.length(), offset += LONG_SPECIES.length() * Long.BYTES) {
+ LongVector.fromMemorySegment(LONG_SPECIES, memorySegment, offset, ByteOrder.LITTLE_ENDIAN)
Review Comment:
It was when reading 16 longs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #13636:
URL: https://github.com/apache/lucene/pull/13636#discussion_r1709358369
##########
lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java:
##########
@@ -16,20 +16,56 @@
*/
package org.apache.lucene.internal.vectorization;
+import java.io.IOException;
+import java.lang.foreign.MemorySegment;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.util.Locale;
import java.util.logging.Logger;
import jdk.incubator.vector.FloatVector;
+import jdk.incubator.vector.VectorShape;
+import jdk.incubator.vector.VectorSpecies;
import org.apache.lucene.codecs.hnsw.FlatVectorsScorer;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.MemorySegmentAccessInput;
import org.apache.lucene.util.Constants;
import org.apache.lucene.util.SuppressForbidden;
/** A vectorization provider that leverages the Panama Vector API. */
final class PanamaVectorizationProvider extends VectorizationProvider {
+ /** Preferred with in bits for vectors. */
+ static final int PREFERRED_VECTOR_BITSIZE;
+
+ static final VectorSpecies<Long> PRERERRED_LONG_SPECIES;
+ static final VectorSpecies<Integer> PRERERRED_INT_SPECIES;
+
+ /** Whether integer vectors can be trusted to actually be fast. */
+ static final boolean HAS_FAST_INTEGER_VECTORS;
+
private final VectorUtilSupport vectorUtilSupport;
+ static {
+ // default to platform supported bitsize
+ int vectorBitSize = VectorShape.preferredShape().vectorBitSize();
Review Comment:
See my other comment. Let's put the all the constants (move all of them in the provider) into a separate class. Still static, but it won't be initialized too early.
The constants need to be static for hotspot to treat them as final.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org