You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "rajat315315 (via GitHub)" <gi...@apache.org> on 2026/08/01 06:52:03 UTC

[I] [ENH]: Optimize DocValues Bulk Decoding using Panama Vector API [lucene]

rajat315315 opened a new issue, #16451:
URL: https://github.com/apache/lucene/issues/16451

   ### Description
   
   ## Motivation
   
   Lucene heavily relies on DocValues for sorting, faceting, and range filtering. For byte-aligned packed formats (8-bit, 16-bit, and 32-bit), the bulk decoding process (`DocValuesBulkDecodeSupport`) decompresses compressed byte arrays back into native Java `long[]` value arrays. 
   
   Currently, the default implementation (`DefaultDocValuesBulkDecodeSupport`) performs this decoding using scalar loops. While modern JVM HotSpot JIT (C2 compiler) tries to auto-vectorize loops, it frequently fails or generates suboptimal code for:
   1. **Widening Type Casts**: Widening 8-bit bytes, 16-bit shorts, or 32-bit integers to 64-bit `long` lanes.
   2. **Strided Memory Access**: Fetching compressed data using strided offset arithmetic.
   
   By leveraging the JDK Panama Vector API (`jdk.incubator.vector`), we can explicitly program vectorized instructions (using AVX2 or AVX-512 vector shapes depending on hardware support) to perform bulk decoding and widening operations in parallel vector lanes.
   
   ---
   
   ## Proposed Implementation Design
   
   The proposed implementation updates `PanamaDocValuesBulkDecodeSupport` to support hardware-accelerated vectorization for byte-aligned bit widths (8, 16, 32) using `LongVector` and statically imported zero-extending operators:
   
   *   **8-Bit Packed Decoding (`decode8`)**: Loads a `ByteVector` and uses `.convertShape(ZERO_EXTEND_B2L, LongVector.SPECIES_PREFERRED, 0)` to widen byte lanes into long lanes in parallel.
   *   **16-Bit Packed Decoding (`decode16`)**: Loads a `ByteVector`, reinterprets it as a `ShortVector` (`.reinterpretAsShorts()`), and widens it using `ZERO_EXTEND_S2L`.
   *   **32-Bit Packed Decoding (`decode32`)**: Loads a `ByteVector`, reinterprets it as an `IntVector` (`.reinterpretAsInts()`), and widens it using `ZERO_EXTEND_I2L`.
   *   **Tail Handling**: Safely bounds-checks the vector loop to process the maximum possible vector lanes without reading past array boundaries, falling back to the scalar decoder for the tail elements.
   *   **Fallback**: Standard fallback to the scalar implementation in non-little-endian environments or when the platform's vector capabilities are insufficient (e.g., preferred species width is less than 32 bytes).
   
   ---
   
   ## Benchmark Results (Wikimedia 1M Dataset)
   
   We evaluated performance on a 1M document Wikimedia dataset using `luceneutil`, executing 5 JVM iterations with 20 warmups per query. The benchmark compared the clean standard `lucene_baseline` (`main` branch) against our candidate branch (`perf/simd-docvalues-decode`).
   
   Here are the averaged throughput (QPS) comparison results for the top-performing tasks:
   
   | Task / Query | Baseline QPS | Panama Candidate QPS | Throughput Change (Pct Diff) | Statistical Significance (p-value) |
   | :--- | :---: | :---: | :---: | :---: |
   | **Respell** (Fuzzy/Edit Distance) | 14.85 | 18.43 | **+24.1%** | 0.027 |
   | **LowPhrase** | 427.70 | 495.32 | **+15.8%** | 0.103 |
   | **Fuzzy1** | 34.47 | 39.51 | **+14.6%** | 0.150 |
   | **IntNRQ** (Numeric Range Query) | 208.01 | 232.59 | **+11.8%** | 0.186 |
   | **HighTermMonthSort** (Sorting) | 270.39 | 301.12 | **+11.4%** | 0.446 |
   | **MedIntervalsOrdered** | 37.82 | 40.30 | **+6.6%** | 0.473 |
   | **Prefix3** | 174.93 | 185.70 | **+6.2%** | 0.375 |
   | **HighPhrase** | 22.86 | 24.08 | **+5.3%** | 0.549 |
   | **LowIntervalsOrdered** | 62.04 | 64.47 | **+3.9%** | 0.612 |
   | **HighTermDayOfYearSort** (Sorting) | 100.37 | 104.13 | **+3.7%** | 0.712 |
   | **Fuzzy2** | 27.03 | 27.91 | **+3.2%** | 0.731 |
   
   ### Summary of Performance Impact
   *   **Fuzzy Terms & Edit Distances**: Operations like `Respell` show a statistically significant **+24.1% QPS speedup** due to heavy DocValues decompression cycles.
   *   **Sorting & Range Queries**: Sorting on compressed fields (`HighTermMonthSort` at **+11.4%**) and range filtering (`IntNRQ` at **+11.8%**) achieve substantial throughput improvements from parallel decompression.
   
   ---
   
   ## Current Status
   
   An active PR/branch `perf/simd-docvalues-decode` is currently working on this proposal. Correctness has been verified with a robust unit test suite (`TestDocValuesBulkDecodeSupport.java`) comparing default scalar outputs against Panama vector outputs across randomized bit widths, byte offsets, and alignment configurations.
   
   ---
   
   ## Code & Next Steps
   
   I have the draft code working in a local branch `perf/simd-docvalues-decode`. Before opening a pull request, I wanted to check if this approach looks reasonable to everyone.
   
   Specifically, I'd love feedback on:
   *   Whether the vector widening casts (`ZERO_EXTEND_B2L`, etc.) are structured cleanly according to our current guidelines.
   *   Any additional edge cases or platforms we should be testing.
   
   Uwe (@uschindler), Chris (@chegar999), Mike (@mikemccand), Adrien (@jpountz), Robert (@rmuir) — since you've worked on vector utility and decoding optimizations, I'd appreciate your thoughts on this!
   
   Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [I] [ENH]: Optimize DocValues Bulk Decoding using Panama Vector API [lucene]

Posted by "CH-Abhinav (via GitHub)" <gi...@apache.org>.
CH-Abhinav commented on issue #16451:
URL: https://github.com/apache/lucene/issues/16451#issuecomment-5308599411

   @rmuir @uschindler  well this issue's author used a fare bit of AI without understanding what he was doing.  Like his test queries and benchmarks do not touch DocValues.  But the Idea of using Panama's `ZERO_EXTEND` isn't bad at all. So I want to give it a shot and prepare a benchmark and a PR regarding this issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [I] [ENH]: Optimize DocValues Bulk Decoding using Panama Vector API [lucene]

Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on issue #16451:
URL: https://github.com/apache/lucene/issues/16451#issuecomment-5151143528

   this is all hallucinated nonsense.
   
   > Fuzzy Terms & Edit Distances: Operations like Respell show a statistically significant +24.1% QPS speedup due to heavy DocValues decompression cycles.
   
   put the AI away


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org