You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Pranshu-S (via GitHub)" <gi...@apache.org> on 2026/10/01 10:59:19 UTC

Re: [PR] Add scalar quantization support for float16 vectors in the dedup KNN Codec [lucene]

Pranshu-S commented on code in PR #16710:
URL: https://github.com/apache/lucene/pull/16710#discussion_r4154626977


##########
lucene/sandbox/src/java/org/apache/lucene/sandbox/codecs/dedup/DedupScalarQuantizedVectorValues.java:
##########
@@ -328,4 +383,88 @@ public VectorScorer rescorer(float[] target) throws IOException {
       return rawValues.rescorer(target);
     }
   }
+
+  /**
+   * FLOAT16 analogue of {@link RawAndQuantizedValues}: exposes a field's raw de-duplicated {@code
+   * short[]} vectors for full-fidelity readback, while {@link #scorer(short[])} scores against the
+   * shared quantized view (the {@code short[]} target is inflated to {@code float[]} to match the
+   * data-blind quantizer). Mirrors the FLOAT16 handling in {@code
+   * Lucene104ScalarQuantizedVectorsReader}.
+   */
+  static final class Float16RawAndQuantizedValues extends Float16VectorValues
+      implements DedupVectorValues {
+    private final DedupVectorValues.Float16Impl rawValues;
+    private final FieldValues quantizedValues;
+
+    Float16RawAndQuantizedValues(
+        DedupVectorValues.Float16Impl rawValues, FieldValues quantizedValues) {
+      this.rawValues = rawValues;
+      this.quantizedValues = quantizedValues;
+    }
+
+    FieldValues getQuantizedValues() {
+      return quantizedValues;
+    }
+
+    @Override
+    public KnnVectorValues getGroupView() {
+      return rawValues.getGroupView();
+    }
+
+    @Override
+    public FieldOrdToGroupOrd getFieldOrdToGroupOrd() {
+      return rawValues.getFieldOrdToGroupOrd();
+    }
+
+    @Override
+    public int dimension() {
+      return rawValues.dimension();
+    }
+
+    @Override
+    public int size() {
+      return rawValues.size();
+    }
+
+    @Override
+    public int ordToDoc(int ord) {
+      return rawValues.ordToDoc(ord);
+    }
+
+    @Override
+    public Bits getAcceptOrds(Bits acceptDocs) {
+      return rawValues.getAcceptOrds(acceptDocs);
+    }
+
+    @Override
+    public void prefetch(int[] ordsToPrefetch, int numOrds) throws IOException {
+      rawValues.prefetch(ordsToPrefetch, numOrds);
+    }
+
+    @Override
+    public short[] vectorValue(int ord) throws IOException {
+      return rawValues.vectorValue(ord);
+    }
+
+    @Override
+    public DocIndexIterator iterator() {
+      return rawValues.iterator();
+    }
+
+    @Override
+    public Float16RawAndQuantizedValues copy() throws IOException {
+      return new Float16RawAndQuantizedValues(rawValues.copy(), quantizedValues.copy());
+    }
+
+    @Override
+    public VectorScorer scorer(short[] target) throws IOException {
+      float[] inflated = DedupUtil.inflateFloat16(target, new float[target.length]);

Review Comment:
   I took a second look, you're right that the manual inflation in Float16RawAndQuantizedValues was redundant.  Doing the inflation again upstream just duplicated that logic and would have silently diverged from the fast path once #16533 lands.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org