You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@paimon.apache.org by "adrian-wang (via GitHub)" <gi...@apache.org> on 2026/10/13 11:19:24 UTC

[PR] fix: skip NaN scores in IVF top-k heaps [paimon-vector-index]

adrian-wang opened a new pull request, #116:
URL: https://github.com/apache/paimon-vector-index/pull/116

   Reject both signs of NaN in FlatTopKHeap, ReaderTopKHeap, IVF-PQ TopKHeap, and the unified TopKHeap before insertion or duplicate updates. Use total_cmp for consistent heap comparisons and sorting, and reject NaN candidates in should_consider.
   
   Add regression coverage for both NaN signs, heap capacity, duplicate IDs, and IVF-FLAT in-memory and reader inner-product searches, including padding when valid neighbors are fewer than k.
   
   Validation: Rust workspace tests, formatting, and Clippy all pass.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix: skip NaN scores in IVF top-k heaps [paimon-vector-index]

Posted by "github-actions[bot] (via GitHub) via issues" <is...@paimon.apache.org>.

github-actions[bot] commented on PR #116:
URL: https://github.com/apache/paimon-vector-index/pull/116#issuecomment-6678790672

   <!-- paimon-pr-benchmark -->
   [Benchmark run](https://github.com/apache/paimon-vector-index/actions/runs/42174558607) · success · attempt 2
   
   ## Vector index benchmark
   
   **🟢 No alerts (5/5 indexes)**
   
   | Index | Recall@10 (Δ) | Single QPS Δ | Batch QPS Δ | Status |
   |---|---:|---:|---:|---|
   | IVF_FLAT | 100.00% (+0.00 pp) | +0.8% | +4.5% | 🟢 OK |
   | IVF_SQ | 99.26% (+0.00 pp) | -0.9% | -1.5% | 🟢 OK |
   | IVF_PQ | 46.60% (+0.00 pp) | -0.2% | -1.7% | 🟢 OK |
   | IVF_RQ | 90.91% (+0.00 pp) | -5.2% | -9.4% | 🟢 OK |
   | DISKANN | 77.19% (+0.00 pp) | -1.0% | -2.0% | 🟢 OK |
   
   🟡 QPS ↓ >10% or Recall ↓ >1pp · 🔴 QPS ↓ >20% or Recall ↓ >3pp. Advisory only; QPS is measured after warmup.
   
   <details><summary>Absolute metrics & environment</summary>
   
   Medians, base → PR. QPS is timed after warmup; P95 uses the first warm sweep.
   
   | Index | Single QPS | Batch QPS | P95 (µs) | Build (ms) | Process RSS (MiB) | Size (MiB) |
   |---|---:|---:|---:|---:|---:|---:|
   | IVF_FLAT | 11,408.40 → 11,504.56 | 62,620.15 → 65,459.65 | 107.38 → 108.30 | 26.00 → 26.50 | 19.79 → 19.79 | 2.47 → 2.47 |
   | IVF_SQ | 63,026.79 → 62,460.74 | 115,809.17 → 114,116.04 | 17.98 → 18.20 | 19.00 → 19.00 | 19.79 → 19.79 | 0.67 → 0.67 |
   | IVF_PQ | 11,072.29 → 11,045.81 | 79,533.47 → 78,189.18 | 109.67 → 109.54 | 189.00 → 190.00 | 19.79 → 19.79 | 0.17 → 0.17 |
   | IVF_RQ | 9,206.76 → 8,723.86 | 34,695.93 → 31,435.31 | 132.44 → 140.01 | 33.50 → 33.50 | 19.79 → 19.79 | 0.52 → 0.52 |
   | DISKANN | 7,024.81 → 6,957.21 | 14,884.19 → 14,586.43 | 168.68 → 168.82 | 1,046.50 → 1,071.50 | 19.83 → 19.88 | 3.22 → 3.22 |
   
   Base `f80e5568dc476f7e0bd56c73c88a0b46bf8018ca` · PR merge `fa10e788031f838b5efb923d33fb4ed7e70a3272`
   rustc 1.99.1 (58cf8f9c3 2026-10-13) · Linux-7.0.0-1017-azure-x86_64-with-glibc2.39 · AMD EPYC 7763 64-Core Processor
   6 samples/version/index, alternating execution order; 2 threads; 10k × 64D vectors; 2,048 queries; ≥1s per timed mode.
   RSS is the process lifetime peak up to build completion, including dataset/ground truth.
   Raw CSVs, sample ranges, workload settings and logs are in the workflow artifact.
   
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix: skip NaN scores in IVF top-k heaps [paimon-vector-index]

Posted by "JingsongLi (via GitHub) via issues" <is...@paimon.apache.org>.

JingsongLi merged PR #116:
URL: https://github.com/apache/paimon-vector-index/pull/116


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org