[jira] [Updated] (OPENNLP-1895) Quantized vector storage and search for static embeddings

["Kristian Rickert (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/OPENNLP-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kristian Rickert updated OPENNLP-1895:
--------------------------------------
    Summary:  Quantized vector storage and search for static embeddings  (was: Quantized static embedding tables (2-4 bits per dimension))

>  Quantized vector storage and search for static embeddings
> ----------------------------------------------------------
>
>                 Key: OPENNLP-1895
>                 URL: https://issues.apache.org/jira/browse/OPENNLP-1895
>             Project: OpenNLP
>          Issue Type: New Feature
>    Affects Versions: 3.0.0-M4
>            Reporter: Kristian Rickert
>            Assignee: Kristian Rickert
>            Priority: Major
>
> Follow-up to OPENNLP-1877 (opennlp-embeddings, static embedding engine).
>  
> Static embedding tables are large: a 500k-row, 300-dim float32 table is roughly 600 MB. Modern scalar quantization brings that to 75 MB at 4 bits per dimension or 38 MB at 2 bits, with near-optimal distortion, which puts full-vocabulary tables in range of memory-constrained deployments and speeds up embedding throughput (the workload is memory-bandwidth-bound, so reading 4-16x fewer bytes is a direct win).
>  
> This ticket adds quantized table storage to opennlp-embeddings using the TurboQuant construction ([arXiv:2504.19874|https://arxiv.org/abs/2504.19874], Zandieh, Daliri, Hadian, Mirrokni, "TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate"): a seeded randomized rotation to make coordinates near-independent with a known distribution, followed by per-coordinate optimal scalar quantizers, with an optional 1-bit quantized-JL residual for unbiased inner product estimation.
>  
> h3. Scope
>  
> * A quantized table implementation behind the same lookup seam as the existing float table, so components are unaware of the storage form.
> * A converter (CLI) that quantizes an existing float table offline.
> * Pure JVM, implemented from the paper (cited in javadoc); no ports of any third-party implementation, no new dependencies. Hot loops use the \{{jdk.incubator.vector}} patterns already accepted in the module.
> * Out of scope: any approximate nearest-neighbor index. This is table storage and lookup only; indexing belongs to search engines.
>  
> h3. Acceptance measurements (will be in the PR)
>  
> * Round-trip distortion and downstream parity: cosine similarity of decoded vs original vectors, and doccat accuracy delta on a fixed fixture, at 2, 3, and 4 bits.
> * Throughput and memory vs the float table.
> * A cross-check against a reference Rust implementation of the same algorithm on identical tables, so the JVM cost is stated honestly.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)