[I] [IEP] Enhance TextQuery for Hybrid Search: Integrate Keyword and Vector Retrieval using Lucene9+ [ignite]

["junphine (via GitHub)" <gi...@apache.org>]

junphine opened a new issue, #13526:
URL: https://github.com/apache/ignite/issues/13526

   ### Proposal
   Extend the existing `TextQuery` API to support hybrid search, combining Lucene-based keyword matching with vector-based Approximate Nearest Neighbor (ANN) search. This aligns with the vision of IEP-141 to enable "hybrid queries, where filtering by metadata (structured data) and ranking by vector happen in a single execution layer" [citation:2][citation:3].
   
   This approach reuses the familiar `TextQuery` interface, making it intuitive for existing users and avoiding API fragmentation.
   
   ### Motivation & Design Goal
   - **Unified Query Experience**: Users can perform a search that considers both exact keyword relevance and semantic similarity in one query.
   - **Leverage Lucene**: Reuse the existing `GridLuceneIndex` infrastructure to manage both inverted indexes (for text) and vector indexes (e.g., Lucene's `KnnFloatVectorQuery`) [citation:11].
   - **Single Execution Layer**: Perform filtering by structured data and ranking by vector similarity within the same query engine, optimizing complex AI workloads like RAG and semantic search [citation:2][citation:3].
   
   ### Proposed Interface: Enhanced TextQuery
   
   The `TextQuery` class will be enhanced with new builder-style methods to accept a vector query component.
   
   ```java
   package org.apache.ignite.cache.query;
   
   /**
    * A hybrid query that performs both text and vector search.
    * By default, it behaves as a standard TextQuery. When a vector is provided,
    * it performs a hybrid search combining Lucene's keyword and KNN queries.
    */
   public final class TextQuery<K, V> extends Query<Cache.Entry<K, V>> {
       /**
        * Standard constructor for pure text search.
        */
       public TextQuery(Class<V> type, String txt) { ... }
   
       /**
        * NEW: Sets the vector field and query vector for hybrid search.
        *
        * @param vectorFieldName Name of the vector field (annotated with @QueryVectorField).
        * @param vector The query vector (float array).
        * @param k Number of nearest neighbors to return from the vector portion.
        * @return this TextQuery instance for chaining.
        */
       public TextQuery<K, V> setVectorQuery(String vectorFieldName, float[] vector, int k) { ... }
   
       /**
        * NEW: Sets the distance metric for the vector part (defaults to COSINE).
        */
       public TextQuery<K, V> setDistanceMetric(DistanceMetric metric) { ... }
   
       /**
        * NEW: Sets the hybrid ranking strategy.
        * - RRF: Reciprocal Rank Fusion (default).
        * - WEIGHTED_SUM: Weighted sum of text and vector scores.
        */
       public TextQuery<K, V> setHybridStrategy(HybridStrategy strategy) { ... }
   
       /**
        * NEW: Sets the weight for the vector portion in a WEIGHTED_SUM strategy.
        */
       public TextQuery<K, V> setVectorWeight(float weight) { ... }
   
       // Existing methods: setPageSize, setLocal, etc. remain unchanged [citation:1][citation:4][citation:5].
   }
   ```
   
   ### Proposed Annotation: @QueryVectorField
   Same as previously proposed. This annotation marks a field for vector indexing, specifying dimension, data type (FP32, INT8, etc.), and the distance metric .
   ```
   java
   package org.apache.ignite.cache.query.annotations;
   
   public @interface QueryVectorField {
       int dimension();
       VectorDataType dataType() default VectorDataType.FP32;
       DistanceMetric metric() default DistanceMetric.COSINE;
   }
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org