[jira] (CAMEL-21587) Langchain4j : add similarity search

["Zineb Bendhiba (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CAMEL-21587 ]


    Zineb Bendhiba deleted comment on CAMEL-21587:
    ----------------------------------------

was (Author: zbendhiba):
This is  one is a not finished yet. I introduced similarity search in Qdrant.

I'm working on the decoding for 4.10, but I still need to improve this. Moving this Jira to next release

> Langchain4j : add similarity search
> -----------------------------------
>
>                 Key: CAMEL-21587
>                 URL: https://issues.apache.org/jira/browse/CAMEL-21587
>             Project: Camel
>          Issue Type: New Feature
>            Reporter: Zineb Bendhiba
>            Assignee: Zineb Bendhiba
>            Priority: Major
>
> +*Problem:*+
> Apache Camel Langchain4j and Vector databases (Qdrant/Pinecone) can't perform similarity search for RAG. This limitation hinders our ability to use Apache Camel with Vector databases with Langchain4j for robust search capabilities.
> While creating an example of RAG using Apache Camel Langchain4j and Apache Camel Vector databases, I realized that the Vector databases components cannot handle similarity search for the Langchain4j scenarios. 
> Despite trying both Qdrant and Pinecone, I encountered issues when attempting to perform RAG (Relevance-based Active Learning) with these databases.
> +*Research*+
> I've analyzed various existing issues related to Search with Qdrant and Pinecone. For example, the Camel Qdrant component doesn't provide similarity search capabilities. In contrast, Pinecone allows for searching based on similarity.
> However, when trying to use Pinecone with Apache Camel, I encountered a challenge: we need to convert text to Embeddings using Camel Langchain4j Embeddings Component. The resulting list of Embeddings cannot be used directly with RAG.
> +*Proposed Solution*+
> To address this limitation, I propose implementing a new feature (or new camel AI component) that enables easy similarity search for RAG using Apache Camel and Vector databases. This could involve leveraging the abstraction from Embedding stores in Langchain4j to provide an easy way to perform similarity searches.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)