Re: Inquiry About GSoC Project - Beam ML Vector DB/Feature Store Integrations

[Danny McCormick via dev <de...@beam.apache.org>]

Yeah, that would be best - thanks!

On Sun, Apr 6, 2025 at 6:34 AM Aditya <ad...@gmail.com> wrote:

> *Subject:* Question Regarding OpenAI Embedding Implementation
>
> Hi Danny,
>
> I have a quick question:
>
> Should the OpenAI embedding implementation handle Chunk objects from the
> RAG framework similarly to how the VertexAI implementation does?
>
> Thanks,
>
> Aditya
>
>>

Re: Inquiry About GSoC Project - Beam ML Vector DB/Feature Store Integrations

[Aditya <ad...@gmail.com>]

**Subject:** Help with OpenAI Embedding Handler Output Type Hint Warning

Hi Danny,

I’m running into a warning related to the `OpenAITextEmbeddings` handler in
Apache Beam and could use your guidance.

Here’s the warning I’m seeing:

```

WARNING:root:This output type hint will be ignored and not used for
type-checking purposes. Typically, output type hints for a PTransform are
single (or nested) types wrapped by a PCollection, PDone, or None. Got:
Union[Tuple[PCollection[~MLTransformOutputT], PCollection[Row]],
PCollection[~MLTransformOutputT]] instead.

```

This shows up multiple times and seems related to how the output type is
declared in the transform.

I’m wondering if this is expected or if the output signature should be
updated to prevent the warning.

Here’s the link to the code I’m working with:

openai
<https://github.com/aditya0yadav/beam-1/blob/master/sdks/python/apache_beam/ml/transforms/embeddings/open_ai.py>


👉
https://github.com/aditya0yadav/beam-1/blob/master/sdks/python/apache_beam/ml/transforms/embeddings/open_ai.py

Would appreciate your thoughts when you get a chance.

Best,

Aditya

>

Re: Inquiry About GSoC Project - Beam ML Vector DB/Feature Store Integrations

[Aditya <ad...@gmail.com>]

https://github.com/apache/beam/pull/34700

On Sat, 26 Apr, 2025, 04:57 Aditya, <ad...@gmail.com> wrote:

> Hi Danny,
>
> I’ve completed the implementation of the OpenAI text embedding handler and
> have opened a PR for it.
> Could you please take a look when you get a chance?
>
> Thanks,
> Aditya
>
>>

Re: Inquiry About GSoC Project - Beam ML Vector DB/Feature Store Integrations

[Aditya <ad...@gmail.com>]

Hi Danny,

I’ve completed the implementation of the OpenAI text embedding handler and
have opened a PR for it.
Could you please take a look when you get a chance?

Thanks,
Aditya

>

Re: Inquiry About GSoC Project - Beam ML Vector DB/Feature Store Integrations

[Aditya <ad...@gmail.com>]

Thanks for the clarification.

On Tue, Apr 8, 2025 at 12:41 AM Danny McCormick <da...@google.com>
wrote:

> Yeah, that would be best - thanks!
>
> On Sun, Apr 6, 2025 at 6:34 AM Aditya <ad...@gmail.com> wrote:
>
>> *Subject:* Question Regarding OpenAI Embedding Implementation
>>
>> Hi Danny,
>>
>> I have a quick question:
>>
>> Should the OpenAI embedding implementation handle Chunk objects from the
>> RAG framework similarly to how the VertexAI implementation does?
>>
>> Thanks,
>>
>> Aditya
>>
>>>