You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Anand Inguva via dev <de...@beam.apache.org> on 2023/10/30 14:00:00 UTC

Embeddings generation in MLTransform

Hi all,

In Apache Beam 2.50.0 Python SDK, we added MLTransform
<https://github.com/apache/beam/blob/c2816c8d9770df3d30eec15a2563c48cb66a66b3/website/www/site/content/en/blog/beam-2.50.0.md?plain=1#L53>,
which is used to pre/post process data using common ML operations. Now, we
are planning to generate embeddings
<https://en.wikipedia.org/wiki/Embedding> with ML models using MLTransform.

I have created a doc
<https://docs.google.com/document/d/1En4bfbTu4rvu7LWJIKV3G33jO-xJfTdbaSFSURmQw_s/edit#heading=h.wskna8eurvjv>
on how we can do this. Please go through the doc if interested and let me
know of any feedback.

Thanks,
Anand

Doc:
https://docs.google.com/document/d/1En4bfbTu4rvu7LWJIKV3G33jO-xJfTdbaSFSURmQw_s/edit#heading=h.wskna8eurvjv

Re: Embeddings generation in MLTransform

Posted by Anand Inguva via dev <de...@beam.apache.org>.
Hi all,

After the initial email, I went ahead and added a few more things as per
the comments on the doc
<https://docs.google.com/document/d/1En4bfbTu4rvu7LWJIKV3G33jO-xJfTdbaSFSURmQw_s/edit?usp=gmail#heading=h.wskna8eurvjv>.
Please take a look and let me know what you think.

Thanks,
Anand

On Mon, Oct 30, 2023 at 10:00 AM Anand Inguva <an...@google.com>
wrote:

> Hi all,
>
> In Apache Beam 2.50.0 Python SDK, we added MLTransform
> <https://github.com/apache/beam/blob/c2816c8d9770df3d30eec15a2563c48cb66a66b3/website/www/site/content/en/blog/beam-2.50.0.md?plain=1#L53>,
> which is used to pre/post process data using common ML operations. Now, we
> are planning to generate embeddings
> <https://en.wikipedia.org/wiki/Embedding> with ML models using
> MLTransform.
>
> I have created a doc
> <https://docs.google.com/document/d/1En4bfbTu4rvu7LWJIKV3G33jO-xJfTdbaSFSURmQw_s/edit#heading=h.wskna8eurvjv>
> on how we can do this. Please go through the doc if interested and let me
> know of any feedback.
>
> Thanks,
> Anand
>
> Doc:
> https://docs.google.com/document/d/1En4bfbTu4rvu7LWJIKV3G33jO-xJfTdbaSFSURmQw_s/edit#heading=h.wskna8eurvjv
>

Re: Embeddings generation in MLTransform

Posted by Alexey Romanenko <ar...@gmail.com>.
You need to send a blank email to dev-unsubscribe@beam.apache.org <ma...@beam.apache.org>

—
Alexey


> On 5 Dec 2023, at 11:57, Divya Sanghi <bi...@gmail.com> wrote:
> 
> Can someone suggest how to unsubscribe?
> 
> On Mon, Oct 30, 2023 at 7:33 PM Anand Inguva via dev <dev@beam.apache.org <ma...@beam.apache.org>> wrote:
>> Hi all,
>> 
>> In Apache Beam 2.50.0 Python SDK, we added MLTransform <https://github.com/apache/beam/blob/c2816c8d9770df3d30eec15a2563c48cb66a66b3/website/www/site/content/en/blog/beam-2.50.0.md?plain=1#L53>, which is used to pre/post process data using common ML operations. Now, we are planning to generate embeddings <https://en.wikipedia.org/wiki/Embedding> with ML models using MLTransform. 
>> 
>> I have created a doc <https://docs.google.com/document/d/1En4bfbTu4rvu7LWJIKV3G33jO-xJfTdbaSFSURmQw_s/edit#heading=h.wskna8eurvjv> on how we can do this. Please go through the doc if interested and let me know of any feedback. 
>> 
>> Thanks,
>> Anand
>> 
>> Doc: https://docs.google.com/document/d/1En4bfbTu4rvu7LWJIKV3G33jO-xJfTdbaSFSURmQw_s/edit#heading=h.wskna8eurvjv

Re: Embeddings generation in MLTransform

Posted by Divya Sanghi <bi...@gmail.com>.
Can someone suggest how to unsubscribe?

On Mon, Oct 30, 2023 at 7:33 PM Anand Inguva via dev <de...@beam.apache.org>
wrote:

> Hi all,
>
> In Apache Beam 2.50.0 Python SDK, we added MLTransform
> <https://github.com/apache/beam/blob/c2816c8d9770df3d30eec15a2563c48cb66a66b3/website/www/site/content/en/blog/beam-2.50.0.md?plain=1#L53>,
> which is used to pre/post process data using common ML operations. Now, we
> are planning to generate embeddings
> <https://en.wikipedia.org/wiki/Embedding> with ML models using
> MLTransform.
>
> I have created a doc
> <https://docs.google.com/document/d/1En4bfbTu4rvu7LWJIKV3G33jO-xJfTdbaSFSURmQw_s/edit#heading=h.wskna8eurvjv>
> on how we can do this. Please go through the doc if interested and let me
> know of any feedback.
>
> Thanks,
> Anand
>
> Doc:
> https://docs.google.com/document/d/1En4bfbTu4rvu7LWJIKV3G33jO-xJfTdbaSFSURmQw_s/edit#heading=h.wskna8eurvjv
>