You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by Ganesh <ga...@gmail.com> on 2025/02/23 05:38:08 UTC

Integrating LLMs into apache flink pipelines

Hi Everyone, I've been experimenting with integrating LLMs into ETL and
data pipelines to leverage the models for data processing.

And I've created a blog post with a example pipeline to integrate openai
models using langchian-beam library's transforms and load data and perform
sentiment analysis in apache flink pipeline runner

Check it out and share your thoughts.

Post -
https://medium.com/@ganxesh/integrating-llms-into-apache-flink-pipelines-8fb433743761

Langchian-Beam - https://github.com/Ganeshsivakumar/langchain-beam

Thanks,
Ganesh.

Re: Integrating LLMs into apache flink pipelines

Posted by Shengkai Fang <fs...@gmail.com>.

Thanks for your sharing!

Could you confirm whether the pipeline is capable of generating structured
summaries for product-specific comments or customer feedback?

Best,
Shengkai

Ganesh <ga...@gmail.com> 于2025年2月23日周日 13:38写道：

> Hi Everyone, I've been experimenting with integrating LLMs into ETL and
> data pipelines to leverage the models for data processing.
>
> And I've created a blog post with a example pipeline to integrate openai
> models using langchian-beam library's transforms and load data and perform
> sentiment analysis in apache flink pipeline runner
>
> Check it out and share your thoughts.
>
> Post -
>
> https://medium.com/@ganxesh/integrating-llms-into-apache-flink-pipelines-8fb433743761
>
> Langchian-Beam - https://github.com/Ganeshsivakumar/langchain-beam
>
> Thanks,
> Ganesh.
>