You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Govindarajan Srinivasaraghavan <go...@gmail.com> on 2016/12/19 04:35:01 UTC
Stream Iterations
Hi All,
I have a use case for which I need some suggestions. It's a streaming
application with kafka source and then groupBy, keyBy and perform some
calculations. The output of each calculation has to be a side input for the
next calculation and also it needs to be sent to a sink.
Right now I'm achieving this by storing the result state in memory and also
save it in redis cache. I was looking at delta iterations in flink
documentation. It would great if someone can help me understand if I can
achieve this using iterations or any other api. Thanks in advance.
Regards,
Govind
Re: Stream Iterations
Posted by Ufuk Celebi <uc...@apache.org>.
The delta iterations are a batch-only feature.
Did you try the DataStream#iterate?
https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/streaming/index.html#iterations
On Mon, Dec 19, 2016 at 5:35 AM, Govindarajan Srinivasaraghavan
<go...@gmail.com> wrote:
> Hi All,
>
> I have a use case for which I need some suggestions. It's a streaming
> application with kafka source and then groupBy, keyBy and perform some
> calculations. The output of each calculation has to be a side input for the
> next calculation and also it needs to be sent to a sink.
>
> Right now I'm achieving this by storing the result state in memory and also
> save it in redis cache. I was looking at delta iterations in flink
> documentation. It would great if someone can help me understand if I can
> achieve this using iterations or any other api. Thanks in advance.
>
> Regards,
> Govind
Re: Stream Iterations
Posted by Ufuk Celebi <uc...@apache.org>.
The delta iterations are a batch-only feature.
Did you try the DataStream#iterate?
https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/streaming/index.html#iterations
On Mon, Dec 19, 2016 at 5:35 AM, Govindarajan Srinivasaraghavan
<go...@gmail.com> wrote:
> Hi All,
>
> I have a use case for which I need some suggestions. It's a streaming
> application with kafka source and then groupBy, keyBy and perform some
> calculations. The output of each calculation has to be a side input for the
> next calculation and also it needs to be sent to a sink.
>
> Right now I'm achieving this by storing the result state in memory and also
> save it in redis cache. I was looking at delta iterations in flink
> documentation. It would great if someone can help me understand if I can
> achieve this using iterations or any other api. Thanks in advance.
>
> Regards,
> Govind