You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Gyula Fóra <gy...@gmail.com> on 2019/10/21 12:15:34 UTC

Comparing Storm and Flink resource requirements

Hi All!

I would like to ask the community for any experience regarding migration
from Storm to Flink production applications.

Specifically I am interested in your experience related to the resource
requirements for the same pipeline as implemented in Flink vs in Storm. The
design of the applications might be significantly different for the 2
systems (state, added complexity by new features enabled) so I don't expect
precise numbers, estimates would be completely fine :)

I understand that this is a quite complex question and I would really
appreciate any information you can provide.

Thank you all!
Gyula

Re: Comparing Storm and Flink resource requirements

Posted by Gyula Fóra <gy...@gmail.com>.
Thanks Vino, this is very helpful.

At the end of the day we are looking for something like:
A job that requires N nodes in Storm, only requires K nodes when
implemented in Flink, where hopefully K < N :)

Cheers,
Gyula

On Tue, Oct 22, 2019 at 2:31 PM vino yang <ya...@gmail.com> wrote:

> Hi Gyula,
>
> Based on our previous experience switching from Storm to Flink. For the
> same business, resources of the same size are completely sufficient, and
> the performance indicators are slightly better than Storm. As you said,
> this may be related to using some of Flink's special features like state.
> Our state was small at the time, and the main business was real-time ETL.
> If it is a different type of business, the problem may be more complicated
> and may require a specific analysis of the specific problem.
>
> Best,
> Vino
>
> Gyula Fóra <gy...@gmail.com> 于2019年10月21日周一 下午8:15写道:
>
>> Hi All!
>>
>> I would like to ask the community for any experience regarding migration
>> from Storm to Flink production applications.
>>
>> Specifically I am interested in your experience related to the resource
>> requirements for the same pipeline as implemented in Flink vs in Storm. The
>> design of the applications might be significantly different for the 2
>> systems (state, added complexity by new features enabled) so I don't expect
>> precise numbers, estimates would be completely fine :)
>>
>> I understand that this is a quite complex question and I would really
>> appreciate any information you can provide.
>>
>> Thank you all!
>> Gyula
>>
>

Re: Comparing Storm and Flink resource requirements

Posted by vino yang <ya...@gmail.com>.
Hi Gyula,

Based on our previous experience switching from Storm to Flink. For the
same business, resources of the same size are completely sufficient, and
the performance indicators are slightly better than Storm. As you said,
this may be related to using some of Flink's special features like state.
Our state was small at the time, and the main business was real-time ETL.
If it is a different type of business, the problem may be more complicated
and may require a specific analysis of the specific problem.

Best,
Vino

Gyula Fóra <gy...@gmail.com> 于2019年10月21日周一 下午8:15写道:

> Hi All!
>
> I would like to ask the community for any experience regarding migration
> from Storm to Flink production applications.
>
> Specifically I am interested in your experience related to the resource
> requirements for the same pipeline as implemented in Flink vs in Storm. The
> design of the applications might be significantly different for the 2
> systems (state, added complexity by new features enabled) so I don't expect
> precise numbers, estimates would be completely fine :)
>
> I understand that this is a quite complex question and I would really
> appreciate any information you can provide.
>
> Thank you all!
> Gyula
>