You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by M Singh <ma...@yahoo.com> on 2019/07/01 10:28:33 UTC
Apache Flink - Multiple Kinesis stream consumers
Hi:
I am trying to understand how does Flink coordinate multiple kinesis consumers and am trying to find out:
1. Is it possible to read same kinesis stream independently multiple times within a single application ? How does Flink coordinate consuming same kinesis multiple times in a single application ? Are there any issues that can arise from this pattern ?2. How does Flink coordinate consumers of same kinesis stream across multiple applications ?
Thanks
Mans
Re: Apache Flink - Multiple Kinesis stream consumers
Posted by Konstantin Knauf <ko...@ververica.com>.
Hi Mans,
Flink does not coordinate the consumption at all (neither within the same
nor across applications). Each FlinkKinesisConsumer will keep track of its
consumption (position in each shared) in order to provide exactly-once
guarantees.
Cheers,
Konstantin
On Mon, Jul 1, 2019 at 12:30 PM M Singh <ma...@yahoo.com> wrote:
> Hi:
>
> I am trying to understand how does Flink coordinate multiple kinesis
> consumers and am trying to find out:
>
> 1. Is it possible to read same kinesis stream independently multiple times
> within a single application ? How does Flink coordinate consuming same
> kinesis multiple times in a single application ? Are there any issues that
> can arise from this pattern ?
> 2. How does Flink coordinate consumers of same kinesis stream across
> multiple applications ?
>
> Thanks
>
> Mans
>
--
Konstantin Knauf | Solutions Architect
+49 160 91394525
Planned Absences: 10.08.2019 - 31.08.2019, 05.09. - 06.09.2010
--
Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen