You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Gaurav Luthra <ga...@gmail.com> on 2018/12/21 06:45:42 UTC

Getting "ProducerFenced" exception while using flink kafka producer

Hi,

I have two flink jobs, both the jobs are using Flink Kafka Producer and
Flink Kafka Consumer running in Exactly-Once mode.
Parallelism of both the jobs is one.
Both the jobs are same in number of operators and type of operators.

When we start one job then that job runs fine. But as soon as we start the
second job then both the jobs start failing with "ProducerFenced" Exception
at runtime.

As per our understanding we think that both the jobs get the same value of
transactional ID (assigned by flink), that is required to run the job in
Exactly-Once mode.

I think, Flink calculates the transactional ID with concatenation of
operator names and subtask ID. For our case in both the jobs we have same
operators, and both jobs runs with parallelism of one, so both jobs get
subtask ID.

Question:
Kindly provide the solution for this exception, Kindly show some light on
our understanding the reason of this exception. And kindly tell how can we
get different transactional ID for two jobs.
Thanks & Regards
Gaurav Luthra
Mob:- +91-9901945206