You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by mayank rathi <ma...@gmail.com> on 2017/08/07 14:02:00 UTC

How to send data stored in Kafka from beginning to a new consumer using NiFi?

Hello All,

How can I add a new NiFi consumer to a Kafka topic and send data from
beginning? I do not see any setting in ConsumerKafkaRecord_0_10 processor
to accomplish this.

I am looking for functionality similar to --from-beginning parameter in
Kafka Console Consumer.

"./bin/kafka-console-consumer --zookeeper zk1:2181,zk2:2181,zk3:2181/kafka
--topic LCL_LANG_DSTRBTN1 --from-beginning"

Thanks!!


-- 
NOTICE: This email message is for the sole use of the intended recipient(s)
and may contain confidential and privileged information. Any unauthorized
review, use, disclosure or distribution is prohibited. If you are not the
intended recipient, please contact the sender by reply email and destroy
all copies of the original message.

Re: How to send data stored in Kafka from beginning to a new consumer using NiFi?

Posted by Mark Payne <ma...@hotmail.com>.
Mayank,

You would just set the "Offset Reset" property to "earliest". Please note, though, that if you have already
used the processor to consume from Kafka, you would need to change your "Group ID" also because if
the Group ID has already been used to consume from Kafka, then the offsets are not reset.

Thanks
-Mark


On Aug 7, 2017, at 10:02 AM, mayank rathi <ma...@gmail.com>> wrote:


Hello All,

How can I add a new NiFi consumer to a Kafka topic and send data from beginning? I do not see any setting in ConsumerKafkaRecord_0_10 processor to accomplish this.

I am looking for functionality similar to --from-beginning parameter in Kafka Console Consumer.

"./bin/kafka-console-consumer --zookeeper zk1:2181,zk2:2181,zk3:2181/kafka --topic LCL_LANG_DSTRBTN1 --from-beginning"

Thanks!!


--
NOTICE: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.