You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by sumit singhal <su...@gmail.com> on 2017/11/21 14:16:00 UTC

Kafka consumer query.

Hi Team,

My Kafka consumer has 2 threads and the number of partitions is let's say
10 so overall 5 partitions per consumer thread. I am saving the time at
which a particular record needs to be processed. Now if record1 on
partition1 needs to be picked 10 hours from now Thread should move to next
partition to see if next partition can be picked.

example :

P1 - 8

P2 - 7

P3 - 6

P4 - 5

P5 - 4

Now data on partition P1 needs to be picked at 8 hours and let's say the
current time is 6 hours if I make my thread to wait for 2 hours I'll wait
for 2 hours although I could process P3, P4, and P5.

Please let me know how should I proceed. Any help is highly appreciable.

Regards,
Sumit.