You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Jens Rantil <je...@tink.se> on 2016/01/27 18:05:53 UTC
Only interested in certain partitions
Hi,
Background: I am using Kafka 0.9 using the Java client. I have a consumer
with a fixed set of keys that it is interested in from a given topic.
Assuming I have many partitions, I could manually assign my consumer to
only listen to the relevant partitions, given my keys.
Question: Given a key and a topic, when using the new Java consumer, how
can I figure out which partition the key will be written to? If not
possible, I will file a JIRA.
Thanks,
Jens
--
Jens Rantil
Backend engineer
Tink AB
Email: jens.rantil@tink.se
Phone: +46 708 84 18 32
Web: www.tink.se
Facebook <https://www.facebook.com/#!/tink.se> Linkedin
<http://www.linkedin.com/company/2735919?trk=vsrp_companies_res_photo&trkInfo=VSRPsearchId%3A1057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary>
Twitter <https://twitter.com/tink>
Re: Only interested in certain partitions
Posted by Ewen Cheslack-Postava <ew...@confluent.io>.
One option is to instantiate and invoke the DefaultPartitioner yourself (or
whatever partitioner you've specified for partitioner.class). However, that
will require passing in a Cluster object, which you'll need to construct
yourself. This is just used to get the number of partitions for the topic
so it shouldn't be a problem to fill it in with basic, dummy data as long
as you have the right number of partitions.
-Ewen
On Wed, Jan 27, 2016 at 9:05 AM, Jens Rantil <je...@tink.se> wrote:
> Hi,
>
> Background: I am using Kafka 0.9 using the Java client. I have a consumer
> with a fixed set of keys that it is interested in from a given topic.
> Assuming I have many partitions, I could manually assign my consumer to
> only listen to the relevant partitions, given my keys.
>
> Question: Given a key and a topic, when using the new Java consumer, how
> can I figure out which partition the key will be written to? If not
> possible, I will file a JIRA.
>
> Thanks,
> Jens
>
> --
> Jens Rantil
> Backend engineer
> Tink AB
>
> Email: jens.rantil@tink.se
> Phone: +46 708 84 18 32
> Web: www.tink.se
>
> Facebook <https://www.facebook.com/#!/tink.se> Linkedin
> <
> http://www.linkedin.com/company/2735919?trk=vsrp_companies_res_photo&trkInfo=VSRPsearchId%3A1057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary
> >
> Twitter <https://twitter.com/tink>
>
--
Thanks,
Ewen