You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Subash K <su...@ericsson.com> on 2020/04/09 15:53:06 UTC
Clarity on Pulsar Consumer API receive
Hi,
I'm new to Pub-Sub systems and currently comparing features of Kafka and Pulsar. I've started with simple producer and consumer and I notice a key difference in the consumer API.
* The Consumer API of Kafka poll ConsumerRecords<Integer, byte[]> records = consumer.poll(0); gets all the unacknowledged messages available in the topic at that point of time.
* But whereas the consumer API of Pulsar receive msg = consumer.receive(0, TimeUnit.SECONDS); gets only one unacknowledged message from the topic.
I'm wondering is there any advantage seen in Pulsar by returning only one message at a time. Can someone please help in clarifying?
P.S: I'm using the default subscription mode in consumer (Exclusive)
Regards,
Subash Kunjupillai
RE: Clarity on Pulsar Consumer API receive
Posted by Subash K <su...@ericsson.com>.
Thanks Sijie!
Regards,
Subash Kunjupillai
From: Sijie Guo <gu...@gmail.com>
Sent: Thursday, April 9, 2020 10:44 PM
To: users@pulsar.apache.org
Subject: Re: Clarity on Pulsar Consumer API receive
Raman Gupta answered your question in Slack. I copied and pasted his answer here.
Firstly, that Kafka call doesn't get all unread messages, it gets an undefined amount of messages which might be all or might be less (there are a bunch of consumer properties that control this like fetch.max.bytes and max.poll.records.Secondly, AIUI the Pulsar consumer does the same thing in the background but presents a simpler one-receive-per-message API to you the user. See receiverQueueSize for configuring<https://pulsar.apache.org/docs/en/client-libraries-java/#configure-consumer> this behavior.If you really want to receive a batch of messages all at once, you can use a batching receive<https://pulsar.apache.org/docs/en/client-libraries-java/#batch-receive>.
- Sijie
On Thu, Apr 9, 2020 at 8:53 AM Subash K <su...@ericsson.com>> wrote:
Hi,
I'm new to Pub-Sub systems and currently comparing features of Kafka and Pulsar. I've started with simple producer and consumer and I notice a key difference in the consumer API.
* The Consumer API of Kafka poll ConsumerRecords<Integer, byte[]> records = consumer.poll(0); gets all the unacknowledged messages available in the topic at that point of time.
* But whereas the consumer API of Pulsar receive msg = consumer.receive(0, TimeUnit.SECONDS); gets only one unacknowledged message from the topic.
I'm wondering is there any advantage seen in Pulsar by returning only one message at a time. Can someone please help in clarifying?
P.S: I'm using the default subscription mode in consumer (Exclusive)
Regards,
Subash Kunjupillai
Re: Clarity on Pulsar Consumer API receive
Posted by Sijie Guo <gu...@gmail.com>.
Raman Gupta answered your question in Slack. I copied and pasted his answer
here.
Firstly, that Kafka call doesn't get *all* unread messages, it gets an
undefined amount of messages which might be all or might be less (there are
a bunch of consumer properties that control this like fetch.max.bytes and
max.poll.records.Secondly, AIUI the Pulsar consumer does the same thing in
the background but presents a simpler one-receive-per-message API to you
the user. See receiverQueueSize for configuring
<https://pulsar.apache.org/docs/en/client-libraries-java/#configure-consumer>
this
behavior.If you really want to receive a batch of messages all at once, you
can use a batching receive
<https://pulsar.apache.org/docs/en/client-libraries-java/#batch-receive>.
- Sijie
On Thu, Apr 9, 2020 at 8:53 AM Subash K <su...@ericsson.com> wrote:
> Hi,
>
>
>
> I'm new to Pub-Sub systems and currently comparing features of Kafka and
> Pulsar. I've started with simple producer and consumer and I notice a key
> difference in the consumer API.
>
> - The Consumer API of Kafka poll ConsumerRecords<Integer, byte[]>
> records = consumer.poll(0); gets all the unacknowledged messages
> available in the topic at that point of time.
> - But whereas the consumer API of Pulsar receive msg =
> consumer.receive(0, TimeUnit.SECONDS); gets only one unacknowledged
> message from the topic.
>
> I'm wondering is there any advantage seen in Pulsar by returning only one
> message at a time. Can someone please help in clarifying?
>
>
>
> *P.S: I'm using the default subscription mode in consumer (Exclusive)*
>
>
>
> Regards,
>
> Subash Kunjupillai
>
>
>