poll(0) after subscribe

[Mansi Shah <ma...@maprtech.com>]

Hi, 

I had a question around expected behavior / API expectations.

Is subscribe a lazy call? What I mean by that is if I subscribe to a topic and then call poll(0), then is this first poll guaranteed to never return data? 
Will the data pre-fetch not start as soon as subscribe is called? Is this behavior also true for assign calls?

I ask this because the 0.9.0 connect code does the following - 

subscribe(topicName); 
poll(0); <— expect this call to not return any data
and then it starts its actual data consuming poll iterator. 


In the above case we can have potential data loss if the that first poll(0) actually did return some records. I see this code has been restructured for 0.10.0 timeframe.

Thanks in advance for any help on this subject !!
Mansi. 

Re: poll(0) after subscribe

[Mansi Shah <ma...@maprtech.com>]

Guozhang
Thanks for the explanation !!

Mansi.

Sent from my iPhone

> On Mar 5, 2016, at 3:38 PM, Guozhang Wang <wa...@gmail.com> wrote:
> 
> Guozhang

Re: poll(0) after subscribe

[Guozhang Wang <wa...@gmail.com>]

Hi Mansi,

1) subscribe is a lazy call: only the subsequent poll() will do the leg
work of talking to broker, registering the subscription, getting assigned
partitions and issuing fetch requests to the partition leaders.

2) currently the timeout value for poll() does not count for the time
elapsed for coordination, but only for fetching requests; that said, if
this poll(0) is the first EVER poll() function call then it should not
return anything as it will only trigger the underlying NIO selector once;
i.e. pre-fetch will not be triggered yet.

Guozhang

On Fri, Mar 4, 2016 at 4:22 PM, Mansi Shah <ma...@maprtech.com> wrote:

> Hi,
>
> I had a question around expected behavior / API expectations.
>
> Is subscribe a lazy call? What I mean by that is if I subscribe to a topic
> and then call poll(0), then is this first poll guaranteed to never return
> data?
> Will the data pre-fetch not start as soon as subscribe is called? Is this
> behavior also true for assign calls?
>
> I ask this because the 0.9.0 connect code does the following -
>
> subscribe(topicName);
> poll(0); <— expect this call to not return any data
> and then it starts its actual data consuming poll iterator.
>
>
> In the above case we can have potential data loss if the that first
> poll(0) actually did return some records. I see this code has been
> restructured for 0.10.0 timeframe.
>
> Thanks in advance for any help on this subject !!
> Mansi.
>
>


-- 
-- Guozhang