blocking ISpout::nextTuple()

["Ramin Farajollah (BLOOMBERG/ 731 LEX)" <rf...@bloomberg.net>]

Hi,

should a spout block on its source to produce a tuple?

For example, should it read its source from a blocking queue before emit?
In this case, the spout will indefinitely block before it can emit a tuple.

I read about how ack and fail are processed on the same thread. 

1) Will blocking in spout's nextTuple result in blocking the downstream bolts?
2) How about when a messageId is not specified?
3) How about when a tuple traverses multiple JVM instances on the same or cross machines?

Kind regards


<< �gA mind is like a parachute. It doesn&#39;t work if it is not open�h Frank Zap >>

Re: blocking ISpout::nextTuple()

[Stephen Powis <sp...@salesforce.com>]

I believe the docs around spouts say nextTuple() should never block.  In
the spout implementations I've done, I typically spin off a separate work
thread in the spout's open() method and push tuples into a shared
concurrent queue.  nextTuple() simply tries to pop the next message off of
that shared queue.

On Tue, Jul 18, 2017 at 8:32 AM, Ramin Farajollah (BLOOMBERG/ 731 LEX) <
rfarajollah@bloomberg.net> wrote:

> Hi,
>
> should a spout block on its source to produce a tuple?
>
> For example, should it read its source from a blocking queue before emit?
> In this case, the spout will indefinitely block before it can emit a tuple.
>
> I read about how ack and fail are processed on the same thread.
>
> 1) Will blocking in spout's nextTuple result in blocking the downstream
> bolts?
> 2) How about when a messageId is not specified?
> 3) How about when a tuple traverses multiple JVM instances on the same or
> cross machines?
>
> Kind regards
>
>
>
>
> << �gA mind is like a parachute. It doesn't work if
> it is not open�h Frank Zap >>
>