how to speed spout.nextTuple method up?

[Link Wang <li...@gmail.com>]

hi all,

I have a trouble about huge message reading in my spout. that is the spout.nextTuple is called so slowly to read message from zmq like this: subscriber.recv(0);

it cause too many message accumulated at the zmq subscriber side, then the memory usage of this worker will grow up continually.

but when i try to use a pure java application based on jzmq to read the same zmq, it worked well. then i add an thread.sleep in my reading loop, the same problem appeared.

so, i think the cause is that spout.nextTuple is call too slowly, am i right? any suggestion?

回复： how to speed spout.nextTuple method up?

["肖康(Kang Xiao)" <kx...@gmail.com>]

hi Link,  

Spout will sleep(default 1ms) if nextTuple emit zero tuple or max-spout-pending is reached. You can check jstack of spout to see what it is doing.  

--  
Best Regards!
肖康(Kang Xiao)


在 2014年3月10日星期一，8:37，Link Wang 写道：

> hi all,
>  
> I have a trouble about huge message reading in my spout. that is the spout.nextTuple is called so slowly to read message from zmq like this: subscriber.recv(0);
>  
> it cause too many message accumulated at the zmq subscriber side, then the memory usage of this worker will grow up continually.
>  
> but when i try to use a pure java application based on jzmq to read the same zmq, it worked well. then i add an thread.sleep in my reading loop, the same problem appeared.
>  
> so, i think the cause is that spout.nextTuple is call too slowly, am i right? any suggestion?