Spout not emitting tuples

[Minqi Jiang <mn...@gmail.com>]

Hi all,

I've written some python code for a spout that emits tuple values based on
user's Twitter timelines. In order to emit the proper tuple values, I
iterate through the user's timeline of historical tweets. However, I find
that if I include the code to iterate through a user's timeline in my
nextTuple method, the spout does not properly emit tuples. Instead, I see
the following debug message printed out whenever a tuple should be emitted:

66861 [Thread-14-counter] INFO  backtype.storm.daemon.task - Emitting:
counter __metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@457cb2f7> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {capacity=1024, write_pos=0,
read_pos=0, population=0}]> #<DataPoint [__receive = {capacity=1024,
write_pos=3, read_pos=2, population=1}]> #<DataPoint [__process-latency =
{}]> #<DataPoint [__transfer-count = {}]> #<DataPoint [__execute-latency =
{}]> #<DataPoint [__fail-count = {}]> #<DataPoint [__emit-count = {}]>
#<DataPoint [__execute-count = {}]>]]

It's not clear to me what is going wrong. Any pointers on how to debug this
would be much appreciated.

Thanks!

Re: Max Spout Pending Ideal Number?

[Vladi Feigin <vl...@gmail.com>]

hi,

It's matter of fine tuning and depends on your topology , there is no one
gold number. Start from 1K , monitor it. If you see that your topology
supports higher throughput , increase it ... or if it's not, decrease it..

  Vladi

On Sat, Nov 15, 2014 at 9:27 AM, Nilesh Chhapru <
nilesh.chhapru@ugamsolutions.com> wrote:

>  Hi All,
>
>
>
> What number should we keep for max spout pending in ideal scenario, as it
> is causing the throughput issues.
>
>
>
> *Regards*,
>
> *Nilesh Chhapru.*
>
> ------------------------------
>
> ---------------------------------------------------------------------------------------Disclaimer----------------------------------------------------------------------------------------------
>
> ****Opinions expressed in this e-mail are those of the author and do not
> necessarily represent those of Ugam. Ugam does not accept any
> responsibility or liability for it. This e-mail message may contain
> proprietary, confidential or legally privileged information for the sole
> use of the person or entity to whom this message was originally addressed.
> Any review, re-transmission, dissemination or other use of or taking of any
> action in reliance upon this information by persons or entities other than
> the intended recipient is prohibited. If you have received this e-mail in
> error, please delete it and all attachments from any servers, hard drives
> or any other media.
>
> Warning: Sufficient measures have been taken to scan any presence of
> viruses however the recipient should check this email and any attachments
> for the presence of viruses. Ugam accepts no liability for any damage
> caused by any virus transmitted by this email. ****
>

Max Spout Pending Ideal Number?

[Nilesh Chhapru <ni...@ugamsolutions.com>]

Hi All,

What number should we keep for max spout pending in ideal scenario, as it is causing the throughput issues.

Regards,
Nilesh Chhapru.

________________________________
---------------------------------------------------------------------------------------Disclaimer----------------------------------------------------------------------------------------------

****Opinions expressed in this e-mail are those of the author and do not necessarily represent those of Ugam. Ugam does not accept any responsibility or liability for it. This e-mail message may contain proprietary, confidential or legally privileged information for the sole use of the person or entity to whom this message was originally addressed. Any review, re-transmission, dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you have received this e-mail in error, please delete it and all attachments from any servers, hard drives or any other media.

Warning: Sufficient measures have been taken to scan any presence of viruses however the recipient should check this email and any attachments for the presence of viruses. Ugam accepts no liability for any damage caused by any virus transmitted by this email. ****

Re: Spout not emitting tuples

[Devang Shah <de...@gmail.com>]

The topology.debug is set to true, please storm.yaml.

Disable that and try putting log statements before and after emit calls and
take it from there. Initially try send a single message to check if the
topology behaves as expected.

Thanks and Regards,
Devang
On 15 Nov 2014 11:07, "Minqi Jiang" <mn...@gmail.com> wrote:

> Hi all,
>
> I've written some python code for a spout that emits tuple values based on
> user's Twitter timelines. In order to emit the proper tuple values, I
> iterate through the user's timeline of historical tweets. However, I find
> that if I include the code to iterate through a user's timeline in my
> nextTuple method, the spout does not properly emit tuples. Instead, I see
> the following debug message printed out whenever a tuple should be emitted:
>
> 66861 [Thread-14-counter] INFO  backtype.storm.daemon.task - Emitting:
> counter __metrics [#<TaskInfo
> backtype.storm.metric.api.IMetricsConsumer$TaskInfo@457cb2f7>
> [#<DataPoint [__ack-count = {}]> #<DataPoint [__sendqueue = {capacity=1024,
> write_pos=0, read_pos=0, population=0}]> #<DataPoint [__receive =
> {capacity=1024, write_pos=3, read_pos=2, population=1}]> #<DataPoint
> [__process-latency = {}]> #<DataPoint [__transfer-count = {}]> #<DataPoint
> [__execute-latency = {}]> #<DataPoint [__fail-count = {}]> #<DataPoint
> [__emit-count = {}]> #<DataPoint [__execute-count = {}]>]]
>
> It's not clear to me what is going wrong. Any pointers on how to debug
> this would be much appreciated.
>
> Thanks!
>
>
>