time in storm

[hjh <ap...@163.com>]

Hi, I need to know exact time delay in storm. For example, I have a 
topology like SpoutA->BoltB>BoltC. First I collect the time when SpoutA 
emit the tuple, and then in BoltB and BoltC, each will collect the time 
when the new tuple arrives and when the tuple is emitted to the next 
tuple. In this way, I can get t-SpoutA-Emit, t-BoltB-NewTuple, 
t-BoltB-Emit, t-BoltC-NewTuple and t-BoltC-Emit. Then the process delay 
and the transmit delay can be calculated. How ever when I use this 
method to get the delays, some delays are negative. For example 
t-BoltC-NewTuple minus t-BoltB-Emit may be negative. I use 
System.currentTimeMillis() to record time. Can anybody help to figure 
out why this happens?

Another question is about the capacity in the Storm UI, if the capacity 
of a bolt is less than 1, say 0.7, does that mean some times the bolt 
has to wait for new tuple?

Thank you very much!!!

Re: time in storm

[hjh <ap...@163.com>]

Thank you very much!! I will try to do that to sync the supervisors in 
my cluster.

On 03/05/2015 12:55 PM, Guillermo López Leal wrote:
> 网易邮箱
>> El 5/3/2015, a las 18:45, hjh <applyhhj@163.com 
>> <ma...@163.com>> escribió:
>>
>> Hi, I need to know exact time delay in storm. For example, I have a 
>> topology like SpoutA->BoltB>BoltC. First I collect the time when 
>> SpoutA emit the tuple, and then in BoltB and BoltC, each will collect 
>> the time when the new tuple arrives and when the tuple is emitted to 
>> the next tuple. In this way, I can get t-SpoutA-Emit, 
>> t-BoltB-NewTuple, t-BoltB-Emit, t-BoltC-NewTuple and t-BoltC-Emit. 
>> Then the process delay and the transmit delay can be calculated. How 
>> ever when I use this method to get the delays, some delays are 
>> negative. For example t-BoltC-NewTuple minus t-BoltB-Emit may be 
>> negative. I use System.currentTimeMillis() to record time. Can 
>> anybody help to figure out why this happens?
>
> Hi there,
>
> We have experienced this on EC2, where we are seeing increasing 
> processing time over the time in a linear increase. Just use NTP to 
> sync all your supervisors, and you will be mostly fine (we do have a 
> cron job each hour).
>
> Cheers,
>
> Guillermo
> 提示图标 邮件带有附件预览链接，若您转发或回复此邮件时不希望对方预览附 
> 件，建议您手动删除链接。
> 共有 *1* 个附件
>
> signature.asc(1K)
>     极速下载
>     <http://preview.mail.163.com/xdownload?filename=signature.asc&mid=1tbiMh6z5FD-zNo0cAAAsG&part=3&sign=6aba5c10aae0be824513a01e2abbfe97&time=1425578552&uid=applyhhj%40163.com>
>     在线预览
>     <http://preview.mail.163.com/preview?mid=1tbiMh6z5FD-zNo0cAAAsG&part=3&sign=6aba5c10aae0be824513a01e2abbfe97&time=1425578552&uid=applyhhj%40163.com>
>
>

Re: time in storm

[Guillermo López Leal <gl...@sentisis.com>]

> El 5/3/2015, a las 18:45, hjh <ap...@163.com> escribió:
> 
> Hi, I need to know exact time delay in storm. For example, I have a topology like SpoutA->BoltB>BoltC. First I collect the time when SpoutA emit the tuple, and then in BoltB and BoltC, each will collect the time when the new tuple arrives and when the tuple is emitted to the next tuple. In this way, I can get t-SpoutA-Emit, t-BoltB-NewTuple, t-BoltB-Emit, t-BoltC-NewTuple and t-BoltC-Emit. Then the process delay and the transmit delay can be calculated. How ever when I use this method to get the delays, some delays are negative. For example t-BoltC-NewTuple minus t-BoltB-Emit may be negative. I use System.currentTimeMillis() to record time. Can anybody help to figure out why this happens?

Hi there,

We have experienced this on EC2, where we are seeing increasing processing time over the time in a linear increase. Just use NTP to sync all your supervisors, and you will be mostly fine (we do have a cron job each hour).

Cheers,

Guillermo