Tuples loss in the topology

[Vladi Feigin <vl...@gmail.com>]

Hi ALL,

We have very weird topology behavior : after a few hours it starts to lose
the tuples.
We read the messages from Kafka. We know how many messages producer sends
so we are sure there are significant tuples loss when comparing send vs.
received.

We don't see the exceptions in our code. We don't use acks. The topology is
not isolated.
Storm version 0.82

We suspect that Storm for some reason reallocates the workers between nodes
 (physical machines) and this causes significant data loss.

Do you have experience with such behavior? How do you cope with this? (We
want to avoid acking because performance is important to us). How to
monitor this?

Thank you,
Vladi

Re: Tuples loss in the topology

[Andrew Xor <an...@gmail.com>]

Hi,

 I am sorry but not ack'ing means there will be data loss... you only deal
with that by using ack's.

On Sunday, October 12, 2014, Vladi Feigin <vl...@gmail.com> wrote:

> Hi ALL,
>
> We have very weird topology behavior : after a few hours it starts to lose
> the tuples.
> We read the messages from Kafka. We know how many messages producer sends
> so we are sure there are significant tuples loss when comparing send vs.
> received.
>
> We don't see the exceptions in our code. We don't use acks. The topology
> is not isolated.
> Storm version 0.82
>
> We suspect that Storm for some reason reallocates the workers between
> nodes  (physical machines) and this causes significant data loss.
>
> Do you have experience with such behavior? How do you cope with this? (We
> want to avoid acking because performance is important to us). How to
> monitor this?
>
> Thank you,
> Vladi
>
>
>


-- 
Kindly yours,

Andrew Grammenos

-- PGP PKey --
​ <https://www.dropbox.com/s/2kcxe59zsi9nrdt/pgpsig.txt>
https://www.dropbox.com/s/ei2nqsen641daei/pgpsig.txt