You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Raphael Hsieh <ra...@gmail.com> on 2015/03/07 00:07:44 UTC
Double counting batches
Hi. I've been noticing in my graphs that every so often I'm receiving
some anomalous spikes in data that contain almost twice the amount of
data that I'm expecting. I believe this has something to do with my
batches being double counted.
I am using an OpaqueTridentKafkaSpout. This supposedly guarantees
exactly once processing.
I am using DynamoDB as an external datastore.
Does anybody have any suggestions on how I can go about debugging this ?
Thanks
--
Raphael Hsieh
Re: Double counting batches
Posted by Raphael Hsieh <ra...@gmail.com>.
bump
On Fri, Mar 6, 2015 at 3:07 PM, Raphael Hsieh <ra...@gmail.com> wrote:
> Hi. I've been noticing in my graphs that every so often I'm receiving
> some anomalous spikes in data that contain almost twice the amount of
> data that I'm expecting. I believe this has something to do with my
> batches being double counted.
>
> I am using an OpaqueTridentKafkaSpout. This supposedly guarantees
> exactly once processing.
> I am using DynamoDB as an external datastore.
>
> Does anybody have any suggestions on how I can go about debugging this ?
>
> Thanks
>
> --
> Raphael Hsieh
--
Raphael Hsieh
Amazon.com
Software Development Engineer I
(978) 764-9014