Negative collection counts

["John D. Ament" <jo...@apache.org>]

Hi,

I have a system with 1.0.1 on it.  Not too high of load was applied, about
25 concurrent users.  Our workflow with Usergrid is add messages for a
little, read them, then delete one at a time.

After running this test, we saw that the collection counts in the
management portal were negative.  There were 0 pending messages, however
the count was showing something like -326.

Any idea how this could happen?

John

Re: Negative collection counts

[Todd Nine <to...@gmail.com>]

Yeah, we have the classic distributed counter problem.  We can't do
immediate counters at scale.  However, what we intend to do is roll up our
counters so that we don't lose ticks during a system failure.  You may have
latency in the getting the value, but if you were to stop writing, it would
always be correct after everything flushes.  We have to get some other
frameworks in place to accomplish this.  Hopefully we'll get this done in a
month or so on the 2.0 deployment.

On Tue Feb 24 2015 at 11:46:12 AM John D. Ament <jo...@apache.org>
wrote:

> Ooooo so if I have two nodes, then they could get out of sync pretty
> quickly huh?
>
> As of now,t he count is still -326, we haven't processed any new data in
> it.
>
>
> On Tue Feb 24 2015 at 12:48:33 PM Todd Nine <to...@gmail.com> wrote:
>
>> Hey John,
>>   Our counts are summed in memory, then flushed every 30 seconds or 10k
>> updates, whichever happens first.  It's possible one of these rollups
>> dropped it's mutation, and your counts are off.  This is on our roadmap to
>> fix in the 2.0 branch.  Unfortunately, there's no great workaround at the
>> moment.
>>
>> Todd
>>
>> On Tue Feb 24 2015 at 10:35:29 AM John D. Ament <jo...@apache.org>
>> wrote:
>>
>>> Hi,
>>>
>>> I have a system with 1.0.1 on it.  Not too high of load was applied,
>>> about 25 concurrent users.  Our workflow with Usergrid is add messages for
>>> a little, read them, then delete one at a time.
>>>
>>> After running this test, we saw that the collection counts in the
>>> management portal were negative.  There were 0 pending messages, however
>>> the count was showing something like -326.
>>>
>>> Any idea how this could happen?
>>>
>>> John
>>>
>>

Re: Negative collection counts

["John D. Ament" <jo...@apache.org>]

Ooooo so if I have two nodes, then they could get out of sync pretty
quickly huh?

As of now,t he count is still -326, we haven't processed any new data in it.

On Tue Feb 24 2015 at 12:48:33 PM Todd Nine <to...@gmail.com> wrote:

> Hey John,
>   Our counts are summed in memory, then flushed every 30 seconds or 10k
> updates, whichever happens first.  It's possible one of these rollups
> dropped it's mutation, and your counts are off.  This is on our roadmap to
> fix in the 2.0 branch.  Unfortunately, there's no great workaround at the
> moment.
>
> Todd
>
> On Tue Feb 24 2015 at 10:35:29 AM John D. Ament <jo...@apache.org>
> wrote:
>
>> Hi,
>>
>> I have a system with 1.0.1 on it.  Not too high of load was applied,
>> about 25 concurrent users.  Our workflow with Usergrid is add messages for
>> a little, read them, then delete one at a time.
>>
>> After running this test, we saw that the collection counts in the
>> management portal were negative.  There were 0 pending messages, however
>> the count was showing something like -326.
>>
>> Any idea how this could happen?
>>
>> John
>>
>

Re: Negative collection counts

[Todd Nine <to...@gmail.com>]

Hey John,
  Our counts are summed in memory, then flushed every 30 seconds or 10k
updates, whichever happens first.  It's possible one of these rollups
dropped it's mutation, and your counts are off.  This is on our roadmap to
fix in the 2.0 branch.  Unfortunately, there's no great workaround at the
moment.

Todd

On Tue Feb 24 2015 at 10:35:29 AM John D. Ament <jo...@apache.org>
wrote:

> Hi,
>
> I have a system with 1.0.1 on it.  Not too high of load was applied, about
> 25 concurrent users.  Our workflow with Usergrid is add messages for a
> little, read them, then delete one at a time.
>
> After running this test, we saw that the collection counts in the
> management portal were negative.  There were 0 pending messages, however
> the count was showing something like -326.
>
> Any idea how this could happen?
>
> John
>