You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@usergrid.apache.org by "John D. Ament" <jo...@apache.org> on 2015/04/17 17:31:22 UTC
High tombstone counts when updating devices
Hi,
We just added a new feature to our app that allows a remote device to
register w/ usergrid. We're seeing very high tombstone counts, our DBAs
have reconfigured to clean up every day with 2 million entries. But this
seems a bit high and we had one environment (our perf environment) max out
after 36 hours. Currently we update the device every minute (in our case,
the device isn't a mobile but some other embedded type of device that we
want to ensure is online).
We're on 1.0.1.
Any comments about how to properly configure GC to clean up everything?
John
Re: High tombstone counts when updating devices
Posted by Todd Nine <to...@gmail.com>.
Hey John,
Every time an entity gets updated, it's old values are removed from this
CF, and the new ones are indexed, even if they're the same. This is
because in a concurrent environment, we insert new properties with the
version that's being written, since we can't gaurentee someone else isn't
already updating with a higher version. In our production cluster, we
turned our GC time down to 2 days on the cassandra CF, and this alleviates
a lot of our issues. Attached is a CQL file we run after UG creates the
CFs to optimize the cassandra storage. Note that you may need to modify
the keyspace replication strategy.
On Tue, 21 Apr 2015 at 09:50 John D. Ament <jo...@apache.org> wrote:
> Todd,
>
> This is on the Entity_Index.
>
> John
>
> On Tue, Apr 21, 2015 at 11:49 AM Todd Nine <to...@gmail.com> wrote:
>
>> Hey John,
>> Which CFs are you seeing this on?
>>
>> On Tue, 21 Apr 2015 at 09:35 John D. Ament <jo...@apache.org> wrote:
>>
>>> Any thoughts on this?
>>>
>>>
>>>
>>> On Fri, Apr 17, 2015 at 11:31 AM John D. Ament <jo...@apache.org>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> We just added a new feature to our app that allows a remote device to
>>>> register w/ usergrid. We're seeing very high tombstone counts, our DBAs
>>>> have reconfigured to clean up every day with 2 million entries. But this
>>>> seems a bit high and we had one environment (our perf environment) max out
>>>> after 36 hours. Currently we update the device every minute (in our case,
>>>> the device isn't a mobile but some other embedded type of device that we
>>>> want to ensure is online).
>>>>
>>>> We're on 1.0.1.
>>>>
>>>> Any comments about how to properly configure GC to clean up everything?
>>>>
>>>> John
>>>>
>>>
Re: High tombstone counts when updating devices
Posted by "John D. Ament" <jo...@apache.org>.
Todd,
This is on the Entity_Index.
John
On Tue, Apr 21, 2015 at 11:49 AM Todd Nine <to...@gmail.com> wrote:
> Hey John,
> Which CFs are you seeing this on?
>
> On Tue, 21 Apr 2015 at 09:35 John D. Ament <jo...@apache.org> wrote:
>
>> Any thoughts on this?
>>
>>
>>
>> On Fri, Apr 17, 2015 at 11:31 AM John D. Ament <jo...@apache.org>
>> wrote:
>>
>>> Hi,
>>>
>>> We just added a new feature to our app that allows a remote device to
>>> register w/ usergrid. We're seeing very high tombstone counts, our DBAs
>>> have reconfigured to clean up every day with 2 million entries. But this
>>> seems a bit high and we had one environment (our perf environment) max out
>>> after 36 hours. Currently we update the device every minute (in our case,
>>> the device isn't a mobile but some other embedded type of device that we
>>> want to ensure is online).
>>>
>>> We're on 1.0.1.
>>>
>>> Any comments about how to properly configure GC to clean up everything?
>>>
>>> John
>>>
>>
Re: High tombstone counts when updating devices
Posted by Todd Nine <to...@gmail.com>.
Hey John,
Which CFs are you seeing this on?
On Tue, 21 Apr 2015 at 09:35 John D. Ament <jo...@apache.org> wrote:
> Any thoughts on this?
>
>
>
> On Fri, Apr 17, 2015 at 11:31 AM John D. Ament <jo...@apache.org>
> wrote:
>
>> Hi,
>>
>> We just added a new feature to our app that allows a remote device to
>> register w/ usergrid. We're seeing very high tombstone counts, our DBAs
>> have reconfigured to clean up every day with 2 million entries. But this
>> seems a bit high and we had one environment (our perf environment) max out
>> after 36 hours. Currently we update the device every minute (in our case,
>> the device isn't a mobile but some other embedded type of device that we
>> want to ensure is online).
>>
>> We're on 1.0.1.
>>
>> Any comments about how to properly configure GC to clean up everything?
>>
>> John
>>
>
Re: High tombstone counts when updating devices
Posted by "John D. Ament" <jo...@apache.org>.
Any thoughts on this?
On Fri, Apr 17, 2015 at 11:31 AM John D. Ament <jo...@apache.org>
wrote:
> Hi,
>
> We just added a new feature to our app that allows a remote device to
> register w/ usergrid. We're seeing very high tombstone counts, our DBAs
> have reconfigured to clean up every day with 2 million entries. But this
> seems a bit high and we had one environment (our perf environment) max out
> after 36 hours. Currently we update the device every minute (in our case,
> the device isn't a mobile but some other embedded type of device that we
> want to ensure is online).
>
> We're on 1.0.1.
>
> Any comments about how to properly configure GC to clean up everything?
>
> John
>