You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Kunal Gangakhedkar <kg...@gmail.com> on 2018/03/05 07:02:28 UTC
system.size_estimates - safe to remove sstables?
Hi all,
I have a 2-node cluster running cassandra 2.1.18.
One of the nodes has run out of disk space and died - almost all of it
shows up as occupied by size_estimates CF.
Out of 296GiB, 288GiB shows up as consumed by size_estimates in 'du -sh'
output.
This is while the other node is chugging along - shows only 25MiB consumed
by size_estimates (du -sh output).
Any idea why this descripancy?
Is it safe to remove the size_estimates sstables from the affected node and
restart the service?
Thanks,
Kunal
RE: system.size_estimates - safe to remove sstables?
Posted by Kunal Gangakhedkar <kg...@gmail.com>.
No, this is a different cluster.
Kunal
On 13-Mar-2018 6:27 AM, "Kenneth Brotman" <ke...@yahoo.com.invalid>
wrote:
Kunal,
Is this the GCE cluster you are speaking of in the “Adding new DC?” thread?
Kenneth Brotman
*From:* Kunal Gangakhedkar [mailto:kgangakhedkar@gmail.com]
*Sent:* Sunday, March 11, 2018 2:18 PM
*To:* user@cassandra.apache.org
*Subject:* Re: system.size_estimates - safe to remove sstables?
Finally, got a chance to work on it over the weekend.
It worked as advertised. :)
Thanks a lot, Chris.
Kunal
On 8 March 2018 at 10:47, Kunal Gangakhedkar <kg...@gmail.com>
wrote:
Thanks a lot, Chris.
Will try it today/tomorrow and update here.
Thanks,
Kunal
On 7 March 2018 at 00:25, Chris Lohfink <cl...@apple.com> wrote:
While its off you can delete the files in the directory yeah
Chris
On Mar 6, 2018, at 2:35 AM, Kunal Gangakhedkar <kg...@gmail.com>
wrote:
Hi Chris,
I checked for snapshots and backups - none found.
Also, we're not using opscenter, hadoop or spark or any such tool.
So, do you think we can just remove the cf and restart the service?
Thanks,
Kunal
On 5 March 2018 at 21:52, Chris Lohfink <cl...@apple.com> wrote:
Any chance space used by snapshots? What files exist there that are taking
up space?
> On Mar 5, 2018, at 1:02 AM, Kunal Gangakhedkar <kg...@gmail.com>
wrote:
>
> Hi all,
>
> I have a 2-node cluster running cassandra 2.1.18.
> One of the nodes has run out of disk space and died - almost all of it
shows up as occupied by size_estimates CF.
> Out of 296GiB, 288GiB shows up as consumed by size_estimates in 'du -sh'
output.
>
> This is while the other node is chugging along - shows only 25MiB
consumed by size_estimates (du -sh output).
>
> Any idea why this descripancy?
> Is it safe to remove the size_estimates sstables from the affected node
and restart the service?
>
> Thanks,
> Kunal
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
<us...@cassandra.apacheorg>
For additional commands, e-mail: user-help@cassandra.apache.org
RE: system.size_estimates - safe to remove sstables?
Posted by Kenneth Brotman <ke...@yahoo.com.INVALID>.
Kunal,
Is this the GCE cluster you are speaking of in the “Adding new DC?” thread?
Kenneth Brotman
From: Kunal Gangakhedkar [mailto:kgangakhedkar@gmail.com]
Sent: Sunday, March 11, 2018 2:18 PM
To: user@cassandra.apache.org
Subject: Re: system.size_estimates - safe to remove sstables?
Finally, got a chance to work on it over the weekend.
It worked as advertised. :)
Thanks a lot, Chris.
Kunal
On 8 March 2018 at 10:47, Kunal Gangakhedkar <kg...@gmail.com> wrote:
Thanks a lot, Chris.
Will try it today/tomorrow and update here.
Thanks,
Kunal
On 7 March 2018 at 00:25, Chris Lohfink <cl...@apple.com> wrote:
While its off you can delete the files in the directory yeah
Chris
On Mar 6, 2018, at 2:35 AM, Kunal Gangakhedkar <kg...@gmail.com> wrote:
Hi Chris,
I checked for snapshots and backups - none found.
Also, we're not using opscenter, hadoop or spark or any such tool.
So, do you think we can just remove the cf and restart the service?
Thanks,
Kunal
On 5 March 2018 at 21:52, Chris Lohfink <cl...@apple.com> wrote:
Any chance space used by snapshots? What files exist there that are taking up space?
> On Mar 5, 2018, at 1:02 AM, Kunal Gangakhedkar <kg...@gmail.com> wrote:
>
> Hi all,
>
> I have a 2-node cluster running cassandra 2.1.18.
> One of the nodes has run out of disk space and died - almost all of it shows up as occupied by size_estimates CF.
> Out of 296GiB, 288GiB shows up as consumed by size_estimates in 'du -sh' output.
>
> This is while the other node is chugging along - shows only 25MiB consumed by size_estimates (du -sh output).
>
> Any idea why this descripancy?
> Is it safe to remove the size_estimates sstables from the affected node and restart the service?
>
> Thanks,
> Kunal
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org <ma...@cassandra.apacheorg>
For additional commands, e-mail: user-help@cassandra.apache.org
Re: system.size_estimates - safe to remove sstables?
Posted by Kunal Gangakhedkar <kg...@gmail.com>.
Finally, got a chance to work on it over the weekend.
It worked as advertised. :)
Thanks a lot, Chris.
Kunal
On 8 March 2018 at 10:47, Kunal Gangakhedkar <kg...@gmail.com>
wrote:
> Thanks a lot, Chris.
>
> Will try it today/tomorrow and update here.
>
> Thanks,
> Kunal
>
> On 7 March 2018 at 00:25, Chris Lohfink <cl...@apple.com> wrote:
>
>> While its off you can delete the files in the directory yeah
>>
>> Chris
>>
>>
>> On Mar 6, 2018, at 2:35 AM, Kunal Gangakhedkar <kg...@gmail.com>
>> wrote:
>>
>> Hi Chris,
>>
>> I checked for snapshots and backups - none found.
>> Also, we're not using opscenter, hadoop or spark or any such tool.
>>
>> So, do you think we can just remove the cf and restart the service?
>>
>> Thanks,
>> Kunal
>>
>> On 5 March 2018 at 21:52, Chris Lohfink <cl...@apple.com> wrote:
>>
>>> Any chance space used by snapshots? What files exist there that are
>>> taking up space?
>>>
>>> > On Mar 5, 2018, at 1:02 AM, Kunal Gangakhedkar <
>>> kgangakhedkar@gmail.com> wrote:
>>> >
>>> > Hi all,
>>> >
>>> > I have a 2-node cluster running cassandra 2.1.18.
>>> > One of the nodes has run out of disk space and died - almost all of it
>>> shows up as occupied by size_estimates CF.
>>> > Out of 296GiB, 288GiB shows up as consumed by size_estimates in 'du
>>> -sh' output.
>>> >
>>> > This is while the other node is chugging along - shows only 25MiB
>>> consumed by size_estimates (du -sh output).
>>> >
>>> > Any idea why this descripancy?
>>> > Is it safe to remove the size_estimates sstables from the affected
>>> node and restart the service?
>>> >
>>> > Thanks,
>>> > Kunal
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>>> For additional commands, e-mail: user-help@cassandra.apache.org
>>>
>>>
>>
>>
>
Re: system.size_estimates - safe to remove sstables?
Posted by Kunal Gangakhedkar <kg...@gmail.com>.
Thanks a lot, Chris.
Will try it today/tomorrow and update here.
Thanks,
Kunal
On 7 March 2018 at 00:25, Chris Lohfink <cl...@apple.com> wrote:
> While its off you can delete the files in the directory yeah
>
> Chris
>
>
> On Mar 6, 2018, at 2:35 AM, Kunal Gangakhedkar <kg...@gmail.com>
> wrote:
>
> Hi Chris,
>
> I checked for snapshots and backups - none found.
> Also, we're not using opscenter, hadoop or spark or any such tool.
>
> So, do you think we can just remove the cf and restart the service?
>
> Thanks,
> Kunal
>
> On 5 March 2018 at 21:52, Chris Lohfink <cl...@apple.com> wrote:
>
>> Any chance space used by snapshots? What files exist there that are
>> taking up space?
>>
>> > On Mar 5, 2018, at 1:02 AM, Kunal Gangakhedkar <kg...@gmail.com>
>> wrote:
>> >
>> > Hi all,
>> >
>> > I have a 2-node cluster running cassandra 2.1.18.
>> > One of the nodes has run out of disk space and died - almost all of it
>> shows up as occupied by size_estimates CF.
>> > Out of 296GiB, 288GiB shows up as consumed by size_estimates in 'du
>> -sh' output.
>> >
>> > This is while the other node is chugging along - shows only 25MiB
>> consumed by size_estimates (du -sh output).
>> >
>> > Any idea why this descripancy?
>> > Is it safe to remove the size_estimates sstables from the affected node
>> and restart the service?
>> >
>> > Thanks,
>> > Kunal
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>> For additional commands, e-mail: user-help@cassandra.apache.org
>>
>>
>
>
Re: system.size_estimates - safe to remove sstables?
Posted by Chris Lohfink <cl...@apple.com>.
While its off you can delete the files in the directory yeah
Chris
> On Mar 6, 2018, at 2:35 AM, Kunal Gangakhedkar <kg...@gmail.com> wrote:
>
> Hi Chris,
>
> I checked for snapshots and backups - none found.
> Also, we're not using opscenter, hadoop or spark or any such tool.
>
> So, do you think we can just remove the cf and restart the service?
>
> Thanks,
> Kunal
>
> On 5 March 2018 at 21:52, Chris Lohfink <clohfink@apple.com <ma...@apple.com>> wrote:
> Any chance space used by snapshots? What files exist there that are taking up space?
>
> > On Mar 5, 2018, at 1:02 AM, Kunal Gangakhedkar <kgangakhedkar@gmail.com <ma...@gmail.com>> wrote:
> >
> > Hi all,
> >
> > I have a 2-node cluster running cassandra 2.1.18.
> > One of the nodes has run out of disk space and died - almost all of it shows up as occupied by size_estimates CF.
> > Out of 296GiB, 288GiB shows up as consumed by size_estimates in 'du -sh' output.
> >
> > This is while the other node is chugging along - shows only 25MiB consumed by size_estimates (du -sh output).
> >
> > Any idea why this descripancy?
> > Is it safe to remove the size_estimates sstables from the affected node and restart the service?
> >
> > Thanks,
> > Kunal
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org <ma...@cassandra.apache.org>
> For additional commands, e-mail: user-help@cassandra.apache.org <ma...@cassandra.apache.org>
>
>
Re: system.size_estimates - safe to remove sstables?
Posted by Kunal Gangakhedkar <kg...@gmail.com>.
Hi Chris,
I checked for snapshots and backups - none found.
Also, we're not using opscenter, hadoop or spark or any such tool.
So, do you think we can just remove the cf and restart the service?
Thanks,
Kunal
On 5 March 2018 at 21:52, Chris Lohfink <cl...@apple.com> wrote:
> Any chance space used by snapshots? What files exist there that are taking
> up space?
>
> > On Mar 5, 2018, at 1:02 AM, Kunal Gangakhedkar <kg...@gmail.com>
> wrote:
> >
> > Hi all,
> >
> > I have a 2-node cluster running cassandra 2.1.18.
> > One of the nodes has run out of disk space and died - almost all of it
> shows up as occupied by size_estimates CF.
> > Out of 296GiB, 288GiB shows up as consumed by size_estimates in 'du -sh'
> output.
> >
> > This is while the other node is chugging along - shows only 25MiB
> consumed by size_estimates (du -sh output).
> >
> > Any idea why this descripancy?
> > Is it safe to remove the size_estimates sstables from the affected node
> and restart the service?
> >
> > Thanks,
> > Kunal
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: user-help@cassandra.apache.org
>
>
Re: system.size_estimates - safe to remove sstables?
Posted by Chris Lohfink <cl...@apple.com>.
Any chance space used by snapshots? What files exist there that are taking up space?
> On Mar 5, 2018, at 1:02 AM, Kunal Gangakhedkar <kg...@gmail.com> wrote:
>
> Hi all,
>
> I have a 2-node cluster running cassandra 2.1.18.
> One of the nodes has run out of disk space and died - almost all of it shows up as occupied by size_estimates CF.
> Out of 296GiB, 288GiB shows up as consumed by size_estimates in 'du -sh' output.
>
> This is while the other node is chugging along - shows only 25MiB consumed by size_estimates (du -sh output).
>
> Any idea why this descripancy?
> Is it safe to remove the size_estimates sstables from the affected node and restart the service?
>
> Thanks,
> Kunal
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org
Re: system.size_estimates - safe to remove sstables?
Posted by Chris Lohfink <cl...@apple.com>.
Unless using spark or hadoop nothing consumes the data in that table (unless you have tooling that may use it like opscenter or something) so your safe to just truncate it or rm the sstables when instance offline you will be fine, if you do use that table you can then do a `nodetool refreshsizeestimates` to readd it or just wait for it to re-run automatically (every 5 min).
Chris
> On Mar 5, 2018, at 1:02 AM, Kunal Gangakhedkar <kg...@gmail.com> wrote:
>
> Hi all,
>
> I have a 2-node cluster running cassandra 2.1.18.
> One of the nodes has run out of disk space and died - almost all of it shows up as occupied by size_estimates CF.
> Out of 296GiB, 288GiB shows up as consumed by size_estimates in 'du -sh' output.
>
> This is while the other node is chugging along - shows only 25MiB consumed by size_estimates (du -sh output).
>
> Any idea why this descripancy?
> Is it safe to remove the size_estimates sstables from the affected node and restart the service?
>
> Thanks,
> Kunal
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org