You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Peng Xiao <25...@qq.com> on 2017/09/30 08:21:03 UTC
space left for compaction
Dear All,
As for STCS,datastax suggest us to keep half of the free space for compaction,this is not strict,could anyone advise how many space should we left for one node?
Thanks,
Peng Xiao
Re: space left for compaction
Posted by Avi Levi <av...@indeni.com>.
Got it. Thanks
On Mon, Oct 2, 2017 at 4:54 AM, Justin Cameron <ju...@instaclustr.com>
wrote:
> Hi Avi,
>
> Actually, in Thomas' example you would need an additional 100G of free
> disk space to complete the compaction, in the worst-case situation (the
> worst-case would be that neither input SSTable contains any overlapping
> data or tombstones, therefore the output SSTable would also be roughly
> 100G).
>
> STCS progressively compacts SSTables of similar size together, with the
> output being a single SSTable containing the data of the input SSTables.
>
> Eventually you may end up with some very large SSTables that combined will
> take up 50% of your total disk space. In order to compact those SSTables
> together, STCS requires an equal amount of free disk space, which would be
> the other (unused) 50% of your total disk space.
>
> Cheers,
> Justin
>
> On Mon, 2 Oct 2017 at 12:42 Avi Levi <av...@indeni.com> wrote:
>
>> Hi Thomas ,
>> So IIUC in this case you should leave at least 50G for compaction (half
>> of the sstables size). Is that makes sense?
>> Cheers
>> Avi
>>
>>
>> On Oct 1, 2017 11:39 AM, "Steinmaurer, Thomas" <
>> thomas.steinmaurer@dynatrace.com> wrote:
>>
>> Hi,
>>
>>
>>
>> half of free space does not make sense. Imagine your SSTables need 100G
>> space and you have 20G free disk. Compaction won’t be able to do its job
>> with 10G.
>>
>>
>>
>> Half free of total disk makes more sense and is what you need for a major
>> compaction worst case.
>>
>>
>>
>> Thomas
>>
>>
>>
>> *From:* Peng Xiao [mailto:2535053@qq.com]
>> *Sent:* Samstag, 30. September 2017 10:21
>> *To:* user <us...@cassandra.apache.org>
>> *Subject:* space left for compaction
>>
>>
>>
>> Dear All,
>>
>>
>>
>> As for STCS,datastax suggest us to keep half of the free space for
>> compaction,this is not strict,could anyone advise how many space should we
>> left for one node?
>>
>>
>>
>> Thanks,
>>
>> Peng Xiao
>> The contents of this e-mail are intended for the named addressee only. It
>> contains information that may be confidential. Unless you are the named
>> addressee or an authorized designee, you may not copy or use it, or
>> disclose it to anyone else. If you received it in error please notify us
>> immediately and then destroy it. Dynatrace Austria GmbH (registration
>> number FN 91482h) is a company registered in Linz whose registered office
>> is at 4040 Linz, Austria, Freist
>> <https://maps.google.com/?q=4040+Linz,+Austria,+Freist%C3%A4dterstra%C3%9Fe+313&entry=gmail&source=g>
>> ädterstra
>> <https://maps.google.com/?q=4040+Linz,+Austria,+Freist%C3%A4dterstra%C3%9Fe+313&entry=gmail&source=g>
>> ße 313
>> <https://maps.google.com/?q=4040+Linz,+Austria,+Freist%C3%A4dterstra%C3%9Fe+313&entry=gmail&source=g>
>>
>>
>> --
>
>
> *Justin Cameron*Senior Software Engineer
>
>
> <https://www.instaclustr.com/>
>
>
> This email has been sent on behalf of Instaclustr Pty. Limited (Australia)
> and Instaclustr Inc (USA).
>
> This email and any attachments may contain confidential and legally
> privileged information. If you are not the intended recipient, do not copy
> or disclose its content, but please reply to this email immediately and
> highlight the error to the sender and then immediately delete the message.
>
Re: space left for compaction
Posted by Justin Cameron <ju...@instaclustr.com>.
Hi Avi,
Actually, in Thomas' example you would need an additional 100G of free disk
space to complete the compaction, in the worst-case situation (the
worst-case would be that neither input SSTable contains any overlapping
data or tombstones, therefore the output SSTable would also be roughly
100G).
STCS progressively compacts SSTables of similar size together, with the
output being a single SSTable containing the data of the input SSTables.
Eventually you may end up with some very large SSTables that combined will
take up 50% of your total disk space. In order to compact those SSTables
together, STCS requires an equal amount of free disk space, which would be
the other (unused) 50% of your total disk space.
Cheers,
Justin
On Mon, 2 Oct 2017 at 12:42 Avi Levi <av...@indeni.com> wrote:
> Hi Thomas ,
> So IIUC in this case you should leave at least 50G for compaction (half
> of the sstables size). Is that makes sense?
> Cheers
> Avi
>
>
> On Oct 1, 2017 11:39 AM, "Steinmaurer, Thomas" <
> thomas.steinmaurer@dynatrace.com> wrote:
>
> Hi,
>
>
>
> half of free space does not make sense. Imagine your SSTables need 100G
> space and you have 20G free disk. Compaction won’t be able to do its job
> with 10G.
>
>
>
> Half free of total disk makes more sense and is what you need for a major
> compaction worst case.
>
>
>
> Thomas
>
>
>
> *From:* Peng Xiao [mailto:2535053@qq.com]
> *Sent:* Samstag, 30. September 2017 10:21
> *To:* user <us...@cassandra.apache.org>
> *Subject:* space left for compaction
>
>
>
> Dear All,
>
>
>
> As for STCS,datastax suggest us to keep half of the free space for
> compaction,this is not strict,could anyone advise how many space should we
> left for one node?
>
>
>
> Thanks,
>
> Peng Xiao
> The contents of this e-mail are intended for the named addressee only. It
> contains information that may be confidential. Unless you are the named
> addressee or an authorized designee, you may not copy or use it, or
> disclose it to anyone else. If you received it in error please notify us
> immediately and then destroy it. Dynatrace Austria GmbH (registration
> number FN 91482h) is a company registered in Linz whose registered office
> is at 4040 Linz, Austria, Freist
> <https://maps.google.com/?q=4040+Linz,+Austria,+Freist%C3%A4dterstra%C3%9Fe+313&entry=gmail&source=g>
> ädterstra
> <https://maps.google.com/?q=4040+Linz,+Austria,+Freist%C3%A4dterstra%C3%9Fe+313&entry=gmail&source=g>
> ße 313
> <https://maps.google.com/?q=4040+Linz,+Austria,+Freist%C3%A4dterstra%C3%9Fe+313&entry=gmail&source=g>
>
>
> --
*Justin Cameron*Senior Software Engineer
<https://www.instaclustr.com/>
This email has been sent on behalf of Instaclustr Pty. Limited (Australia)
and Instaclustr Inc (USA).
This email and any attachments may contain confidential and legally
privileged information. If you are not the intended recipient, do not copy
or disclose its content, but please reply to this email immediately and
highlight the error to the sender and then immediately delete the message.
RE: space left for compaction
Posted by Avi Levi <av...@indeni.com>.
Hi Thomas ,
So IIUC in this case you should leave at least 50G for compaction (half of
the sstables size). Is that makes sense?
Cheers
Avi
On Oct 1, 2017 11:39 AM, "Steinmaurer, Thomas" <
thomas.steinmaurer@dynatrace.com> wrote:
Hi,
half of free space does not make sense. Imagine your SSTables need 100G
space and you have 20G free disk. Compaction won’t be able to do its job
with 10G.
Half free of total disk makes more sense and is what you need for a major
compaction worst case.
Thomas
*From:* Peng Xiao [mailto:2535053@qq.com]
*Sent:* Samstag, 30. September 2017 10:21
*To:* user <us...@cassandra.apache.org>
*Subject:* space left for compaction
Dear All,
As for STCS,datastax suggest us to keep half of the free space for
compaction,this is not strict,could anyone advise how many space should we
left for one node?
Thanks,
Peng Xiao
The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or
disclose it to anyone else. If you received it in error please notify us
immediately and then destroy it. Dynatrace Austria GmbH (registration
number FN 91482h) is a company registered in Linz whose registered office
is at 4040 Linz, Austria, Freist
<https://maps.google.com/?q=4040+Linz,+Austria,+Freist%C3%A4dterstra%C3%9Fe+313&entry=gmail&source=g>
ädterstra
<https://maps.google.com/?q=4040+Linz,+Austria,+Freist%C3%A4dterstra%C3%9Fe+313&entry=gmail&source=g>
ße 313
<https://maps.google.com/?q=4040+Linz,+Austria,+Freist%C3%A4dterstra%C3%9Fe+313&entry=gmail&source=g>
RE: space left for compaction
Posted by "Steinmaurer, Thomas" <th...@dynatrace.com>.
Hi,
half of free space does not make sense. Imagine your SSTables need 100G space and you have 20G free disk. Compaction won't be able to do its job with 10G.
Half free of total disk makes more sense and is what you need for a major compaction worst case.
Thomas
From: Peng Xiao [mailto:2535053@qq.com]
Sent: Samstag, 30. September 2017 10:21
To: user <us...@cassandra.apache.org>
Subject: space left for compaction
Dear All,
As for STCS,datastax suggest us to keep half of the free space for compaction,this is not strict,could anyone advise how many space should we left for one node?
Thanks,
Peng Xiao
The contents of this e-mail are intended for the named addressee only. It contains information that may be confidential. Unless you are the named addressee or an authorized designee, you may not copy or use it, or disclose it to anyone else. If you received it in error please notify us immediately and then destroy it. Dynatrace Austria GmbH (registration number FN 91482h) is a company registered in Linz whose registered office is at 4040 Linz, Austria, Freist?dterstra?e 313