You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by "Steinmaurer, Thomas" <th...@dynatrace.com> on 2018/09/06 05:45:46 UTC
nodetool cleanup - compaction remaining time
Hello,
is it a known issue / limitation that cleanup compactions aren't counted in the compaction remaining time?
nodetool compactionstats -H
pending tasks: 1
compaction type keyspace table completed total unit progress
Cleanup XXX YYY 908.16 GB 1.13 TB bytes 78.63%
Active compaction remaining time : 0h00m00s
This is with 2.1.18.
Thanks,
Thomas
The contents of this e-mail are intended for the named addressee only. It contains information that may be confidential. Unless you are the named addressee or an authorized designee, you may not copy or use it, or disclose it to anyone else. If you received it in error please notify us immediately and then destroy it. Dynatrace Austria GmbH (registration number FN 91482h) is a company registered in Linz whose registered office is at 4040 Linz, Austria, Freist?dterstra?e 313
RE: nodetool cleanup - compaction remaining time
Posted by "Steinmaurer, Thomas" <th...@dynatrace.com>.
I have created https://issues.apache.org/jira/browse/CASSANDRA-14701
Please adapt as needed. Thanks!
Thomas
From: Jeff Jirsa <jj...@gmail.com>
Sent: Donnerstag, 06. September 2018 07:52
To: cassandra <us...@cassandra.apache.org>
Subject: Re: nodetool cleanup - compaction remaining time
Probably worth a JIRA (especially if you can repro in 3.0 or higher, since 2.1 is critical fixes only)
On Wed, Sep 5, 2018 at 10:46 PM Steinmaurer, Thomas <th...@dynatrace.com>> wrote:
Hello,
is it a known issue / limitation that cleanup compactions aren’t counted in the compaction remaining time?
nodetool compactionstats -H
pending tasks: 1
compaction type keyspace table completed total unit progress
Cleanup XXX YYY 908.16 GB 1.13 TB bytes 78.63%
Active compaction remaining time : 0h00m00s
This is with 2.1.18.
Thanks,
Thomas
The contents of this e-mail are intended for the named addressee only. It contains information that may be confidential. Unless you are the named addressee or an authorized designee, you may not copy or use it, or disclose it to anyone else. If you received it in error please notify us immediately and then destroy it. Dynatrace Austria GmbH (registration number FN 91482h) is a company registered in Linz whose registered office is at 4040 Linz, Austria, Freistädterstraße 313
The contents of this e-mail are intended for the named addressee only. It contains information that may be confidential. Unless you are the named addressee or an authorized designee, you may not copy or use it, or disclose it to anyone else. If you received it in error please notify us immediately and then destroy it. Dynatrace Austria GmbH (registration number FN 91482h) is a company registered in Linz whose registered office is at 4040 Linz, Austria, Freistädterstraße 313
RE: nodetool cleanup - compaction remaining time
Posted by "Steinmaurer, Thomas" <th...@dynatrace.com>.
Alain,
compaction throughput is set to 32.
Regards,
Thomas
From: Alain RODRIGUEZ <ar...@gmail.com>
Sent: Donnerstag, 06. September 2018 11:50
To: user cassandra.apache.org <us...@cassandra.apache.org>
Subject: Re: nodetool cleanup - compaction remaining time
Hello Thomas.
Be aware that this behavior happens when the compaction throughput is set to 0 (unthrottled/unlimited). I believe the estimate uses the speed limit for calculation (which is often very much wrong anyway).
I just meant to say, you might want to make sure that it's due to cleanup type of compaction indeed and not due to some changes you could have made in the compaction throughput threshold.
C*heers,
-----------------------
Alain Rodriguez - @arodream - alain@thelastpickle.com<ma...@thelastpickle.com>
France / Spain
The Last Pickle - Apache Cassandra Consulting
http://www.thelastpickle.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.thelastpickle.com&data=01%7C01%7Cthomas.steinmaurer%40dynatrace.com%7Cc2bc41a121c24575fbd208d613df2772%7C70ebe3a35b30435d9d677716d74ca190%7C1&sdata=uJumOQxj3HEkZZl1Q%2BNrGoPGvMVTnvJgRO1Q%2Bdoot5M%3D&reserved=0>
Le jeu. 6 sept. 2018 à 06:51, Jeff Jirsa <jj...@gmail.com>> a écrit :
Probably worth a JIRA (especially if you can repro in 3.0 or higher, since 2.1 is critical fixes only)
On Wed, Sep 5, 2018 at 10:46 PM Steinmaurer, Thomas <th...@dynatrace.com>> wrote:
Hello,
is it a known issue / limitation that cleanup compactions aren’t counted in the compaction remaining time?
nodetool compactionstats -H
pending tasks: 1
compaction type keyspace table completed total unit progress
Cleanup XXX YYY 908.16 GB 1.13 TB bytes 78.63%
Active compaction remaining time : 0h00m00s
This is with 2.1.18.
Thanks,
Thomas
The contents of this e-mail are intended for the named addressee only. It contains information that may be confidential. Unless you are the named addressee or an authorized designee, you may not copy or use it, or disclose it to anyone else. If you received it in error please notify us immediately and then destroy it. Dynatrace Austria GmbH (registration number FN 91482h) is a company registered in Linz whose registered office is at 4040 Linz, Austria, Freistädterstraße 313
The contents of this e-mail are intended for the named addressee only. It contains information that may be confidential. Unless you are the named addressee or an authorized designee, you may not copy or use it, or disclose it to anyone else. If you received it in error please notify us immediately and then destroy it. Dynatrace Austria GmbH (registration number FN 91482h) is a company registered in Linz whose registered office is at 4040 Linz, Austria, Freistädterstraße 313
Re: nodetool cleanup - compaction remaining time
Posted by Alain RODRIGUEZ <ar...@gmail.com>.
>
> As far as I can remember, if you have unthrottled compaction, then the
> message is different: it says "n/a".
Ah right!
I am now completely convinced this needs a JIRA as well (indeed, if it's
not fixed in C*3+, as Jeff mentioned).
Thanks for the feedback Alex.
Le jeu. 6 sept. 2018 à 11:06, Oleksandr Shulgin <
oleksandr.shulgin@zalando.de> a écrit :
> On Thu, Sep 6, 2018 at 11:50 AM Alain RODRIGUEZ <ar...@gmail.com>
> wrote:
>
>>
>> Be aware that this behavior happens when the compaction throughput is set
>> to *0 *(unthrottled/unlimited). I believe the estimate uses the speed
>> limit for calculation (which is often very much wrong anyway).
>>
>
> As far as I can remember, if you have unthrottled compaction, then the
> message is different: it says "n/a". The all zeroes you usually see when
> you only have Validation compactions, and apparently Cleanup work the same
> way, at least in the 2.1 version.
>
>
> https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java#L102
>
> Actually, if you look closely, it's obvious that only real Compaction
> tasks count toward remainingBytes, so all Validation/Clenaup/Upgrade don't
> count. The reason must be that only actual compaction is affected by the
> throttling parameter. Is that assumption correct?
>
> In any case it would make more sense to measure the actual throughput to
> provide an accurate estimate. Not sure if there is JIRA issue for that
> already.
>
> --
> Alex
>
>
Re: nodetool cleanup - compaction remaining time
Posted by Oleksandr Shulgin <ol...@zalando.de>.
On Thu, Sep 6, 2018 at 11:50 AM Alain RODRIGUEZ <ar...@gmail.com> wrote:
>
> Be aware that this behavior happens when the compaction throughput is set
> to *0 *(unthrottled/unlimited). I believe the estimate uses the speed
> limit for calculation (which is often very much wrong anyway).
>
As far as I can remember, if you have unthrottled compaction, then the
message is different: it says "n/a". The all zeroes you usually see when
you only have Validation compactions, and apparently Cleanup work the same
way, at least in the 2.1 version.
https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java#L102
Actually, if you look closely, it's obvious that only real Compaction tasks
count toward remainingBytes, so all Validation/Clenaup/Upgrade don't
count. The reason must be that only actual compaction is affected by the
throttling parameter. Is that assumption correct?
In any case it would make more sense to measure the actual throughput to
provide an accurate estimate. Not sure if there is JIRA issue for that
already.
--
Alex
Re: nodetool cleanup - compaction remaining time
Posted by Alain RODRIGUEZ <ar...@gmail.com>.
Hello Thomas.
Be aware that this behavior happens when the compaction throughput is set
to *0 *(unthrottled/unlimited). I believe the estimate uses the speed limit
for calculation (which is often very much wrong anyway).
I just meant to say, you might want to make sure that it's due to cleanup
type of compaction indeed and not due to some changes you could have made
in the compaction throughput threshold.
C*heers,
-----------------------
Alain Rodriguez - @arodream - alain@thelastpickle.com
France / Spain
The Last Pickle - Apache Cassandra Consulting
http://www.thelastpickle.com
Le jeu. 6 sept. 2018 à 06:51, Jeff Jirsa <jj...@gmail.com> a écrit :
> Probably worth a JIRA (especially if you can repro in 3.0 or higher, since
> 2.1 is critical fixes only)
>
> On Wed, Sep 5, 2018 at 10:46 PM Steinmaurer, Thomas <
> thomas.steinmaurer@dynatrace.com> wrote:
>
>> Hello,
>>
>>
>>
>> is it a known issue / limitation that cleanup compactions aren’t counted
>> in the compaction remaining time?
>>
>>
>>
>> nodetool compactionstats -H
>>
>>
>>
>> pending tasks: 1
>>
>> compaction type keyspace table completed total
>> unit progress
>>
>> Cleanup XXX YYY 908.16 GB 1.13 TB
>> bytes 78.63%
>>
>> Active compaction remaining time : 0h00m00s
>>
>>
>>
>>
>>
>> This is with 2.1.18.
>>
>>
>>
>>
>>
>> Thanks,
>>
>> Thomas
>>
>>
>> The contents of this e-mail are intended for the named addressee only. It
>> contains information that may be confidential. Unless you are the named
>> addressee or an authorized designee, you may not copy or use it, or
>> disclose it to anyone else. If you received it in error please notify us
>> immediately and then destroy it. Dynatrace Austria GmbH (registration
>> number FN 91482h) is a company registered in Linz whose registered office
>> is at 4040 Linz, Austria, Freistädterstraße 313
>>
>
Re: nodetool cleanup - compaction remaining time
Posted by Jeff Jirsa <jj...@gmail.com>.
Probably worth a JIRA (especially if you can repro in 3.0 or higher, since
2.1 is critical fixes only)
On Wed, Sep 5, 2018 at 10:46 PM Steinmaurer, Thomas <
thomas.steinmaurer@dynatrace.com> wrote:
> Hello,
>
>
>
> is it a known issue / limitation that cleanup compactions aren’t counted
> in the compaction remaining time?
>
>
>
> nodetool compactionstats -H
>
>
>
> pending tasks: 1
>
> compaction type keyspace table completed total
> unit progress
>
> Cleanup XXX YYY 908.16 GB 1.13 TB
> bytes 78.63%
>
> Active compaction remaining time : 0h00m00s
>
>
>
>
>
> This is with 2.1.18.
>
>
>
>
>
> Thanks,
>
> Thomas
>
>
> The contents of this e-mail are intended for the named addressee only. It
> contains information that may be confidential. Unless you are the named
> addressee or an authorized designee, you may not copy or use it, or
> disclose it to anyone else. If you received it in error please notify us
> immediately and then destroy it. Dynatrace Austria GmbH (registration
> number FN 91482h) is a company registered in Linz whose registered office
> is at 4040 Linz, Austria, Freistädterstraße 313
>