You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jon Haddad (JIRA)" <ji...@apache.org> on 2017/03/02 00:26:45 UTC
[jira] [Comment Edited] (CASSANDRA-13010) nodetool compactionstats
should say which disk a compaction is writing to
[ https://issues.apache.org/jira/browse/CASSANDRA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891358#comment-15891358 ]
Jon Haddad edited comment on CASSANDRA-13010 at 3/2/17 12:26 AM:
-----------------------------------------------------------------
I set up 6 data directories and dropped my memtable_heap_space_in_mb to 32 MB to force a ton of flushes and lots of compactions. I do see directories in the compactionstats, indicating that it's working, but i haven't verified the right directories are listed.
The output is a little hard to read, since the directories make the output look like this:
{code}
jhaddad@rustyrazorblade ~/dev/cassandra$ bin/nodetool compactionstats
pending tasks: 6
- keyspace1.standard1: 6
id compaction type keyspace table target directory completed total unit progress
7f3e7a40-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 /Users/jhaddad/var/lib/cassandra/data1/keyspace1/standard1-f867e470fedb11e6a2c121962153156a 17696623 38340460 bytes 46.16%
83c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 /Users/jhaddad/var/lib/cassandra/data4/keyspace1/standard1-f867e470fedb11e6a2c121962153156a 3651789 33841385 bytes 10.79%
7a1beed0-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 /Users/jhaddad/var/lib/cassandra/data3/keyspace1/standard1-f867e470fedb11e6a2c121962153156a 34012548 37166390 bytes 91.51%
7a2e6560-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 /Users/jhaddad/var/lib/cassandra/data6/keyspace1/standard1-f867e470fedb11e6a2c121962153156a 33334698 36996530 bytes 90.10%
7dabcc50-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 /Users/jhaddad/var/lib/cassandra/data2/keyspace1/standard1-f867e470fedb11e6a2c121962153156a 22993893 37875910 bytes 60.71%
Active compaction remaining time : 0h00m04s
{code}
From a user interface perspective, It would be great if the tasks could be separated by directory, rather than inlining the directory in the table. So in my example something more like:
{code}
jhaddad@rustyrazorblade ~/dev/cassandra$ bin/nodetool compactionstats
pending tasks: 6
- keyspace1.standard1: 6
id compaction type keyspace table completed total unit progress
/Users/jhaddad/var/lib/cassandra/data1/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7f3e7a40-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 17696623 38340460 bytes 46.16%
/Users/jhaddad/var/lib/cassandra/data4/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
83c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 3651789 33841385 bytes 10.79%
/Users/jhaddad/var/lib/cassandra/data3/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7a1beed0-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 34012548 37166390 bytes 91.51%
/Users/jhaddad/var/lib/cassandra/data6/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7a2e6560-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 33334698 36996530 bytes 90.10%
/Users/jhaddad/var/lib/cassandra/data2/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7dabcc50-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 22993893 37875910 bytes 60.71%
Active compaction remaining time : 0h00m04s
{code}
If there are multiple compactions in a single directory, you would see this:
{code}
/Users/jhaddad/var/lib/cassandra/data3/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7a1beed0-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 34012548 37166390 bytes 91.51%
7a2e6560-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 33334698 36996530 bytes 90.10%
7dabcc50-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 22993893 37875910 bytes 60.71%
{code}
I'm also not sure if there's value in including the keyspace & table directory, since that information is duplicated. If we limit the result to the data directory we would end up with at most N sections, 1 per data directory specified in the yaml. I think that would be the *most* useful form of this output (note the line between the directories):
{code}
/Users/jhaddad/var/lib/cassandra/data4
83c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 3651789 33841385 bytes 10.79%
03c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard2 3651789 13841385 bytes 10.79%
/Users/jhaddad/var/lib/cassandra/data3
7a1beed0-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard3 34012548 57166390 bytes 91.51%
83c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard5 3651789 43841385 bytes 10.79%
{code}
What do you think?
was (Author: rustyrazorblade):
I set up 6 data directories and dropped my memtable_heap_space_in_mb to 32 MB to force a ton of flushes and lots of compactions. I do see directories in the compactionstats, indicating that it's working, but i haven't verified the right directories are listed.
The output is a little hard to read, since the directories make the output look like this:
{code}
jhaddad@rustyrazorblade ~/dev/cassandra$ bin/nodetool compactionstats
pending tasks: 6
- keyspace1.standard1: 6
id compaction type keyspace table target directory completed total unit progress
7f3e7a40-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 /Users/jhaddad/var/lib/cassandra/data1/keyspace1/standard1-f867e470fedb11e6a2c121962153156a 17696623 38340460 bytes 46.16%
83c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 /Users/jhaddad/var/lib/cassandra/data4/keyspace1/standard1-f867e470fedb11e6a2c121962153156a 3651789 33841385 bytes 10.79%
7a1beed0-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 /Users/jhaddad/var/lib/cassandra/data3/keyspace1/standard1-f867e470fedb11e6a2c121962153156a 34012548 37166390 bytes 91.51%
7a2e6560-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 /Users/jhaddad/var/lib/cassandra/data6/keyspace1/standard1-f867e470fedb11e6a2c121962153156a 33334698 36996530 bytes 90.10%
7dabcc50-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 /Users/jhaddad/var/lib/cassandra/data2/keyspace1/standard1-f867e470fedb11e6a2c121962153156a 22993893 37875910 bytes 60.71%
Active compaction remaining time : 0h00m04s
{code}
From a user interface perspective, It would be great if the tasks could be separated by directory, rather than inlining the directory in the table. So in my example something more like:
{code}
jhaddad@rustyrazorblade ~/dev/cassandra$ bin/nodetool compactionstats
pending tasks: 6
- keyspace1.standard1: 6
id compaction type keyspace table completed total unit progress
/Users/jhaddad/var/lib/cassandra/data1/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7f3e7a40-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 17696623 38340460 bytes 46.16%
/Users/jhaddad/var/lib/cassandra/data4/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
83c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 3651789 33841385 bytes 10.79%
/Users/jhaddad/var/lib/cassandra/data3/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7a1beed0-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 34012548 37166390 bytes 91.51%
/Users/jhaddad/var/lib/cassandra/data6/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7a2e6560-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 33334698 36996530 bytes 90.10%
/Users/jhaddad/var/lib/cassandra/data2/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7dabcc50-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 22993893 37875910 bytes 60.71%
Active compaction remaining time : 0h00m04s
{code}
If there are multiple compactions in a single directory, you would see this:
{code}
/Users/jhaddad/var/lib/cassandra/data3/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7a1beed0-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 34012548 37166390 bytes 91.51%
7a2e6560-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 33334698 36996530 bytes 90.10%
7dabcc50-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 22993893 37875910 bytes 60.71%
{code}
I'm also not sure if there's value in including the keyspace & table directory, since that information is duplicated. If we limit the result to the data directory we would end up with at most N sections, 1 per data directory specified in the yaml. I think that would be the *most* useful form of this output. I believe this would be the most helpful output (note the line between the directories):
{code}
/Users/jhaddad/var/lib/cassandra/data4
83c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1 3651789 33841385 bytes 10.79%
03c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard2 3651789 13841385 bytes 10.79%
/Users/jhaddad/var/lib/cassandra/data3
7a1beed0-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard3 34012548 57166390 bytes 91.51%
83c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard5 3651789 43841385 bytes 10.79%
{code}
What do you think?
> nodetool compactionstats should say which disk a compaction is writing to
> -------------------------------------------------------------------------
>
> Key: CASSANDRA-13010
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13010
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jon Haddad
> Assignee: Alex Lourie
> Labels: lhf
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)