You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Brandon Williams (Created) (JIRA)" <ji...@apache.org> on 2011/10/04 21:45:34 UTC
[jira] [Created] (CASSANDRA-3308) Add compaction_thread_priority
back
Add compaction_thread_priority back
-----------------------------------
Key: CASSANDRA-3308
URL: https://issues.apache.org/jira/browse/CASSANDRA-3308
Project: Cassandra
Issue Type: Bug
Reporter: Brandon Williams
Fix For: 1.0.0
In CASSANDRA-3104, this was removed with the following reasoning:
bq. compaction_throughput_mb_per_sec is a more effective throttle on compaction.
This turns out to be false in the majority of deployments. In many (if not most) situations, compaction is actually CPU bound, not IO bound, so multithreaded compaction is generally helpful, but the priority needs to be lowered in order to prevent it from stealing CPU used for reads/writes.
Compaction is always CPU bound on both real hardware (sw raid0 with two SATA disks) and on a rackspace cloud server (though my understanding is they are back by a raid10 array underneath) however I suspect even a single drive is fast enough to handle the ~20MB/s that compaction is currently performing when unthrottled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3308) Add compaction_thread_priority
back
Posted by "Jonathan Ellis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3308:
--------------------------------------
Component/s: Core
Priority: Minor (was: Major)
Issue Type: Improvement (was: Bug)
> Add compaction_thread_priority back
> -----------------------------------
>
> Key: CASSANDRA-3308
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3308
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 1.0.0
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Priority: Minor
> Labels: compaction
> Fix For: 1.0.0
>
> Attachments: 3308.txt
>
>
> In CASSANDRA-3104, this was removed with the following reasoning:
> bq. compaction_throughput_mb_per_sec is a more effective throttle on compaction.
> This turns out to be false in the majority of deployments. In many (if not most) situations, compaction is actually CPU bound, not IO bound, so multithreaded compaction is generally helpful, but the priority needs to be lowered in order to prevent it from stealing CPU used for reads/writes.
> Compaction is always CPU bound on both real hardware (sw raid0 with two SATA disks) and on a rackspace cloud server (though my understanding is they are back by a raid10 array underneath) however I suspect even a single drive is fast enough to handle the ~20MB/s that compaction is currently performing when unthrottled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3308) Add compaction_thread_priority
back
Posted by "Jonathan Ellis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3308:
--------------------------------------
Attachment: 3308.txt
patch to use MIN_PRIORITY for compaction and HH threads. (users who don't want this can disable it by removing the UseThreadPriorities JVM option from cassandra-env.sh)
> Add compaction_thread_priority back
> -----------------------------------
>
> Key: CASSANDRA-3308
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3308
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Brandon Williams
> Labels: compaction
> Fix For: 1.0.0
>
> Attachments: 3308.txt
>
>
> In CASSANDRA-3104, this was removed with the following reasoning:
> bq. compaction_throughput_mb_per_sec is a more effective throttle on compaction.
> This turns out to be false in the majority of deployments. In many (if not most) situations, compaction is actually CPU bound, not IO bound, so multithreaded compaction is generally helpful, but the priority needs to be lowered in order to prevent it from stealing CPU used for reads/writes.
> Compaction is always CPU bound on both real hardware (sw raid0 with two SATA disks) and on a rackspace cloud server (though my understanding is they are back by a raid10 array underneath) however I suspect even a single drive is fast enough to handle the ~20MB/s that compaction is currently performing when unthrottled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3308) Add compaction_thread_priority
back
Posted by "Brandon Williams (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122087#comment-13122087 ]
Brandon Williams commented on CASSANDRA-3308:
---------------------------------------------
+1
> Add compaction_thread_priority back
> -----------------------------------
>
> Key: CASSANDRA-3308
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3308
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Labels: compaction
> Fix For: 1.0.0
>
> Attachments: 3308.txt
>
>
> In CASSANDRA-3104, this was removed with the following reasoning:
> bq. compaction_throughput_mb_per_sec is a more effective throttle on compaction.
> This turns out to be false in the majority of deployments. In many (if not most) situations, compaction is actually CPU bound, not IO bound, so multithreaded compaction is generally helpful, but the priority needs to be lowered in order to prevent it from stealing CPU used for reads/writes.
> Compaction is always CPU bound on both real hardware (sw raid0 with two SATA disks) and on a rackspace cloud server (though my understanding is they are back by a raid10 array underneath) however I suspect even a single drive is fast enough to handle the ~20MB/s that compaction is currently performing when unthrottled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3308) Add compaction_thread_priority
back
Posted by "Brandon Williams (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120462#comment-13120462 ]
Brandon Williams commented on CASSANDRA-3308:
---------------------------------------------
bq. Even so, I think mbps is a easier tuning handle than thread priority (which is notorious for becoming a no-op depending on scheduler/platform).
Actually, it's not, you have to experiment to know where to set it, and it's hardware-specific. If I have a platform where I can just lower the priority, I'd like to at least have the knob to turn.
> Add compaction_thread_priority back
> -----------------------------------
>
> Key: CASSANDRA-3308
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3308
> Project: Cassandra
> Issue Type: Bug
> Reporter: Brandon Williams
> Fix For: 1.0.0
>
>
> In CASSANDRA-3104, this was removed with the following reasoning:
> bq. compaction_throughput_mb_per_sec is a more effective throttle on compaction.
> This turns out to be false in the majority of deployments. In many (if not most) situations, compaction is actually CPU bound, not IO bound, so multithreaded compaction is generally helpful, but the priority needs to be lowered in order to prevent it from stealing CPU used for reads/writes.
> Compaction is always CPU bound on both real hardware (sw raid0 with two SATA disks) and on a rackspace cloud server (though my understanding is they are back by a raid10 array underneath) however I suspect even a single drive is fast enough to handle the ~20MB/s that compaction is currently performing when unthrottled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3308) Add compaction_thread_priority
back
Posted by "Jonathan Ellis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120460#comment-13120460 ]
Jonathan Ellis commented on CASSANDRA-3308:
-------------------------------------------
bq. compaction is actually CPU bound, not IO bound
Even so, I think mbps is a easier tuning handle than thread priority (which is notorious for becoming a no-op depending on scheduler/platform).
> Add compaction_thread_priority back
> -----------------------------------
>
> Key: CASSANDRA-3308
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3308
> Project: Cassandra
> Issue Type: Bug
> Reporter: Brandon Williams
> Fix For: 1.0.0
>
>
> In CASSANDRA-3104, this was removed with the following reasoning:
> bq. compaction_throughput_mb_per_sec is a more effective throttle on compaction.
> This turns out to be false in the majority of deployments. In many (if not most) situations, compaction is actually CPU bound, not IO bound, so multithreaded compaction is generally helpful, but the priority needs to be lowered in order to prevent it from stealing CPU used for reads/writes.
> Compaction is always CPU bound on both real hardware (sw raid0 with two SATA disks) and on a rackspace cloud server (though my understanding is they are back by a raid10 array underneath) however I suspect even a single drive is fast enough to handle the ~20MB/s that compaction is currently performing when unthrottled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira