You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Nick Bailey (JIRA)" <ji...@apache.org> on 2011/01/25 20:16:45 UTC
[jira] Created: (CASSANDRA-2053) Make cache saving less contentious
Make cache saving less contentious
----------------------------------
Key: CASSANDRA-2053
URL: https://issues.apache.org/jira/browse/CASSANDRA-2053
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.0
Reporter: Nick Bailey
Fix For: 0.7.2
The current default for saving key caches is every hour. Additionally the default timeout for flushing memtables is every hour. I've seen situations where both of these occuring at the same time every hour causes enough pressure on the node to have it drop messages and other nodes mark it dead. This happens across the cluster and results in flapping.
We should do something to spread this out. Perhaps staggering cache saves/flushes that occur due to timeouts.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-2053) Make cache saving less contentious
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2053:
--------------------------------------
Attachment: 2053.txt
Moves CacheWriter to a top-level class and runs it on the CompactionManager executor. Reads and writes udpated to use BRAF cache-non-polluting mode.
> Make cache saving less contentious
> ----------------------------------
>
> Key: CASSANDRA-2053
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2053
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.0
> Reporter: Nick Bailey
> Fix For: 0.7.2
>
> Attachments: 2053.txt
>
>
> The current default for saving key caches is every hour. Additionally the default timeout for flushing memtables is every hour. I've seen situations where both of these occuring at the same time every hour causes enough pressure on the node to have it drop messages and other nodes mark it dead. This happens across the cluster and results in flapping.
> We should do something to spread this out. Perhaps staggering cache saves/flushes that occur due to timeouts.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2053) Make cache saving less contentious
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2053:
--------------------------------------
Remaining Estimate: 4h
Original Estimate: 4h
> Make cache saving less contentious
> ----------------------------------
>
> Key: CASSANDRA-2053
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2053
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.0
> Reporter: Nick Bailey
> Assignee: Jonathan Ellis
> Fix For: 0.7.1
>
> Attachments: 2053.txt
>
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> The current default for saving key caches is every hour. Additionally the default timeout for flushing memtables is every hour. I've seen situations where both of these occuring at the same time every hour causes enough pressure on the node to have it drop messages and other nodes mark it dead. This happens across the cluster and results in flapping.
> We should do something to spread this out. Perhaps staggering cache saves/flushes that occur due to timeouts.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2053) Make cache saving less
contentious
Posted by "Nick Bailey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12991049#comment-12991049 ]
Nick Bailey commented on CASSANDRA-2053:
----------------------------------------
+1
> Make cache saving less contentious
> ----------------------------------
>
> Key: CASSANDRA-2053
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2053
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.0
> Reporter: Nick Bailey
> Assignee: Jonathan Ellis
> Fix For: 0.7.2
>
> Attachments: 2053.txt
>
>
> The current default for saving key caches is every hour. Additionally the default timeout for flushing memtables is every hour. I've seen situations where both of these occuring at the same time every hour causes enough pressure on the node to have it drop messages and other nodes mark it dead. This happens across the cluster and results in flapping.
> We should do something to spread this out. Perhaps staggering cache saves/flushes that occur due to timeouts.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2053) Make cache saving less
contentious
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12991070#comment-12991070 ]
Hudson commented on CASSANDRA-2053:
-----------------------------------
Integrated in Cassandra-0.7 #251 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/251/])
cache writing moved to CompactionManager to reduce i/o contention and updated to use non-cache-polluting writes
patch by jbellis; reviewed by nickmbailey for CASSANDRA-2053
> Make cache saving less contentious
> ----------------------------------
>
> Key: CASSANDRA-2053
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2053
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.0
> Reporter: Nick Bailey
> Assignee: Jonathan Ellis
> Fix For: 0.7.2
>
> Attachments: 2053.txt
>
>
> The current default for saving key caches is every hour. Additionally the default timeout for flushing memtables is every hour. I've seen situations where both of these occuring at the same time every hour causes enough pressure on the node to have it drop messages and other nodes mark it dead. This happens across the cluster and results in flapping.
> We should do something to spread this out. Perhaps staggering cache saves/flushes that occur due to timeouts.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira