You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jeremy Hanna (JIRA)" <ji...@apache.org> on 2010/08/31 00:15:54 UTC
[jira] Created: (CASSANDRA-1445) Make keys/rows cached more safe
Make keys/rows cached more safe
-------------------------------
Key: CASSANDRA-1445
URL: https://issues.apache.org/jira/browse/CASSANDRA-1445
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Jeremy Hanna
Priority: Minor
Currently when a user sets keys/rows cached for their column families, they do at their own risk. If you cache too much information, you will get an OOM error on the server and crash the server.
This improvement would be to make that configuration safer.
In order to accomplish this, we could add a configuration option for size of cache (for either). The server would be able to see how much data is expected to be cached based on column family settings and make a best effort to cache it. However, it would not exceed that maximum value for the node (by some form of approximation). Instead it would weight the different values for caching and satisfy the expectation as best it can. So it would potentially cache less than was requested, but at least it would not crash the server.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1445) Make keys/rows cached more safe
Posted by "Jeremy Hanna (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904832#action_12904832 ]
Jeremy Hanna commented on CASSANDRA-1445:
-----------------------------------------
seems like we have a good bit of information about sizing already. for example, for each sstable, we have estimates for row sizes in the EstimatedHistogram objects.
I wonder if we would be able to estimate how much memory could be dedicated to rowcache. Then calculate what the requested rowcache is for the various column families along with how much memory it would take up. Then it could override the rowcache capacity value for those sstables. The system could could take into account the requested row cache number for that column family as well as the estimated row width in order to calculate the adjusted rowcache capacity values.
This calculation could update the rowcache capacity at startup and periodically.
> Make keys/rows cached more safe
> -------------------------------
>
> Key: CASSANDRA-1445
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1445
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jeremy Hanna
> Assignee: Jeremy Hanna
> Priority: Minor
>
> Currently when a user sets keys/rows cached for their column families, they do at their own risk. If you cache too much information, you will get an OOM error on the server and crash the server.
> This improvement would be to make that configuration safer.
> In order to accomplish this, we could add a configuration option for size of cache (for either). The server would be able to see how much data is expected to be cached based on column family settings and make a best effort to cache it. However, it would not exceed that maximum value for the node (by some form of approximation). Instead it would weight the different values for caching and satisfy the expectation as best it can. So it would potentially cache less than was requested, but at least it would not crash the server.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CASSANDRA-1445) Make keys/rows cached more safe
Posted by "Jeremy Hanna (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeremy Hanna reassigned CASSANDRA-1445:
---------------------------------------
Assignee: Jeremy Hanna
> Make keys/rows cached more safe
> -------------------------------
>
> Key: CASSANDRA-1445
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1445
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jeremy Hanna
> Assignee: Jeremy Hanna
> Priority: Minor
>
> Currently when a user sets keys/rows cached for their column families, they do at their own risk. If you cache too much information, you will get an OOM error on the server and crash the server.
> This improvement would be to make that configuration safer.
> In order to accomplish this, we could add a configuration option for size of cache (for either). The server would be able to see how much data is expected to be cached based on column family settings and make a best effort to cache it. However, it would not exceed that maximum value for the node (by some form of approximation). Instead it would weight the different values for caching and satisfy the expectation as best it can. So it would potentially cache less than was requested, but at least it would not crash the server.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1445) Make keys/rows cached more safe
Posted by "Jeremy Hanna (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12905955#action_12905955 ]
Jeremy Hanna commented on CASSANDRA-1445:
-----------------------------------------
After talking with Matt Dennis, it sounds like the estimated business might be too hand wavy and inaccurate. Also Matt mentioned that there are cache settings on RDBMS systems that basically have the same issue - used incorrectly, bad things can happen.
This ticket was meant as a precursor for safe management of the cache memory, like for a multi-tenant situation where there can be several users competing for memory. If that's the case though, it may be that memory is better spent with only having key cache enabled. Even then, it would probably be necessary to have a setting for a global override limit for number of keys cached for a particular column family or maybe the keyspace level (the total of the column families' keys cached.
I think that would remove flexibility but give more stability for a situation with potentially a lot of column families where you want to have some more isolation (for multi-tenant).
> Make keys/rows cached more safe
> -------------------------------
>
> Key: CASSANDRA-1445
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1445
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jeremy Hanna
> Assignee: Jeremy Hanna
> Priority: Minor
>
> Currently when a user sets keys/rows cached for their column families, they do at their own risk. If you cache too much information, you will get an OOM error on the server and crash the server.
> This improvement would be to make that configuration safer.
> In order to accomplish this, we could add a configuration option for size of cache (for either). The server would be able to see how much data is expected to be cached based on column family settings and make a best effort to cache it. However, it would not exceed that maximum value for the node (by some form of approximation). Instead it would weight the different values for caching and satisfy the expectation as best it can. So it would potentially cache less than was requested, but at least it would not crash the server.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CASSANDRA-1445) Make keys/rows cached more safe
Posted by "Jeremy Hanna (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeremy Hanna resolved CASSANDRA-1445.
-------------------------------------
Resolution: Won't Fix
Created CASSANDRA-1461 to specify a plan for certain situations.
> Make keys/rows cached more safe
> -------------------------------
>
> Key: CASSANDRA-1445
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1445
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jeremy Hanna
> Assignee: Jeremy Hanna
> Priority: Minor
>
> Currently when a user sets keys/rows cached for their column families, they do at their own risk. If you cache too much information, you will get an OOM error on the server and crash the server.
> This improvement would be to make that configuration safer.
> In order to accomplish this, we could add a configuration option for size of cache (for either). The server would be able to see how much data is expected to be cached based on column family settings and make a best effort to cache it. However, it would not exceed that maximum value for the node (by some form of approximation). Instead it would weight the different values for caching and satisfy the expectation as best it can. So it would potentially cache less than was requested, but at least it would not crash the server.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.