You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Yang Yang (Created) (JIRA)" <ji...@apache.org> on 2011/09/30 22:26:45 UTC
[jira] [Created] (CASSANDRA-3287) limit row cache size as a
proportion of sstable size
limit row cache size as a proportion of sstable size
----------------------------------------------------
Key: CASSANDRA-3287
URL: https://issues.apache.org/jira/browse/CASSANDRA-3287
Project: Cassandra
Issue Type: Bug
Reporter: Yang Yang
Priority: Minor
I was debugging some problem,
then I found that my memory size increased extremely fast,
and finally reached my old gen total, and then CMS was starting all the time, back-to-back.
even after I force a full GC, the old gen space is not decreasing any.
so I invoked invalidate cache from jmx, then did GC, now the old gen size went down.
so it's due to row cache.
but my row cache is really not that big.
then I realized that it's due to too many small sstables. each sstable has the same size row cache. then for the smaller sstables, it's too favorable: they have a higher retention ratio, while in fact we can assume that the hit ratio should be the same for all sstables, particularly if you have a random access pattern.
if this makes sense, I can put in a simple patch on calling the constructor of the rowcache
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3287) limit row cache size as a
proportion of sstable size
Posted by "Jonathan Ellis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118414#comment-13118414 ]
Jonathan Ellis commented on CASSANDRA-3287:
-------------------------------------------
Key cache and row cache are both per-CF, not per-SSTable.
We plan to make the caches global in CASSANDRA-3143.
> limit row cache size as a proportion of sstable size
> ----------------------------------------------------
>
> Key: CASSANDRA-3287
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3287
> Project: Cassandra
> Issue Type: Bug
> Reporter: Yang Yang
> Priority: Minor
>
> I was debugging some problem,
> then I found that my memory size increased extremely fast,
> and finally reached my old gen total, and then CMS was starting all the time, back-to-back.
> even after I force a full GC, the old gen space is not decreasing any.
> so I invoked invalidate cache from jmx, then did GC, now the old gen size went down.
> so it's due to row cache.
> but my row cache is really not that big.
> then I realized that it's due to too many small sstables. each sstable has the same size row cache. then for the smaller sstables, it's too favorable: they have a higher retention ratio, while in fact we can assume that the hit ratio should be the same for all sstables, particularly if you have a random access pattern.
> if this makes sense, I can put in a simple patch on calling the constructor of the rowcache
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3287) limit row cache size as a
proportion of sstable size
Posted by "Yang Yang (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yang Yang resolved CASSANDRA-3287.
----------------------------------
Resolution: Invalid
> limit row cache size as a proportion of sstable size
> ----------------------------------------------------
>
> Key: CASSANDRA-3287
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3287
> Project: Cassandra
> Issue Type: Bug
> Reporter: Yang Yang
> Priority: Minor
>
> I was debugging some problem,
> then I found that my memory size increased extremely fast,
> and finally reached my old gen total, and then CMS was starting all the time, back-to-back.
> even after I force a full GC, the old gen space is not decreasing any.
> so I invoked invalidate cache from jmx, then did GC, now the old gen size went down.
> so it's due to row cache.
> but my row cache is really not that big.
> then I realized that it's due to too many small sstables. each sstable has the same size row cache. then for the smaller sstables, it's too favorable: they have a higher retention ratio, while in fact we can assume that the hit ratio should be the same for all sstables, particularly if you have a random access pattern.
> if this makes sense, I can put in a simple patch on calling the constructor of the rowcache
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3287) limit row cache size as a
proportion of sstable size
Posted by "Yang Yang (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118399#comment-13118399 ]
Yang Yang commented on CASSANDRA-3287:
--------------------------------------
also a cap on the total size of all row caches would also be helpful
> limit row cache size as a proportion of sstable size
> ----------------------------------------------------
>
> Key: CASSANDRA-3287
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3287
> Project: Cassandra
> Issue Type: Bug
> Reporter: Yang Yang
> Priority: Minor
>
> I was debugging some problem,
> then I found that my memory size increased extremely fast,
> and finally reached my old gen total, and then CMS was starting all the time, back-to-back.
> even after I force a full GC, the old gen space is not decreasing any.
> so I invoked invalidate cache from jmx, then did GC, now the old gen size went down.
> so it's due to row cache.
> but my row cache is really not that big.
> then I realized that it's due to too many small sstables. each sstable has the same size row cache. then for the smaller sstables, it's too favorable: they have a higher retention ratio, while in fact we can assume that the hit ratio should be the same for all sstables, particularly if you have a random access pattern.
> if this makes sense, I can put in a simple patch on calling the constructor of the rowcache
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira