You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Daniel Doubleday (Created) (JIRA)" <ji...@apache.org> on 2012/03/20 09:39:45 UTC
[jira] [Created] (CASSANDRA-4065) Bogus MemoryMeter liveRatio
calculations
Bogus MemoryMeter liveRatio calculations
----------------------------------------
Key: CASSANDRA-4065
URL: https://issues.apache.org/jira/browse/CASSANDRA-4065
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0.8
Reporter: Daniel Doubleday
Priority: Minor
I get strange cfs.liveRatios.
A couple of mem meter runs seem to calculate bogus results:
{noformat}
Tue 09:14:48 dd@blnrzh045:~$ grep 'setting live ratio to maximum of 64 instead of' /var/log/cassandra/system.log
WARN [MemoryMeter:1] 2012-03-20 08:08:07,253 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
WARN [MemoryMeter:1] 2012-03-20 08:08:09,160 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
WARN [MemoryMeter:1] 2012-03-20 08:08:13,274 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
WARN [MemoryMeter:1] 2012-03-20 08:08:22,032 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
WARN [MemoryMeter:1] 2012-03-20 08:12:41,057 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 67.11787351054079
WARN [MemoryMeter:1] 2012-03-20 08:13:50,877 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 112.58547951925435
WARN [MemoryMeter:1] 2012-03-20 08:15:29,021 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 193.36945063589877
WARN [MemoryMeter:1] 2012-03-20 08:17:50,716 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 348.45008340969434
{noformat}
Because meter runs never decrease liveRatio in Memtable (Which seems strange to me. If past calcs should be included for any reason wouldn't averaging make more sense?):
{noformat}
cfs.liveRatio = Math.max(cfs.liveRatio, newRatio);
{noformat}
Memtables are flushed every couple of secs:
{noformat}
ColumnFamilyStore.java (line 712) Enqueuing flush of Memtable-BlobStore@935814661(1874540/149963200 serialized/live bytes, 202 ops)
{noformat}
Even though a saner liveRatio has been calculated after the bogus runs:
{noformat}
INFO [MemoryMeter:1] 2012-03-20 08:19:55,934 Memtable.java (line 198) CFS(Keyspace='SmeetBlob', ColumnFamily='BlobStore')
liveRatio is 64.0 (just-counted was 2.97165811895841). calculation took 124ms for 58 columns
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4065) Bogus MemoryMeter liveRatio
calculations
Posted by "Jonathan Ellis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233504#comment-13233504 ]
Jonathan Ellis commented on CASSANDRA-4065:
-------------------------------------------
I wonder if we could use some kind of constantly-updated estimate instead of scanning the whole memtable periodically...
> Bogus MemoryMeter liveRatio calculations
> ----------------------------------------
>
> Key: CASSANDRA-4065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4065
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.8
> Reporter: Daniel Doubleday
> Priority: Minor
>
> I get strange cfs.liveRatios.
> A couple of mem meter runs seem to calculate bogus results:
> {noformat}
> Tue 09:14:48 dd@blnrzh045:~$ grep 'setting live ratio to maximum of 64 instead of' /var/log/cassandra/system.log
> WARN [MemoryMeter:1] 2012-03-20 08:08:07,253 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:09,160 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:13,274 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:22,032 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:12:41,057 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 67.11787351054079
> WARN [MemoryMeter:1] 2012-03-20 08:13:50,877 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 112.58547951925435
> WARN [MemoryMeter:1] 2012-03-20 08:15:29,021 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 193.36945063589877
> WARN [MemoryMeter:1] 2012-03-20 08:17:50,716 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 348.45008340969434
> {noformat}
> Because meter runs never decrease liveRatio in Memtable (Which seems strange to me. If past calcs should be included for any reason wouldn't averaging make more sense?):
> {noformat}
> cfs.liveRatio = Math.max(cfs.liveRatio, newRatio);
> {noformat}
> Memtables are flushed every couple of secs:
> {noformat}
> ColumnFamilyStore.java (line 712) Enqueuing flush of Memtable-BlobStore@935814661(1874540/149963200 serialized/live bytes, 202 ops)
> {noformat}
> Even though a saner liveRatio has been calculated after the bogus runs:
> {noformat}
> INFO [MemoryMeter:1] 2012-03-20 08:19:55,934 Memtable.java (line 198) CFS(Keyspace='SmeetBlob', ColumnFamily='BlobStore')
> liveRatio is 64.0 (just-counted was 2.97165811895841). calculation took 124ms for 58 columns
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4065) Bogus MemoryMeter liveRatio
calculations
Posted by "Jonathan Ellis (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-4065.
---------------------------------------
Resolution: Fixed
sounds reasonable to me. committed, thanks!
> Bogus MemoryMeter liveRatio calculations
> ----------------------------------------
>
> Key: CASSANDRA-4065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4065
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.0
> Reporter: Daniel Doubleday
> Assignee: Daniel Doubleday
> Priority: Minor
> Fix For: 1.1.0
>
>
> I get strange cfs.liveRatios.
> A couple of mem meter runs seem to calculate bogus results:
> {noformat}
> Tue 09:14:48 dd@blnrzh045:~$ grep 'setting live ratio to maximum of 64 instead of' /var/log/cassandra/system.log
> WARN [MemoryMeter:1] 2012-03-20 08:08:07,253 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:09,160 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:13,274 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:22,032 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:12:41,057 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 67.11787351054079
> WARN [MemoryMeter:1] 2012-03-20 08:13:50,877 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 112.58547951925435
> WARN [MemoryMeter:1] 2012-03-20 08:15:29,021 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 193.36945063589877
> WARN [MemoryMeter:1] 2012-03-20 08:17:50,716 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 348.45008340969434
> {noformat}
> Because meter runs never decrease liveRatio in Memtable (Which seems strange to me. If past calcs should be included for any reason wouldn't averaging make more sense?):
> {noformat}
> cfs.liveRatio = Math.max(cfs.liveRatio, newRatio);
> {noformat}
> Memtables are flushed every couple of secs:
> {noformat}
> ColumnFamilyStore.java (line 712) Enqueuing flush of Memtable-BlobStore@935814661(1874540/149963200 serialized/live bytes, 202 ops)
> {noformat}
> Even though a saner liveRatio has been calculated after the bogus runs:
> {noformat}
> INFO [MemoryMeter:1] 2012-03-20 08:19:55,934 Memtable.java (line 198) CFS(Keyspace='SmeetBlob', ColumnFamily='BlobStore')
> liveRatio is 64.0 (just-counted was 2.97165811895841). calculation took 124ms for 58 columns
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4065) Bogus MemoryMeter liveRatio
calculations
Posted by "Jonathan Ellis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233503#comment-13233503 ]
Jonathan Ellis commented on CASSANDRA-4065:
-------------------------------------------
bq. meter runs never decrease liveRatio in Memtable, which seems strange to me. If past calcs should be included for any reason wouldn't averaging make more sense?
I'm open to ways to improve this, but the idea is that the penalty for overestimating how big the memtable is (frequent flushes) is less severe than the penalty for underestimating (running out of memory and dying).
> Bogus MemoryMeter liveRatio calculations
> ----------------------------------------
>
> Key: CASSANDRA-4065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4065
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.8
> Reporter: Daniel Doubleday
> Priority: Minor
>
> I get strange cfs.liveRatios.
> A couple of mem meter runs seem to calculate bogus results:
> {noformat}
> Tue 09:14:48 dd@blnrzh045:~$ grep 'setting live ratio to maximum of 64 instead of' /var/log/cassandra/system.log
> WARN [MemoryMeter:1] 2012-03-20 08:08:07,253 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:09,160 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:13,274 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:22,032 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:12:41,057 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 67.11787351054079
> WARN [MemoryMeter:1] 2012-03-20 08:13:50,877 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 112.58547951925435
> WARN [MemoryMeter:1] 2012-03-20 08:15:29,021 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 193.36945063589877
> WARN [MemoryMeter:1] 2012-03-20 08:17:50,716 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 348.45008340969434
> {noformat}
> Because meter runs never decrease liveRatio in Memtable (Which seems strange to me. If past calcs should be included for any reason wouldn't averaging make more sense?):
> {noformat}
> cfs.liveRatio = Math.max(cfs.liveRatio, newRatio);
> {noformat}
> Memtables are flushed every couple of secs:
> {noformat}
> ColumnFamilyStore.java (line 712) Enqueuing flush of Memtable-BlobStore@935814661(1874540/149963200 serialized/live bytes, 202 ops)
> {noformat}
> Even though a saner liveRatio has been calculated after the bogus runs:
> {noformat}
> INFO [MemoryMeter:1] 2012-03-20 08:19:55,934 Memtable.java (line 198) CFS(Keyspace='SmeetBlob', ColumnFamily='BlobStore')
> liveRatio is 64.0 (just-counted was 2.97165811895841). calculation took 124ms for 58 columns
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4065) Bogus MemoryMeter liveRatio
calculations
Posted by "Jonathan Ellis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233592#comment-13233592 ]
Jonathan Ellis commented on CASSANDRA-4065:
-------------------------------------------
That's probably possible in theory, but there isn't a fixed-size overhead for structures like CLHM.
> Bogus MemoryMeter liveRatio calculations
> ----------------------------------------
>
> Key: CASSANDRA-4065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4065
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.8
> Reporter: Daniel Doubleday
> Priority: Minor
>
> I get strange cfs.liveRatios.
> A couple of mem meter runs seem to calculate bogus results:
> {noformat}
> Tue 09:14:48 dd@blnrzh045:~$ grep 'setting live ratio to maximum of 64 instead of' /var/log/cassandra/system.log
> WARN [MemoryMeter:1] 2012-03-20 08:08:07,253 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:09,160 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:13,274 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:22,032 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:12:41,057 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 67.11787351054079
> WARN [MemoryMeter:1] 2012-03-20 08:13:50,877 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 112.58547951925435
> WARN [MemoryMeter:1] 2012-03-20 08:15:29,021 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 193.36945063589877
> WARN [MemoryMeter:1] 2012-03-20 08:17:50,716 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 348.45008340969434
> {noformat}
> Because meter runs never decrease liveRatio in Memtable (Which seems strange to me. If past calcs should be included for any reason wouldn't averaging make more sense?):
> {noformat}
> cfs.liveRatio = Math.max(cfs.liveRatio, newRatio);
> {noformat}
> Memtables are flushed every couple of secs:
> {noformat}
> ColumnFamilyStore.java (line 712) Enqueuing flush of Memtable-BlobStore@935814661(1874540/149963200 serialized/live bytes, 202 ops)
> {noformat}
> Even though a saner liveRatio has been calculated after the bogus runs:
> {noformat}
> INFO [MemoryMeter:1] 2012-03-20 08:19:55,934 Memtable.java (line 198) CFS(Keyspace='SmeetBlob', ColumnFamily='BlobStore')
> liveRatio is 64.0 (just-counted was 2.97165811895841). calculation took 124ms for 58 columns
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4065) Bogus MemoryMeter liveRatio
calculations
Posted by "Daniel Doubleday (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233560#comment-13233560 ]
Daniel Doubleday commented on CASSANDRA-4065:
---------------------------------------------
Maybe a simple solution would suffice:
Don't use jamm but do some simplified estimation as
estimate_size = $raw_size + $row_count * ROW_OVERHEAD + $column_count * COL_OVERHEAD
Since we know the used data structures ... and to keep it easy use 64b overhead since this will be the usual case anyway.
> Bogus MemoryMeter liveRatio calculations
> ----------------------------------------
>
> Key: CASSANDRA-4065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4065
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.8
> Reporter: Daniel Doubleday
> Priority: Minor
>
> I get strange cfs.liveRatios.
> A couple of mem meter runs seem to calculate bogus results:
> {noformat}
> Tue 09:14:48 dd@blnrzh045:~$ grep 'setting live ratio to maximum of 64 instead of' /var/log/cassandra/system.log
> WARN [MemoryMeter:1] 2012-03-20 08:08:07,253 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:09,160 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:13,274 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:22,032 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:12:41,057 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 67.11787351054079
> WARN [MemoryMeter:1] 2012-03-20 08:13:50,877 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 112.58547951925435
> WARN [MemoryMeter:1] 2012-03-20 08:15:29,021 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 193.36945063589877
> WARN [MemoryMeter:1] 2012-03-20 08:17:50,716 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 348.45008340969434
> {noformat}
> Because meter runs never decrease liveRatio in Memtable (Which seems strange to me. If past calcs should be included for any reason wouldn't averaging make more sense?):
> {noformat}
> cfs.liveRatio = Math.max(cfs.liveRatio, newRatio);
> {noformat}
> Memtables are flushed every couple of secs:
> {noformat}
> ColumnFamilyStore.java (line 712) Enqueuing flush of Memtable-BlobStore@935814661(1874540/149963200 serialized/live bytes, 202 ops)
> {noformat}
> Even though a saner liveRatio has been calculated after the bogus runs:
> {noformat}
> INFO [MemoryMeter:1] 2012-03-20 08:19:55,934 Memtable.java (line 198) CFS(Keyspace='SmeetBlob', ColumnFamily='BlobStore')
> liveRatio is 64.0 (just-counted was 2.97165811895841). calculation took 124ms for 58 columns
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4065) Bogus MemoryMeter liveRatio
calculations
Posted by "Daniel Doubleday (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233847#comment-13233847 ]
Daniel Doubleday commented on CASSANDRA-4065:
---------------------------------------------
bq. That's probably possible in theory, but there isn't a fixed-size overhead for structures like CLHM.
Well yes - I though of an approximation.
But maybe even easier (still conservative but able to heal):
{noformat}
if (newRatio > cfs.liveRatio) {
cfs.liveRatio = newRatio;
} else {
cfs.liveRatio = (cfs.liveRatio + newRatio) / 2.0;
}
{noformat}
> Bogus MemoryMeter liveRatio calculations
> ----------------------------------------
>
> Key: CASSANDRA-4065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4065
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.8
> Reporter: Daniel Doubleday
> Priority: Minor
>
> I get strange cfs.liveRatios.
> A couple of mem meter runs seem to calculate bogus results:
> {noformat}
> Tue 09:14:48 dd@blnrzh045:~$ grep 'setting live ratio to maximum of 64 instead of' /var/log/cassandra/system.log
> WARN [MemoryMeter:1] 2012-03-20 08:08:07,253 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:09,160 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:13,274 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:22,032 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:12:41,057 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 67.11787351054079
> WARN [MemoryMeter:1] 2012-03-20 08:13:50,877 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 112.58547951925435
> WARN [MemoryMeter:1] 2012-03-20 08:15:29,021 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 193.36945063589877
> WARN [MemoryMeter:1] 2012-03-20 08:17:50,716 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 348.45008340969434
> {noformat}
> Because meter runs never decrease liveRatio in Memtable (Which seems strange to me. If past calcs should be included for any reason wouldn't averaging make more sense?):
> {noformat}
> cfs.liveRatio = Math.max(cfs.liveRatio, newRatio);
> {noformat}
> Memtables are flushed every couple of secs:
> {noformat}
> ColumnFamilyStore.java (line 712) Enqueuing flush of Memtable-BlobStore@935814661(1874540/149963200 serialized/live bytes, 202 ops)
> {noformat}
> Even though a saner liveRatio has been calculated after the bogus runs:
> {noformat}
> INFO [MemoryMeter:1] 2012-03-20 08:19:55,934 Memtable.java (line 198) CFS(Keyspace='SmeetBlob', ColumnFamily='BlobStore')
> liveRatio is 64.0 (just-counted was 2.97165811895841). calculation took 124ms for 58 columns
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4065) Bogus MemoryMeter liveRatio
calculations
Posted by "Jonathan Ellis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-4065:
--------------------------------------
Reviewer: jbellis
Affects Version/s: (was: 1.0.8)
0.8.0
Fix Version/s: 1.1.0
Assignee: Daniel Doubleday
> Bogus MemoryMeter liveRatio calculations
> ----------------------------------------
>
> Key: CASSANDRA-4065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4065
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.0
> Reporter: Daniel Doubleday
> Assignee: Daniel Doubleday
> Priority: Minor
> Fix For: 1.1.0
>
>
> I get strange cfs.liveRatios.
> A couple of mem meter runs seem to calculate bogus results:
> {noformat}
> Tue 09:14:48 dd@blnrzh045:~$ grep 'setting live ratio to maximum of 64 instead of' /var/log/cassandra/system.log
> WARN [MemoryMeter:1] 2012-03-20 08:08:07,253 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:09,160 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:13,274 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:08:22,032 Memtable.java (line 193) setting live ratio to maximum of 64 instead of Infinity
> WARN [MemoryMeter:1] 2012-03-20 08:12:41,057 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 67.11787351054079
> WARN [MemoryMeter:1] 2012-03-20 08:13:50,877 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 112.58547951925435
> WARN [MemoryMeter:1] 2012-03-20 08:15:29,021 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 193.36945063589877
> WARN [MemoryMeter:1] 2012-03-20 08:17:50,716 Memtable.java (line 193) setting live ratio to maximum of 64 instead of 348.45008340969434
> {noformat}
> Because meter runs never decrease liveRatio in Memtable (Which seems strange to me. If past calcs should be included for any reason wouldn't averaging make more sense?):
> {noformat}
> cfs.liveRatio = Math.max(cfs.liveRatio, newRatio);
> {noformat}
> Memtables are flushed every couple of secs:
> {noformat}
> ColumnFamilyStore.java (line 712) Enqueuing flush of Memtable-BlobStore@935814661(1874540/149963200 serialized/live bytes, 202 ops)
> {noformat}
> Even though a saner liveRatio has been calculated after the bogus runs:
> {noformat}
> INFO [MemoryMeter:1] 2012-03-20 08:19:55,934 Memtable.java (line 198) CFS(Keyspace='SmeetBlob', ColumnFamily='BlobStore')
> liveRatio is 64.0 (just-counted was 2.97165811895841). calculation took 124ms for 58 columns
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira