You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Matthew F. Dennis (JIRA)" <ji...@apache.org> on 2011/02/02 23:43:29 UTC
[jira] Created: (CASSANDRA-2102) saved row cache doesn't save the
cache
saved row cache doesn't save the cache
--------------------------------------
Key: CASSANDRA-2102
URL: https://issues.apache.org/jira/browse/CASSANDRA-2102
Project: Cassandra
Issue Type: Bug
Reporter: Matthew F. Dennis
Assignee: Matthew F. Dennis
saving row caches works by periodically iterating of the keySet() on the caches and writing the keys for the cached contents to disk. The cache keys are DecoratedKeys. DecoratedKeys contain a Token token and a ByteBuffer key. The underlying buffer on the key gets reused so the contents change. This means that all the cache entries have distinct tokens but only a handful of distinct key values. This means that when the cache is loaded you only end up loading a handful of keys instead of the ones actually in your cache.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2102) saved row cache doesn't save the
cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990276#comment-12990276 ]
Jonathan Ellis commented on CASSANDRA-2102:
-------------------------------------------
Let's clone in storageproxy since it's not necessary for keys read over MessagingService
> saved row cache doesn't save the cache
> --------------------------------------
>
> Key: CASSANDRA-2102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2102
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Matthew F. Dennis
> Assignee: Matthew F. Dennis
> Fix For: 0.7.2
>
> Attachments: 2102-cassandra-0.7-v2.txt, 2102-cassandra-0.7.txt
>
>
> saving row caches works by periodically iterating of the keySet() on the caches and writing the keys for the cached contents to disk. The cache keys are DecoratedKeys. DecoratedKeys contain a Token token and a ByteBuffer key. The underlying buffer on the key gets reused so the contents change. This means that all the cache entries have distinct tokens but only a handful of distinct key values. This means that when the cache is loaded you only end up loading a handful of keys instead of the ones actually in your cache.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (CASSANDRA-2102) saved row cache doesn't save the
cache
Posted by "Matthew F. Dennis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthew F. Dennis reopened CASSANDRA-2102:
------------------------------------------
Assignee: Matthew F. Dennis
not really a duplicate.
This deals with writing the cache correctly.
CASSANDRA-2076 should better handle in general when a cache file is corrupt.
> saved row cache doesn't save the cache
> --------------------------------------
>
> Key: CASSANDRA-2102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2102
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Matthew F. Dennis
> Assignee: Matthew F. Dennis
> Fix For: 0.7.2
>
> Attachments: 2102-cassandra-0.7.txt
>
>
> saving row caches works by periodically iterating of the keySet() on the caches and writing the keys for the cached contents to disk. The cache keys are DecoratedKeys. DecoratedKeys contain a Token token and a ByteBuffer key. The underlying buffer on the key gets reused so the contents change. This means that all the cache entries have distinct tokens but only a handful of distinct key values. This means that when the cache is loaded you only end up loading a handful of keys instead of the ones actually in your cache.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2102) saved row cache doesn't save the
cache
Posted by "Matthew F. Dennis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthew F. Dennis updated CASSANDRA-2102:
-----------------------------------------
Affects Version/s: 0.7.0
Fix Version/s: 0.7.2
> saved row cache doesn't save the cache
> --------------------------------------
>
> Key: CASSANDRA-2102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2102
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Matthew F. Dennis
> Assignee: Matthew F. Dennis
> Fix For: 0.7.2
>
>
> saving row caches works by periodically iterating of the keySet() on the caches and writing the keys for the cached contents to disk. The cache keys are DecoratedKeys. DecoratedKeys contain a Token token and a ByteBuffer key. The underlying buffer on the key gets reused so the contents change. This means that all the cache entries have distinct tokens but only a handful of distinct key values. This means that when the cache is loaded you only end up loading a handful of keys instead of the ones actually in your cache.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2102) saved row cache doesn't save the
cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12993114#comment-12993114 ]
Jonathan Ellis commented on CASSANDRA-2102:
-------------------------------------------
I think you're right, if your cache is doing its job (high hit rate) then copy-before-insert will be less copies than copy-for-local-operation.
> saved row cache doesn't save the cache
> --------------------------------------
>
> Key: CASSANDRA-2102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2102
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Matthew F. Dennis
> Assignee: Matthew F. Dennis
> Fix For: 0.7.2
>
> Attachments: 2102-cassandra-0.7-v2.txt, 2102-cassandra-0.7.txt
>
>
> saving row caches works by periodically iterating of the keySet() on the caches and writing the keys for the cached contents to disk. The cache keys are DecoratedKeys. DecoratedKeys contain a Token token and a ByteBuffer key. The underlying buffer on the key gets reused so the contents change. This means that all the cache entries have distinct tokens but only a handful of distinct key values. This means that when the cache is loaded you only end up loading a handful of keys instead of the ones actually in your cache.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2102) saved row cache doesn't save the
cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2102:
--------------------------------------
Remaining Estimate: 4m
Original Estimate: 4m
> saved row cache doesn't save the cache
> --------------------------------------
>
> Key: CASSANDRA-2102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2102
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Matthew F. Dennis
> Assignee: Matthew F. Dennis
> Fix For: 0.7.1
>
> Attachments: 2102-cassandra-0.7-v2.txt, 2102-cassandra-0.7.txt
>
> Original Estimate: 4m
> Remaining Estimate: 4m
>
> saving row caches works by periodically iterating of the keySet() on the caches and writing the keys for the cached contents to disk. The cache keys are DecoratedKeys. DecoratedKeys contain a Token token and a ByteBuffer key. The underlying buffer on the key gets reused so the contents change. This means that all the cache entries have distinct tokens but only a handful of distinct key values. This means that when the cache is loaded you only end up loading a handful of keys instead of the ones actually in your cache.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2102) saved row cache doesn't save the
cache
Posted by "Matthew F. Dennis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthew F. Dennis updated CASSANDRA-2102:
-----------------------------------------
Attachment: 2102-cassandra-0.7-v2.txt
per IRC discussion v2 patch just copies the key when inserting into the cache.
Since CASSANDRA-1034 looks likely in the somewhat near future, it's pointless to make the changes in the original patch.
> saved row cache doesn't save the cache
> --------------------------------------
>
> Key: CASSANDRA-2102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2102
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Matthew F. Dennis
> Assignee: Matthew F. Dennis
> Fix For: 0.7.2
>
> Attachments: 2102-cassandra-0.7-v2.txt, 2102-cassandra-0.7.txt
>
>
> saving row caches works by periodically iterating of the keySet() on the caches and writing the keys for the cached contents to disk. The cache keys are DecoratedKeys. DecoratedKeys contain a Token token and a ByteBuffer key. The underlying buffer on the key gets reused so the contents change. This means that all the cache entries have distinct tokens but only a handful of distinct key values. This means that when the cache is loaded you only end up loading a handful of keys instead of the ones actually in your cache.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2102) saved row cache doesn't save the
cache
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12993238#comment-12993238 ]
Hudson commented on CASSANDRA-2102:
-----------------------------------
Integrated in Cassandra-0.7 #276 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/276/])
> saved row cache doesn't save the cache
> --------------------------------------
>
> Key: CASSANDRA-2102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2102
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Matthew F. Dennis
> Assignee: Matthew F. Dennis
> Fix For: 0.7.1
>
> Attachments: 2102-cassandra-0.7-v2.txt, 2102-cassandra-0.7.txt
>
>
> saving row caches works by periodically iterating of the keySet() on the caches and writing the keys for the cached contents to disk. The cache keys are DecoratedKeys. DecoratedKeys contain a Token token and a ByteBuffer key. The underlying buffer on the key gets reused so the contents change. This means that all the cache entries have distinct tokens but only a handful of distinct key values. This means that when the cache is loaded you only end up loading a handful of keys instead of the ones actually in your cache.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2102) saved row cache doesn't save the
cache
Posted by "Matthew F. Dennis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992241#comment-12992241 ]
Matthew F. Dennis commented on CASSANDRA-2102:
----------------------------------------------
I really think safety is the way to go here (e.g. clone right before we put it into the cache). Not to mention cloning early would be generating memcopies for things we don't need to if they never actually end up in the cache (e.g. things already in the cache, rejected by bloom filters).
> saved row cache doesn't save the cache
> --------------------------------------
>
> Key: CASSANDRA-2102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2102
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Matthew F. Dennis
> Assignee: Matthew F. Dennis
> Fix For: 0.7.2
>
> Attachments: 2102-cassandra-0.7-v2.txt, 2102-cassandra-0.7.txt
>
>
> saving row caches works by periodically iterating of the keySet() on the caches and writing the keys for the cached contents to disk. The cache keys are DecoratedKeys. DecoratedKeys contain a Token token and a ByteBuffer key. The underlying buffer on the key gets reused so the contents change. This means that all the cache entries have distinct tokens but only a handful of distinct key values. This means that when the cache is loaded you only end up loading a handful of keys instead of the ones actually in your cache.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2102) saved row cache doesn't save the
cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989857#comment-12989857 ]
Jonathan Ellis commented on CASSANDRA-2102:
-------------------------------------------
bq. The underlying buffer on the key gets reused
Where does this reuse happen? RowMutation.deepCopy should be taking care of it on the local side, and sent-over-the-network buffers are not reused.
> saved row cache doesn't save the cache
> --------------------------------------
>
> Key: CASSANDRA-2102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2102
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Matthew F. Dennis
> Assignee: Matthew F. Dennis
> Fix For: 0.7.2
>
> Attachments: 2102-cassandra-0.7.txt
>
>
> saving row caches works by periodically iterating of the keySet() on the caches and writing the keys for the cached contents to disk. The cache keys are DecoratedKeys. DecoratedKeys contain a Token token and a ByteBuffer key. The underlying buffer on the key gets reused so the contents change. This means that all the cache entries have distinct tokens but only a handful of distinct key values. This means that when the cache is loaded you only end up loading a handful of keys instead of the ones actually in your cache.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2102) saved row cache doesn't save the
cache
Posted by "Matthew F. Dennis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthew F. Dennis updated CASSANDRA-2102:
-----------------------------------------
Attachment: 2102-cassandra-0.7.txt
attached patch switches to saving tokens to disk instead of keys, removes DecoratedKey.key from the entries in caches (fixing the bug and reducing memory consumption) and versions (kind of) the saved cache files.
> saved row cache doesn't save the cache
> --------------------------------------
>
> Key: CASSANDRA-2102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2102
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Matthew F. Dennis
> Assignee: Matthew F. Dennis
> Fix For: 0.7.2
>
> Attachments: 2102-cassandra-0.7.txt
>
>
> saving row caches works by periodically iterating of the keySet() on the caches and writing the keys for the cached contents to disk. The cache keys are DecoratedKeys. DecoratedKeys contain a Token token and a ByteBuffer key. The underlying buffer on the key gets reused so the contents change. This means that all the cache entries have distinct tokens but only a handful of distinct key values. This means that when the cache is loaded you only end up loading a handful of keys instead of the ones actually in your cache.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira