[jira] [Comment Edited] (CASSANDRA-10155) 2i key cache load fails

["Ariel Weisberg (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-10155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14726103#comment-14726103 ] 

Ariel Weisberg edited comment on CASSANDRA-10155 at 9/1/15 8:20 PM:
--------------------------------------------------------------------

[Have a 3.0 change for this.|https://github.com/apache/cassandra/compare/trunk...aweisberg:C-10155-3.0?expand=1] I need to make sure existing tests run, and then make sure all the new code is tested.

My code for consuming reads during cache load is probably uncovered and there are dropped table/keyspace cases that need coverage.

I also need to rebase it to 2.1 and then figure out the right way to merge forward.

* There is now only one saved cache file for each kind of key (row, key, counter)
* This means that loading of each kind of cache is single threaded. Previous code loaded caches synchronously at startup so it was probably single threaded anyways.
* Loading is async in its own thread for each cache so key and row cache can load at the same time
* Caches could load concurrently with other startup activities, but there didn't seem to be a desire to have that happen so startup blocks until they are loaded
* Cache keys now contain the keyspace and CF name instead of UUID. There is no footprint change for on heap caches because they are POJOs pointing to a canonical object. Off heap will potentially see an increase in size and time since it will have to store and decode two strings instead of a UUID.
* Schema now has a function that will look up a CF, even if it is an index, correctly.
* Cache loading used to collect a future per cache entry in a list! For an off heap cache that is larger than the Java heap this is a problem. Modified to bound the number of pending reads and consume read results by loading them into the cache as they come in.
* Promoted errors in loading caches to info level that were debug.



was (Author: aweisberg):
[Have a 3.0 change for this.|https://github.com/apache/cassandra/compare/trunk...aweisberg:C-10155-3.0?expand=1] I need to make sure existing tests run, and then make sure all the new code is tested.

I need to rebase it to 2.1 and then figure out the right way to get to merge forward.

* There is now only one saved cache file for each kind of key (row, key, counter)
* This means that loading of each kind of cache is single threaded. Previous code loaded caches synchronously at startup so it was probably single threaded anyways.
* Loading is async in its own thread for each cache so key and row cache can load at the same time
* Caches could load concurrently with other startup activities, but there didn't seem to be a desire to have that happen so startup blocks until they are loaded
* Cache keys now contain the keyspace and CF name instead of UUID. There is no footprint change for on heap caches because they are POJOs pointing to a canonical object. Off heap will potentially see an increase in size and time since it will have to store and decode two strings instead of a UUID.
* Schema now has a function that will look up a CF, even if it is an index, correctly.
* Cache loading used to collect a future per cache entry in a list! For an off heap cache that is larger than the Java heap this is a problem. Modified to bound the number of pending reads and consume read results by loading them into the cache as they come in.
* Promoted errors in loading caches to info level that were debug.


> 2i key cache load fails
> -----------------------
>
>                 Key: CASSANDRA-10155
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10155
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Robert Stupp
>            Assignee: Ariel Weisberg
>             Fix For: 3.0.0 rc1, 2.1.10, 2.2.2
>
>
> CASSANDRA-9265 changed how key cache content is serialized to disk. It uses {{UUID cfId}} to generate the file path for each {{ColumnFamilyStore}}.
> Since {{cfId}} of a secondary index is the same as for the base table, the key-cache files for 2i's and the base are the same. This will/may lead to deserialization failures on restart for tables with at least one 2i.
> /cc [~aweisberg] [~danchia]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)