[jira] [Commented] (CASSANDRA-10990) Support streaming of older version sstables in 3.0

["Sylvain Lebresne (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-10990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110707#comment-15110707 ] 

Sylvain Lebresne commented on CASSANDRA-10990:
----------------------------------------------

The problem will be compact tables. With them, old nodes have their cells for declared columns intermingled with other cells and we need to get all the former ones first for the new format (as they become static columns). When reading old sstables, we end up having to sometimes double read partitions during compaction (thankfully it's only for some tables and it's a one time thing) and do some tricks for reads (which are too long to explain here but one can look at {{ThriftResultsMerger}} if interested).

Anyway, we can't easily do this for streaming. The only viable solution I can see would be to first write all the "dynamic" cells to some temporary location collecting the "static" ones while doing it, then writing the static ones and then copying back the dynamic ones. Doable, though it's getting complicated.

I also want to note that streaming sstables is in itself useless. What we care is the features that use streaming, i.e. bootstrap, moving and repair. And at least for repair, this won't work out of the box because 3.0 don't compute the same digest than 2.X nodes. Now, 3.0 nodes are actually able to compute the 2.X ones (which we use for digest queries), but we need to add the code so they pick the right version to compute. As for bootstrap/moving, they "should" work out of the box in mixed cluster (providing streaming works) but we've changed enough in 3.0, including with schemas, that we need a good battery of tests before claiming it does.

So anyway, we can likely solve all those problems, but it's worth noting that this ticket is probably a rather big one if you include the testing necessary (and I think it's better to say "sorry but you can't do streaming based operations during upgrade" than pretending it works but having it break easily in practice).


> Support streaming of older version sstables in 3.0
> --------------------------------------------------
>
>                 Key: CASSANDRA-10990
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10990
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Streaming and Messaging
>            Reporter: Jeremy Hanna
>            Assignee: Paulo Motta
>
> In 2.0 we introduced support for streaming older versioned sstables (CASSANDRA-5772).  In 3.0, because of the rewrite of the storage layer, this became no longer supported.  So currently, while 3.0 can read sstables in the 2.1/2.2 format, it cannot stream the older versioned sstables.  We should do some work to make this still possible to be consistent with what CASSANDRA-5772 provided.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)