[jira] [Comment Edited] (CASSANDRA-4047) Bulk hinting

["Brandon Williams (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-4047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13817758#comment-13817758 ] 

Brandon Williams edited comment on CASSANDRA-4047 at 11/8/13 10:28 PM:
-----------------------------------------------------------------------

Attaching what I had here, mostly-rebased against 2.0, before I hit a snag.  It doesn't quite compile against 2.0 because now streaming is totally different, but on 1.2 it would work if you manually inserted the bulk hints with cqlsh.  The snag I hit was actually inserting the hints from the bulk loader.  While it's fairly simple to explain, in that you just need to insert a hint for a failure on any replica that does succeed, I couldn't get the information I needed (ks/cf name for the insert, and range from the filename) out of the streaming callback at the time.

Can you take over, Yuki?


was (Author: brandon.williams):
Attaching what I had here, mostly-rebased against 2.0, before I hit a snag.  It doesn't quite compile against 2.0 because now streaming is totally different, but on 1.2 it would work if you manually inserted the bulk hints with cqlsh.  The snag I hit was actually inserting the hints from the bulk loader.  While it's fairly simple to explain, in that you just need to insert a hint for a failure on any replica that does succeed, I couldn't get the information I needed (ks/cf name for the insert, and range from the filename) out of the streaming callback at the time.

Can you over, Yuki?

> Bulk hinting
> ------------
>
>                 Key: CASSANDRA-4047
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4047
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Brandon Williams
>            Assignee: Yuki Morishita
>             Fix For: 2.0.3
>
>         Attachments: 4047-wip.txt
>
>
> With the introduction of the BulkOutputFormat, there may be cases where someone would like to tolerate node failures and have the job complete, but afterwards since we streamed they have to repair or rely on read repair.  We don't currently have any way of hinting streams, but a node could take a snapshot before acknowledging the stream session, then remember to send the files in the snapshot to the unavailable nodes when they come back up.  This isn't quite ideal since of course the node may have compacted these files, however it's much simpler than any sort of key tracking at this scale.



--
This message was sent by Atlassian JIRA
(v6.1#6144)