You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Stu Hood (JIRA)" <ji...@apache.org> on 2009/10/28 15:58:59 UTC
[jira] Created: (CASSANDRA-520) Implement Range Repairs
Implement Range Repairs
-----------------------
Key: CASSANDRA-520
URL: https://issues.apache.org/jira/browse/CASSANDRA-520
Project: Cassandra
Issue Type: New Feature
Reporter: Stu Hood
Fix For: 0.5
After CASSANDRA-193, the TreeRequest/Response conversation will have generated a full list of Ranges that disagree between nodes. We need an operation that can efficiently batch repair the ranges, similar to what happens during read repair for a single key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-520) Implement Range Repairs
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12773578#action_12773578 ]
Stu Hood commented on CASSANDRA-520:
------------------------------------
One straightforward way to implement the repair would be to perform a range read for each disagreeing range, and letting read repairs handle repairing individual keys. This would require a command similar/identical to RangeCommand except that it would use Tokens as the 'startWith' and 'stopAt' parameters.
> Implement Range Repairs
> -----------------------
>
> Key: CASSANDRA-520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-520
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Stu Hood
> Assignee: Stu Hood
> Fix For: 0.5
>
>
> After CASSANDRA-193, the TreeRequest/Response conversation will have generated a full list of Ranges that disagree between nodes. We need an operation that can efficiently batch repair the ranges, similar to what happens during read repair for a single key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-520) Implement Range Repairs
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stu Hood updated CASSANDRA-520:
-------------------------------
Attachment: 520-bulk-2-repair-strategy-by-fraction.diff
520-bulk-1-use-streaming-api.diff
Rebased 520-bulk to apply on top of the latest edition of 193.
> Implement Range Repairs
> -----------------------
>
> Key: CASSANDRA-520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-520
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Stu Hood
> Assignee: Stu Hood
> Fix For: 0.5
>
> Attachments: 520-bulk-1-use-streaming-api.diff, 520-bulk-1-use-streaming-api.diff, 520-bulk-2-repair-strategy-by-fraction.diff, 520-bulk-2-repair-strategy-by-fraction.diff, 520-selective-1-decorated-keys-for-range-command.diff
>
>
> After CASSANDRA-193, the TreeRequest/Response conversation will have generated a full list of Ranges that disagree between nodes. We need an operation that can efficiently batch repair the ranges, similar to what happens during read repair for a single key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-520) Implement Range Repairs
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12787186#action_12787186 ]
Jonathan Ellis commented on CASSANDRA-520:
------------------------------------------
committed, w/ additional changes to get it to build, and breaking out the cleanup pieces into separate commits.
> Implement Range Repairs
> -----------------------
>
> Key: CASSANDRA-520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-520
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Stu Hood
> Assignee: Stu Hood
> Fix For: 0.5
>
> Attachments: 520-bulk-1-use-streaming-api.diff, 520-bulk-1-use-streaming-api.diff, 520-bulk-1-use-streaming-api.diff, 520-bulk-2-repair-strategy-by-fraction.diff, 520-bulk-2-repair-strategy-by-fraction.diff, 520-bulk-2-repair-strategy-by-fraction.diff
>
>
> After CASSANDRA-193, the TreeRequest/Response conversation will have generated a full list of Ranges that disagree between nodes. We need an operation that can efficiently batch repair the ranges, similar to what happens during read repair for a single key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-520) Implement Range Repairs
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788073#action_12788073 ]
Hudson commented on CASSANDRA-520:
----------------------------------
Integrated in Cassandra #282 (See [http://hudson.zones.apache.org/hudson/job/Cassandra/282/])
> Implement Range Repairs
> -----------------------
>
> Key: CASSANDRA-520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-520
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Stu Hood
> Assignee: Stu Hood
> Fix For: 0.5
>
> Attachments: 520-bulk-1-use-streaming-api.diff, 520-bulk-1-use-streaming-api.diff, 520-bulk-1-use-streaming-api.diff, 520-bulk-2-repair-strategy-by-fraction.diff, 520-bulk-2-repair-strategy-by-fraction.diff, 520-bulk-2-repair-strategy-by-fraction.diff
>
>
> After CASSANDRA-193, the TreeRequest/Response conversation will have generated a full list of Ranges that disagree between nodes. We need an operation that can efficiently batch repair the ranges, similar to what happens during read repair for a single key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-520) Implement Range Repairs
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stu Hood updated CASSANDRA-520:
-------------------------------
Attachment: (was: 520-tokens-for-range-command.diff)
> Implement Range Repairs
> -----------------------
>
> Key: CASSANDRA-520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-520
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Stu Hood
> Assignee: Stu Hood
> Fix For: 0.5
>
> Attachments: 520-bulk-range-repair.diff
>
>
> After CASSANDRA-193, the TreeRequest/Response conversation will have generated a full list of Ranges that disagree between nodes. We need an operation that can efficiently batch repair the ranges, similar to what happens during read repair for a single key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-520) Implement Range Repairs
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stu Hood updated CASSANDRA-520:
-------------------------------
Attachment: 520-selective-1-decorated-keys-for-range-command.diff
520-bulk-2-repair-strategy-by-fraction.diff
520-bulk-1-use-streaming-api.diff
I've rebased these patches, and added selection of 'bulk' or 'selective' (range command) style repair. They are completely disjoint, and should probably be merged separately: '520-bulk' depends on the latest patchset from #193, but '520-selective' applies to trunk.
I think 520-bulk is ready for merge, and since repair will still be a manual process in 0.5, requiring an anti-compaction for repair seems reasonable.
> Implement Range Repairs
> -----------------------
>
> Key: CASSANDRA-520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-520
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Stu Hood
> Assignee: Stu Hood
> Fix For: 0.5
>
> Attachments: 520-bulk-1-use-streaming-api.diff, 520-bulk-2-repair-strategy-by-fraction.diff, 520-selective-1-decorated-keys-for-range-command.diff
>
>
> After CASSANDRA-193, the TreeRequest/Response conversation will have generated a full list of Ranges that disagree between nodes. We need an operation that can efficiently batch repair the ranges, similar to what happens during read repair for a single key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-520) Implement Range Repairs
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12786320#action_12786320 ]
Stu Hood commented on CASSANDRA-520:
------------------------------------
520-selective was merged as part of 568, so I've removed it.
> Implement Range Repairs
> -----------------------
>
> Key: CASSANDRA-520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-520
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Stu Hood
> Assignee: Stu Hood
> Fix For: 0.5
>
> Attachments: 520-bulk-1-use-streaming-api.diff, 520-bulk-1-use-streaming-api.diff, 520-bulk-2-repair-strategy-by-fraction.diff, 520-bulk-2-repair-strategy-by-fraction.diff
>
>
> After CASSANDRA-193, the TreeRequest/Response conversation will have generated a full list of Ranges that disagree between nodes. We need an operation that can efficiently batch repair the ranges, similar to what happens during read repair for a single key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-520) Implement Range Repairs
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12773581#action_12773581 ]
Jonathan Ellis commented on CASSANDRA-520:
------------------------------------------
you mean, using CASSANDRA-344?
> Implement Range Repairs
> -----------------------
>
> Key: CASSANDRA-520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-520
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Stu Hood
> Assignee: Stu Hood
> Fix For: 0.5
>
>
> After CASSANDRA-193, the TreeRequest/Response conversation will have generated a full list of Ranges that disagree between nodes. We need an operation that can efficiently batch repair the ranges, similar to what happens during read repair for a single key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-520) Implement Range Repairs
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12787071#action_12787071 ]
Jonathan Ellis commented on CASSANDRA-520:
------------------------------------------
patch 2 does not compile
src/java/org/apache/cassandra/service/AntiEntropyService.java:631: submitAnti(org.apache.cassandra.db.ColumnFamilyStore,java.util.Collection<org.apache.cassandra.dht.Range>,java.net.InetAddress) in org.apache.cassandra.db.CompactionManager cannot be applied to (org.apache.cassandra.db.ColumnFamilyStore,java.util.List<org.apache.cassandra.utils.MerkleTree.TreeRange>,java.net.InetAddress)
[javac] List<SSTableReader> sstables = CompactionManager.instance().submitAnti(cfstore, differences, remote).get();
> Implement Range Repairs
> -----------------------
>
> Key: CASSANDRA-520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-520
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Stu Hood
> Assignee: Stu Hood
> Fix For: 0.5
>
> Attachments: 520-bulk-1-use-streaming-api.diff, 520-bulk-1-use-streaming-api.diff, 520-bulk-2-repair-strategy-by-fraction.diff, 520-bulk-2-repair-strategy-by-fraction.diff
>
>
> After CASSANDRA-193, the TreeRequest/Response conversation will have generated a full list of Ranges that disagree between nodes. We need an operation that can efficiently batch repair the ranges, similar to what happens during read repair for a single key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CASSANDRA-520) Implement Range Repairs
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reassigned CASSANDRA-520:
----------------------------------------
Assignee: Stu Hood
> Implement Range Repairs
> -----------------------
>
> Key: CASSANDRA-520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-520
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Stu Hood
> Assignee: Stu Hood
> Fix For: 0.5
>
>
> After CASSANDRA-193, the TreeRequest/Response conversation will have generated a full list of Ranges that disagree between nodes. We need an operation that can efficiently batch repair the ranges, similar to what happens during read repair for a single key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-520) Implement Range Repairs
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stu Hood updated CASSANDRA-520:
-------------------------------
Attachment: (was: 520-bulk-range-repair.diff)
> Implement Range Repairs
> -----------------------
>
> Key: CASSANDRA-520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-520
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Stu Hood
> Assignee: Stu Hood
> Fix For: 0.5
>
>
> After CASSANDRA-193, the TreeRequest/Response conversation will have generated a full list of Ranges that disagree between nodes. We need an operation that can efficiently batch repair the ranges, similar to what happens during read repair for a single key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-520) Implement Range Repairs
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stu Hood updated CASSANDRA-520:
-------------------------------
Attachment: 520-tokens-for-range-command.diff
This patch changes RangeCommand to take a DecoratedKey for startWith and stopAt. The idea is that a RangeCommand for a Token Range would provide DecoratedKeys with null keys. Along with 344, this allows for a naive implementation of range repairs.
> Implement Range Repairs
> -----------------------
>
> Key: CASSANDRA-520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-520
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Stu Hood
> Assignee: Stu Hood
> Fix For: 0.5
>
> Attachments: 520-tokens-for-range-command.diff
>
>
> After CASSANDRA-193, the TreeRequest/Response conversation will have generated a full list of Ranges that disagree between nodes. We need an operation that can efficiently batch repair the ranges, similar to what happens during read repair for a single key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-520) Implement Range Repairs
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stu Hood updated CASSANDRA-520:
-------------------------------
Attachment: (was: 520-selective-1-decorated-keys-for-range-command.diff)
> Implement Range Repairs
> -----------------------
>
> Key: CASSANDRA-520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-520
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Stu Hood
> Assignee: Stu Hood
> Fix For: 0.5
>
> Attachments: 520-bulk-1-use-streaming-api.diff, 520-bulk-1-use-streaming-api.diff, 520-bulk-2-repair-strategy-by-fraction.diff, 520-bulk-2-repair-strategy-by-fraction.diff
>
>
> After CASSANDRA-193, the TreeRequest/Response conversation will have generated a full list of Ranges that disagree between nodes. We need an operation that can efficiently batch repair the ranges, similar to what happens during read repair for a single key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-520) Implement Range Repairs
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stu Hood updated CASSANDRA-520:
-------------------------------
Attachment: 520-bulk-range-repair.diff
Here's an implementation of the efficient bulk range repair (thanks to jbellis' cleanup of the Streaming API, the actual transfer only takes 10 lines!) but it depends on #193.
Once we have one of these repair methods merged, I can work on calculation of percent difference between two trees so that we can use different methods based on thresholds.
> Implement Range Repairs
> -----------------------
>
> Key: CASSANDRA-520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-520
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Stu Hood
> Assignee: Stu Hood
> Fix For: 0.5
>
> Attachments: 520-bulk-range-repair.diff, 520-tokens-for-range-command.diff
>
>
> After CASSANDRA-193, the TreeRequest/Response conversation will have generated a full list of Ranges that disagree between nodes. We need an operation that can efficiently batch repair the ranges, similar to what happens during read repair for a single key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-520) Implement Range Repairs
Posted by "Stu Hood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stu Hood updated CASSANDRA-520:
-------------------------------
Attachment: 520-bulk-2-repair-strategy-by-fraction.diff
520-bulk-1-use-streaming-api.diff
Rebased to apply to trunk.
Triggering RangeCommand based repairs is a FIXME at AntiEntropyService:617, which means that this patch won't actually trigger repairs for differences of less than 5% between nodes.
> Implement Range Repairs
> -----------------------
>
> Key: CASSANDRA-520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-520
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Stu Hood
> Assignee: Stu Hood
> Fix For: 0.5
>
> Attachments: 520-bulk-1-use-streaming-api.diff, 520-bulk-1-use-streaming-api.diff, 520-bulk-1-use-streaming-api.diff, 520-bulk-2-repair-strategy-by-fraction.diff, 520-bulk-2-repair-strategy-by-fraction.diff, 520-bulk-2-repair-strategy-by-fraction.diff
>
>
> After CASSANDRA-193, the TreeRequest/Response conversation will have generated a full list of Ranges that disagree between nodes. We need an operation that can efficiently batch repair the ranges, similar to what happens during read repair for a single key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.