You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jonathan Ellis (JIRA)" <ji...@apache.org> on 2009/06/03 23:36:07 UTC
[jira] Created: (CASSANDRA-213) complete bootstrap code
complete bootstrap code
-----------------------
Key: CASSANDRA-213
URL: https://issues.apache.org/jira/browse/CASSANDRA-213
Project: Cassandra
Issue Type: New Feature
Reporter: Jonathan Ellis
Priority: Critical
Fix For: 0.4
The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738414#action_12738414 ]
Jonathan Ellis commented on CASSANDRA-213:
------------------------------------------
Also, the right approach for nodeprobe is to tell the new node "bootstrap yourself" and have it tell the actual bootstrap commands to the other nodes.
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 213-v1.patch, 213-v2.patch, 213-v3.patch
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-213) complete bootstrap code
Posted by "Sandeep Tata (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sandeep Tata updated CASSANDRA-213:
-----------------------------------
Attachment: 213-v2.patch
Rebased to trunk (the newline fixes clobbered the patch).
And made other minor changes.
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 213-v1.patch, 213-v2.patch
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-213) complete bootstrap code
Posted by "Sammy Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sammy Yu updated CASSANDRA-213:
-------------------------------
Attachment: 0003-Patch-for-Cassandra-279-3rd.patch
snapshots directory are now created under the data directory so the structure looks like this:
<data directory>/snapshots/<snapshotname>/<table>/<sstable>
added support for clearsnapshot which will wipe the snapshots directory under each data directory
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sammy Yu
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 0003-Patch-for-Cassandra-279-3rd.patch
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-213) complete bootstrap code
Posted by "Sandeep Tata (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sandeep Tata updated CASSANDRA-213:
-----------------------------------
Attachment: 213-v3-addendum.patch
A change to contentstream that slipped through the rebase. (StreamManager needs to see ip addresses as keys, not hostnames)
Got rid of the 0==1 code. Added some logging and a comment that the files should sent in an order that sends the -Data.db file last.
Verified that nodeprobe -host <oldnode> bootstrap <newnode> works for normal situations.
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 213-v1.patch, 213-v2.patch, 213-v3-addendum.patch, 213-v3.patch
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Sandeep Tata (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733725#action_12733725 ]
Sandeep Tata commented on CASSANDRA-213:
----------------------------------------
I'm beginning to look at it -- I'll post here as I make any progress so we should be able to work on it in parallel.
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sammy Yu
> Priority: Critical
> Fix For: 0.4
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Chris Goffinet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724898#action_12724898 ]
Chris Goffinet commented on CASSANDRA-213:
------------------------------------------
+1 as well. Is anyone tackling this at the moment? Wanted to make sure since Sammy Yu had it on his list.
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Priority: Critical
> Fix For: 0.4
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Sandeep Tata (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724304#action_12724304 ]
Sandeep Tata commented on CASSANDRA-213:
----------------------------------------
+1 on moving bootstrap logic to its own package.
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Priority: Critical
> Fix For: 0.4
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CASSANDRA-213) complete bootstrap code
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reassigned CASSANDRA-213:
----------------------------------------
Assignee: Sandeep Tata (was: Sammy Yu)
Assigning to Sandeep for now.
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CASSANDRA-213) complete bootstrap code
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reassigned CASSANDRA-213:
----------------------------------------
Assignee: Sammy Yu
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Assignee: Sammy Yu
> Priority: Critical
> Fix For: 0.4
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-213) complete bootstrap code
Posted by "Sammy Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sammy Yu updated CASSANDRA-213:
-------------------------------
Comment: was deleted
(was: snapshots directory are now created under the data directory so the structure looks like this:
<data directory>/snapshots/<snapshotname>/<table>/<sstable>
added support for clearsnapshot which will wipe the snapshots directory under each data directory)
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sammy Yu
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 0003-Patch-for-Cassandra-279-3rd.patch
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-213) complete bootstrap code
Posted by "Sammy Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sammy Yu updated CASSANDRA-213:
-------------------------------
Attachment: (was: 0003-Patch-for-Cassandra-279-3rd.patch)
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sammy Yu
> Priority: Critical
> Fix For: 0.4
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Jun Rao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738402#action_12738402 ]
Jun Rao commented on CASSANDRA-213:
-----------------------------------
I am able to apply v2 to trunk, using git apply.
I generate patch using svn diff in cygwin.
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 213-v1.patch, 213-v2.patch
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Sandeep Tata (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738463#action_12738463 ]
Sandeep Tata commented on CASSANDRA-213:
----------------------------------------
>>Also, the right approach for nodeprobe is to tell the new node "bootstrap yourself" and have it tell the actual bootstrap commands to the other nodes.
Yeah, this is probably cleaner, but this also means
1) You have to bootstrap each node separately instead of supplying a list of nodes to an existing node.
2) I'm not sure how long it would take for a new node to get a stable version of the token-to-endpoint map. Each gossip message might send a random subset of the endpoint map to the new node (although it currently sends everything, I think).
I think it is safer for an existing, stable node to send out the bootstrap commands.
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 213-v1.patch, 213-v2.patch, 213-v3-addendum.patch, 213-v3.patch
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Sammy Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733434#action_12733434 ]
Sammy Yu commented on CASSANDRA-213:
------------------------------------
Sandeep -- I have started looking at the verb handlers, but I haven't gotten very far yet. Are you working on it as wll?
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sammy Yu
> Priority: Critical
> Fix For: 0.4
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737597#action_12737597 ]
Jonathan Ellis commented on CASSANDRA-213:
------------------------------------------
have you tried applying your own patches?
you could try "get show HEAD > ..." but I'm not sure if that would be any different.
what does Jun do differently? is he on a different OS?
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 213-v1.patch
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738951#action_12738951 ]
Hudson commented on CASSANDRA-213:
----------------------------------
Integrated in Cassandra #157 (See [http://hudson.zones.apache.org/hudson/job/Cassandra/157/])
Bootstrap code + tests. Patch by Sandeep Tata; reviewed by jbellis for
forgot SSTable part of last.
patch by Sandeep Tata; reviewed by jbellis for
renames and cleanup for bootstrap-related code.
patch by Sandeep Tata; reviewed by jbellis for
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 213-v1.patch, 213-v2.patch, 213-v3-addendum.patch, 213-v3.patch, 213-v4.patch
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-213) complete bootstrap code
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-213:
-------------------------------------
Attachment: 213-v3.patch
Committed the miscellaneous cleanup and renames.
Remaining part is attached as v3.
Let's get this to where it at least works in the "everything went as expected" case before committing the rest.
Is it simply a matter of removing the don't-do-anything-here if code?
if (0==1)
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 213-v1.patch, 213-v2.patch, 213-v3.patch
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Sandeep Tata (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12735958#action_12735958 ]
Sandeep Tata commented on CASSANDRA-213:
----------------------------------------
Ah, that makes sense.
I was confused by "Eventually we want the old node to keep serving requests until the new node has the data." meant. I guess you're talking about extra coordination to make sure a new node doesn't start serving reads until it gets the data from all its bootstrap buddies.
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Sandeep Tata (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737417#action_12737417 ]
Sandeep Tata commented on CASSANDRA-213:
----------------------------------------
Attached a preliminary patch. There are a few corner cases (such as failures during bootstrap) that don't work yet.
I should be able to iron out the rest of this over the weekend.
Sammy -- perhaps you want to get a head start on reviewing this.
I haven't switched to the more IO efficient approach that Jonathan suggested above yet.
You can test the current patch by running something like:
bin/nodeprobe -host 1.2.3.4 bootstrap 1.2.3.5
^^^ ^^^
an existing node new node
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 213-v1.patch
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733833#action_12733833 ]
Jonathan Ellis commented on CASSANDRA-213:
------------------------------------------
Some thoughts on bootstrap:
For this ticket let's just get the code we inherited from FB working: split the sstables up, send the right segments to the new node, (eventually) clean up.
I think that the existing approach of moving or copying the files into a special to-send directory is the wrong one. Eventually we want the old node to keep serving requests until the new node has the data. (Not for this ticket though, I will make another.) At which point moving things around will disrupt normal operations. But if we split it and leave it in the old data dir, then we can scan it for its beginning and end keys easily enough to see if it needs to be part of the sendfile operation.
So for this ticket, I recommend that the bootstrap consist of
0) get rid of the special bootstrap staging area code
1) anticompaction
2) iterate over sstable files and see if they are in the right range
3) send the files
4) assemble the files (index + bloom filter + data) into sstable objects on the bootstraped node
we have 1 and 3 and a start at 4; we need 2 which is relatively small. and the other parts will probably need debugging. :)
make sense?
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sammy Yu
> Priority: Critical
> Fix For: 0.4
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737542#action_12737542 ]
Jonathan Ellis commented on CASSANDRA-213:
------------------------------------------
How are you generating your patches? I have never had trouble with anyone else's but I consistently have to use git-apply for yours.
Here though even get-apply didn't work:
fatal: git-apply: bad git-diff - inconsistent new filename on line 356
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 213-v1.patch
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Sandeep Tata (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733424#action_12733424 ]
Sandeep Tata commented on CASSANDRA-213:
----------------------------------------
Sammy -- are you working on this?
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sammy Yu
> Priority: Critical
> Fix For: 0.4
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Sandeep Tata (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737593#action_12737593 ]
Sandeep Tata commented on CASSANDRA-213:
----------------------------------------
Hmm .. I used "git diff trunk..HEAD > 213-v1.patch"
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 213-v1.patch
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12723757#action_12723757 ]
Jonathan Ellis commented on CASSANDRA-213:
------------------------------------------
someone pointed out on IRC that bootstrap* probably doesn't belong in the dht package. I suggest just moving it to its own.
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Priority: Critical
> Fix For: 0.4
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12735892#action_12735892 ]
Jonathan Ellis commented on CASSANDRA-213:
------------------------------------------
> Why do you want to get rid of the bootstrap staging code?
Because it's a lot less IO (50% less reads, to be exact) to have anticompact split the old file into new files, each of which are served live until bootstrap is complete, at which point the sent file is removed, than to copy the to-send files out to a staging dir, and later re-scan the original file for cleanup & writing what is left.
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724901#action_12724901 ]
Jonathan Ellis commented on CASSANDRA-213:
------------------------------------------
Sammy is the first AFAIK.
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Priority: Critical
> Fix For: 0.4
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-213) complete bootstrap code
Posted by "Sandeep Tata (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sandeep Tata updated CASSANDRA-213:
-----------------------------------
Attachment: 213-v1.patch
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 213-v1.patch
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Sandeep Tata (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12735891#action_12735891 ]
Sandeep Tata commented on CASSANDRA-213:
----------------------------------------
The current code relies on CFS.doCleanupCompaction() to get rid of the old keys that no longer belong to the node.
The anticompaction call for bootstrap simply makes a copy of the keys meant for the target in separate files in the "bootstrap" directory.
This seems reasonable to me. Why do you want to get rid of the bootstrap staging code?
(There are a bunch of minor bugs I'm working through .. I should be able to get to more testing tomorrow)
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Sandeep Tata (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738127#action_12738127 ]
Sandeep Tata commented on CASSANDRA-213:
----------------------------------------
patch -p 1 < 213-v1.patch applies cleanly on trunk (edf2226079d61a0e23f803bc3f064b9e360fb141) for me
Hmm ... I think Jun generates his patches on a windows machine.
Jun: does this patch apply for you?
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 213-v1.patch
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-213) complete bootstrap code
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738475#action_12738475 ]
Jonathan Ellis commented on CASSANDRA-213:
------------------------------------------
this is the right thing to do though -- eventually, we want to have a --bootstrap-myself option to start a new node, so it immediately goes into bootstrap mode (including waiting a minute or so for a stable map) instead of participating immediately and serving up incorrect "no, there's no data for that key" replies.
yes, this means it's slightly less straightforward to split a single existing node across multiple new nodes in a single boostrap operation, but (a) the code isn't designed to do that anyway so I don't see that as giving up much and (b) if we really need this it's easy to add a delay on the existing node to wait a few seconds to see if more than one node wants to be be a target.
the common op is "I'm adding a group of nodes mixed around the ring" so let's optimize for that and make it as foolproof as possible.
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 213-v1.patch, 213-v2.patch, 213-v3-addendum.patch, 213-v3.patch
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-213) complete bootstrap code
Posted by "Sandeep Tata (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sandeep Tata updated CASSANDRA-213:
-----------------------------------
Attachment: 213-v4.patch
Rebased v3+addendum
Refactored hinted-handoff delivery and bootstrap into separate methods
Got rid of unused filtering in the nodelist
Allowed hostnames in the bootstrap command (resolved them to ip addresses before creating EndPoints)
> complete bootstrap code
> -----------------------
>
> Key: CASSANDRA-213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-213
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Sandeep Tata
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 213-v1.patch, 213-v2.patch, 213-v3-addendum.patch, 213-v3.patch, 213-v4.patch
>
>
> The bootstrap code sends files from DatabaseDescriptor.getBootstrapFileLocation to the node that is joining the ring, but nothing actually puts files into that directory.
> There are probably other bugs / omissions we will find as we flesh this out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.