You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Gary Dusbabek (JIRA)" <ji...@apache.org> on 2010/01/13 23:19:54 UTC
[jira] Created: (CASSANDRA-696) Bootstrapping doesn't work on new
clusters
Bootstrapping doesn't work on new clusters
------------------------------------------
Key: CASSANDRA-696
URL: https://issues.apache.org/jira/browse/CASSANDRA-696
Project: Cassandra
Issue Type: Bug
Reporter: Gary Dusbabek
Assignee: Gary Dusbabek
Priority: Minor
This is an edge case.
1. start a clean 3 node cluster with autobootstrap on.
2. load some data.
3. bootstrap in a 4th node.
the logs in the 4th node will indicate that data was not received. If you restart the cluster in between steps 1 and 2, or 2 and 3, boot strapping works fine.
I find that waiting on the table flush when making the streaming request solves the problem (see patch).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-696) Bootstrapping doesn't work on new
clusters
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800007#action_12800007 ]
Jonathan Ellis commented on CASSANDRA-696:
------------------------------------------
+1. can you commit to 0.5 and merge to trunk?
> Bootstrapping doesn't work on new clusters
> ------------------------------------------
>
> Key: CASSANDRA-696
> URL: https://issues.apache.org/jira/browse/CASSANDRA-696
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Attachments: wait_for_memtable_flush.patch
>
>
> This is an edge case.
> 1. start a clean 3 node cluster with autobootstrap on.
> 2. load some data.
> 3. bootstrap in a 4th node.
> the logs in the 4th node will indicate that data was not received. If you restart the cluster in between steps 1 and 2, or 2 and 3, boot strapping works fine.
> I find that waiting on the table flush when making the streaming request solves the problem (see patch).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-696) Bootstrapping doesn't work on new
clusters
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799974#action_12799974 ]
Jonathan Ellis commented on CASSANDRA-696:
------------------------------------------
is this a 0.5 bug?
> Bootstrapping doesn't work on new clusters
> ------------------------------------------
>
> Key: CASSANDRA-696
> URL: https://issues.apache.org/jira/browse/CASSANDRA-696
> Project: Cassandra
> Issue Type: Bug
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Attachments: wait_for_memtable_flush.patch
>
>
> This is an edge case.
> 1. start a clean 3 node cluster with autobootstrap on.
> 2. load some data.
> 3. bootstrap in a 4th node.
> the logs in the 4th node will indicate that data was not received. If you restart the cluster in between steps 1 and 2, or 2 and 3, boot strapping works fine.
> I find that waiting on the table flush when making the streaming request solves the problem (see patch).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-696) Bootstrapping doesn't work on new
clusters
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800003#action_12800003 ]
Gary Dusbabek commented on CASSANDRA-696:
-----------------------------------------
Just tested on 0.5... yes, the bug is there too.
> Bootstrapping doesn't work on new clusters
> ------------------------------------------
>
> Key: CASSANDRA-696
> URL: https://issues.apache.org/jira/browse/CASSANDRA-696
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Attachments: wait_for_memtable_flush.patch
>
>
> This is an edge case.
> 1. start a clean 3 node cluster with autobootstrap on.
> 2. load some data.
> 3. bootstrap in a 4th node.
> the logs in the 4th node will indicate that data was not received. If you restart the cluster in between steps 1 and 2, or 2 and 3, boot strapping works fine.
> I find that waiting on the table flush when making the streaming request solves the problem (see patch).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-696) Bootstrapping doesn't work on new
clusters
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800008#action_12800008 ]
Jonathan Ellis commented on CASSANDRA-696:
------------------------------------------
(minor tweak: can you change the re-throw from Interrupted to AssertionError?)
> Bootstrapping doesn't work on new clusters
> ------------------------------------------
>
> Key: CASSANDRA-696
> URL: https://issues.apache.org/jira/browse/CASSANDRA-696
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Attachments: wait_for_memtable_flush.patch
>
>
> This is an edge case.
> 1. start a clean 3 node cluster with autobootstrap on.
> 2. load some data.
> 3. bootstrap in a 4th node.
> the logs in the 4th node will indicate that data was not received. If you restart the cluster in between steps 1 and 2, or 2 and 3, boot strapping works fine.
> I find that waiting on the table flush when making the streaming request solves the problem (see patch).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-696) Bootstrapping doesn't work on new
clusters
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Dusbabek updated CASSANDRA-696:
------------------------------------
Component/s: Core
Affects Version/s: 0.5
> Bootstrapping doesn't work on new clusters
> ------------------------------------------
>
> Key: CASSANDRA-696
> URL: https://issues.apache.org/jira/browse/CASSANDRA-696
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Attachments: wait_for_memtable_flush.patch
>
>
> This is an edge case.
> 1. start a clean 3 node cluster with autobootstrap on.
> 2. load some data.
> 3. bootstrap in a 4th node.
> the logs in the 4th node will indicate that data was not received. If you restart the cluster in between steps 1 and 2, or 2 and 3, boot strapping works fine.
> I find that waiting on the table flush when making the streaming request solves the problem (see patch).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-696) Bootstrapping doesn't work on new
clusters
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Dusbabek updated CASSANDRA-696:
------------------------------------
Attachment: wait_for_memtable_flush.patch
block on the table flush.
> Bootstrapping doesn't work on new clusters
> ------------------------------------------
>
> Key: CASSANDRA-696
> URL: https://issues.apache.org/jira/browse/CASSANDRA-696
> Project: Cassandra
> Issue Type: Bug
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Attachments: wait_for_memtable_flush.patch
>
>
> This is an edge case.
> 1. start a clean 3 node cluster with autobootstrap on.
> 2. load some data.
> 3. bootstrap in a 4th node.
> the logs in the 4th node will indicate that data was not received. If you restart the cluster in between steps 1 and 2, or 2 and 3, boot strapping works fine.
> I find that waiting on the table flush when making the streaming request solves the problem (see patch).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-696) Bootstrapping doesn't work on new
clusters
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800005#action_12800005 ]
Gary Dusbabek commented on CASSANDRA-696:
-----------------------------------------
To be precise, it looks like anything still in the memtables will not be streamed to the bootstrapping node since we're not blocking on the table.flush() call. I haven't researched it enough, but I suspect the same thing will happen in an established cluster: everything already committed to SSTables gets streamed, and anything still left in the memtable is left behind. It's just that in a brand new cluster, there never are any SSTables in the first place--it makes the bug more obvious.
> Bootstrapping doesn't work on new clusters
> ------------------------------------------
>
> Key: CASSANDRA-696
> URL: https://issues.apache.org/jira/browse/CASSANDRA-696
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Reporter: Gary Dusbabek
> Assignee: Gary Dusbabek
> Priority: Minor
> Attachments: wait_for_memtable_flush.patch
>
>
> This is an edge case.
> 1. start a clean 3 node cluster with autobootstrap on.
> 2. load some data.
> 3. bootstrap in a 4th node.
> the logs in the 4th node will indicate that data was not received. If you restart the cluster in between steps 1 and 2, or 2 and 3, boot strapping works fine.
> I find that waiting on the table flush when making the streaming request solves the problem (see patch).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.