You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2011/07/28 01:23:09 UTC
[jira] [Commented] (CASSANDRA-2948) Nodetool move fails to stream
out data from moved node to new endpoint.
[ https://issues.apache.org/jira/browse/CASSANDRA-2948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072094#comment-13072094 ]
Hudson commented on CASSANDRA-2948:
-----------------------------------
Integrated in Cassandra-0.8 #242 (See [https://builds.apache.org/job/Cassandra-0.8/242/])
Gossiper notifies of local state changes.
Patch by Sam Overton, reviewed by brandonwilliams for CASSANDRA-2948
brandonwilliams : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1151659
Files :
* /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/gms/Gossiper.java
* /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageService.java
> Nodetool move fails to stream out data from moved node to new endpoint.
> -----------------------------------------------------------------------
>
> Key: CASSANDRA-2948
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2948
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.2
> Reporter: Sam Overton
> Assignee: Sam Overton
> Priority: Critical
> Labels: move, streaming
> Fix For: 0.8.3
>
> Attachments: 2948.patch
>
>
> When moving a node in the ring with nodetool move, that node streams its data to itself instead of to the new endpoint responsible for its old range.
> Steps to reproduce:
> * Create a cluster (A,B,C,D) with tokens (0,4,8,C) using ByteOrderedPartitioner
> * Create a keyspace and CF with RF=1
> * Insert keys (2,6,A,E). This should put one key on each node.
> * Move node A to token 7. This should cause:
> + node C streams key 6 to node A
> + node A streams key E to node B
> However instead, node A streams key E to itself.
> Selected log messages from node A:
> INFO [RMI TCP Connection(6)-10.2.129.41] 2011-07-26 16:29:17,075 StorageService.java (line 1878) Moving miles/10.2.129.41 from Token(bytes[00]) to Token(bytes[07]).
> DEBUG [RMI TCP Connection(6)-10.2.129.41] 2011-07-26 16:29:17,080 StorageService.java (line 1941) Table ks: work map {/10.2.129.16=[(Token(bytes[04]),Token(bytes[07])]]}.
> INFO [RMI TCP Connection(6)-10.2.129.41] 2011-07-26 16:29:17,080 StorageService.java (line 1946) Sleeping 30000 ms before start streaming/fetching ranges.
> ...
> INFO [RMI TCP Connection(6)-10.2.129.41] 2011-07-26 16:29:46,728 StorageService.java (line 522) Moving: fetching new ranges and streaming old ranges
> DEBUG [RMI TCP Connection(6)-10.2.129.41] 2011-07-26 16:29:46,728 StorageService.java (line 1960) [Move->STREAMING] Work Map: {ks={(Token(bytes[0c]),Token(bytes[00])]=[miles/10.2.129.41]}}
> DEBUG [RMI TCP Connection(6)-10.2.129.41] 2011-07-26 16:29:46,729 StorageService.java (line 1965) [Move->FETCHING] Work Map: {ks={/10.2.129.16=[(Token(bytes[04]),Token(bytes[07])]]}}
> DEBUG [RMI TCP Connection(6)-10.2.129.41] 2011-07-26 16:29:46,730 StorageService.java (line 2411) Requesting from /10.2.129.16 ranges (Token(bytes[04]),Token(bytes[07])]
> ...
> INFO [StreamStage:1] 2011-07-26 16:29:46,737 StreamOut.java (line 90) Beginning transfer to miles/10.2.129.41
> DEBUG [StreamStage:1] 2011-07-26 16:29:46,737 StreamOut.java (line 91) Ranges are (Token(bytes[0c]),Token(bytes[00])]
> This appears to be caused because in StorageService.move we call
> Gossiper.instance.addLocalApplicationState(ApplicationState.STATUS, valueFactory.moving(newToken));
> and then get the new token metadata in order to calculate where the new endpoint is that we should stream to
> TokenMetadata tokenMetaClone = tokenMetadata_.cloneAfterAllSettled();
> however, in addLocalApplicationState there is no notification broadcast for the change in local state, so tokenMetadata_ never updates the list of moving nodes, and the tokenMetaClone is still the state of the ring from before the move.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira