You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Eric Evans (JIRA)" <ji...@apache.org> on 2012/08/20 22:33:38 UTC

[jira] [Created] (CASSANDRA-4559) implement token relocation

Eric Evans created CASSANDRA-4559:
-------------------------------------

             Summary: implement token relocation
                 Key: CASSANDRA-4559
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
             Project: Cassandra
          Issue Type: Sub-task
          Components: Core, Tools
            Reporter: Eric Evans
            Assignee: Eric Evans


Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.

----

h3. Patches
||Compare||Raw diff||Description||
|[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
|[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
|[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|

----

_Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4559) implement token relocation

Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440530#comment-13440530 ] 

Eric Evans commented on CASSANDRA-4559:
---------------------------------------

bq. It seems like that would be handy, though now you need to be able to communicate multiple tokens in the status... which almost brings us back around to the status-appending game we want to avoid in CASSANDRA-4383

I agree, but propose we do it anyway (append to the status), if it's important to support more than one range in an operation.  Taking the approach being worked on in CASSANDRA-4383 seems like overkill, and _relocate_ is limited enough in scope that is seems less likely to matter.  The impact of any future changes would be more limited as well.

I've updated the patches accordingly; Let me know what you think!
                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4559) implement token relocation

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13439719#comment-13439719 ] 

Brandon Williams commented on CASSANDRA-4559:
---------------------------------------------

Won't we want to be able to relocate more than one token at a time?

Also, it seems like we re-use the moving status, or remove it if we want to rename the functionality, since doing it in a mixed cluster will never be supported.
                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4559) implement token relocation

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13439841#comment-13439841 ] 

Brandon Williams commented on CASSANDRA-4559:
---------------------------------------------

bq. It doesn't make a huge difference to me either way; I can make the operation support multiple ranges in a call if you prefer.

It seems like that would be handy, though now you need to be able to communicate multiple tokens in the status... which almost brings us back around to the status-appending game we want to avoid in CASSANDRA-4383 :(

bq. Don't we need to continue supporting the old-style move for users that want to continue running with 1 token per node?

You're right, there's a difference between "I'm moving to that token" (move) and "I'm taking this token from this other node" (relocate)
                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (CASSANDRA-4559) implement token relocation

Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eric Evans updated CASSANDRA-4559:
----------------------------------

    Reviewer: brandon.williams
    
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4559) implement token relocation

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455274#comment-13455274 ] 

Brandon Williams commented on CASSANDRA-4559:
---------------------------------------------

+1, ship it!
                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>         Attachments: 4559.txt
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-4559) implement token relocation

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450947#comment-13450947 ] 

Brandon Williams edited comment on CASSANDRA-4559 at 9/8/12 7:33 AM:
---------------------------------------------------------------------

Every once in a long while you can see this exception:

{noformat}
ERROR 20:30:18,355 Exception in thread Thread[OptionalTasks:1,5,main]
java.lang.NullPointerException
        at org.apache.cassandra.locator.TokenMetadata.removeFromRelocating(TokenMetadata.java:443)
        at org.apache.cassandra.service.StorageService$2.run(StorageService.java:1252)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
{noformat}

I suspect this is because we're using the same interval (RING_DELAY) and sometimes they coincide.  We can just double the relocation removal.
                
      was (Author: brandon.williams):
    Every once in a long while you can see this exception:

{{noformat}}
ERROR 20:30:18,355 Exception in thread Thread[OptionalTasks:1,5,main]
java.lang.NullPointerException
        at org.apache.cassandra.locator.TokenMetadata.removeFromRelocating(TokenMetadata.java:443)
        at org.apache.cassandra.service.StorageService$2.run(StorageService.java:1252)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
{{noformat}

I suspect this is because we're using the same interval (RING_DELAY) and sometimes they coincide.  We can just double the relocation removal.
                  
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>         Attachments: 4559.txt
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-4559) implement token relocation

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Brandon Williams updated CASSANDRA-4559:
----------------------------------------

    Attachment: 4559.txt

At least one problem is that cloneAfterAllSettled doesn't take into account relocating tokens, only moving tokens.  Updated patch fixes that, but there's still something wrong, however nodes at least stream out now.
                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>         Attachments: 4559.txt
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-4559) implement token relocation

Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eric Evans updated CASSANDRA-4559:
----------------------------------

    Description: 
Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.

_Edit0: Linked in new patch containing tests._
----

h3. Patches
||Compare||Raw diff||Description||
|[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
|[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
|[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
|[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|

----

_Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

  was:
Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.

----

h3. Patches
||Compare||Raw diff||Description||
|[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
|[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
|[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|

----

_Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

    
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Comment Edited] (CASSANDRA-4559) implement token relocation

Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448096#comment-13448096 ] 

Eric Evans edited comment on CASSANDRA-4559 at 9/5/12 9:00 AM:
---------------------------------------------------------------

Since this is a freeze-dependency, let me provide a bit of a status update:

These patches seemed "mostly" stable prior to merging w/ trunk after CASSANDRA-4383 landed.  I don't know if the problems since are the result of a faulty merge, a problem with what was committed for CASSANDRA-4383, or something else entirely (I'm still looking into it).

I say "mostly" stable because occasionally one node would seem to miss the {{NORMAL}} sent by the destination node after a successful relocation, and so that nodes idea of the state of the ring would differ from the rest of the cluster.  I'm still looking into this, or I will be after sorting out the new spate of issues (see above).

Most of the challenges here have related to network concurrency, so if you wish to test, you really need to use the shuffle utility in order to set more than one in motion at once.  The shuffle tool isn't quite done yet either, but (if interested )you can checkout at https://github.com/acunu/cassandra/tree/p/4443/060_shuffle_utility (which includes the token relocation).
                
      was (Author: urandom):
    Since this is a freeze-dependency, let me provide a bit of a status update:

These patches seemed "mostly" stable prior to merging w/ trunk after CASSANDRA-4383 landed.  I don't know if the problems since are the result of a faulty merge, a problem with what was committed for CASSANDRA-4383, or something else entirely (I'm still looking into it).

I say "mostly" stable because occasionally one node would seem to miss the {{NORMAL}} sent by the destination node after a successful relocation, and so that nodes idea of the state of the ring would differ from the rest of the cluster.  I'm still looking into this, or I will be after sorting out the new spate of issues (see above).

Most of the challenges here have related to network concurrency, so you if wish to test, you really need to use the shuffle utility in order to set more than one in motion at once.  The shuffle tool isn't quite done yet either, but (if interested )you can checkout at https://github.com/acunu/cassandra/tree/p/4443/060_shuffle_utility (which includes the token relocation).
                  
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4559) implement token relocation

Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448096#comment-13448096 ] 

Eric Evans commented on CASSANDRA-4559:
---------------------------------------

Since this is a freeze-dependency, let me provide a bit of a status update:

These patches seemed "mostly" stable prior to merging w/ trunk after CASSANDRA-4383 landed.  I don't know if the problems since are the result of a faulty merge, a problem with what was committed for CASSANDRA-4383, or something else entirely (I'm still looking into it).

I say "mostly" stable because occasionally one node would seem to miss the {{NORMAL}} sent by the destination node after a successful relocation, and so that nodes idea of the state of the ring would differ from the rest of the cluster.  I'm still looking into this, or I will be after sorting out the new spate of issues (see above).

Most of the challenges here have related to network concurrency, so if wish to test, you really need to use the shuffle utility in order to set more than one in motion at once.  The shuffle tool isn't quite done yet either, but (if interested )you can checkout at https://github.com/acunu/cassandra/tree/p/4443/060_shuffle_utility (which includes the token relocation).
                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4559) implement token relocation

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453579#comment-13453579 ] 

Brandon Williams commented on CASSANDRA-4559:
---------------------------------------------

bq. Updated patch fixes that, but there's still something wrong, however nodes at least stream out now.

I take it back that something is still wrong, replicas are all going to right places now.  The remaining problems that I see are the aforementioned NPE (very rare) and sometimes I see nodes incorrectly fall back to generation comparison on token conflicts instead of relocating, leading to different views of the ring on different nodes.  I'm not really sure what is causing either of these yet.
                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>         Attachments: 4559.txt
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4559) implement token relocation

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450649#comment-13450649 ] 

Brandon Williams commented on CASSANDRA-4559:
---------------------------------------------

I'm pretty sure any gossip-related issues were fixed with the last patch on CASSANDRA-4383.  But now there are other problems with missing data occasionally.  It's not entirely easy to replicate since it seems to depend on what shuffle decides to do which is random, but here is the simplest example I've distilled from the logs with a small amount of vnodes.

Given nodes A through C with the following tokens (which were evenly split single tokens, but now are contiguous two-token ranges after upgrading):

A - 0,5
B - 1,2
C - 3,4

shuffle attempts to move them to:

A - 1,3
B - 0,5
C - 2,4  

The first step begins as follows:

A begins taking 1 from B
B begins taking 5 from A
C begins taking 2 from B

At this point, one interesting is that pendingRanges on all nodes look like this:

A: (4,1]
B: (5,5]
C: (2,2]

B and C both have the entire ring pending, which may or may not be significant.

Now A correctly requests (0,1] from B, B correctly requests (4,5] from A, and C correctly requests (1,2] from B.

Next, A takes (2,3] from C, B takes (5,0] from A, but C already owns (3,4] and does nothing.  

At this point shuffle is complete, but there is at least one key missing from each node, which is odd because it means C's first and only step was faulty.  Other shuffle combinations with these same tokens/layout can succeed, so I'm not entirely sure what makes the failures special, but I'll keep looking into it.
                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-4559) implement token relocation

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450935#comment-13450935 ] 

Brandon Williams edited comment on CASSANDRA-4559 at 9/8/12 7:19 AM:
---------------------------------------------------------------------

For all the trouble this was to figure out, the cause and solution are remarkably simple.

calculateStreamAndFetchRanges breaks on the first intersection while iterating the range, which works fine with a single token, but not with many since there can be further intersections.  Simply removing the breaks here so the loops continue makes everything work correctly.

Also, since RangeRelocator calculates ranges upon instantiation, I moved this until after the ring delay sleep for announcement, purely as an optimization so the node can catch more relocation announcements while it sleeps before deciding what to stream.

                
      was (Author: brandon.williams):
    For all the trouble this was the figure out, the cause and solution are remarkably simple.

calculateStreamAndFetchRanges breaks on the first intersection while iterating the range, which works fine with a single token, but not with many since there can be further intersections.  Simply removing the breaks here so the loops continue makes everything work correctly.

Also, since RangeRelocator calculates ranges upon instantiation, I moved this until after the ring delay sleep for announcement, purely as an optimization so the node can catch more relocation announcements while it sleeps before deciding what to stream.

                  
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>         Attachments: 4559.txt
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4559) implement token relocation

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450968#comment-13450968 ] 

Brandon Williams commented on CASSANDRA-4559:
---------------------------------------------

Hmm, you're right... and that's the only place we remove it.
                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>         Attachments: 4559.txt
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-4559) implement token relocation

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13439719#comment-13439719 ] 

Brandon Williams edited comment on CASSANDRA-4559 at 8/23/12 5:01 AM:
----------------------------------------------------------------------

Won't we want to be able to relocate more than one token at a time?

Also, it seems like we could re-use the moving status, or remove it if we want to rename the functionality, since doing it in a mixed cluster will never be supported.
                
      was (Author: brandon.williams):
    Won't we want to be able to relocate more than one token at a time?

Also, it seems like we re-use the moving status, or remove it if we want to rename the functionality, since doing it in a mixed cluster will never be supported.
                  
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4559) implement token relocation

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450962#comment-13450962 ] 

Jonathan Ellis commented on CASSANDRA-4559:
-------------------------------------------

bq. I suspect this is because we're using the same interval (RING_DELAY) and sometimes they coincide. We can just double the relocation removal.

I don't follow how the interval means we can remove the same token twice.
                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>         Attachments: 4559.txt
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4559) implement token relocation

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451341#comment-13451341 ] 

Brandon Williams commented on CASSANDRA-4559:
---------------------------------------------

Somehow the first replica is correct now, but some of the subsequent ones are not. :(
                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>         Attachments: 4559.txt
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4559) implement token relocation

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13443522#comment-13443522 ] 

Brandon Williams commented on CASSANDRA-4559:
---------------------------------------------

LGTM, +1
                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-4559) implement token relocation

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis updated CASSANDRA-4559:
--------------------------------------

    Affects Version/s: 1.2.0 beta 1
        Fix Version/s: 1.2.0 beta 1
    
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-4559) implement token relocation

Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448096#comment-13448096 ] 

Eric Evans edited comment on CASSANDRA-4559 at 9/5/12 8:59 AM:
---------------------------------------------------------------

Since this is a freeze-dependency, let me provide a bit of a status update:

These patches seemed "mostly" stable prior to merging w/ trunk after CASSANDRA-4383 landed.  I don't know if the problems since are the result of a faulty merge, a problem with what was committed for CASSANDRA-4383, or something else entirely (I'm still looking into it).

I say "mostly" stable because occasionally one node would seem to miss the {{NORMAL}} sent by the destination node after a successful relocation, and so that nodes idea of the state of the ring would differ from the rest of the cluster.  I'm still looking into this, or I will be after sorting out the new spate of issues (see above).

Most of the challenges here have related to network concurrency, so you if wish to test, you really need to use the shuffle utility in order to set more than one in motion at once.  The shuffle tool isn't quite done yet either, but (if interested )you can checkout at https://github.com/acunu/cassandra/tree/p/4443/060_shuffle_utility (which includes the token relocation).
                
      was (Author: urandom):
    Since this is a freeze-dependency, let me provide a bit of a status update:

These patches seemed "mostly" stable prior to merging w/ trunk after CASSANDRA-4383 landed.  I don't know if the problems since are the result of a faulty merge, a problem with what was committed for CASSANDRA-4383, or something else entirely (I'm still looking into it).

I say "mostly" stable because occasionally one node would seem to miss the {{NORMAL}} sent by the destination node after a successful relocation, and so that nodes idea of the state of the ring would differ from the rest of the cluster.  I'm still looking into this, or I will be after sorting out the new spate of issues (see above).

Most of the challenges here have related to network concurrency, so if wish to test, you really need to use the shuffle utility in order to set more than one in motion at once.  The shuffle tool isn't quite done yet either, but (if interested )you can checkout at https://github.com/acunu/cassandra/tree/p/4443/060_shuffle_utility (which includes the token relocation).
                  
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4559) implement token relocation

Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13439831#comment-13439831 ] 

Eric Evans commented on CASSANDRA-4559:
---------------------------------------

bq. Won't we want to be able to relocate more than one token at a time?

Sure, and you could do that by invoking the operation once per token.  Though now that you mention it, I had planned to make it non-blocking for that reason.

It doesn't make a huge difference to me either way; I can make the operation support multiple ranges in a call if you prefer.

bq. Also, it seems like we could re-use the moving status, or remove it if we want to rename the functionality, since doing it in a mixed cluster will never be supported.

Don't we need to continue supporting the old-style move for users that want to continue running with 1 token per node?
                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (CASSANDRA-4559) implement token relocation

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Brandon Williams updated CASSANDRA-4559:
----------------------------------------

    Attachment: 4559.txt

For all the trouble this was the figure out, the cause and solution are remarkably simple.

calculateStreamAndFetchRanges breaks on the first intersection while iterating the range, which works fine with a single token, but not with many since there can be further intersections.  Simply removing the breaks here so the loops continue makes everything work correctly.

Also, since RangeRelocator calculates ranges upon instantiation, I moved this until after the ring delay sleep for announcement, purely as an optimization so the node can catch more relocation announcements while it sleeps before deciding what to stream.

                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>         Attachments: 4559.txt
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4559) implement token relocation

Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13454980#comment-13454980 ] 

Eric Evans commented on CASSANDRA-4559:
---------------------------------------

I've merged in these changes, and it looks good in my testing; Thanks!
                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>         Attachments: 4559.txt
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-4559) implement token relocation

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Brandon Williams updated CASSANDRA-4559:
----------------------------------------

    Attachment: 4559.txt

During token conflict resolution, we were 'protecting' the relocating host by making it win, however this only occurred when received a normal state from it.  Since under concurrent relocations we have no guarantee of the order we'll receive messages in, it was possible for the 'loser' to send a normal first, causing generation comparison which could ultimately make the token disappear from that host until it restarted.  Updated patch addresses this and ignores messages from the loser.
                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>         Attachments: 4559.txt
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-4559) implement token relocation

Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eric Evans resolved CASSANDRA-4559.
-----------------------------------

    Resolution: Fixed

committed
                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>         Attachments: 4559.txt
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4559) implement token relocation

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450947#comment-13450947 ] 

Brandon Williams commented on CASSANDRA-4559:
---------------------------------------------

Every once in a long while you can see this exception:

{{noformat}}
ERROR 20:30:18,355 Exception in thread Thread[OptionalTasks:1,5,main]
java.lang.NullPointerException
        at org.apache.cassandra.locator.TokenMetadata.removeFromRelocating(TokenMetadata.java:443)
        at org.apache.cassandra.service.StorageService$2.run(StorageService.java:1252)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
{{noformat}

I suspect this is because we're using the same interval (RING_DELAY) and sometimes they coincide.  We can just double the relocation removal.
                
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>         Attachments: 4559.txt
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-4559) implement token relocation

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Brandon Williams updated CASSANDRA-4559:
----------------------------------------

    Attachment:     (was: 4559.txt)
    
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-4559) implement token relocation

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Brandon Williams updated CASSANDRA-4559:
----------------------------------------

    Attachment:     (was: 4559.txt)
    
> implement token relocation
> --------------------------
>
>                 Key: CASSANDRA-4559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4559
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Core, Tools
>    Affects Versions: 1.2.0 beta 1
>            Reporter: Eric Evans
>            Assignee: Eric Evans
>              Labels: vnodes
>             Fix For: 1.2.0 beta 1
>
>
> Whatever the specifics of a _shuffle_ (see CASSANDRA-4443), it will be necessary to relocate a range from one node to another.
> _Edit0: Linked in new patch containing tests._
> ----
> h3. Patches
> ||Compare||Raw diff||Description||
> |[010_refactor_range_move|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move]|[010_refactor_range_move.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/010_refactor_range_move...p/4443/010_refactor_range_move.diff]|No Description|
> |[020_calculate_pending|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending]|[020_calculate_pending.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/020_calculate_pending...p/4443/020_calculate_pending.diff]|No Description|
> |[030_relocate_token|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token]|[030_relocate_token.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/030_relocate_token...p/4443/030_relocate_token.diff]|No Description|
> |[040_tests|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests]|[040_tests.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4443/040_tests...p/4443/040_tests.diff]|No Description|
> ----
> _Note: These are branches managed with TopGit. If you are applying the patch output manually, you will either need to filter the TopGit metadata files (i.e. {{wget -O - <url> | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}), or remove them afterward ({{rm .topmsg .topdeps}})._

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira