You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Christian Spriegel (Created) (JIRA)" <ji...@apache.org> on 2011/10/16 18:36:11 UTC

[jira] [Created] (CASSANDRA-3369) AssertionError when adding a node and doing repair, repair hangs

AssertionError when adding a node and doing repair, repair hangs
----------------------------------------------------------------

                 Key: CASSANDRA-3369
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3369
             Project: Cassandra
          Issue Type: Bug
          Components: Core
    Affects Versions: 1.0.0
         Environment: Ubuntu Linux amd64, 1.0.0-rc2
            Reporter: Christian Spriegel


Hi again,

I was playing aroung with Cassandra 1.0.0-rc2 and got an AssertionError. The cluster I set up was two cassandra nodes on one laptop using different 127.0.0.* loopback devices. Both nodes have separate folders on the harddisk.


Here is what I did:


1. Started node1 and inserted some data into it using a simple singlethreaded testprogram (uses hector 0.8.0-2):
127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       100.00% Token(bytes[63e5b6995466cd3221cba16646ae19ed])

2. I started another node, node 2 = 127.0.0.2:
127.0.0.2       datacenter1 rack1       Up     Normal  147.57 KB       50.00%  Token(bytes[4d6ccfeaa8bb59551751a2816fde9343])
127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       50.00%  Token(bytes[63e5b6995466cd3221cba16646ae19ed])

3. I triggered a "nodetool -h 127.0.0.1  repair" on the first node that had the data from my test.


This repair does not seem to ever end. The nodetool is hanging now but my computer is idle. I get an AssertionError on the first node:
java.lang.AssertionError
	at org.apache.cassandra.service.AntiEntropyService$Validator.prepare(AntiEntropyService.java:283)
	at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:825)
	at org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:63)
	at org.apache.cassandra.db.compaction.CompactionManager$6.call(CompactionManager.java:432)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)


kind regards,
Christian

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (CASSANDRA-3369) AssertionError when adding a node and doing repair, repair hangs

Posted by "Christian Spriegel (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Christian Spriegel updated CASSANDRA-3369:
------------------------------------------

    Attachment: NODE2_system.log
                NODE2_cassandra.yaml
                NODE1_system.log
                NODE1_cassandra.yaml

Added config and log files from both cassandra nodes.
                
> AssertionError when adding a node and doing repair, repair hangs
> ----------------------------------------------------------------
>
>                 Key: CASSANDRA-3369
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3369
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.0
>         Environment: Ubuntu Linux amd64, 1.0.0-rc2
>            Reporter: Christian Spriegel
>         Attachments: NODE1_cassandra.yaml, NODE1_system.log, NODE2_cassandra.yaml, NODE2_system.log
>
>
> Hi again,
> I was playing aroung with Cassandra 1.0.0-rc2 and got an AssertionError. The cluster I set up was two cassandra nodes on one laptop using different 127.0.0.* loopback devices. Both nodes have separate folders on the harddisk.
> Here is what I did:
> 1. Started node1 and inserted some data into it using a simple singlethreaded testprogram (uses hector 0.8.0-2):
> 127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       100.00% Token(bytes[63e5b6995466cd3221cba16646ae19ed])
> 2. I started another node, node 2 = 127.0.0.2:
> 127.0.0.2       datacenter1 rack1       Up     Normal  147.57 KB       50.00%  Token(bytes[4d6ccfeaa8bb59551751a2816fde9343])
> 127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       50.00%  Token(bytes[63e5b6995466cd3221cba16646ae19ed])
> 3. I triggered a "nodetool -h 127.0.0.1  repair" on the first node that had the data from my test.
> This repair does not seem to ever end. The nodetool is hanging now but my computer is idle. I get an AssertionError on the first node:
> java.lang.AssertionError
> 	at org.apache.cassandra.service.AntiEntropyService$Validator.prepare(AntiEntropyService.java:283)
> 	at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:825)
> 	at org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:63)
> 	at org.apache.cassandra.db.compaction.CompactionManager$6.call(CompactionManager.java:432)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> 	at java.lang.Thread.run(Thread.java:662)
> kind regards,
> Christian

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-3369) AssertionError when adding a node and doing repair, repair hangs

Posted by "Christian Spriegel (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128541#comment-13128541 ] 

Christian Spriegel commented on CASSANDRA-3369:
-----------------------------------------------

Update: I also had problems because I was using Defalte compression, which seems to be broken. The AssertionError during repair still occurs with Snappy Compression.
                
> AssertionError when adding a node and doing repair, repair hangs
> ----------------------------------------------------------------
>
>                 Key: CASSANDRA-3369
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3369
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.0
>         Environment: Ubuntu Linux amd64, 1.0.0-rc2
>            Reporter: Christian Spriegel
>            Assignee: Sylvain Lebresne
>         Attachments: NODE1_cassandra.yaml, NODE1_system.log, NODE2_cassandra.yaml, NODE2_system.log
>
>
> Hi again,
> I was playing aroung with Cassandra 1.0.0-rc2 and got an AssertionError. The cluster I set up was two cassandra nodes on one laptop using different 127.0.0.* loopback devices. Both nodes have separate folders on the harddisk.
> Here is what I did:
> 1. Started node1 and inserted some data into it using a simple singlethreaded testprogram (uses hector 0.8.0-2):
> 127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       100.00% Token(bytes[63e5b6995466cd3221cba16646ae19ed])
> 2. I started another node, node 2 = 127.0.0.2:
> 127.0.0.2       datacenter1 rack1       Up     Normal  147.57 KB       50.00%  Token(bytes[4d6ccfeaa8bb59551751a2816fde9343])
> 127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       50.00%  Token(bytes[63e5b6995466cd3221cba16646ae19ed])
> 3. I triggered a "nodetool -h 127.0.0.1  repair" on the first node that had the data from my test.
> This repair does not seem to ever end. The nodetool is hanging now but my computer is idle. I get an AssertionError on the first node:
> java.lang.AssertionError
> 	at org.apache.cassandra.service.AntiEntropyService$Validator.prepare(AntiEntropyService.java:283)
> 	at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:825)
> 	at org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:63)
> 	at org.apache.cassandra.db.compaction.CompactionManager$6.call(CompactionManager.java:432)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> 	at java.lang.Thread.run(Thread.java:662)
> Update: I dont know if it is important but here is the schema of my test:
> create keyspace Test with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:2};
> CREATE COLUMN FAMILY Response WITH key_validation_class=BytesType AND compression_options={sstable_compression:DeflateCompressor};
> kind regards,
> Christian

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (CASSANDRA-3369) AssertionError when adding a node and doing repair, repair hangs

Posted by "Sylvain Lebresne (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sylvain Lebresne updated CASSANDRA-3369:
----------------------------------------

    Attachment: 3369.patch

This is due to the code not handling correctly a case of having no sample keys for one of it's token range.

Let's not that:
  * this is not specific to 1.0.0, 0.8 is affected too.
  * this only affect order preserving partitioner (not because the order matter but because this code path is not taken with RandomPartitioner since a few releases now, at least as far as repair is concerned).
  * this would be unlikely to show in any serious production setting, since having no sample for a given range means that either you have almost no rows in the cluster or that the cluster is extremely badly balanced.

Patch attached to fix (against 0.8).
                
> AssertionError when adding a node and doing repair, repair hangs
> ----------------------------------------------------------------
>
>                 Key: CASSANDRA-3369
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3369
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.8.7, 1.0.0
>         Environment: Ubuntu Linux amd64, 1.0.0-rc2
>            Reporter: Christian Spriegel
>            Assignee: Sylvain Lebresne
>             Fix For: 0.8.8, 1.0.1
>
>         Attachments: 3369.patch, NODE1_cassandra.yaml, NODE1_system.log, NODE2_cassandra.yaml, NODE2_system.log
>
>
> Hi again,
> I was playing aroung with Cassandra 1.0.0-rc2 and got an AssertionError. The cluster I set up was two cassandra nodes on one laptop using different 127.0.0.* loopback devices. Both nodes have separate folders on the harddisk.
> Here is what I did:
> 1. Started node1 and inserted some data into it using a simple singlethreaded testprogram (uses hector 0.8.0-2):
> 127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       100.00% Token(bytes[63e5b6995466cd3221cba16646ae19ed])
> 2. I started another node, node 2 = 127.0.0.2:
> 127.0.0.2       datacenter1 rack1       Up     Normal  147.57 KB       50.00%  Token(bytes[4d6ccfeaa8bb59551751a2816fde9343])
> 127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       50.00%  Token(bytes[63e5b6995466cd3221cba16646ae19ed])
> 3. I triggered a "nodetool -h 127.0.0.1  repair" on the first node that had the data from my test.
> This repair does not seem to ever end. The nodetool is hanging now but my computer is idle. I get an AssertionError on the first node:
> java.lang.AssertionError
> 	at org.apache.cassandra.service.AntiEntropyService$Validator.prepare(AntiEntropyService.java:283)
> 	at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:825)
> 	at org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:63)
> 	at org.apache.cassandra.db.compaction.CompactionManager$6.call(CompactionManager.java:432)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> 	at java.lang.Thread.run(Thread.java:662)
> Update: I dont know if it is important but here is the schema of my test:
> create keyspace Test with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:2};
> CREATE COLUMN FAMILY Response WITH key_validation_class=BytesType AND compression_options={sstable_compression:DeflateCompressor};
> kind regards,
> Christian

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-3369) AssertionError when adding a node and doing repair, repair hangs

Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13132769#comment-13132769 ] 

Hudson commented on CASSANDRA-3369:
-----------------------------------

Integrated in Cassandra-0.8 #384 (See [https://builds.apache.org/job/Cassandra-0.8/384/])
    fix assertion error during repair with ordered partitioners
patch by slebresne; reviewed by jbellis for CASSANDRA-3369

slebresne : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1187333
Files : 
* /cassandra/branches/cassandra-0.8/CHANGES.txt
* /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
* /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/AntiEntropyService.java

                
> AssertionError when adding a node and doing repair, repair hangs
> ----------------------------------------------------------------
>
>                 Key: CASSANDRA-3369
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3369
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.8.7, 1.0.0
>         Environment: Ubuntu Linux amd64, 1.0.0-rc2
>            Reporter: Christian Spriegel
>            Assignee: Sylvain Lebresne
>             Fix For: 0.8.8, 1.0.1
>
>         Attachments: 3369.patch, NODE1_cassandra.yaml, NODE1_system.log, NODE2_cassandra.yaml, NODE2_system.log
>
>
> Hi again,
> I was playing aroung with Cassandra 1.0.0-rc2 and got an AssertionError. The cluster I set up was two cassandra nodes on one laptop using different 127.0.0.* loopback devices. Both nodes have separate folders on the harddisk.
> Here is what I did:
> 1. Started node1 and inserted some data into it using a simple singlethreaded testprogram (uses hector 0.8.0-2):
> 127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       100.00% Token(bytes[63e5b6995466cd3221cba16646ae19ed])
> 2. I started another node, node 2 = 127.0.0.2:
> 127.0.0.2       datacenter1 rack1       Up     Normal  147.57 KB       50.00%  Token(bytes[4d6ccfeaa8bb59551751a2816fde9343])
> 127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       50.00%  Token(bytes[63e5b6995466cd3221cba16646ae19ed])
> 3. I triggered a "nodetool -h 127.0.0.1  repair" on the first node that had the data from my test.
> This repair does not seem to ever end. The nodetool is hanging now but my computer is idle. I get an AssertionError on the first node:
> java.lang.AssertionError
> 	at org.apache.cassandra.service.AntiEntropyService$Validator.prepare(AntiEntropyService.java:283)
> 	at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:825)
> 	at org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:63)
> 	at org.apache.cassandra.db.compaction.CompactionManager$6.call(CompactionManager.java:432)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> 	at java.lang.Thread.run(Thread.java:662)
> Update: I dont know if it is important but here is the schema of my test:
> create keyspace Test with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:2};
> CREATE COLUMN FAMILY Response WITH key_validation_class=BytesType AND compression_options={sstable_compression:DeflateCompressor};
> kind regards,
> Christian

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (CASSANDRA-3369) AssertionError when adding a node and doing repair, repair hangs

Posted by "Christian Spriegel (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Christian Spriegel updated CASSANDRA-3369:
------------------------------------------

    Description: 
Hi again,

I was playing aroung with Cassandra 1.0.0-rc2 and got an AssertionError. The cluster I set up was two cassandra nodes on one laptop using different 127.0.0.* loopback devices. Both nodes have separate folders on the harddisk.


Here is what I did:


1. Started node1 and inserted some data into it using a simple singlethreaded testprogram (uses hector 0.8.0-2):
127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       100.00% Token(bytes[63e5b6995466cd3221cba16646ae19ed])

2. I started another node, node 2 = 127.0.0.2:
127.0.0.2       datacenter1 rack1       Up     Normal  147.57 KB       50.00%  Token(bytes[4d6ccfeaa8bb59551751a2816fde9343])
127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       50.00%  Token(bytes[63e5b6995466cd3221cba16646ae19ed])

3. I triggered a "nodetool -h 127.0.0.1  repair" on the first node that had the data from my test.


This repair does not seem to ever end. The nodetool is hanging now but my computer is idle. I get an AssertionError on the first node:
java.lang.AssertionError
	at org.apache.cassandra.service.AntiEntropyService$Validator.prepare(AntiEntropyService.java:283)
	at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:825)
	at org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:63)
	at org.apache.cassandra.db.compaction.CompactionManager$6.call(CompactionManager.java:432)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)


Update: I dont know if it is important but here is the schema of my test:
create keyspace Test with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:2};
CREATE COLUMN FAMILY Response WITH key_validation_class=BytesType AND compression_options={sstable_compression:DeflateCompressor};


kind regards,
Christian

  was:
Hi again,

I was playing aroung with Cassandra 1.0.0-rc2 and got an AssertionError. The cluster I set up was two cassandra nodes on one laptop using different 127.0.0.* loopback devices. Both nodes have separate folders on the harddisk.


Here is what I did:


1. Started node1 and inserted some data into it using a simple singlethreaded testprogram (uses hector 0.8.0-2):
127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       100.00% Token(bytes[63e5b6995466cd3221cba16646ae19ed])

2. I started another node, node 2 = 127.0.0.2:
127.0.0.2       datacenter1 rack1       Up     Normal  147.57 KB       50.00%  Token(bytes[4d6ccfeaa8bb59551751a2816fde9343])
127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       50.00%  Token(bytes[63e5b6995466cd3221cba16646ae19ed])

3. I triggered a "nodetool -h 127.0.0.1  repair" on the first node that had the data from my test.


This repair does not seem to ever end. The nodetool is hanging now but my computer is idle. I get an AssertionError on the first node:
java.lang.AssertionError
	at org.apache.cassandra.service.AntiEntropyService$Validator.prepare(AntiEntropyService.java:283)
	at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:825)
	at org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:63)
	at org.apache.cassandra.db.compaction.CompactionManager$6.call(CompactionManager.java:432)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)


kind regards,
Christian

    
> AssertionError when adding a node and doing repair, repair hangs
> ----------------------------------------------------------------
>
>                 Key: CASSANDRA-3369
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3369
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.0
>         Environment: Ubuntu Linux amd64, 1.0.0-rc2
>            Reporter: Christian Spriegel
>         Attachments: NODE1_cassandra.yaml, NODE1_system.log, NODE2_cassandra.yaml, NODE2_system.log
>
>
> Hi again,
> I was playing aroung with Cassandra 1.0.0-rc2 and got an AssertionError. The cluster I set up was two cassandra nodes on one laptop using different 127.0.0.* loopback devices. Both nodes have separate folders on the harddisk.
> Here is what I did:
> 1. Started node1 and inserted some data into it using a simple singlethreaded testprogram (uses hector 0.8.0-2):
> 127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       100.00% Token(bytes[63e5b6995466cd3221cba16646ae19ed])
> 2. I started another node, node 2 = 127.0.0.2:
> 127.0.0.2       datacenter1 rack1       Up     Normal  147.57 KB       50.00%  Token(bytes[4d6ccfeaa8bb59551751a2816fde9343])
> 127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       50.00%  Token(bytes[63e5b6995466cd3221cba16646ae19ed])
> 3. I triggered a "nodetool -h 127.0.0.1  repair" on the first node that had the data from my test.
> This repair does not seem to ever end. The nodetool is hanging now but my computer is idle. I get an AssertionError on the first node:
> java.lang.AssertionError
> 	at org.apache.cassandra.service.AntiEntropyService$Validator.prepare(AntiEntropyService.java:283)
> 	at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:825)
> 	at org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:63)
> 	at org.apache.cassandra.db.compaction.CompactionManager$6.call(CompactionManager.java:432)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> 	at java.lang.Thread.run(Thread.java:662)
> Update: I dont know if it is important but here is the schema of my test:
> create keyspace Test with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:2};
> CREATE COLUMN FAMILY Response WITH key_validation_class=BytesType AND compression_options={sstable_compression:DeflateCompressor};
> kind regards,
> Christian

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-3369) AssertionError when adding a node and doing repair, repair hangs

Posted by "Jonathan Ellis (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13132092#comment-13132092 ] 

Jonathan Ellis commented on CASSANDRA-3369:
-------------------------------------------

+1
                
> AssertionError when adding a node and doing repair, repair hangs
> ----------------------------------------------------------------
>
>                 Key: CASSANDRA-3369
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3369
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.8.7, 1.0.0
>         Environment: Ubuntu Linux amd64, 1.0.0-rc2
>            Reporter: Christian Spriegel
>            Assignee: Sylvain Lebresne
>             Fix For: 0.8.8, 1.0.1
>
>         Attachments: 3369.patch, NODE1_cassandra.yaml, NODE1_system.log, NODE2_cassandra.yaml, NODE2_system.log
>
>
> Hi again,
> I was playing aroung with Cassandra 1.0.0-rc2 and got an AssertionError. The cluster I set up was two cassandra nodes on one laptop using different 127.0.0.* loopback devices. Both nodes have separate folders on the harddisk.
> Here is what I did:
> 1. Started node1 and inserted some data into it using a simple singlethreaded testprogram (uses hector 0.8.0-2):
> 127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       100.00% Token(bytes[63e5b6995466cd3221cba16646ae19ed])
> 2. I started another node, node 2 = 127.0.0.2:
> 127.0.0.2       datacenter1 rack1       Up     Normal  147.57 KB       50.00%  Token(bytes[4d6ccfeaa8bb59551751a2816fde9343])
> 127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       50.00%  Token(bytes[63e5b6995466cd3221cba16646ae19ed])
> 3. I triggered a "nodetool -h 127.0.0.1  repair" on the first node that had the data from my test.
> This repair does not seem to ever end. The nodetool is hanging now but my computer is idle. I get an AssertionError on the first node:
> java.lang.AssertionError
> 	at org.apache.cassandra.service.AntiEntropyService$Validator.prepare(AntiEntropyService.java:283)
> 	at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:825)
> 	at org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:63)
> 	at org.apache.cassandra.db.compaction.CompactionManager$6.call(CompactionManager.java:432)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> 	at java.lang.Thread.run(Thread.java:662)
> Update: I dont know if it is important but here is the schema of my test:
> create keyspace Test with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:2};
> CREATE COLUMN FAMILY Response WITH key_validation_class=BytesType AND compression_options={sstable_compression:DeflateCompressor};
> kind regards,
> Christian

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Assigned] (CASSANDRA-3369) AssertionError when adding a node and doing repair, repair hangs

Posted by "Brandon Williams (Assigned) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Brandon Williams reassigned CASSANDRA-3369:
-------------------------------------------

    Assignee: Sylvain Lebresne
    
> AssertionError when adding a node and doing repair, repair hangs
> ----------------------------------------------------------------
>
>                 Key: CASSANDRA-3369
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3369
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.0
>         Environment: Ubuntu Linux amd64, 1.0.0-rc2
>            Reporter: Christian Spriegel
>            Assignee: Sylvain Lebresne
>         Attachments: NODE1_cassandra.yaml, NODE1_system.log, NODE2_cassandra.yaml, NODE2_system.log
>
>
> Hi again,
> I was playing aroung with Cassandra 1.0.0-rc2 and got an AssertionError. The cluster I set up was two cassandra nodes on one laptop using different 127.0.0.* loopback devices. Both nodes have separate folders on the harddisk.
> Here is what I did:
> 1. Started node1 and inserted some data into it using a simple singlethreaded testprogram (uses hector 0.8.0-2):
> 127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       100.00% Token(bytes[63e5b6995466cd3221cba16646ae19ed])
> 2. I started another node, node 2 = 127.0.0.2:
> 127.0.0.2       datacenter1 rack1       Up     Normal  147.57 KB       50.00%  Token(bytes[4d6ccfeaa8bb59551751a2816fde9343])
> 127.0.0.1       datacenter1 rack1       Up     Normal  583.55 MB       50.00%  Token(bytes[63e5b6995466cd3221cba16646ae19ed])
> 3. I triggered a "nodetool -h 127.0.0.1  repair" on the first node that had the data from my test.
> This repair does not seem to ever end. The nodetool is hanging now but my computer is idle. I get an AssertionError on the first node:
> java.lang.AssertionError
> 	at org.apache.cassandra.service.AntiEntropyService$Validator.prepare(AntiEntropyService.java:283)
> 	at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:825)
> 	at org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:63)
> 	at org.apache.cassandra.db.compaction.CompactionManager$6.call(CompactionManager.java:432)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> 	at java.lang.Thread.run(Thread.java:662)
> Update: I dont know if it is important but here is the schema of my test:
> create keyspace Test with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:2};
> CREATE COLUMN FAMILY Response WITH key_validation_class=BytesType AND compression_options={sstable_compression:DeflateCompressor};
> kind regards,
> Christian

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira