You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Yuki Morishita (JIRA)" <ji...@apache.org> on 2014/04/08 23:31:16 UTC
[jira] [Commented] (CASSANDRA-7005) repair_test dtest fails on 2.1
[ https://issues.apache.org/jira/browse/CASSANDRA-7005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13963466#comment-13963466 ]
Yuki Morishita commented on CASSANDRA-7005:
-------------------------------------------
Validation is throwing exception:
{code}
ERROR [ValidationExecutor:1] 2014-04-08 16:25:11,669 CassandraDaemon.java:166 - Exception in thread Thread[ValidationExecutor:1,1,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.sstable.SSTableReader.tidy(SSTableReader.java:549) ~[main/:na]
at org.apache.cassandra.io.sstable.SSTableReader.close(SSTableReader.java:622) ~[main/:na]
at org.apache.cassandra.io.util.FileUtils.closeQuietly(FileUtils.java:217) ~[main/:na]
at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:956) ~[main/:na]
at org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:96) ~[main/:na]
at org.apache.cassandra.db.compaction.CompactionManager$8.call(CompactionManager.java:548) ~[main/:na]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
{code}
Possibly related to CASSANDRA-7000?
> repair_test dtest fails on 2.1
> ------------------------------
>
> Key: CASSANDRA-7005
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7005
> Project: Cassandra
> Issue Type: Test
> Components: Tests
> Reporter: Michael Shuler
>
> {noformat}
> $ PRINT_DEBUG=true nosetests --nocapture --nologcapture --verbosity=3 repair_test.py
> nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$']
> simple_repair_order_preserving_test (repair_test.TestRepair) ... cluster ccm directory: /tmp/dtest-BVfye7
> Starting cluster..
> Inserting data...
> Checking data on node3...
> Checking data on node1...
> Checking data on node2...
> starting repair...
> [2014-04-08 13:44:31,424] Starting repair command #1, repairing 3 ranges for keyspace ks (seq=true, full=true)
> [2014-04-08 13:44:32,748] Repair session d262e390-bf4d-11e3-a482-75998baadb41 for range (00,0113427455640312821154458202477256070484] failed with error org.apache.cassandra.exceptions.RepairException: [repair #d262e390-bf4d-11e3-a482-75998baadb41 on ks/cf, (00,0113427455640312821154458202477256070484]] Validation failed in /127.0.0.2
> [2014-04-08 13:44:32,751] Repair session d2b98f10-bf4d-11e3-a482-75998baadb41 for range (0113427455640312821154458202477256070484,56713727820156410577229101238628035242] failed with error org.apache.cassandra.exceptions.RepairException: [repair #d2b98f10-bf4d-11e3-a482-75998baadb41 on ks/cf, (0113427455640312821154458202477256070484,56713727820156410577229101238628035242]] Validation failed in /127.0.0.2
> [2014-04-08 13:44:32,753] Repair session d2dca770-bf4d-11e3-a482-75998baadb41 for range (56713727820156410577229101238628035242,00] failed with error org.apache.cassandra.exceptions.RepairException: [repair #d2dca770-bf4d-11e3-a482-75998baadb41 on ks/cf, (56713727820156410577229101238628035242,00]] Validation failed in /127.0.0.2
> [2014-04-08 13:44:32,753] Repair command #1 finished
> [2014-04-08 13:44:32,770] Nothing to repair for keyspace 'system'
> [2014-04-08 13:44:32,783] Starting repair command #2, repairing 2 ranges for keyspace system_traces (seq=true, full=true)
> [2014-04-08 13:44:34,635] Repair session d3310900-bf4d-11e3-a482-75998baadb41 for range (0113427455640312821154458202477256070484,56713727820156410577229101238628035242] finished
> [2014-04-08 13:44:34,640] Repair session d3f80280-bf4d-11e3-a482-75998baadb41 for range (56713727820156410577229101238628035242,00] finished
> [2014-04-08 13:44:34,640] Repair command #2 finished
> Repair time: 4.63053512573
> FAIL
> ERROR
> simple_repair_test (repair_test.TestRepair) ... cluster ccm directory: /tmp/dtest-_L5lTP
> Starting cluster..
> Inserting data...
> Checking data on node3...
> Checking data on node1...
> Checking data on node2...
> starting repair...
> [2014-04-08 13:47:14,109] Starting repair command #1, repairing 3 ranges for keyspace ks (seq=true, full=true)
> [2014-04-08 13:47:15,291] Repair session 335a5840-bf4e-11e3-b691-75998baadb41 for range (-3074457345618258603,3074457345618258602] failed with error org.apache.cassandra.exceptions.RepairException: [repair #335a5840-bf4e-11e3-b691-75998baadb41 on ks/cf, (-3074457345618258603,3074457345618258602]] Validation failed in /127.0.0.2
> [2014-04-08 13:47:15,292] Repair session 33ad0c20-bf4e-11e3-b691-75998baadb41 for range (-9223372036854775808,-3074457345618258603] failed with error org.apache.cassandra.exceptions.RepairException: [repair #33ad0c20-bf4e-11e3-b691-75998baadb41 on ks/cf, (-9223372036854775808,-3074457345618258603]] Validation failed in /127.0.0.2
> [2014-04-08 13:47:15,295] Repair session 33e978e0-bf4e-11e3-b691-75998baadb41 for range (3074457345618258602,-9223372036854775808] failed with error org.apache.cassandra.exceptions.RepairException: [repair #33e978e0-bf4e-11e3-b691-75998baadb41 on ks/cf, (3074457345618258602,-9223372036854775808]] Validation failed in /127.0.0.2
> [2014-04-08 13:47:15,295] Repair command #1 finished
> [2014-04-08 13:47:15,307] Nothing to repair for keyspace 'system'
> [2014-04-08 13:47:15,322] Starting repair command #2, repairing 2 ranges for keyspace system_traces (seq=true, full=true)
> [2014-04-08 13:47:15,983] Repair session 3412f9e0-bf4e-11e3-b691-75998baadb41 for range (-3074457345618258603,3074457345618258602] finished
> [2014-04-08 13:47:15,988] Repair session 345d9770-bf4e-11e3-b691-75998baadb41 for range (3074457345618258602,-9223372036854775808] finished
> [2014-04-08 13:47:15,988] Repair command #2 finished
> Repair time: 3.25424218178
> FAIL
> ERROR
> ======================================================================
> ERROR: simple_repair_order_preserving_test (repair_test.TestRepair)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/home/mshuler/git/cassandra-dtest/dtest.py", line 202, in tearDown
> raise AssertionError('Unexpected error in %s node log: %s' % (node.name, errors))
> AssertionError: Unexpected error in node1 node log: ['ERROR [AntiEntropySessions:1] 2014-04-08 13:44:32,011 RepairSession.java:289 - [repair #d262e390-bf4d-11e3-a482-75998baadb41] session completed with the following error\n', 'ERROR [AntiEntropySessions:1] 2014-04-08 13:44:32,013 CassandraDaemon.java:166 - Exception in thread Thread[AntiEntropySessions:1,5,RMI Runtime]\n', 'ERROR [AntiEntropySessions:2] 2014-04-08 13:44:32,247 RepairSession.java:289 - [repair #d2b98f10-bf4d-11e3-a482-75998baadb41] session completed with the following error\n', 'ERROR [AntiEntropySessions:2] 2014-04-08 13:44:32,249 CassandraDaemon.java:166 - Exception in thread Thread[AntiEntropySessions:2,5,RMI Runtime]\n', 'ERROR [Thread-8] 2014-04-08 13:44:32,747 StorageService.java:2659 - Repair session d262e390-bf4d-11e3-a482-75998baadb41 for range (00,0113427455640312821154458202477256070484] failed with error org.apache.cassandra.exceptions.RepairException: [repair #d262e390-bf4d-11e3-a482-75998baadb41 on ks/cf, (00,0113427455640312821154458202477256070484]] Validation failed in /127.0.0.2\n', 'ERROR [AntiEntropySessions:3] 2014-04-08 13:44:32,749 RepairSession.java:289 - [repair #d2dca770-bf4d-11e3-a482-75998baadb41] session completed with the following error\n', 'ERROR [AntiEntropySessions:3] 2014-04-08 13:44:32,750 CassandraDaemon.java:166 - Exception in thread Thread[AntiEntropySessions:3,5,RMI Runtime]\n', 'ERROR [Thread-8] 2014-04-08 13:44:32,751 StorageService.java:2659 - Repair session d2b98f10-bf4d-11e3-a482-75998baadb41 for range (0113427455640312821154458202477256070484,56713727820156410577229101238628035242] failed with error org.apache.cassandra.exceptions.RepairException: [repair #d2b98f10-bf4d-11e3-a482-75998baadb41 on ks/cf, (0113427455640312821154458202477256070484,56713727820156410577229101238628035242]] Validation failed in /127.0.0.2\n', 'ERROR [Thread-8] 2014-04-08 13:44:32,752 StorageService.java:2659 - Repair session d2dca770-bf4d-11e3-a482-75998baadb41 for range (56713727820156410577229101238628035242,00] failed with error org.apache.cassandra.exceptions.RepairException: [repair #d2dca770-bf4d-11e3-a482-75998baadb41 on ks/cf, (56713727820156410577229101238628035242,00]] Validation failed in /127.0.0.2\n']
> ======================================================================
> ERROR: simple_repair_test (repair_test.TestRepair)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/home/mshuler/git/cassandra-dtest/dtest.py", line 202, in tearDown
> raise AssertionError('Unexpected error in %s node log: %s' % (node.name, errors))
> AssertionError: Unexpected error in node1 node log: ['ERROR [AntiEntropySessions:1] 2014-04-08 13:47:14,712 RepairSession.java:289 - [repair #335a5840-bf4e-11e3-b691-75998baadb41] session completed with the following error\n', 'ERROR [AntiEntropySessions:1] 2014-04-08 13:47:14,714 CassandraDaemon.java:166 - Exception in thread Thread[AntiEntropySessions:1,5,RMI Runtime]\n', 'ERROR [AntiEntropySessions:2] 2014-04-08 13:47:15,058 RepairSession.java:289 - [repair #33ad0c20-bf4e-11e3-b691-75998baadb41] session completed with the following error\n', 'ERROR [AntiEntropySessions:2] 2014-04-08 13:47:15,058 CassandraDaemon.java:166 - Exception in thread Thread[AntiEntropySessions:2,5,RMI Runtime]\n', 'ERROR [AntiEntropySessions:3] 2014-04-08 13:47:15,288 RepairSession.java:289 - [repair #33e978e0-bf4e-11e3-b691-75998baadb41] session completed with the following error\n', 'ERROR [AntiEntropySessions:3] 2014-04-08 13:47:15,289 CassandraDaemon.java:166 - Exception in thread Thread[AntiEntropySessions:3,5,RMI Runtime]\n', 'ERROR [Thread-8] 2014-04-08 13:47:15,290 StorageService.java:2659 - Repair session 335a5840-bf4e-11e3-b691-75998baadb41 for range (-3074457345618258603,3074457345618258602] failed with error org.apache.cassandra.exceptions.RepairException: [repair #335a5840-bf4e-11e3-b691-75998baadb41 on ks/cf, (-3074457345618258603,3074457345618258602]] Validation failed in /127.0.0.2\n', 'ERROR [Thread-8] 2014-04-08 13:47:15,291 StorageService.java:2659 - Repair session 33ad0c20-bf4e-11e3-b691-75998baadb41 for range (-9223372036854775808,-3074457345618258603] failed with error org.apache.cassandra.exceptions.RepairException: [repair #33ad0c20-bf4e-11e3-b691-75998baadb41 on ks/cf, (-9223372036854775808,-3074457345618258603]] Validation failed in /127.0.0.2\n', 'ERROR [Thread-8] 2014-04-08 13:47:15,294 StorageService.java:2659 - Repair session 33e978e0-bf4e-11e3-b691-75998baadb41 for range (3074457345618258602,-9223372036854775808] failed with error org.apache.cassandra.exceptions.RepairException: [repair #33e978e0-bf4e-11e3-b691-75998baadb41 on ks/cf, (3074457345618258602,-9223372036854775808]] Validation failed in /127.0.0.2\n']
> ======================================================================
> FAIL: simple_repair_order_preserving_test (repair_test.TestRepair)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/home/mshuler/git/cassandra-dtest/repair_test.py", line 39, in simple_repair_order_preserving_test
> self._simple_repair(order_preserving_partitioner=True)
> File "/home/mshuler/git/cassandra-dtest/repair_test.py", line 95, in _simple_repair
> assert len(l) == 2, "Lines matching: " + str([elt[0] for elt in l])
> AssertionError: Lines matching: []
> ======================================================================
> FAIL: simple_repair_test (repair_test.TestRepair)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/home/mshuler/git/cassandra-dtest/repair_test.py", line 36, in simple_repair_test
> self._simple_repair()
> File "/home/mshuler/git/cassandra-dtest/repair_test.py", line 95, in _simple_repair
> assert len(l) == 2, "Lines matching: " + str([elt[0] for elt in l])
> AssertionError: Lines matching: []
> ----------------------------------------------------------------------
> Ran 2 tests in 324.740s
> FAILED (errors=2, failures=2)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)