You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Stefania (JIRA)" <ji...@apache.org> on 2016/07/08 03:46:11 UTC
[jira] [Commented] (CASSANDRA-11950) dtest failure in
consistency_test.TestAvailability.test_network_topology_strategy_each_quorum
[ https://issues.apache.org/jira/browse/CASSANDRA-11950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15367172#comment-15367172 ]
Stefania commented on CASSANDRA-11950:
--------------------------------------
At the moment this test is consistently failing for an entirely different reason, sample [here|http://cassci.datastax.com/job/trunk_large_dtest/14/testReport/junit/consistency_test/TestAvailability/test_network_topology_strategy_each_quorum/]. Despite the timeout, this is not related to the recent Netty upgrade (CASSANDRA-12032), rather this was introduced by CASSANDRA-11971. We cannot recycle the buffer in SP.sendMessagesToNonlocalDC, since its backing array will be used asynchronously by the message.
Patch for 3.9 and trunk:
||3.9||trunk||
|[patch|https://github.com/stef1927/cassandra/commits/11950-3.9]|[patch|https://github.com/stef1927/cassandra/commits/11950]|
|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11950-3.9-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11950-testall/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11950-3.9-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11950-dtest/]|
[~tjake] could you review? We may also want to put it in 3.8 if we are re-tagging.
--
As for the original issue, this looks like the race that was fixed by CASSANDRA-12083, cc [~beobal]. Relevant log messages:
{code}
DEBUG [MigrationStage:1] 2016-06-03 14:25:27,380 Schema.java:465 - Adding org.apache.cassandra.config.CFMetaData@5fa5b473[cfId=03b14ad0-2997-11e6-b8c7-01c3aea11be7,ksName=mytestks,cfName=users,flags=[],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.1, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={max_threshold=32, min_threshold=4}}, compression=org.apache.cassandra.schema.CompressionParams@2c86b999, extensions={}},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[age firstname lastname] | [value]],partitionKeyColumns=[ColumnDefinition{name=userid, type=org.apache.cassandra.db.marshal.Int32Type, kind=PARTITION_KEY, position=0}],clusteringColumns=[ColumnDefinition{name=column1, type=org.apache.cassandra.db.marshal.UTF8Type, kind=CLUSTERING, position=0}],keyValidator=org.apache.cassandra.db.marshal.Int32Type,columnMetadata=[ColumnDefinition{name=column1, type=org.apache.cassandra.db.marshal.UTF8Type, kind=CLUSTERING, position=0}, ColumnDefinition{name=firstname, type=org.apache.cassandra.db.marshal.UTF8Type, kind=STATIC, position=-1}, ColumnDefinition{name=value, type=org.apache.cassandra.db.marshal.BytesType, kind=REGULAR, position=-1}, ColumnDefinition{name=userid, type=org.apache.cassandra.db.marshal.Int32Type, kind=PARTITION_KEY, position=0}, ColumnDefinition{name=lastname, type=org.apache.cassandra.db.marshal.UTF8Type, kind=STATIC, position=-1}, ColumnDefinition{name=age, type=org.apache.cassandra.db.marshal.Int32Type, kind=STATIC, position=-1}],droppedColumns={},triggers=[],indexes=[]] to cfIdMap
ERROR [SharedPool-Worker-1] 2016-06-03 14:25:27,460 Keyspace.java:504 - Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 (mytestks.users)
{code}
--
The test should pass consistently provided the problem above is fixed, but there are still several warnings because the schema hasn't fully propagated yet, I wonder if we should add a pause to the test after creating the tables or if there is something else that we can do to remove the warnings.
> dtest failure in consistency_test.TestAvailability.test_network_topology_strategy_each_quorum
> ---------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-11950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11950
> Project: Cassandra
> Issue Type: Bug
> Reporter: Sean McCarthy
> Assignee: Stefania
> Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, node3.log, node3_debug.log, node4.log, node4_debug.log, node5.log, node5_debug.log, node6.log, node6_debug.log, node7.log, node7_debug.log, node8.log, node8_debug.log, node9.log, node9_debug.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_large_dtest/10/testReport/consistency_test/TestAvailability/test_network_topology_strategy_each_quorum
> Failed on CassCI build trunk_large_dtest #10
> Logs are attached.
> {code}
> Stacktrace
> File "/usr/lib/python2.7/unittest/case.py", line 358, in run
> self.tearDown()
> File "/home/automaton/cassandra-dtest/dtest.py", line 719, in tearDown
> raise AssertionError('Unexpected error in log, see stdout')
> Standard Output
> Unexpected error in node3 log, error:
> ERROR [SharedPool-Worker-1] 2016-06-03 14:25:27,460 Keyspace.java:504 - Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 (mytestks.users)
> ERROR [SharedPool-Worker-2] 2016-06-03 14:25:27,460 Keyspace.java:504 - Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 (mytestks.users)
> ERROR [SharedPool-Worker-3] 2016-06-03 14:25:27,462 Keyspace.java:504 - Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 (mytestks.users)
> ERROR [SharedPool-Worker-2] 2016-06-03 14:25:27,464 Keyspace.java:504 - Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 (mytestks.users)
> ERROR [SharedPool-Worker-3] 2016-06-03 14:25:27,464 Keyspace.java:504 - Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 (mytestks.users)
> ERROR [SharedPool-Worker-1] 2016-06-03 14:25:27,465 Keyspace.java:504 - Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 (mytestks.users)
> ERROR [SharedPool-Worker-4] 2016-06-03 14:25:27,465 Keyspace.java:504 - Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 (mytestks.users)
> ERROR [SharedPool-Worker-5] 2016-06-03 14:25:27,465 Keyspace.java:504 - Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 (mytestks.users)
> ERROR [SharedPool-Worker-7] 2016-06-03 14:25:27,465 Keyspace.java:504 - Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 (mytestks.users)
> ERROR [SharedPool-Worker-6] 2016-06-03 14:25:27,465 Keyspace.java:504 - Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 (mytestks.users)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)