You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jason Brown (JIRA)" <ji...@apache.org> on 2018/07/24 15:26:00 UTC
[jira] [Updated] (CASSANDRA-13768) auth_test.py dtest fails
intermittently due to thread race condition
[ https://issues.apache.org/jira/browse/CASSANDRA-13768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Brown updated CASSANDRA-13768:
------------------------------------
Labels: dtest (was: )
> auth_test.py dtest fails intermittently due to thread race condition
> --------------------------------------------------------------------
>
> Key: CASSANDRA-13768
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13768
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
> Reporter: Jaydeepkumar Chovatia
> Assignee: Jaydeepkumar Chovatia
> Priority: Minor
> Labels: dtest
>
> *Root cause:* dtest {{auth_test.TestAuth.system_auth_ks_is_alterable_test}} is failing intermittently and it is due to race condition between Python driver continuously sending cql query (by thread t1) to know status of cluster, and {{ccm}} is stopping the nodes (by thread t2)
> *Failure details:* http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/lastCompletedBuild/testReport/auth_test/TestAuth/system_auth_ks_is_alterable_test/
> *Solution:* Ignore {{UnavailableException}} for this particular test as it is not going to harm test case in anyway.
> Please find snippet of C* log when test fails:
> {{INFO [InternalResponseStage:8] 2017-08-15 10:44:13,213 RepairRunnable.java:337 - Repair command #2 finished in 0 seconds INFO [GossipStage:1] 2017-08-15 10:44:13,601 Gossiper.java:1019 - InetAddress /127.0.0.1 is now DOWN INFO [HANDSHAKE-/127.0.0.1] 2017-08-15 10:44:13,715 OutboundTcpConnection.java:564 - Handshaking version with /127.0.0.1 INFO [SharedPool-Worker-2] 2017-08-15 10:44:16,712 Message.java:512 - Received: OPTIONS, v=4 INFO [SharedPool-Worker-2] 2017-08-15 10:44:16,713 Message.java:512 - Received: STARTUP {CQL_VERSION=3.4.0, COMPRESSION=snappy}, v=4 INFO [SharedPool-Worker-2] 2017-08-15 10:44:16,719 Message.java:512 - Received: org.apache.cassandra.transport.messages.AuthResponse@4474c719, v=4 INFO [GossipStage:1] 2017-08-15 10:44:16,723 Gossiper.java:1019 - InetAddress /127.0.0.3 is now DOWN ERROR [SharedPool-Worker-2] 2017-08-15 10:44:16,822 Message.java:621 - Unexpected exception during request; channel = [id: 0xffc2fc9a, L:/127.0.0.2:9042 - R:/127.0.0.2:51369] java.lang.RuntimeException: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM at org.apache.cassandra.auth.CassandraRoleManager.getRole(CassandraRoleManager.java:516) ~[main/:na] at org.apache.cassandra.auth.CassandraRoleManager.canLogin(CassandraRoleManager.java:310) ~[main/:na] at org.apache.cassandra.service.ClientState.login(ClientState.java:270) ~[main/:na] at org.apache.cassandra.transport.messages.AuthResponse.execute(AuthResponse.java:79) ~[main/:na] at org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:513) [main/:na] at org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:407) [main/:na] at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-all-4.0.44.Final.jar:4.0.44.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357) [netty-all-4.0.44.Final.jar:4.0.44.Final] at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:35) [netty-all-4.0.44.Final.jar:4.0.44.Final] at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:348) [netty-all-4.0.44.Final.jar:4.0.44.Final] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_144] at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) [main/:na] at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) [main/:na] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_144] Caused by: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM at org.apache.cassandra.db.ConsistencyLevel.assureSufficientLiveNodes(ConsistencyLevel.java:334) ~[main/:na] at org.apache.cassandra.service.AbstractReadExecutor.getReadExecutor(AbstractReadExecutor.java:162) ~[main/:na] at org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.<init>(StorageProxy.java:1733) ~[main/:na] at org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1696) ~[main/:na] at org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1640) ~[main/:na] at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1544) ~[main/:na] at org.apache.cassandra.db.SinglePartitionReadCommand$Group.execute(SinglePartitionReadCommand.java:965) ~[main/:na] at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:263) ~[main/:na] at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:224) ~[main/:na] at org.apache.cassandra.auth.CassandraRoleManager.getRoleFromTable(CassandraRoleManager.java:524) ~[main/:na] at org.apache.cassandra.auth.CassandraRoleManager.getRole(CassandraRoleManager.java:506) ~[main/:na] ... 13 common frames omitted INFO [HANDSHAKE-/127.0.0.3] 2017-08-15 10:44:17,678 OutboundTcpConnection.java:564 - Handshaking version with /127.0.0.3 INFO [SharedPool-Worker-1] 2017-08-15 10:44:17,927 Message.java:512 - Received: OPTIONS, v=4 INFO [SharedPool-Worker-1] 2017-08-15 10:44:17,928 Message.java:512 - Received: STARTUP {CQL_VERSION=3.4.0, COMPRESSION=snappy}, v=4 INFO [SharedPool-Worker-2] 2017-08-15 10:44:17,932 Message.java:512 - Received: org.apache.cassandra.transport.messages.AuthResponse@3cc2a95c, v=4 INFO [StorageServiceShutdownHook] 2017-08-15 10:44:19,814 HintsService.java:212 - Paused hints dispatch INFO [StorageServiceShutdownHook] 2017-08-15 10:44:19,818 Server.java:180 - Stop listening for CQL clients INFO [StorageServiceShutdownHook] 2017-08-15 10:44:19,818 Gossiper.java:1490 - Announcing shutdown INFO [StorageServiceShutdownHook] 2017-08-15 10:44:19,819 StorageService.java:2011 - Node /127.0.0.2 state jump to shutdown}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org