You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kudu.apache.org by "Grant Henke (Jira)" <ji...@apache.org> on 2020/06/03 15:21:00 UTC
[jira] [Updated] (KUDU-2880) TestSecurity is flaky
[ https://issues.apache.org/jira/browse/KUDU-2880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Henke updated KUDU-2880:
------------------------------
Component/s: test
> TestSecurity is flaky
> ---------------------
>
> Key: KUDU-2880
> URL: https://issues.apache.org/jira/browse/KUDU-2880
> Project: Kudu
> Issue Type: Test
> Components: test
> Reporter: Hao Hao
> Priority: Major
> Attachments: test-output.txt
>
>
> A recent run of TestSecurity failed with the following error:
> {noformat}
> There was 1 failure:
> 1) testExternallyProvidedSubjectRefreshedExternally(org.apache.kudu.client.TestSecurity)
> org.apache.kudu.client.NonRecoverableException: cannot complete before timeout: KuduRpc(method=ListTabletServers, tablet=null, attempt=26, TimeoutTracker(timeout=30000, elapsed=29608), Traces: [0ms] refreshing cache from master, [46ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.126:46581, [63ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.124:43241, [69ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.126:46581: Network error: Failed to connect to peer master-127.0.202.126:46581(127.0.202.126:46581): Connection refused: /127.0.202.126:46581, [70ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.125:43873, [250ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.125:43873: Network error: [peer master-127.0.202.125:43873(127.0.202.125:43873)] unexpected exception from downstream on [id: 0x2fae7299, /127.0.0.1:57014 => /127.0.202.125:43873], [282ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.124:43241: OK, [336ms] delaying RPC due to: Service unavailable: Master config (127.0.202.126:46581,127.0.202.124:43241,127.0.202.125:43873) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: Failed to connect to peer master-127.0.202.126:46581(127.0.202.126:46581): Connection refused: /127.0.202.126:46581,org.apache.kudu.client.RecoverableException: [peer master-127.0.202.125:43873(127.0.202.125:43873)] unexpected exception from downstream on [id: 0x2fae7299, /127.0.0.1:57014 => /127.0.202.125:43873], [357ms] refreshing cache from master, [358ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.126:46581, [358ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.124:43241, [360ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.126:46581: Network error: java.net.ConnectException: Connection refused: /127.0.202.126:46581, [360ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.125:43873, [361ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.125:43873: Network error: Failed to connect to peer master-127.0.202.125:43873(127.0.202.125:43873): Connection refused: /127.0.202.125:43873, [363ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.124:43241: OK, [364ms] delaying RPC due to: Service unavailable: Master config (127.0.202.126:46581,127.0.202.124:43241,127.0.202.125:43873) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.126:46581,org.apache.kudu.client.RecoverableException: Failed to connect to peer master-127.0.202.125:43873(127.0.202.125:43873): Connection refused: /127.0.202.125:43873, [376ms] refreshing cache from master, [377ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.126:46581, [377ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.124:43241, [378ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.125:43873, [379ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.126:46581: Network error: Failed to connect to peer master-127.0.202.126:46581(127.0.202.126:46581): Connection refused: /127.0.202.126:46581, [381ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.125:43873: Network error: java.net.ConnectException: Connection refused: /127.0.202.125:43873, [382ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.124:43241: OK, [383ms] delaying RPC due to: Service unavailable: Master config (127.0.202.126:46581,127.0.202.124:43241,127.0.202.125:43873) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: Failed to connect to peer master-127.0.202.126:46581(127.0.202.126:46581): Connection refused: /127.0.202.126:46581,org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.125:43873, [397ms] refreshing cache from master, [397ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.126:46581, [398ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.124:43241, [399ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.126:46581: Network error: Failed to connect to peer master-127.0.202.126:46581(127.0.202.126:46581): Connection refused: /127.0.202.126:46581, [402ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.124:43241: OK, [408ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.125:43873, [410ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.125:43873: Network error: Failed to connect to peer master-127.0.202.125:43873(127.0.202.125:43873): Connection refused: /127.0.202.125:43873, [411ms] delaying RPC due to: Service unavailable: Master config (127.0.202.126:46581,127.0.202.124:43241,127.0.202.125:43873) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: Failed to connect to peer master-127.0.202.126:46581(127.0.202.126:46581): Connection refused: /127.0.202.126:46581,org.apache.kudu.client.RecoverableException: Failed to connect to peer master-127.0.202.125:43873(127.0.202.125:43873): Connection refused: /127.0.202.125:43873, [416ms] refreshing cache from master, [416ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.126:46581, [417ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.124:43241, [418ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.126:46581: Network error: java.net.ConnectException: Connection refused: /127.0.202.126:46581, [418ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.125:43873, [419ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.125:43873: Network error: java.net.ConnectException: Connection refused: /127.0.202.125:43873, [421ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.124:43241: OK, [422ms] delaying RPC due to: Service unavailable: Master config (127.0.202.126:46581,127.0.202.124:43241,127.0.202.125:43873) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.126:46581,org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.125:43873, [457ms] refreshing cache from master, [457ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.126:46581, [457ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.124:43241, [460ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.125:43873, [460ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.126:46581: Network error: java.net.ConnectException: Connection refused: /127.0.202.126:46581, [462ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.125:43873: Network error: Failed to connect to peer master-127.0.202.125:43873(127.0.202.125:43873): Connection refused: /127.0.202.125:43873, [463ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.124:43241: OK, [465ms] delaying RPC due to: Service unavailable: Master config (127.0.202.126:46581,127.0.202.124:43241,127.0.202.125:43873) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.126:46581,org.apache.kudu.client.RecoverableException: Failed to connect to peer master-127.0.202.125:43873(127.0.202.125:43873): Connection refused: /127.0.202.125:43873, [517ms] refreshing cache from master, [517ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.126:46581, [517ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.124:43241, [518ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.125:43873, [520ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.126:46581: Network error: java.net.ConnectException: Connection refused: /127.0.202.126:46581, [523ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.124:43241: OK, [526ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.125:43873: Network error: java.net.ConnectException: Connection refused: /127.0.202.125:43873, [527ms] delaying RPC due to: Service unavailable: Master config (127.0.202.126:46581,127.0.202.124:43241,127.0.202.125:43873) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.126:46581,org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.125:43873, [597ms] refreshing cache from master, [597ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.126:46581, [598ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.124:43241, [599ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.125:43873, [601ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.126:46581: Network error: Failed to connect to peer master-127.0.202.126:46581(127.0.202.126:46581): Connection refused: /127.0.202.126:46581, [602ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.124:43241: OK, [604ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.125:43873: Network error: java.net.ConnectException: Connection refused: /127.0.202.125:43873, [605ms] delaying RPC due to: Service unavailable: Master config (127.0.202.126:46581,127.0.202.124:43241,127.0.202.125:43873) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: Failed to connect to peer master-127.0.202.126:46581(127.0.202.126:46581): Connection refused: /127.0.202.126:46581,org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.125:43873, [736ms] refreshing cache from master, [737ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.126:46581, [737ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.124:43241, [738ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.125:43873, [739ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.126:46581: Network error: java.net.ConnectException: Connection refused: /127.0.202.126:46581, [741ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.125:43873: Network error: java.net.ConnectException: Connection refused: /127.0.202.125:43873, [743ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.124:43241: OK, [744ms] delaying RPC due to: Service unavailable: Master config (127.0.202.126:46581,127.0.202.124:43241,127.0.202.125:43873) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.126:46581,org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.125:43873, [797ms] refreshing cache from master, [797ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.126:46581, [798ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.124:43241, [798ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.125:43873, [801ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.126:46581: Network error: java.net.ConnectException: Connection refused: /127.0.202.126:46581, [802ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.124:43241: OK, [802ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.125:43873: Network error: java.net.ConnectException: Connection refused: /127.0.202.125:43873, [804ms] delaying RPC due to: Service unavailable: Master config (127.0.202.126:46581,127.0.202.124:43241,127.0.202.125:43873) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.126:46581,org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.125:43873, [1417ms] refreshing cache from master, [1417ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.126:46581, [1417ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.124:43241, [1421ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.125:43873, [1421ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.126:46581: Network error: java.net.ConnectException: Connection refused: /127.0.202.126:46581, [1423ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.125:43873: Network error: java.net.ConnectException: Connection refused: /127.0.202.125:43873, [1423ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.124:43241: OK, [1424ms] delaying RPC due to: Service unavailable: Master config (127.0.202.126:46581,127.0.202.124:43241,127.0.202.125:43873) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.126:46581,org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.125:43873, [2537ms] refreshing cache from master, [2537ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.126:46581, [2537ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.124:43241, [2540ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.125:43873, [2541ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.126:46581: Network error: java.net.ConnectException: Connection refused: /127.0.202.126:46581, [2542ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.125:43873: Network error: Failed to connect to peer master-127.0.202.125:43873(127.0.202.125:43873): Connection refused: /127.0.202.125:43873, [2543ms] Sub RPC ConnectToMaster: received response from server master-127.0.202.124:43241: OK, [2543ms] delaying RPC due to: Service unavailable: Master config (127.0.202.126:46581,127.0.202.124:43241,127.0.202.125:43873) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.126:46581,org.apache.kudu.client.RecoverableException: Failed to connect to peer master-127.0.202.125:43873(127.0.202.125:43873): Connection refused: /127.0.202.125:43873, [3857ms] refreshing cache from master, [3857ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.126:46581, [3858ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.124:43241, [3861ms] Sub RPC ConnectToMaster: sending RPC to server master-127.0.202.125:43873, [3861ms] too many traces: truncated at 100 traces, deferred=Deferred@531938157(state=PENDING, result=null, callback=(continuation of Deferred@691636556 after retry RPC after error@610503197) -> (continuation of Deferred@1438562118 after retry RPC after error@1877676367) -> (continuation of Deferred@176002583 after retry RPC after error@855033383) -> (continuation of Deferred@315773880 after retry RPC after error@621038020) -> (continuation of Deferred@1127198301 after retry RPC after error@1206685665) -> (continuation of Deferred@1201883959 after retry RPC after error@1089368174) -> (continuation of Deferred@2114810728 after retry RPC after error@1564305615) -> (continuation of Deferred@1735340 after retry RPC after error@868489153) -> (continuation of Deferred@642229474 after retry RPC after error@269147125) -> (continuation of Deferred@1125522498 after retry RPC after error@1358918142) -> (continuation of Deferred@1258602423 after retry RPC after error@219615313) -> (continuation of Deferred@2115041725 after retry RPC after error@1430980352) -> (continuation of Deferred@1004335897 after retry RPC after error@243071403) -> (continuation of Deferred@2002269384 after retry RPC after error@2049334820) -> (continuation of Deferred@650001986 after retry RPC after error@1340484255) -> (continuation of Deferred@2033105375 after retry RPC after error@825298848) -> (continuation of Deferred@1722444778 after retry RPC after error@417732466) -> (continuation of Deferred@344634864 after retry RPC after error@763356366) -> (continuation of Deferred@406794160 after retry RPC after error@810214978) -> (continuation of Deferred@1907286158 after retry RPC after error@589636577) -> (continuation of Deferred@1107745434 after retry RPC after error@630803566) -> (continuation of Deferred@590705347 after retry RPC after error@696608623) -> (continuation of Deferred@1571007183 after retry RPC after error@850814761) -> (continuation of Deferred@983414998 after retry RPC after error@950083317) -> (continuation of Deferred@2014588668 after retry RPC after error@2040173048), errback=(continuation of Deferred@691636556 after retry RPC after error@610503197) -> (continuation of Deferred@1438562118 after retry RPC after error@1877676367) -> (continuation of Deferred@176002583 after retry RPC after error@855033383) -> (continuation of Deferred@315773880 after retry RPC after error@621038020) -> (continuation of Deferred@1127198301 after retry RPC after error@1206685665) -> (continuation of Deferred@1201883959 after retry RPC after error@1089368174) -> (continuation of Deferred@2114810728 after retry RPC after error@1564305615) -> (continuation of Deferred@1735340 after retry RPC after error@868489153) -> (continuation of Deferred@642229474 after retry RPC after error@269147125) -> (continuation of Deferred@1125522498 after retry RPC after error@1358918142) -> (continuation of Deferred@1258602423 after retry RPC after error@219615313) -> (continuation of Deferred@2115041725 after retry RPC after error@1430980352) -> (continuation of Deferred@1004335897 after retry RPC after error@243071403) -> (continuation of Deferred@2002269384 after retry RPC after error@2049334820) -> (continuation of Deferred@650001986 after retry RPC after error@1340484255) -> (continuation of Deferred@2033105375 after retry RPC after error@825298848) -> (continuation of Deferred@1722444778 after retry RPC after error@417732466) -> (continuation of Deferred@344634864 after retry RPC after error@763356366) -> (continuation of Deferred@406794160 after retry RPC after error@810214978) -> (continuation of Deferred@1907286158 after retry RPC after error@589636577) -> (continuation of Deferred@1107745434 after retry RPC after error@630803566) -> (continuation of Deferred@590705347 after retry RPC after error@696608623) -> (continuation of Deferred@1571007183 after retry RPC after error@850814761) -> (continuation of Deferred@983414998 after retry RPC after error@950083317) -> (continuation of Deferred@2014588668 after retry RPC after error@2040173048)))
> at org.apache.kudu.client.KuduException.transformException(KuduException.java:110)
> at org.apache.kudu.client.KuduClient.joinAndHandleException(KuduClient.java:413)
> at org.apache.kudu.client.KuduClient.listTabletServers(KuduClient.java:197)
> at org.apache.kudu.client.TestSecurity.checkClientCanReconnect(TestSecurity.java:135)
> at org.apache.kudu.client.TestSecurity.testExternallyProvidedSubjectRefreshedExternally(TestSecurity.java:431)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> Suppressed: org.apache.kudu.client.KuduException$OriginalException: Original asynchronous stack trace
> at org.apache.kudu.client.AsyncKuduClient.tooManyAttemptsOrTimeout(AsyncKuduClient.java:1676)
> at org.apache.kudu.client.AsyncKuduClient.delayedSendRpcToTablet(AsyncKuduClient.java:2121)
> at org.apache.kudu.client.AsyncKuduClient.access$1200(AsyncKuduClient.java:261)
> at org.apache.kudu.client.AsyncKuduClient$RetryRpcErrback.call(AsyncKuduClient.java:1328)
> at org.apache.kudu.client.AsyncKuduClient$RetryRpcErrback.call(AsyncKuduClient.java:1318)
> at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
> at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
> at com.stumbleupon.async.Deferred.callback(Deferred.java:1002)
> at org.apache.kudu.client.ConnectToCluster.incrementCountAndCheckExhausted(ConnectToCluster.java:289)
> at org.apache.kudu.client.ConnectToCluster.access$100(ConnectToCluster.java:49)
> at org.apache.kudu.client.ConnectToCluster$ConnectToMasterCB.call(ConnectToCluster.java:323)
> at org.apache.kudu.client.ConnectToCluster$ConnectToMasterCB.call(ConnectToCluster.java:312)
> at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
> at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
> at com.stumbleupon.async.Deferred.callback(Deferred.java:1002)
> at org.apache.kudu.client.KuduRpc.handleCallback(KuduRpc.java:275)
> at org.apache.kudu.client.KuduRpc.callback(KuduRpc.java:322)
> at org.apache.kudu.client.RpcProxy.responseReceived(RpcProxy.java:296)
> at org.apache.kudu.client.RpcProxy.access$000(RpcProxy.java:59)
> at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:149)
> at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:145)
> at org.apache.kudu.client.Connection.messageReceived(Connection.java:390)
> at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at org.apache.kudu.client.Connection.handleUpstream(Connection.java:238)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
> at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
> at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
> at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
> at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more
> Caused by: org.apache.kudu.client.NoLeaderFoundException: Master config (127.0.202.126:46581,127.0.202.124:43241,127.0.202.125:43873) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.126:46581,org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.125:43873
> at org.apache.kudu.client.ConnectToCluster.incrementCountAndCheckExhausted(ConnectToCluster.java:287)
> at org.apache.kudu.client.ConnectToCluster.access$100(ConnectToCluster.java:49)
> at org.apache.kudu.client.ConnectToCluster$ConnectToMasterCB.call(ConnectToCluster.java:323)
> at org.apache.kudu.client.ConnectToCluster$ConnectToMasterCB.call(ConnectToCluster.java:312)
> at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
> at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
> at com.stumbleupon.async.Deferred.callback(Deferred.java:1002)
> at org.apache.kudu.client.KuduRpc.handleCallback(KuduRpc.java:275)
> at org.apache.kudu.client.KuduRpc.callback(KuduRpc.java:322)
> at org.apache.kudu.client.RpcProxy.responseReceived(RpcProxy.java:296)
> at org.apache.kudu.client.RpcProxy.access$000(RpcProxy.java:59)
> at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:149)
> at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:145)
> at org.apache.kudu.client.Connection.messageReceived(Connection.java:390)
> at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at org.apache.kudu.client.Connection.handleUpstream(Connection.java:238)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
> at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
> at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
> at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
> at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more
> Caused by: org.apache.kudu.client.RecoverableException: java.net.ConnectException: Connection refused: /127.0.202.125:43873
> at org.apache.kudu.client.Connection.channelClosed(Connection.java:266)
> at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:88)
> at org.apache.kudu.client.Connection.handleUpstream(Connection.java:238)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at org.jboss.netty.handler.timeout.ReadTimeoutHandler.channelClosed(ReadTimeoutHandler.java:176)
> at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:88)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:60)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.cleanup(FrameDecoder.java:493)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.channelClosed(FrameDecoder.java:371)
> at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:88)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> at org.jboss.netty.channel.Channels.fireChannelClosed(Channels.java:468)
> at org.jboss.netty.channel.Channels$6.run(Channels.java:457)
> at org.jboss.netty.channel.socket.ChannelRunnableWrapper.run(ChannelRunnableWrapper.java:40)
> at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:391)
> at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:315)
> ... 7 more
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)