You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Abhi (JIRA)" <ji...@apache.org> on 2019/02/22 13:08:00 UTC

[jira] [Created] (KAFKA-7982) ConcurrentModificationException and Continuous warnings "Attempting to send response via channel for which there is no open connection"

Abhi created KAFKA-7982:
---------------------------

             Summary: ConcurrentModificationException and Continuous warnings "Attempting to send response via channel for which there is no open connection"
                 Key: KAFKA-7982
                 URL: https://issues.apache.org/jira/browse/KAFKA-7982
             Project: Kafka
          Issue Type: Bug
          Components: core
    Affects Versions: 2.1.1
            Reporter: Abhi


Hi,

I am getting follow warnings in server.log continuosly and due to this client consumer is not able to consumer messages.

[2019-02-20 10:26:30,312] WARN Attempting to send response via channel for which there is no open connection, connection id 10.218.27.45:9092-10.219.25.239:35248-6259 (kafka.network.Processor)
[2019-02-20 10:26:56,760] WARN Attempting to send response via channel for which there is no open connection, connection id 10.218.27.45:9092-10.219.25.239:35604-6261 (kafka.network.Processor)

I also noticed that before these warnings started to appear, following concurrent modification exception for the same IP address:

[2019-02-20 09:01:11,175] INFO Initiating logout for kafka/u-kafkatst-kafkadev-1.sd.com@UNIX.COM (org.apache.kafka.common.security.kerberos.KerberosLogin)
[2019-02-20 09:01:11,176] WARN [SocketServer brokerId=1] Unexpected error from /10.219.25.239; closing connection (org.apache.kafka.common.network.Selector)
java.util.ConcurrentModificationException
        at java.base/java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:970)
        at java.base/java.util.LinkedList$ListItr.next(LinkedList.java:892)
        at java.base/javax.security.auth.Subject$SecureSet$1.next(Subject.java:1096)
        at java.base/javax.security.auth.Subject$ClassSet$1.run(Subject.java:1501)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.base/javax.security.auth.Subject$ClassSet.populateSet(Subject.java:1499)
        at java.base/javax.security.auth.Subject$ClassSet.<init>(Subject.java:1472)
        at java.base/javax.security.auth.Subject.getPrivateCredentials(Subject.java:764)
        at java.security.jgss/sun.security.jgss.GSSUtil$1.run(GSSUtil.java:336)
        at java.security.jgss/sun.security.jgss.GSSUtil$1.run(GSSUtil.java:328)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.security.jgss/sun.security.jgss.GSSUtil.searchSubject(GSSUtil.java:328)
        at java.security.jgss/sun.security.jgss.wrapper.NativeGSSFactory.getCredFromSubject(NativeGSSFactory.java:53)
        at java.security.jgss/sun.security.jgss.wrapper.NativeGSSFactory.getCredentialElement(NativeGSSFactory.java:116)
        at java.security.jgss/sun.security.jgss.GSSManagerImpl.getCredentialElement(GSSManagerImpl.java:187)
        at java.security.jgss/sun.security.jgss.GSSCredentialImpl.add(GSSCredentialImpl.java:439)
        at java.security.jgss/sun.security.jgss.GSSCredentialImpl.<init>(GSSCredentialImpl.java:74)
        at java.security.jgss/sun.security.jgss.GSSManagerImpl.createCredential(GSSManagerImpl.java:148)
        at jdk.security.jgss/com.sun.security.sasl.gsskerb.GssKrb5Server.<init>(GssKrb5Server.java:108)
        at jdk.security.jgss/com.sun.security.sasl.gsskerb.FactoryImpl.createSaslServer(FactoryImpl.java:85)
        at java.security.sasl/javax.security.sasl.Sasl.createSaslServer(Sasl.java:537)
        at org.apache.kafka.common.security.authenticator.SaslServerAuthenticator.lambda$createSaslKerberosServer$12(SaslServerAuthenticator.java:212)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
        at org.apache.kafka.common.security.authenticator.SaslServerAuthenticator.createSaslKerberosServer(SaslServerAuthenticator.java:211)
        at org.apache.kafka.common.security.authenticator.SaslServerAuthenticator.createSaslServer(SaslServerAuthenticator.java:164)
        at org.apache.kafka.common.security.authenticator.SaslServerAuthenticator.handleKafkaRequest(SaslServerAuthenticator.java:450)
        at org.apache.kafka.common.security.authenticator.SaslServerAuthenticator.authenticate(SaslServerAuthenticator.java:248)
        at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:132)
        at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:532)
        at org.apache.kafka.common.network.Selector.poll(Selector.java:467)
        at kafka.network.Processor.poll(SocketServer.scala:689)
        at kafka.network.Processor.run(SocketServer.scala:594)
        at java.base/java.lang.Thread.run(Thread.java:834)
[2019-02-22 00:18:29,439] INFO Initiating re-login for kafka/u-kafkatst-kafkadev-1.sd.deshaw.com@UNIX.DESHAW.COM (org.apache.kafka.common.security.kerberos.KerberosLogin)
[2019-02-22 00:18:29,440] WARN [SocketServer brokerId=1] Unexpected error from /10.219.25.239; closing connection (org.apache.kafka.common.network.Selector)
org.apache.kafka.common.KafkaException: Principal could not be determined from Subject, this may be a transient failure due to Kerberos re-login
        at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.firstPrincipal(SaslClientAuthenticator.java:435)
        at org.apache.kafka.common.security.authenticator.SaslServerAuthenticator.createSaslKerberosServer(SaslServerAuthenticator.java:177)
        at org.apache.kafka.common.security.authenticator.SaslServerAuthenticator.createSaslServer(SaslServerAuthenticator.java:164)
        at org.apache.kafka.common.security.authenticator.SaslServerAuthenticator.handleKafkaRequest(SaslServerAuthenticator.java:450)
        at org.apache.kafka.common.security.authenticator.SaslServerAuthenticator.authenticate(SaslServerAuthenticator.java:248)
        at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:132)
        at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:532)
        at org.apache.kafka.common.network.Selector.poll(Selector.java:467)
        at kafka.network.Processor.poll(SocketServer.scala:689)
        at kafka.network.Processor.run(SocketServer.scala:594)
        at java.base/java.lang.Thread.run(Thread.java:834)
[2019-02-22 00:18:29,440] WARN [SocketServer brokerId=1] Unexpected error from /10.219.25.239; closing connection (org.apache.kafka.common.network.Selector)
org.apache.kafka.common.KafkaException: Principal could not be determined from Subject, this may be a transient failure due to Kerberos re-login
        at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.firstPrincipal(SaslClientAuthenticator.java:435)
        at org.apache.kafka.common.security.authenticator.SaslServerAuthenticator.createSaslKerberosServer(SaslServerAuthenticator.java:177)
        at org.apache.kafka.common.security.authenticator.SaslServerAuthenticator.createSaslServer(SaslServerAuthenticator.java:164)
        at org.apache.kafka.common.security.authenticator.SaslServerAuthenticator.handleKafkaRequest(SaslServerAuthenticator.java:450)
        at org.apache.kafka.common.security.authenticator.SaslServerAuthenticator.authenticate(SaslServerAuthenticator.java:248)
        at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:132)
        at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:532)
        at org.apache.kafka.common.network.Selector.poll(Selector.java:467)
        at kafka.network.Processor.poll(SocketServer.scala:689)
        at kafka.network.Processor.run(SocketServer.scala:594)
        at java.base/java.lang.Thread.run(Thread.java:834)






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)