You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by ASHOK MACHERLA <ia...@outlook.com> on 2019/05/01 21:35:21 UTC
RE: Required guidelines for kafka upgrade
Dear Senthil
Please update on this mail.
Please check my previous email.
Can I follow that same steps to upgrade Kafka cluster.
Or else suggest any changes required
Thanks
Sent from Outlook
Re: Required guidelines for kafka upgrade
Posted by SenthilKumar K <se...@gmail.com>.
Hi, if you see SSL issue try setting ssl.endpoint.identification.algorithm=
Simply leave it empty no double quote .
It would be good if you share error message from broker logs.
--Senthil
On Fri, May 3, 2019, 9:36 PM Harper Henn <ha...@datto.com> wrote:
> What specific errors are you seeing in the server logs of the broker you
> upgraded (can you copy/paste them)?
>
> On Fri, May 3, 2019 at 7:29 AM ASHOK MACHERLA <iA...@outlook.com> wrote:
>
> > *Dear Senthil*
> >
> > As you suggested , I follow but I’m facing errors
> >
> > This is my old configurations which is Kafka (0.10.1) version
> >
> > *****************************************************************
> >
> > broker.id=0
> >
> > port=9092
> >
> > delete.topic.enable=true
> >
> > message.max.bytes=100000
> >
> > listeners=SSL://192.168.175.128:9092
> >
> > advertised.listeners=SSL://192.168.175.128:9092
> >
> > num.network.threads=3
> >
> > num.io.threads=8
> >
> > socket.send.buffer.bytes=102400
> >
> > socket.receive.buffer.bytes=102400
> >
> > socket.request.max.bytes=104857600
> >
> > log.dirs=/opt/kafka/kafka-logs
> >
> > num.partitions=3
> >
> > default.replication.factor=3
> >
> > auto.topic.creation.enable=false
> >
> > num.recovery.threads.per.data.dir=1
> >
> > log.retention.hours=168
> >
> > log.segment.bytes=1073741824
> >
> > log.retention.check.interval.ms=300000
> >
> > ssl.keystore.location=/opt/kafka/certificate/kafka.keystore.jks
> >
> > ssl.keystore.password=Sbi#123
> >
> > ssl.key.password=Sbi#123
> >
> > ssl.truststore.location=/opt/kafka/certificate/kafka.truststore.jks
> >
> > ssl.truststore.password=Sbi#123
> >
> > security.inter.broker.protocol=SSL
> >
> > zookeeper.connect=192.168.175.128:2181,192.168.175.129:2181,
> > 192.168.175.130:2181
> >
> > zookeeper.connection.timeout.ms=6000
> >
> > *****************************************************************
> >
> > After that i added three parameters into server.properties which is *new
> > kafka version (2.2.0)*
> >
> > inter.broker.protocol.version=0.10.1
> >
> > log.message.format.version=0.10.1
> >
> > ssl.endpoint.identification.algorithm=""
> >
> > After that I stopped one Kafka node, and then I started new Kafka (2.2.0)
> > version in same node.
> >
> > in this port is opening , it's showing 9092 port number
> >
> > but getting errors due to SSL issues
> >
> > I this position first node is running with new version (2.2.0) and
> > remaining two nodes are running with previous version (0.10.1)
> >
> > I checked topic describe command on second node, here ISR is not syncing
> > with new version,
> >
> > it's showing only 1,2, here "0" is missing it means first nodes was not
> > syncing with remaining nodes,
> >
> > it’s should show like 0,1,2.
> >
> > Please help Senthil
> >
> > I tried so many options like below ,
> >
> > ssl.endpoint.identification.algorithm=""
> >
> > ssl.endpoint.identification.algorithm=" "
> >
> > ssl.endpoint.identification.algorithm="none"
> >
> > ssl.endpoint.identification.algorithm="null"
> >
> > ssl.endpoint.identification.algorithm=null
> >
> > ssl.endpoint.identification.algorithm=https
> >
> > please tell what correct value I should mention, and port is is showing
> > but why it's ISR showing only 1,2 instead of 0,1,2
> >
> > is there any firewall settings problems?
> >
> > Please help us to fix this Senthil
> >
> > thanks
> >
> >
> > Sent from Outlook <http://aka.ms/weboutlook>
> > ------------------------------
> > *From:* ASHOK MACHERLA <ia...@outlook.com>
> > *Sent:* 02 May 2019 13:28
> > *To:* users@kafka.apache.org
> > *Subject:* Re: Required guidelines for kafka upgrade
> >
> > OK Senthil
> >
> > Thanks for your support and cooperation
> >
> > Sent from Outlook
> >
>
Re: Required guidelines for kafka upgrade
Posted by ASHOK MACHERLA <ia...@outlook.com>.
Dear Senthil
Thanks a lot your support and help in time without delay.
Thanks senthill
Sent from Outlook
Re: Required guidelines for kafka upgrade
Posted by SenthilKumar K <se...@gmail.com>.
Can you verify your producer and consumer commands ?
Console Producer :
./bin/kafka-console-producer.sh --broker-list xxxx:9093 --producer.config
/kafka/client-ssl.properties --topic kafka_220
Console Consumer:
./bin/kafka-console-consumer.sh --bootstrap-server xxxx:9093
--consumer.config /kafka/client-ssl.properties --topic kafka_220
cat /kafka/client-ssl.properties
security.protocol=SSL
ssl.truststore.location=xxxx
ssl.truststore.password=xxxx
ssl.endpoint.identification.algorithm=
/opt/kafka-new$ sh bin/kafka-console-producer.sh --broker-list
192.168.175.128:9092<http://192.168.175.128:9092/> --producer.config
producer-ssl.config --topic otp-email
Can you share the contents of producer-ssl.config ?
--Senthil
On Sat, May 4, 2019 at 11:14 AM ASHOK MACHERLA <iA...@outlook.com> wrote:
> Dear Senthil
>
> when I tried produce messages into topic ,this type errors coming
> continuously
>
> ashok@Node-1:/opt/kafka-new$ sh bin/kafka-console-producer.sh
> --broker-list 192.168.175.128:9092<http://192.168.175.128:9092/>
> --producer.config producer-ssl.config --topic otp-email
>
> >[2019-05-03 22:37:34,382] ERROR [Producer clientId=console-producer]
> Connection to node -1 (/192.168.175.128:9092<http://192.168.175.128:9092/>)
> failed authentication due to: SSL handshake failed
> (org.apache.kafka.clients.NetworkClient)
> [2019-05-03 22:37:34,689] ERROR [Producer clientId=console-producer]
> Connection to node -1 (/192.168.175.128:9092<http://192.168.175.128:9092/>)
> failed authentication due to: SSL handshake failed
> (org.apache.kafka.clients.NetworkClient)
>
> Please help us to fix this
>
> if anything changes required in server.properties????
>
> Sent from Outlook<http://aka.ms/weboutlook>
> ________________________________
> From: ASHOK MACHERLA <ia...@outlook.com>
> Sent: 04 May 2019 00:44
> To: users@kafka.apache.org
> Subject: Re: Required guidelines for kafka upgrade
>
> Dear Senthil
>
> Could you please explain clearly
>
> Consumer client properties means ???
>
>
> Where can I set that parameter.
>
> I checked within the Kafka cluster, I pushed some messages and when I
> tried to pulling from same topic, it's not printing any messages
>
> Please tell me senthil.
>
> How can we solve this???
>
> Sent from Outlook
>
Re: Re: Required guidelines for kafka upgrade
Posted by "15332318109@189.cn" <15...@189.cn>.
Dear
How Can I send question
From: ASHOK MACHERLA
Date: 2019-05-04 13:44
To: users@kafka.apache.org
Subject: Re: Required guidelines for kafka upgrade
Dear Senthil
when I tried produce messages into topic ,this type errors coming continuously
ashok@Node-1:/opt/kafka-new$ sh bin/kafka-console-producer.sh --broker-list 192.168.175.128:9092<http://192.168.175.128:9092/> --producer.config producer-ssl.config --topic otp-email
>[2019-05-03 22:37:34,382] ERROR [Producer clientId=console-producer] Connection to node -1 (/192.168.175.128:9092<http://192.168.175.128:9092/>) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
[2019-05-03 22:37:34,689] ERROR [Producer clientId=console-producer] Connection to node -1 (/192.168.175.128:9092<http://192.168.175.128:9092/>) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
Please help us to fix this
if anything changes required in server.properties????
Sent from Outlook<http://aka.ms/weboutlook>
________________________________
From: ASHOK MACHERLA <ia...@outlook.com>
Sent: 04 May 2019 00:44
To: users@kafka.apache.org
Subject: Re: Required guidelines for kafka upgrade
Dear Senthil
Could you please explain clearly
Consumer client properties means ???
Where can I set that parameter.
I checked within the Kafka cluster, I pushed some messages and when I tried to pulling from same topic, it's not printing any messages
Please tell me senthil.
How can we solve this???
Sent from Outlook
Re: Required guidelines for kafka upgrade
Posted by ASHOK MACHERLA <iA...@outlook.com>.
Dear Senthil
when I tried produce messages into topic ,this type errors coming continuously
ashok@Node-1:/opt/kafka-new$ sh bin/kafka-console-producer.sh --broker-list 192.168.175.128:9092<http://192.168.175.128:9092/> --producer.config producer-ssl.config --topic otp-email
>[2019-05-03 22:37:34,382] ERROR [Producer clientId=console-producer] Connection to node -1 (/192.168.175.128:9092<http://192.168.175.128:9092/>) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
[2019-05-03 22:37:34,689] ERROR [Producer clientId=console-producer] Connection to node -1 (/192.168.175.128:9092<http://192.168.175.128:9092/>) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
Please help us to fix this
if anything changes required in server.properties????
Sent from Outlook<http://aka.ms/weboutlook>
________________________________
From: ASHOK MACHERLA <ia...@outlook.com>
Sent: 04 May 2019 00:44
To: users@kafka.apache.org
Subject: Re: Required guidelines for kafka upgrade
Dear Senthil
Could you please explain clearly
Consumer client properties means ???
Where can I set that parameter.
I checked within the Kafka cluster, I pushed some messages and when I tried to pulling from same topic, it's not printing any messages
Please tell me senthil.
How can we solve this???
Sent from Outlook
Re: Required guidelines for kafka upgrade
Posted by ASHOK MACHERLA <ia...@outlook.com>.
Dear Senthil
Could you please explain clearly
Consumer client properties means ???
Where can I set that parameter.
I checked within the Kafka cluster, I pushed some messages and when I tried to pulling from same topic, it's not printing any messages
Please tell me senthil.
How can we solve this???
Sent from Outlook
Re: Required guidelines for kafka upgrade
Posted by SenthilKumar K <se...@gmail.com>.
You have to set the same endpoint algorithm (empty) in consumer client
properties.
On Sat, May 4, 2019, 12:15 AM ASHOK MACHERLA <iA...@outlook.com> wrote:
> Dear Senthil
>
> as you suggested I follow, Kafka Cluster is fine ISR showing 0,1,2
>
> but getting SSL error logs
>
> [2019-05-03 11:01:19,611] INFO [SocketServer brokerId=0] Failed
> authentication with /192.168.175.128<http://192.168.175.128/> (SSL
> handshake failed) (org.apache.kafka.common.network.Selector)
>
>
> when I tried to send some messages to one topic
> Producer::
> $ sh bin/kafka-console-producer.sh --broker-list 192.168.175.128:9092
> --topic otp-sms
> aaaaaaaaaaaaaaaaaaa
> bbbbbbbbbbbbbbbbbbbb
> ccccccccccccccccccccccccccc
>
> Consumer::
> $ sh bin/kafka-console-consumer.sh --bootstrap-server 192.168.175.128:9092
> --topic otp-sms
>
> it's not printing any messages , even when I try to pull messages from
> beginning also not working
>
> same above error getting
>
> Please help us Senthil, to fix this issue
>
> Thanks for your support...
>
>
> Sent from Outlook<http://aka.ms/weboutlook>
> ________________________________
> From: ASHOK MACHERLA <iA...@outlook.com>
> Sent: 03 May 2019 23:52
> To: users@kafka.apache.org
> Subject: Re: Required guidelines for kafka upgrade
>
> Dear Senthil
>
> first of all thanks for help,
>
> after I set like ssl.endpoint.identification.algorithm = and then
> restart it's working fine.
>
> after that I changed below parameters in all brokers
>
> inter.broker.protocol.version=2.2.0
> log.message.format.version=2.2.0
>
> after that restarted one by one. ISR is showing 0,1,2 it's good,
> but getting some SSL error, please look into these logs
>
> [2019-05-03 11:00:37,789] INFO [SocketServer brokerId=0] Failed
> authentication with /192.168.175.128<http://192.168.175.128/> (SSL
> handshake failed) (org.apache.kafka.common.network.Selector)
> [2019-05-03 11:01:19,611] INFO [SocketServer brokerId=0] Failed
> authentication with /192.168.175.128<http://192.168.175.128/> (SSL
> handshake failed) (org.apache.kafka.common.network.Selector)
>
> getting logs all brokers,
>
> I tried to produce sample messages to topic,
> consumer is not print that messages .
>
> please help us to fix this
>
> Thanks a lot senthill
>
>
> Sent from Outlook<http://aka.ms/weboutlook>
> ________________________________
> From: SenthilKumar K <se...@gmail.com>
> Sent: 03 May 2019 22:26
> To: users@kafka.apache.org
> Subject: Re: Required guidelines for kafka upgrade
>
> Hi Ashok , From the logs its clear that problem with identification
> algorithm.
>
> at
>
> org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
> ... 15 more
> Caused by: java.security.cert.CertificateException: Unknown identification
> algorithm: " "
>
> Set empty and restart your broker.
>
> --Senthil
>
> On Fri, May 3, 2019 at 10:20 PM SenthilKumar K <se...@gmail.com>
> wrote:
>
> > Here is my server.properties.
> >
> >
> > reserved.broker.max.id = 2147483647
> > log.retention.bytes = 68719476736
> > listeners = SSL://xxxxxx:9093
> > socket.receive.buffer.bytes = 102400
> > broker.id = xxx
> > ssl.truststore.password = xxxxx
> > auto.create.topics.enable = true
> > ssl.enabled.protocols = TLSv1.2
> > zookeeper.connect = xxxxx
> > default.replication.factor = 2
> > offsets.topic.replication.factor = 2
> > log.message.timestamp.type = CreateTime
> > min.insync.replicas = 2
> > transaction.state.log.replication.factor = 2
> > security.inter.broker.protocol = SSL
> > socket.send.buffer.bytes = 102400
> > num.partitions = 40
> > ssl.secure.random.implementation = xxxxx
> > ssl.key.password = xxxx
> > log.retention.ms = 3600000
> > log.cleaner.delete.retention.ms = 300000
> > message.max.bytes = 67108864
> > ssl.endpoint.identification.algorithm =
> > log.roll.ms = 1800000
> > log.message.timestamp.difference.max.ms = 14400000
> > ssl.keystore.location = xxxx
> > log.retention.hours = 168
> > log.retention.check.interval.ms = 180000
> > inter.broker.protocol.version = 2.2.0
> > socket.request.max.bytes = 104857600
> > log.dirs = xxxxx
> > ssl.keystore.password = xxxx
> > ssl.truststore.location = xxxx
> >
> > Pls set `ssl.endpoint.identification.algorithm = ` and restart your
> > broker.
> >
> >
> > On Fri, May 3, 2019 at 10:09 PM ASHOK MACHERLA <iA...@outlook.com>
> > wrote:
> >
> >> Dear
> >>
> >>
> >> Please find this below error
> >>
> >> org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake
> >> failed
> >> Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine
> problem
> >> at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1521)
> >> at
> sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:528)
> >> at
> sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1197)
> >> at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1165)
> >> at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
> >> at
> >>
> org.apache.kafka.common.network.SslTransportLayer.handshakeWrap(SslTransportLayer.java:448)
> >> at
> >>
> org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:313)
> >> at
> >>
> org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:265)
> >> at
> >>
> org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:170)
> >> at
> >>
> org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:536)
> >> at org.apache.kafka.common.network.Selector.poll(Selector.java:472)
> >> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
> >> at
> >>
> org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:74)
> >> at
> >>
> kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:95)
> >> at
> >>
> kafka.server.ReplicaFetcherThread.fetchFromLeader(ReplicaFetcherThread.scala:193)
> >> at
> >>
> kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:280)
> >> at
> >>
> kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3(AbstractFetcherThread.scala:132)
> >> at
> >>
> kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3$adapted(AbstractFetcherThread.scala:131)
> >> at scala.Option.foreach(Option.scala:274)
> >> at
> >>
> kafka.server.AbstractFetcherThread.maybeFetch(AbstractFetcherThread.scala:131)
> >> at
> >>
> kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:113)
> >> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82)
> >> Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine
> problem
> >> at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
> >> at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1709)
> >> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:318)
> >> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
> >> at
> >>
> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
> >> at
> >>
> sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
> >> at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
> >> at sun.security.ssl.Handshaker$1.run(Handshaker.java:970)
> >> at sun.security.ssl.Handshaker$1.run(Handshaker.java:967)
> >> at java.security.AccessController.doPrivileged(Native Method)
> >> at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1459)
> >> at
> >>
> org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:402)
> >> at
> >>
> org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:484)
> >> at
> >>
> org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
> >> ... 15 more
> >> Caused by: java.security.cert.CertificateException: Unknown
> >> identification algorithm: " "
> >> at
> >>
> sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:462)
> >> at
> >>
> sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:436)
> >> at
> >>
> sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:252)
> >> at
> >>
> sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
> >> at
> >>
> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1626)
> >> ... 24 more
> >> [2019-05-03 06:36:23,840] INFO [ReplicaFetcher replicaId=0, leaderId=2,
> >> fetcherId=0] Failed authentication with /192.168.175.130 (SSL handshake
> >> failed) (org.apache.kafka.common.network.Selector)
> >> [2019-05-03 06:36:23,842] ERROR [ReplicaFetcher replicaId=0, leaderId=2,
> >> fetcherId=0] Connection to node 2 (/192.168.175.130:9092) failed
> >> authentication due to: SSL handshake failed
> >> (org.apache.kafka.clients.NetworkClient)
> >> Sent from Outlook<http://aka.ms/weboutlook>
> >> ________________________________
> >> From: Harper Henn <ha...@datto.com>
> >> Sent: 03 May 2019 21:35
> >> To: users@kafka.apache.org
> >> Subject: Re: Required guidelines for kafka upgrade
> >>
> >> What specific errors are you seeing in the server logs of the broker you
> >> upgraded (can you copy/paste them)?
> >>
> >> On Fri, May 3, 2019 at 7:29 AM ASHOK MACHERLA <iA...@outlook.com>
> >> wrote:
> >>
> >> > *Dear Senthil*
> >> >
> >> > As you suggested , I follow but I’m facing errors
> >> >
> >> > This is my old configurations which is Kafka (0.10.1) version
> >> >
> >> > *****************************************************************
> >> >
> >> > broker.id=0
> >> >
> >> > port=9092
> >> >
> >> > delete.topic.enable=true
> >> >
> >> > message.max.bytes=100000
> >> >
> >> > listeners=SSL://192.168.175.128:9092
> >> >
> >> > advertised.listeners=SSL://192.168.175.128:9092
> >> >
> >> > num.network.threads=3
> >> >
> >> > num.io.threads=8
> >> >
> >> > socket.send.buffer.bytes=102400
> >> >
> >> > socket.receive.buffer.bytes=102400
> >> >
> >> > socket.request.max.bytes=104857600
> >> >
> >> > log.dirs=/opt/kafka/kafka-logs
> >> >
> >> > num.partitions=3
> >> >
> >> > default.replication.factor=3
> >> >
> >> > auto.topic.creation.enable=false
> >> >
> >> > num.recovery.threads.per.data.dir=1
> >> >
> >> > log.retention.hours=168
> >> >
> >> > log.segment.bytes=1073741824
> >> >
> >> > log.retention.check.interval.ms=300000
> >> >
> >> > ssl.keystore.location=/opt/kafka/certificate/kafka.keystore.jks
> >> >
> >> > ssl.keystore.password=Sbi#123
> >> >
> >> > ssl.key.password=Sbi#123
> >> >
> >> > ssl.truststore.location=/opt/kafka/certificate/kafka.truststore.jks
> >> >
> >> > ssl.truststore.password=Sbi#123
> >> >
> >> > security.inter.broker.protocol=SSL
> >> >
> >> > zookeeper.connect=192.168.175.128:2181,192.168.175.129:2181,
> >> > 192.168.175.130:2181
> >> >
> >> > zookeeper.connection.timeout.ms=6000
> >> >
> >> > *****************************************************************
> >> >
> >> > After that i added three parameters into server.properties which is
> *new
> >> > kafka version (2.2.0)*
> >> >
> >> > inter.broker.protocol.version=0.10.1
> >> >
> >> > log.message.format.version=0.10.1
> >> >
> >> > ssl.endpoint.identification.algorithm=""
> >> >
> >> > After that I stopped one Kafka node, and then I started new Kafka
> >> (2.2.0)
> >> > version in same node.
> >> >
> >> > in this port is opening , it's showing 9092 port number
> >> >
> >> > but getting errors due to SSL issues
> >> >
> >> > I this position first node is running with new version (2.2.0) and
> >> > remaining two nodes are running with previous version (0.10.1)
> >> >
> >> > I checked topic describe command on second node, here ISR is not
> syncing
> >> > with new version,
> >> >
> >> > it's showing only 1,2, here "0" is missing it means first nodes was
> not
> >> > syncing with remaining nodes,
> >> >
> >> > it’s should show like 0,1,2.
> >> >
> >> > Please help Senthil
> >> >
> >> > I tried so many options like below ,
> >> >
> >> > ssl.endpoint.identification.algorithm=""
> >> >
> >> > ssl.endpoint.identification.algorithm=" "
> >> >
> >> > ssl.endpoint.identification.algorithm="none"
> >> >
> >> > ssl.endpoint.identification.algorithm="null"
> >> >
> >> > ssl.endpoint.identification.algorithm=null
> >> >
> >> > ssl.endpoint.identification.algorithm=https
> >> >
> >> > please tell what correct value I should mention, and port is is
> showing
> >> > but why it's ISR showing only 1,2 instead of 0,1,2
> >> >
> >> > is there any firewall settings problems?
> >> >
> >> > Please help us to fix this Senthil
> >> >
> >> > thanks
> >> >
> >> >
> >> > Sent from Outlook <http://aka.ms/weboutlook>
> >> > ------------------------------
> >> > *From:* ASHOK MACHERLA <ia...@outlook.com>
> >> > *Sent:* 02 May 2019 13:28
> >> > *To:* users@kafka.apache.org
> >> > *Subject:* Re: Required guidelines for kafka upgrade
> >> >
> >> > OK Senthil
> >> >
> >> > Thanks for your support and cooperation
> >> >
> >> > Sent from Outlook
> >> >
> >>
> >
>
Re: Required guidelines for kafka upgrade
Posted by ASHOK MACHERLA <iA...@outlook.com>.
Dear Senthil
as you suggested I follow, Kafka Cluster is fine ISR showing 0,1,2
but getting SSL error logs
[2019-05-03 11:01:19,611] INFO [SocketServer brokerId=0] Failed authentication with /192.168.175.128<http://192.168.175.128/> (SSL handshake failed) (org.apache.kafka.common.network.Selector)
when I tried to send some messages to one topic
Producer::
$ sh bin/kafka-console-producer.sh --broker-list 192.168.175.128:9092 --topic otp-sms
aaaaaaaaaaaaaaaaaaa
bbbbbbbbbbbbbbbbbbbb
ccccccccccccccccccccccccccc
Consumer::
$ sh bin/kafka-console-consumer.sh --bootstrap-server 192.168.175.128:9092 --topic otp-sms
it's not printing any messages , even when I try to pull messages from beginning also not working
same above error getting
Please help us Senthil, to fix this issue
Thanks for your support...
Sent from Outlook<http://aka.ms/weboutlook>
________________________________
From: ASHOK MACHERLA <iA...@outlook.com>
Sent: 03 May 2019 23:52
To: users@kafka.apache.org
Subject: Re: Required guidelines for kafka upgrade
Dear Senthil
first of all thanks for help,
after I set like ssl.endpoint.identification.algorithm = and then restart it's working fine.
after that I changed below parameters in all brokers
inter.broker.protocol.version=2.2.0
log.message.format.version=2.2.0
after that restarted one by one. ISR is showing 0,1,2 it's good,
but getting some SSL error, please look into these logs
[2019-05-03 11:00:37,789] INFO [SocketServer brokerId=0] Failed authentication with /192.168.175.128<http://192.168.175.128/> (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2019-05-03 11:01:19,611] INFO [SocketServer brokerId=0] Failed authentication with /192.168.175.128<http://192.168.175.128/> (SSL handshake failed) (org.apache.kafka.common.network.Selector)
getting logs all brokers,
I tried to produce sample messages to topic,
consumer is not print that messages .
please help us to fix this
Thanks a lot senthill
Sent from Outlook<http://aka.ms/weboutlook>
________________________________
From: SenthilKumar K <se...@gmail.com>
Sent: 03 May 2019 22:26
To: users@kafka.apache.org
Subject: Re: Required guidelines for kafka upgrade
Hi Ashok , From the logs its clear that problem with identification
algorithm.
at
org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
... 15 more
Caused by: java.security.cert.CertificateException: Unknown identification
algorithm: " "
Set empty and restart your broker.
--Senthil
On Fri, May 3, 2019 at 10:20 PM SenthilKumar K <se...@gmail.com>
wrote:
> Here is my server.properties.
>
>
> reserved.broker.max.id = 2147483647
> log.retention.bytes = 68719476736
> listeners = SSL://xxxxxx:9093
> socket.receive.buffer.bytes = 102400
> broker.id = xxx
> ssl.truststore.password = xxxxx
> auto.create.topics.enable = true
> ssl.enabled.protocols = TLSv1.2
> zookeeper.connect = xxxxx
> default.replication.factor = 2
> offsets.topic.replication.factor = 2
> log.message.timestamp.type = CreateTime
> min.insync.replicas = 2
> transaction.state.log.replication.factor = 2
> security.inter.broker.protocol = SSL
> socket.send.buffer.bytes = 102400
> num.partitions = 40
> ssl.secure.random.implementation = xxxxx
> ssl.key.password = xxxx
> log.retention.ms = 3600000
> log.cleaner.delete.retention.ms = 300000
> message.max.bytes = 67108864
> ssl.endpoint.identification.algorithm =
> log.roll.ms = 1800000
> log.message.timestamp.difference.max.ms = 14400000
> ssl.keystore.location = xxxx
> log.retention.hours = 168
> log.retention.check.interval.ms = 180000
> inter.broker.protocol.version = 2.2.0
> socket.request.max.bytes = 104857600
> log.dirs = xxxxx
> ssl.keystore.password = xxxx
> ssl.truststore.location = xxxx
>
> Pls set `ssl.endpoint.identification.algorithm = ` and restart your
> broker.
>
>
> On Fri, May 3, 2019 at 10:09 PM ASHOK MACHERLA <iA...@outlook.com>
> wrote:
>
>> Dear
>>
>>
>> Please find this below error
>>
>> org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake
>> failed
>> Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
>> at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1521)
>> at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:528)
>> at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1197)
>> at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1165)
>> at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.handshakeWrap(SslTransportLayer.java:448)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:313)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:265)
>> at
>> org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:170)
>> at
>> org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:536)
>> at org.apache.kafka.common.network.Selector.poll(Selector.java:472)
>> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
>> at
>> org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:74)
>> at
>> kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:95)
>> at
>> kafka.server.ReplicaFetcherThread.fetchFromLeader(ReplicaFetcherThread.scala:193)
>> at
>> kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:280)
>> at
>> kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3(AbstractFetcherThread.scala:132)
>> at
>> kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3$adapted(AbstractFetcherThread.scala:131)
>> at scala.Option.foreach(Option.scala:274)
>> at
>> kafka.server.AbstractFetcherThread.maybeFetch(AbstractFetcherThread.scala:131)
>> at
>> kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:113)
>> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82)
>> Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
>> at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
>> at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1709)
>> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:318)
>> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
>> at
>> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
>> at
>> sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
>> at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
>> at sun.security.ssl.Handshaker$1.run(Handshaker.java:970)
>> at sun.security.ssl.Handshaker$1.run(Handshaker.java:967)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1459)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:402)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:484)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
>> ... 15 more
>> Caused by: java.security.cert.CertificateException: Unknown
>> identification algorithm: " "
>> at
>> sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:462)
>> at
>> sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:436)
>> at
>> sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:252)
>> at
>> sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
>> at
>> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1626)
>> ... 24 more
>> [2019-05-03 06:36:23,840] INFO [ReplicaFetcher replicaId=0, leaderId=2,
>> fetcherId=0] Failed authentication with /192.168.175.130 (SSL handshake
>> failed) (org.apache.kafka.common.network.Selector)
>> [2019-05-03 06:36:23,842] ERROR [ReplicaFetcher replicaId=0, leaderId=2,
>> fetcherId=0] Connection to node 2 (/192.168.175.130:9092) failed
>> authentication due to: SSL handshake failed
>> (org.apache.kafka.clients.NetworkClient)
>> Sent from Outlook<http://aka.ms/weboutlook>
>> ________________________________
>> From: Harper Henn <ha...@datto.com>
>> Sent: 03 May 2019 21:35
>> To: users@kafka.apache.org
>> Subject: Re: Required guidelines for kafka upgrade
>>
>> What specific errors are you seeing in the server logs of the broker you
>> upgraded (can you copy/paste them)?
>>
>> On Fri, May 3, 2019 at 7:29 AM ASHOK MACHERLA <iA...@outlook.com>
>> wrote:
>>
>> > *Dear Senthil*
>> >
>> > As you suggested , I follow but I’m facing errors
>> >
>> > This is my old configurations which is Kafka (0.10.1) version
>> >
>> > *****************************************************************
>> >
>> > broker.id=0
>> >
>> > port=9092
>> >
>> > delete.topic.enable=true
>> >
>> > message.max.bytes=100000
>> >
>> > listeners=SSL://192.168.175.128:9092
>> >
>> > advertised.listeners=SSL://192.168.175.128:9092
>> >
>> > num.network.threads=3
>> >
>> > num.io.threads=8
>> >
>> > socket.send.buffer.bytes=102400
>> >
>> > socket.receive.buffer.bytes=102400
>> >
>> > socket.request.max.bytes=104857600
>> >
>> > log.dirs=/opt/kafka/kafka-logs
>> >
>> > num.partitions=3
>> >
>> > default.replication.factor=3
>> >
>> > auto.topic.creation.enable=false
>> >
>> > num.recovery.threads.per.data.dir=1
>> >
>> > log.retention.hours=168
>> >
>> > log.segment.bytes=1073741824
>> >
>> > log.retention.check.interval.ms=300000
>> >
>> > ssl.keystore.location=/opt/kafka/certificate/kafka.keystore.jks
>> >
>> > ssl.keystore.password=Sbi#123
>> >
>> > ssl.key.password=Sbi#123
>> >
>> > ssl.truststore.location=/opt/kafka/certificate/kafka.truststore.jks
>> >
>> > ssl.truststore.password=Sbi#123
>> >
>> > security.inter.broker.protocol=SSL
>> >
>> > zookeeper.connect=192.168.175.128:2181,192.168.175.129:2181,
>> > 192.168.175.130:2181
>> >
>> > zookeeper.connection.timeout.ms=6000
>> >
>> > *****************************************************************
>> >
>> > After that i added three parameters into server.properties which is *new
>> > kafka version (2.2.0)*
>> >
>> > inter.broker.protocol.version=0.10.1
>> >
>> > log.message.format.version=0.10.1
>> >
>> > ssl.endpoint.identification.algorithm=""
>> >
>> > After that I stopped one Kafka node, and then I started new Kafka
>> (2.2.0)
>> > version in same node.
>> >
>> > in this port is opening , it's showing 9092 port number
>> >
>> > but getting errors due to SSL issues
>> >
>> > I this position first node is running with new version (2.2.0) and
>> > remaining two nodes are running with previous version (0.10.1)
>> >
>> > I checked topic describe command on second node, here ISR is not syncing
>> > with new version,
>> >
>> > it's showing only 1,2, here "0" is missing it means first nodes was not
>> > syncing with remaining nodes,
>> >
>> > it’s should show like 0,1,2.
>> >
>> > Please help Senthil
>> >
>> > I tried so many options like below ,
>> >
>> > ssl.endpoint.identification.algorithm=""
>> >
>> > ssl.endpoint.identification.algorithm=" "
>> >
>> > ssl.endpoint.identification.algorithm="none"
>> >
>> > ssl.endpoint.identification.algorithm="null"
>> >
>> > ssl.endpoint.identification.algorithm=null
>> >
>> > ssl.endpoint.identification.algorithm=https
>> >
>> > please tell what correct value I should mention, and port is is showing
>> > but why it's ISR showing only 1,2 instead of 0,1,2
>> >
>> > is there any firewall settings problems?
>> >
>> > Please help us to fix this Senthil
>> >
>> > thanks
>> >
>> >
>> > Sent from Outlook <http://aka.ms/weboutlook>
>> > ------------------------------
>> > *From:* ASHOK MACHERLA <ia...@outlook.com>
>> > *Sent:* 02 May 2019 13:28
>> > *To:* users@kafka.apache.org
>> > *Subject:* Re: Required guidelines for kafka upgrade
>> >
>> > OK Senthil
>> >
>> > Thanks for your support and cooperation
>> >
>> > Sent from Outlook
>> >
>>
>
Re: Required guidelines for kafka upgrade
Posted by ASHOK MACHERLA <iA...@outlook.com>.
Dear Senthil
first of all thanks for help,
after I set like ssl.endpoint.identification.algorithm = and then restart it's working fine.
after that I changed below parameters in all brokers
inter.broker.protocol.version=2.2.0
log.message.format.version=2.2.0
after that restarted one by one. ISR is showing 0,1,2 it's good,
but getting some SSL error, please look into these logs
[2019-05-03 11:00:37,789] INFO [SocketServer brokerId=0] Failed authentication with /192.168.175.128<http://192.168.175.128/> (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2019-05-03 11:01:19,611] INFO [SocketServer brokerId=0] Failed authentication with /192.168.175.128<http://192.168.175.128/> (SSL handshake failed) (org.apache.kafka.common.network.Selector)
getting logs all brokers,
I tried to produce sample messages to topic,
consumer is not print that messages .
please help us to fix this
Thanks a lot senthill
Sent from Outlook<http://aka.ms/weboutlook>
________________________________
From: SenthilKumar K <se...@gmail.com>
Sent: 03 May 2019 22:26
To: users@kafka.apache.org
Subject: Re: Required guidelines for kafka upgrade
Hi Ashok , From the logs its clear that problem with identification
algorithm.
at
org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
... 15 more
Caused by: java.security.cert.CertificateException: Unknown identification
algorithm: " "
Set empty and restart your broker.
--Senthil
On Fri, May 3, 2019 at 10:20 PM SenthilKumar K <se...@gmail.com>
wrote:
> Here is my server.properties.
>
>
> reserved.broker.max.id = 2147483647
> log.retention.bytes = 68719476736
> listeners = SSL://xxxxxx:9093
> socket.receive.buffer.bytes = 102400
> broker.id = xxx
> ssl.truststore.password = xxxxx
> auto.create.topics.enable = true
> ssl.enabled.protocols = TLSv1.2
> zookeeper.connect = xxxxx
> default.replication.factor = 2
> offsets.topic.replication.factor = 2
> log.message.timestamp.type = CreateTime
> min.insync.replicas = 2
> transaction.state.log.replication.factor = 2
> security.inter.broker.protocol = SSL
> socket.send.buffer.bytes = 102400
> num.partitions = 40
> ssl.secure.random.implementation = xxxxx
> ssl.key.password = xxxx
> log.retention.ms = 3600000
> log.cleaner.delete.retention.ms = 300000
> message.max.bytes = 67108864
> ssl.endpoint.identification.algorithm =
> log.roll.ms = 1800000
> log.message.timestamp.difference.max.ms = 14400000
> ssl.keystore.location = xxxx
> log.retention.hours = 168
> log.retention.check.interval.ms = 180000
> inter.broker.protocol.version = 2.2.0
> socket.request.max.bytes = 104857600
> log.dirs = xxxxx
> ssl.keystore.password = xxxx
> ssl.truststore.location = xxxx
>
> Pls set `ssl.endpoint.identification.algorithm = ` and restart your
> broker.
>
>
> On Fri, May 3, 2019 at 10:09 PM ASHOK MACHERLA <iA...@outlook.com>
> wrote:
>
>> Dear
>>
>>
>> Please find this below error
>>
>> org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake
>> failed
>> Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
>> at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1521)
>> at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:528)
>> at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1197)
>> at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1165)
>> at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.handshakeWrap(SslTransportLayer.java:448)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:313)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:265)
>> at
>> org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:170)
>> at
>> org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:536)
>> at org.apache.kafka.common.network.Selector.poll(Selector.java:472)
>> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
>> at
>> org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:74)
>> at
>> kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:95)
>> at
>> kafka.server.ReplicaFetcherThread.fetchFromLeader(ReplicaFetcherThread.scala:193)
>> at
>> kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:280)
>> at
>> kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3(AbstractFetcherThread.scala:132)
>> at
>> kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3$adapted(AbstractFetcherThread.scala:131)
>> at scala.Option.foreach(Option.scala:274)
>> at
>> kafka.server.AbstractFetcherThread.maybeFetch(AbstractFetcherThread.scala:131)
>> at
>> kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:113)
>> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82)
>> Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
>> at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
>> at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1709)
>> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:318)
>> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
>> at
>> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
>> at
>> sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
>> at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
>> at sun.security.ssl.Handshaker$1.run(Handshaker.java:970)
>> at sun.security.ssl.Handshaker$1.run(Handshaker.java:967)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1459)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:402)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:484)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
>> ... 15 more
>> Caused by: java.security.cert.CertificateException: Unknown
>> identification algorithm: " "
>> at
>> sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:462)
>> at
>> sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:436)
>> at
>> sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:252)
>> at
>> sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
>> at
>> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1626)
>> ... 24 more
>> [2019-05-03 06:36:23,840] INFO [ReplicaFetcher replicaId=0, leaderId=2,
>> fetcherId=0] Failed authentication with /192.168.175.130 (SSL handshake
>> failed) (org.apache.kafka.common.network.Selector)
>> [2019-05-03 06:36:23,842] ERROR [ReplicaFetcher replicaId=0, leaderId=2,
>> fetcherId=0] Connection to node 2 (/192.168.175.130:9092) failed
>> authentication due to: SSL handshake failed
>> (org.apache.kafka.clients.NetworkClient)
>> Sent from Outlook<http://aka.ms/weboutlook>
>> ________________________________
>> From: Harper Henn <ha...@datto.com>
>> Sent: 03 May 2019 21:35
>> To: users@kafka.apache.org
>> Subject: Re: Required guidelines for kafka upgrade
>>
>> What specific errors are you seeing in the server logs of the broker you
>> upgraded (can you copy/paste them)?
>>
>> On Fri, May 3, 2019 at 7:29 AM ASHOK MACHERLA <iA...@outlook.com>
>> wrote:
>>
>> > *Dear Senthil*
>> >
>> > As you suggested , I follow but I’m facing errors
>> >
>> > This is my old configurations which is Kafka (0.10.1) version
>> >
>> > *****************************************************************
>> >
>> > broker.id=0
>> >
>> > port=9092
>> >
>> > delete.topic.enable=true
>> >
>> > message.max.bytes=100000
>> >
>> > listeners=SSL://192.168.175.128:9092
>> >
>> > advertised.listeners=SSL://192.168.175.128:9092
>> >
>> > num.network.threads=3
>> >
>> > num.io.threads=8
>> >
>> > socket.send.buffer.bytes=102400
>> >
>> > socket.receive.buffer.bytes=102400
>> >
>> > socket.request.max.bytes=104857600
>> >
>> > log.dirs=/opt/kafka/kafka-logs
>> >
>> > num.partitions=3
>> >
>> > default.replication.factor=3
>> >
>> > auto.topic.creation.enable=false
>> >
>> > num.recovery.threads.per.data.dir=1
>> >
>> > log.retention.hours=168
>> >
>> > log.segment.bytes=1073741824
>> >
>> > log.retention.check.interval.ms=300000
>> >
>> > ssl.keystore.location=/opt/kafka/certificate/kafka.keystore.jks
>> >
>> > ssl.keystore.password=Sbi#123
>> >
>> > ssl.key.password=Sbi#123
>> >
>> > ssl.truststore.location=/opt/kafka/certificate/kafka.truststore.jks
>> >
>> > ssl.truststore.password=Sbi#123
>> >
>> > security.inter.broker.protocol=SSL
>> >
>> > zookeeper.connect=192.168.175.128:2181,192.168.175.129:2181,
>> > 192.168.175.130:2181
>> >
>> > zookeeper.connection.timeout.ms=6000
>> >
>> > *****************************************************************
>> >
>> > After that i added three parameters into server.properties which is *new
>> > kafka version (2.2.0)*
>> >
>> > inter.broker.protocol.version=0.10.1
>> >
>> > log.message.format.version=0.10.1
>> >
>> > ssl.endpoint.identification.algorithm=""
>> >
>> > After that I stopped one Kafka node, and then I started new Kafka
>> (2.2.0)
>> > version in same node.
>> >
>> > in this port is opening , it's showing 9092 port number
>> >
>> > but getting errors due to SSL issues
>> >
>> > I this position first node is running with new version (2.2.0) and
>> > remaining two nodes are running with previous version (0.10.1)
>> >
>> > I checked topic describe command on second node, here ISR is not syncing
>> > with new version,
>> >
>> > it's showing only 1,2, here "0" is missing it means first nodes was not
>> > syncing with remaining nodes,
>> >
>> > it’s should show like 0,1,2.
>> >
>> > Please help Senthil
>> >
>> > I tried so many options like below ,
>> >
>> > ssl.endpoint.identification.algorithm=""
>> >
>> > ssl.endpoint.identification.algorithm=" "
>> >
>> > ssl.endpoint.identification.algorithm="none"
>> >
>> > ssl.endpoint.identification.algorithm="null"
>> >
>> > ssl.endpoint.identification.algorithm=null
>> >
>> > ssl.endpoint.identification.algorithm=https
>> >
>> > please tell what correct value I should mention, and port is is showing
>> > but why it's ISR showing only 1,2 instead of 0,1,2
>> >
>> > is there any firewall settings problems?
>> >
>> > Please help us to fix this Senthil
>> >
>> > thanks
>> >
>> >
>> > Sent from Outlook <http://aka.ms/weboutlook>
>> > ------------------------------
>> > *From:* ASHOK MACHERLA <ia...@outlook.com>
>> > *Sent:* 02 May 2019 13:28
>> > *To:* users@kafka.apache.org
>> > *Subject:* Re: Required guidelines for kafka upgrade
>> >
>> > OK Senthil
>> >
>> > Thanks for your support and cooperation
>> >
>> > Sent from Outlook
>> >
>>
>
Re: Required guidelines for kafka upgrade
Posted by SenthilKumar K <se...@gmail.com>.
Hi Ashok , From the logs its clear that problem with identification
algorithm.
at
org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
... 15 more
Caused by: java.security.cert.CertificateException: Unknown identification
algorithm: " "
Set empty and restart your broker.
--Senthil
On Fri, May 3, 2019 at 10:20 PM SenthilKumar K <se...@gmail.com>
wrote:
> Here is my server.properties.
>
>
> reserved.broker.max.id = 2147483647
> log.retention.bytes = 68719476736
> listeners = SSL://xxxxxx:9093
> socket.receive.buffer.bytes = 102400
> broker.id = xxx
> ssl.truststore.password = xxxxx
> auto.create.topics.enable = true
> ssl.enabled.protocols = TLSv1.2
> zookeeper.connect = xxxxx
> default.replication.factor = 2
> offsets.topic.replication.factor = 2
> log.message.timestamp.type = CreateTime
> min.insync.replicas = 2
> transaction.state.log.replication.factor = 2
> security.inter.broker.protocol = SSL
> socket.send.buffer.bytes = 102400
> num.partitions = 40
> ssl.secure.random.implementation = xxxxx
> ssl.key.password = xxxx
> log.retention.ms = 3600000
> log.cleaner.delete.retention.ms = 300000
> message.max.bytes = 67108864
> ssl.endpoint.identification.algorithm =
> log.roll.ms = 1800000
> log.message.timestamp.difference.max.ms = 14400000
> ssl.keystore.location = xxxx
> log.retention.hours = 168
> log.retention.check.interval.ms = 180000
> inter.broker.protocol.version = 2.2.0
> socket.request.max.bytes = 104857600
> log.dirs = xxxxx
> ssl.keystore.password = xxxx
> ssl.truststore.location = xxxx
>
> Pls set `ssl.endpoint.identification.algorithm = ` and restart your
> broker.
>
>
> On Fri, May 3, 2019 at 10:09 PM ASHOK MACHERLA <iA...@outlook.com>
> wrote:
>
>> Dear
>>
>>
>> Please find this below error
>>
>> org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake
>> failed
>> Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
>> at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1521)
>> at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:528)
>> at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1197)
>> at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1165)
>> at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.handshakeWrap(SslTransportLayer.java:448)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:313)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:265)
>> at
>> org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:170)
>> at
>> org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:536)
>> at org.apache.kafka.common.network.Selector.poll(Selector.java:472)
>> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
>> at
>> org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:74)
>> at
>> kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:95)
>> at
>> kafka.server.ReplicaFetcherThread.fetchFromLeader(ReplicaFetcherThread.scala:193)
>> at
>> kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:280)
>> at
>> kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3(AbstractFetcherThread.scala:132)
>> at
>> kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3$adapted(AbstractFetcherThread.scala:131)
>> at scala.Option.foreach(Option.scala:274)
>> at
>> kafka.server.AbstractFetcherThread.maybeFetch(AbstractFetcherThread.scala:131)
>> at
>> kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:113)
>> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82)
>> Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
>> at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
>> at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1709)
>> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:318)
>> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
>> at
>> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
>> at
>> sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
>> at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
>> at sun.security.ssl.Handshaker$1.run(Handshaker.java:970)
>> at sun.security.ssl.Handshaker$1.run(Handshaker.java:967)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1459)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:402)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:484)
>> at
>> org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
>> ... 15 more
>> Caused by: java.security.cert.CertificateException: Unknown
>> identification algorithm: " "
>> at
>> sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:462)
>> at
>> sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:436)
>> at
>> sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:252)
>> at
>> sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
>> at
>> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1626)
>> ... 24 more
>> [2019-05-03 06:36:23,840] INFO [ReplicaFetcher replicaId=0, leaderId=2,
>> fetcherId=0] Failed authentication with /192.168.175.130 (SSL handshake
>> failed) (org.apache.kafka.common.network.Selector)
>> [2019-05-03 06:36:23,842] ERROR [ReplicaFetcher replicaId=0, leaderId=2,
>> fetcherId=0] Connection to node 2 (/192.168.175.130:9092) failed
>> authentication due to: SSL handshake failed
>> (org.apache.kafka.clients.NetworkClient)
>> Sent from Outlook<http://aka.ms/weboutlook>
>> ________________________________
>> From: Harper Henn <ha...@datto.com>
>> Sent: 03 May 2019 21:35
>> To: users@kafka.apache.org
>> Subject: Re: Required guidelines for kafka upgrade
>>
>> What specific errors are you seeing in the server logs of the broker you
>> upgraded (can you copy/paste them)?
>>
>> On Fri, May 3, 2019 at 7:29 AM ASHOK MACHERLA <iA...@outlook.com>
>> wrote:
>>
>> > *Dear Senthil*
>> >
>> > As you suggested , I follow but I’m facing errors
>> >
>> > This is my old configurations which is Kafka (0.10.1) version
>> >
>> > *****************************************************************
>> >
>> > broker.id=0
>> >
>> > port=9092
>> >
>> > delete.topic.enable=true
>> >
>> > message.max.bytes=100000
>> >
>> > listeners=SSL://192.168.175.128:9092
>> >
>> > advertised.listeners=SSL://192.168.175.128:9092
>> >
>> > num.network.threads=3
>> >
>> > num.io.threads=8
>> >
>> > socket.send.buffer.bytes=102400
>> >
>> > socket.receive.buffer.bytes=102400
>> >
>> > socket.request.max.bytes=104857600
>> >
>> > log.dirs=/opt/kafka/kafka-logs
>> >
>> > num.partitions=3
>> >
>> > default.replication.factor=3
>> >
>> > auto.topic.creation.enable=false
>> >
>> > num.recovery.threads.per.data.dir=1
>> >
>> > log.retention.hours=168
>> >
>> > log.segment.bytes=1073741824
>> >
>> > log.retention.check.interval.ms=300000
>> >
>> > ssl.keystore.location=/opt/kafka/certificate/kafka.keystore.jks
>> >
>> > ssl.keystore.password=Sbi#123
>> >
>> > ssl.key.password=Sbi#123
>> >
>> > ssl.truststore.location=/opt/kafka/certificate/kafka.truststore.jks
>> >
>> > ssl.truststore.password=Sbi#123
>> >
>> > security.inter.broker.protocol=SSL
>> >
>> > zookeeper.connect=192.168.175.128:2181,192.168.175.129:2181,
>> > 192.168.175.130:2181
>> >
>> > zookeeper.connection.timeout.ms=6000
>> >
>> > *****************************************************************
>> >
>> > After that i added three parameters into server.properties which is *new
>> > kafka version (2.2.0)*
>> >
>> > inter.broker.protocol.version=0.10.1
>> >
>> > log.message.format.version=0.10.1
>> >
>> > ssl.endpoint.identification.algorithm=""
>> >
>> > After that I stopped one Kafka node, and then I started new Kafka
>> (2.2.0)
>> > version in same node.
>> >
>> > in this port is opening , it's showing 9092 port number
>> >
>> > but getting errors due to SSL issues
>> >
>> > I this position first node is running with new version (2.2.0) and
>> > remaining two nodes are running with previous version (0.10.1)
>> >
>> > I checked topic describe command on second node, here ISR is not syncing
>> > with new version,
>> >
>> > it's showing only 1,2, here "0" is missing it means first nodes was not
>> > syncing with remaining nodes,
>> >
>> > it’s should show like 0,1,2.
>> >
>> > Please help Senthil
>> >
>> > I tried so many options like below ,
>> >
>> > ssl.endpoint.identification.algorithm=""
>> >
>> > ssl.endpoint.identification.algorithm=" "
>> >
>> > ssl.endpoint.identification.algorithm="none"
>> >
>> > ssl.endpoint.identification.algorithm="null"
>> >
>> > ssl.endpoint.identification.algorithm=null
>> >
>> > ssl.endpoint.identification.algorithm=https
>> >
>> > please tell what correct value I should mention, and port is is showing
>> > but why it's ISR showing only 1,2 instead of 0,1,2
>> >
>> > is there any firewall settings problems?
>> >
>> > Please help us to fix this Senthil
>> >
>> > thanks
>> >
>> >
>> > Sent from Outlook <http://aka.ms/weboutlook>
>> > ------------------------------
>> > *From:* ASHOK MACHERLA <ia...@outlook.com>
>> > *Sent:* 02 May 2019 13:28
>> > *To:* users@kafka.apache.org
>> > *Subject:* Re: Required guidelines for kafka upgrade
>> >
>> > OK Senthil
>> >
>> > Thanks for your support and cooperation
>> >
>> > Sent from Outlook
>> >
>>
>
Re: Required guidelines for kafka upgrade
Posted by SenthilKumar K <se...@gmail.com>.
Here is my server.properties.
reserved.broker.max.id = 2147483647
log.retention.bytes = 68719476736
listeners = SSL://xxxxxx:9093
socket.receive.buffer.bytes = 102400
broker.id = xxx
ssl.truststore.password = xxxxx
auto.create.topics.enable = true
ssl.enabled.protocols = TLSv1.2
zookeeper.connect = xxxxx
default.replication.factor = 2
offsets.topic.replication.factor = 2
log.message.timestamp.type = CreateTime
min.insync.replicas = 2
transaction.state.log.replication.factor = 2
security.inter.broker.protocol = SSL
socket.send.buffer.bytes = 102400
num.partitions = 40
ssl.secure.random.implementation = xxxxx
ssl.key.password = xxxx
log.retention.ms = 3600000
log.cleaner.delete.retention.ms = 300000
message.max.bytes = 67108864
ssl.endpoint.identification.algorithm =
log.roll.ms = 1800000
log.message.timestamp.difference.max.ms = 14400000
ssl.keystore.location = xxxx
log.retention.hours = 168
log.retention.check.interval.ms = 180000
inter.broker.protocol.version = 2.2.0
socket.request.max.bytes = 104857600
log.dirs = xxxxx
ssl.keystore.password = xxxx
ssl.truststore.location = xxxx
Pls set `ssl.endpoint.identification.algorithm = ` and restart your broker.
On Fri, May 3, 2019 at 10:09 PM ASHOK MACHERLA <iA...@outlook.com> wrote:
> Dear
>
>
> Please find this below error
>
> org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake
> failed
> Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
> at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1521)
> at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:528)
> at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1197)
> at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1165)
> at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
> at
> org.apache.kafka.common.network.SslTransportLayer.handshakeWrap(SslTransportLayer.java:448)
> at
> org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:313)
> at
> org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:265)
> at
> org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:170)
> at
> org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:536)
> at org.apache.kafka.common.network.Selector.poll(Selector.java:472)
> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
> at
> org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:74)
> at
> kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:95)
> at
> kafka.server.ReplicaFetcherThread.fetchFromLeader(ReplicaFetcherThread.scala:193)
> at
> kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:280)
> at
> kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3(AbstractFetcherThread.scala:132)
> at
> kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3$adapted(AbstractFetcherThread.scala:131)
> at scala.Option.foreach(Option.scala:274)
> at
> kafka.server.AbstractFetcherThread.maybeFetch(AbstractFetcherThread.scala:131)
> at
> kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:113)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82)
> Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
> at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
> at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1709)
> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:318)
> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
> at
> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
> at
> sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
> at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
> at sun.security.ssl.Handshaker$1.run(Handshaker.java:970)
> at sun.security.ssl.Handshaker$1.run(Handshaker.java:967)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1459)
> at
> org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:402)
> at
> org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:484)
> at
> org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
> ... 15 more
> Caused by: java.security.cert.CertificateException: Unknown identification
> algorithm: " "
> at
> sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:462)
> at
> sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:436)
> at
> sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:252)
> at
> sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
> at
> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1626)
> ... 24 more
> [2019-05-03 06:36:23,840] INFO [ReplicaFetcher replicaId=0, leaderId=2,
> fetcherId=0] Failed authentication with /192.168.175.130 (SSL handshake
> failed) (org.apache.kafka.common.network.Selector)
> [2019-05-03 06:36:23,842] ERROR [ReplicaFetcher replicaId=0, leaderId=2,
> fetcherId=0] Connection to node 2 (/192.168.175.130:9092) failed
> authentication due to: SSL handshake failed
> (org.apache.kafka.clients.NetworkClient)
> Sent from Outlook<http://aka.ms/weboutlook>
> ________________________________
> From: Harper Henn <ha...@datto.com>
> Sent: 03 May 2019 21:35
> To: users@kafka.apache.org
> Subject: Re: Required guidelines for kafka upgrade
>
> What specific errors are you seeing in the server logs of the broker you
> upgraded (can you copy/paste them)?
>
> On Fri, May 3, 2019 at 7:29 AM ASHOK MACHERLA <iA...@outlook.com> wrote:
>
> > *Dear Senthil*
> >
> > As you suggested , I follow but I’m facing errors
> >
> > This is my old configurations which is Kafka (0.10.1) version
> >
> > *****************************************************************
> >
> > broker.id=0
> >
> > port=9092
> >
> > delete.topic.enable=true
> >
> > message.max.bytes=100000
> >
> > listeners=SSL://192.168.175.128:9092
> >
> > advertised.listeners=SSL://192.168.175.128:9092
> >
> > num.network.threads=3
> >
> > num.io.threads=8
> >
> > socket.send.buffer.bytes=102400
> >
> > socket.receive.buffer.bytes=102400
> >
> > socket.request.max.bytes=104857600
> >
> > log.dirs=/opt/kafka/kafka-logs
> >
> > num.partitions=3
> >
> > default.replication.factor=3
> >
> > auto.topic.creation.enable=false
> >
> > num.recovery.threads.per.data.dir=1
> >
> > log.retention.hours=168
> >
> > log.segment.bytes=1073741824
> >
> > log.retention.check.interval.ms=300000
> >
> > ssl.keystore.location=/opt/kafka/certificate/kafka.keystore.jks
> >
> > ssl.keystore.password=Sbi#123
> >
> > ssl.key.password=Sbi#123
> >
> > ssl.truststore.location=/opt/kafka/certificate/kafka.truststore.jks
> >
> > ssl.truststore.password=Sbi#123
> >
> > security.inter.broker.protocol=SSL
> >
> > zookeeper.connect=192.168.175.128:2181,192.168.175.129:2181,
> > 192.168.175.130:2181
> >
> > zookeeper.connection.timeout.ms=6000
> >
> > *****************************************************************
> >
> > After that i added three parameters into server.properties which is *new
> > kafka version (2.2.0)*
> >
> > inter.broker.protocol.version=0.10.1
> >
> > log.message.format.version=0.10.1
> >
> > ssl.endpoint.identification.algorithm=""
> >
> > After that I stopped one Kafka node, and then I started new Kafka (2.2.0)
> > version in same node.
> >
> > in this port is opening , it's showing 9092 port number
> >
> > but getting errors due to SSL issues
> >
> > I this position first node is running with new version (2.2.0) and
> > remaining two nodes are running with previous version (0.10.1)
> >
> > I checked topic describe command on second node, here ISR is not syncing
> > with new version,
> >
> > it's showing only 1,2, here "0" is missing it means first nodes was not
> > syncing with remaining nodes,
> >
> > it’s should show like 0,1,2.
> >
> > Please help Senthil
> >
> > I tried so many options like below ,
> >
> > ssl.endpoint.identification.algorithm=""
> >
> > ssl.endpoint.identification.algorithm=" "
> >
> > ssl.endpoint.identification.algorithm="none"
> >
> > ssl.endpoint.identification.algorithm="null"
> >
> > ssl.endpoint.identification.algorithm=null
> >
> > ssl.endpoint.identification.algorithm=https
> >
> > please tell what correct value I should mention, and port is is showing
> > but why it's ISR showing only 1,2 instead of 0,1,2
> >
> > is there any firewall settings problems?
> >
> > Please help us to fix this Senthil
> >
> > thanks
> >
> >
> > Sent from Outlook <http://aka.ms/weboutlook>
> > ------------------------------
> > *From:* ASHOK MACHERLA <ia...@outlook.com>
> > *Sent:* 02 May 2019 13:28
> > *To:* users@kafka.apache.org
> > *Subject:* Re: Required guidelines for kafka upgrade
> >
> > OK Senthil
> >
> > Thanks for your support and cooperation
> >
> > Sent from Outlook
> >
>
Re: Required guidelines for kafka upgrade
Posted by ASHOK MACHERLA <iA...@outlook.com>.
Dear
Please find this below error
org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1521)
at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:528)
at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1197)
at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1165)
at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
at org.apache.kafka.common.network.SslTransportLayer.handshakeWrap(SslTransportLayer.java:448)
at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:313)
at org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:265)
at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:170)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:536)
at org.apache.kafka.common.network.Selector.poll(Selector.java:472)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:74)
at kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:95)
at kafka.server.ReplicaFetcherThread.fetchFromLeader(ReplicaFetcherThread.scala:193)
at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:280)
at kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3(AbstractFetcherThread.scala:132)
at kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3$adapted(AbstractFetcherThread.scala:131)
at scala.Option.foreach(Option.scala:274)
at kafka.server.AbstractFetcherThread.maybeFetch(AbstractFetcherThread.scala:131)
at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:113)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82)
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1709)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:318)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:970)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:967)
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1459)
at org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:402)
at org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:484)
at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
... 15 more
Caused by: java.security.cert.CertificateException: Unknown identification algorithm: " "
at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:462)
at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:436)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:252)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1626)
... 24 more
[2019-05-03 06:36:23,840] INFO [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=0] Failed authentication with /192.168.175.130 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2019-05-03 06:36:23,842] ERROR [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=0] Connection to node 2 (/192.168.175.130:9092) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
Sent from Outlook<http://aka.ms/weboutlook>
________________________________
From: Harper Henn <ha...@datto.com>
Sent: 03 May 2019 21:35
To: users@kafka.apache.org
Subject: Re: Required guidelines for kafka upgrade
What specific errors are you seeing in the server logs of the broker you
upgraded (can you copy/paste them)?
On Fri, May 3, 2019 at 7:29 AM ASHOK MACHERLA <iA...@outlook.com> wrote:
> *Dear Senthil*
>
> As you suggested , I follow but I’m facing errors
>
> This is my old configurations which is Kafka (0.10.1) version
>
> *****************************************************************
>
> broker.id=0
>
> port=9092
>
> delete.topic.enable=true
>
> message.max.bytes=100000
>
> listeners=SSL://192.168.175.128:9092
>
> advertised.listeners=SSL://192.168.175.128:9092
>
> num.network.threads=3
>
> num.io.threads=8
>
> socket.send.buffer.bytes=102400
>
> socket.receive.buffer.bytes=102400
>
> socket.request.max.bytes=104857600
>
> log.dirs=/opt/kafka/kafka-logs
>
> num.partitions=3
>
> default.replication.factor=3
>
> auto.topic.creation.enable=false
>
> num.recovery.threads.per.data.dir=1
>
> log.retention.hours=168
>
> log.segment.bytes=1073741824
>
> log.retention.check.interval.ms=300000
>
> ssl.keystore.location=/opt/kafka/certificate/kafka.keystore.jks
>
> ssl.keystore.password=Sbi#123
>
> ssl.key.password=Sbi#123
>
> ssl.truststore.location=/opt/kafka/certificate/kafka.truststore.jks
>
> ssl.truststore.password=Sbi#123
>
> security.inter.broker.protocol=SSL
>
> zookeeper.connect=192.168.175.128:2181,192.168.175.129:2181,
> 192.168.175.130:2181
>
> zookeeper.connection.timeout.ms=6000
>
> *****************************************************************
>
> After that i added three parameters into server.properties which is *new
> kafka version (2.2.0)*
>
> inter.broker.protocol.version=0.10.1
>
> log.message.format.version=0.10.1
>
> ssl.endpoint.identification.algorithm=""
>
> After that I stopped one Kafka node, and then I started new Kafka (2.2.0)
> version in same node.
>
> in this port is opening , it's showing 9092 port number
>
> but getting errors due to SSL issues
>
> I this position first node is running with new version (2.2.0) and
> remaining two nodes are running with previous version (0.10.1)
>
> I checked topic describe command on second node, here ISR is not syncing
> with new version,
>
> it's showing only 1,2, here "0" is missing it means first nodes was not
> syncing with remaining nodes,
>
> it’s should show like 0,1,2.
>
> Please help Senthil
>
> I tried so many options like below ,
>
> ssl.endpoint.identification.algorithm=""
>
> ssl.endpoint.identification.algorithm=" "
>
> ssl.endpoint.identification.algorithm="none"
>
> ssl.endpoint.identification.algorithm="null"
>
> ssl.endpoint.identification.algorithm=null
>
> ssl.endpoint.identification.algorithm=https
>
> please tell what correct value I should mention, and port is is showing
> but why it's ISR showing only 1,2 instead of 0,1,2
>
> is there any firewall settings problems?
>
> Please help us to fix this Senthil
>
> thanks
>
>
> Sent from Outlook <http://aka.ms/weboutlook>
> ------------------------------
> *From:* ASHOK MACHERLA <ia...@outlook.com>
> *Sent:* 02 May 2019 13:28
> *To:* users@kafka.apache.org
> *Subject:* Re: Required guidelines for kafka upgrade
>
> OK Senthil
>
> Thanks for your support and cooperation
>
> Sent from Outlook
>
Re: Required guidelines for kafka upgrade
Posted by Harper Henn <ha...@datto.com>.
What specific errors are you seeing in the server logs of the broker you
upgraded (can you copy/paste them)?
On Fri, May 3, 2019 at 7:29 AM ASHOK MACHERLA <iA...@outlook.com> wrote:
> *Dear Senthil*
>
> As you suggested , I follow but I’m facing errors
>
> This is my old configurations which is Kafka (0.10.1) version
>
> *****************************************************************
>
> broker.id=0
>
> port=9092
>
> delete.topic.enable=true
>
> message.max.bytes=100000
>
> listeners=SSL://192.168.175.128:9092
>
> advertised.listeners=SSL://192.168.175.128:9092
>
> num.network.threads=3
>
> num.io.threads=8
>
> socket.send.buffer.bytes=102400
>
> socket.receive.buffer.bytes=102400
>
> socket.request.max.bytes=104857600
>
> log.dirs=/opt/kafka/kafka-logs
>
> num.partitions=3
>
> default.replication.factor=3
>
> auto.topic.creation.enable=false
>
> num.recovery.threads.per.data.dir=1
>
> log.retention.hours=168
>
> log.segment.bytes=1073741824
>
> log.retention.check.interval.ms=300000
>
> ssl.keystore.location=/opt/kafka/certificate/kafka.keystore.jks
>
> ssl.keystore.password=Sbi#123
>
> ssl.key.password=Sbi#123
>
> ssl.truststore.location=/opt/kafka/certificate/kafka.truststore.jks
>
> ssl.truststore.password=Sbi#123
>
> security.inter.broker.protocol=SSL
>
> zookeeper.connect=192.168.175.128:2181,192.168.175.129:2181,
> 192.168.175.130:2181
>
> zookeeper.connection.timeout.ms=6000
>
> *****************************************************************
>
> After that i added three parameters into server.properties which is *new
> kafka version (2.2.0)*
>
> inter.broker.protocol.version=0.10.1
>
> log.message.format.version=0.10.1
>
> ssl.endpoint.identification.algorithm=""
>
> After that I stopped one Kafka node, and then I started new Kafka (2.2.0)
> version in same node.
>
> in this port is opening , it's showing 9092 port number
>
> but getting errors due to SSL issues
>
> I this position first node is running with new version (2.2.0) and
> remaining two nodes are running with previous version (0.10.1)
>
> I checked topic describe command on second node, here ISR is not syncing
> with new version,
>
> it's showing only 1,2, here "0" is missing it means first nodes was not
> syncing with remaining nodes,
>
> it’s should show like 0,1,2.
>
> Please help Senthil
>
> I tried so many options like below ,
>
> ssl.endpoint.identification.algorithm=""
>
> ssl.endpoint.identification.algorithm=" "
>
> ssl.endpoint.identification.algorithm="none"
>
> ssl.endpoint.identification.algorithm="null"
>
> ssl.endpoint.identification.algorithm=null
>
> ssl.endpoint.identification.algorithm=https
>
> please tell what correct value I should mention, and port is is showing
> but why it's ISR showing only 1,2 instead of 0,1,2
>
> is there any firewall settings problems?
>
> Please help us to fix this Senthil
>
> thanks
>
>
> Sent from Outlook <http://aka.ms/weboutlook>
> ------------------------------
> *From:* ASHOK MACHERLA <ia...@outlook.com>
> *Sent:* 02 May 2019 13:28
> *To:* users@kafka.apache.org
> *Subject:* Re: Required guidelines for kafka upgrade
>
> OK Senthil
>
> Thanks for your support and cooperation
>
> Sent from Outlook
>
Re: Required guidelines for kafka upgrade
Posted by ASHOK MACHERLA <iA...@outlook.com>.
Dear Senthil
As you suggested , I follow but I’m facing errors
This is my old configurations which is Kafka (0.10.1) version
*****************************************************************
broker.id=0
port=9092
delete.topic.enable=true
message.max.bytes=100000
listeners=SSL://192.168.175.128:9092
advertised.listeners=SSL://192.168.175.128:9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/opt/kafka/kafka-logs
num.partitions=3
default.replication.factor=3
auto.topic.creation.enable=false
num.recovery.threads.per.data.dir=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
ssl.keystore.location=/opt/kafka/certificate/kafka.keystore.jks
ssl.keystore.password=Sbi#123
ssl.key.password=Sbi#123
ssl.truststore.location=/opt/kafka/certificate/kafka.truststore.jks
ssl.truststore.password=Sbi#123
security.inter.broker.protocol=SSL
zookeeper.connect=192.168.175.128:2181,192.168.175.129:2181,192.168.175.130:2181
zookeeper.connection.timeout.ms=6000
*****************************************************************
After that i added three parameters into server.properties which is new kafka version (2.2.0)
inter.broker.protocol.version=0.10.1
log.message.format.version=0.10.1
ssl.endpoint.identification.algorithm=""
After that I stopped one Kafka node, and then I started new Kafka (2.2.0) version in same node.
in this port is opening , it's showing 9092 port number
but getting errors due to SSL issues
I this position first node is running with new version (2.2.0) and remaining two nodes are running with previous version (0.10.1)
I checked topic describe command on second node, here ISR is not syncing with new version,
it's showing only 1,2, here "0" is missing it means first nodes was not syncing with remaining nodes,
it’s should show like 0,1,2.
Please help Senthil
I tried so many options like below ,
ssl.endpoint.identification.algorithm=""
ssl.endpoint.identification.algorithm=" "
ssl.endpoint.identification.algorithm="none"
ssl.endpoint.identification.algorithm="null"
ssl.endpoint.identification.algorithm=null
ssl.endpoint.identification.algorithm=https
please tell what correct value I should mention, and port is is showing but why it's ISR showing only 1,2 instead of 0,1,2
is there any firewall settings problems?
Please help us to fix this Senthil
thanks
[cid:e4673732-efe9-451f-b306-90f18a5ebcf9][cid:90aa38c4-276d-47c3-b745-54d9bc142598]
Sent from Outlook<http://aka.ms/weboutlook>
________________________________
From: ASHOK MACHERLA <ia...@outlook.com>
Sent: 02 May 2019 13:28
To: users@kafka.apache.org
Subject: Re: Required guidelines for kafka upgrade
OK Senthil
Thanks for your support and cooperation
Sent from Outlook
Re: Required guidelines for kafka upgrade
Posted by ASHOK MACHERLA <ia...@outlook.com>.
OK Senthil
Thanks for your support and cooperation
Sent from Outlook
Re: Required guidelines for kafka upgrade
Posted by ASHOK MACHERLA <ia...@outlook.com>.
OK Senthil
Thanks for your support and cooperation
Sent from Outlook
Re: Required guidelines for kafka upgrade
Posted by SenthilKumar K <se...@gmail.com>.
Hi Ashok , I'd suggest you to do this exercise on your SQA environment
before making any change to Prod. Thanks!
--Senthil
On Thu, May 2, 2019 at 11:35 AM SenthilKumar K <se...@gmail.com>
wrote:
> Hi ,
> #1 - Download stable version 2.2.0 [kafka_2.11-2.2.0.tgz
> <https://www.apache.org/dyn/closer.cgi?path=/kafka/2.2.0/kafka_2.11-2.2.0.tgz>
> ]
> #2 - Update server.properties with below values.
> inter.broker.protocol.version=0.10.1
> log.message.format.version=0.10.1
> #3 - Make sure to compare the server.properties config of your existing
> cluster 0.10.1 and new server.properties
> #4 - Restart the Brokers one by one . After restarting broker you have to
> verify the overall cluster functionality
> #5 - If you are not setting ssl.endpoint.identification.algorithm value
> you would
> see org.apache.kafka.common.errors.SslAuthenticationException: SSL
> handshake failed.
> You need to make sure you have proper certs ( applicable only for SSL
> cluster)
> #6 - Update server.properties with below values and restart brokers one by
> one.
> inter.broker.protocol.version=2.2.0
> log.message.format.version=2.2.0
> Prerequisites : Kafka Producer and Consumer Compatibility Check. Make sure
> your current clients are compatible with Kafka 2.2.0.
> --Senthil
>
> On Thu, May 2, 2019 at 3:05 AM ASHOK MACHERLA <ia...@outlook.com> wrote:
>
>> Dear Senthil
>>
>> Please update on this mail.
>>
>> Please check my previous email.
>>
>> Can I follow that same steps to upgrade Kafka cluster.
>>
>> Or else suggest any changes required
>>
>> Thanks
>>
>> Sent from Outlook
>>
>
Re: Required guidelines for kafka upgrade
Posted by SenthilKumar K <se...@gmail.com>.
Hi ,
#1 - Download stable version 2.2.0 [kafka_2.11-2.2.0.tgz
<https://www.apache.org/dyn/closer.cgi?path=/kafka/2.2.0/kafka_2.11-2.2.0.tgz>
]
#2 - Update server.properties with below values.
inter.broker.protocol.version=0.10.1
log.message.format.version=0.10.1
#3 - Make sure to compare the server.properties config of your existing
cluster 0.10.1 and new server.properties
#4 - Restart the Brokers one by one . After restarting broker you have to
verify the overall cluster functionality
#5 - If you are not setting ssl.endpoint.identification.algorithm value
you would
see org.apache.kafka.common.errors.SslAuthenticationException: SSL
handshake failed.
You need to make sure you have proper certs ( applicable only for SSL
cluster)
#6 - Update server.properties with below values and restart brokers one by
one.
inter.broker.protocol.version=2.2.0
log.message.format.version=2.2.0
Prerequisites : Kafka Producer and Consumer Compatibility Check. Make sure
your current clients are compatible with Kafka 2.2.0.
--Senthil
On Thu, May 2, 2019 at 3:05 AM ASHOK MACHERLA <ia...@outlook.com> wrote:
> Dear Senthil
>
> Please update on this mail.
>
> Please check my previous email.
>
> Can I follow that same steps to upgrade Kafka cluster.
>
> Or else suggest any changes required
>
> Thanks
>
> Sent from Outlook
>