You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Sachit Murarka <co...@gmail.com> on 2021/01/18 11:46:31 UTC
Unable to connect to SSL enabled kafka
Hey Users,
I am getting the following error. Can anyone suggest?
Error in attempt 3 getting Kafka offsets:
org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake
failed
Caused by: javax.net.ssl.SSLProtocolException: Unexpected handshake
message: server_hello
at sun.security.ssl.Alert.createSSLException(Alert.java:129)
at sun.security.ssl.Alert.createSSLException(Alert.java:117)
at
sun.security.ssl.TransportContext.fatal(TransportContext.java:314)
at
sun.security.ssl.TransportContext.fatal(TransportContext.java:270)
at
sun.security.ssl.TransportContext.fatal(TransportContext.java:261)
at
sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
at
sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:987)
at
sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:974)
at java.security.AccessController.doPrivileged(Native Method)
at
sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:921)
at
org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:402)
at
org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:484)
at
org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
at
org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:265)
at
org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:170)
at
org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:547)
at org.apache.kafka.common.network.Selector.poll(Selector.java:483)
at
org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:547)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:262)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:212)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:230)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:444)
at
org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1267)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1235)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1168)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$partitionsAssignedToConsumer$2(KafkaOffsetReader.scala:538)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$withRetriesWithoutInterrupt$1(KafkaOffsetReader.scala:600)
at
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.util.UninterruptibleThread.runUninterruptibly(UninterruptibleThread.scala:77)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.withRetriesWithoutInterrupt(KafkaOffsetReader.scala:599)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$partitionsAssignedToConsumer$1(KafkaOffsetReader.scala:536)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.runUninterruptibly(KafkaOffsetReader.scala:567)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.partitionsAssignedToConsumer(KafkaOffsetReader.scala:536)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.fetchEarliestOffsets(KafkaOffsetReader.scala:298)
at
org.apache.spark.sql.kafka010.KafkaMicroBatchStream.$anonfun$getOrCreateInitialPartitionOffsets$1(KafkaMicroBatchStream.scala:151)
at scala.Option.getOrElse(Option.scala:189)
at
org.apache.spark.sql.kafka010.KafkaMicroBatchStream.getOrCreateInitialPartitionOffsets(KafkaMicroBatchStream.scala:148)
at
org.apache.spark.sql.kafka010.KafkaMicroBatchStream.initialOffset(KafkaMicroBatchStream.scala:76)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$5(MicroBatchExecution.scala:378)
at scala.Option.getOrElse(Option.scala:189)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$3(MicroBatchExecution.scala:378)
at
org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken(ProgressReporter.scala:352)
at
org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken$(ProgressReporter.scala:350)
at
org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:69)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$2(MicroBatchExecution.scala:371)
at
scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
at scala.collection.immutable.Map$Map1.foreach(Map.scala:128)
at scala.collection.TraversableLike.map(TraversableLike.scala:238)
at scala.collection.TraversableLike.map$(TraversableLike.scala:231)
at scala.collection.AbstractTraversable.map(Traversable.scala:108)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$1(MicroBatchExecution.scala:368)
at
scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.withProgressLocked(MicroBatchExecution.scala:597)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.constructNextBatch(MicroBatchExecution.scala:364)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$2(MicroBatchExecution.scala:208)
at
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken(ProgressReporter.scala:352)
at
org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken$(ProgressReporter.scala:350)
at
org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:69)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$1(MicroBatchExecution.scala:191)
at
org.apache.spark.sql.execution.streaming.ProcessingTimeExecutor.execute(TriggerExecutor.scala:57)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.runActivatedStream(MicroBatchExecution.scala:185)
at org.apache.spark.sql.execution.streaming.StreamExecution.org
$apache$spark$sql$execution$streaming$StreamExecution$$runStream(StreamExecution.sca
la:334)
Kind Regards,
Sachit Murarka
Re: [EXTERNAL] Unable to connect to SSL enabled kafka
Posted by Sachit Murarka <co...@gmail.com>.
Hello,
Getting follow error:
UTC|TransportContext.java:345|Fatal (UNKNOWN_CA): Received fatal
alert: unknown_ca (
2021-01-18_14:41:22.26584 "throwable" : {
2021-01-18_14:41:22.26584 javax.net.ssl.SSLHandshakeException:
Received fatal alert: unknown_ca
Kind Regards,
Sachit Murarka
On Mon, Jan 18, 2021 at 6:32 PM Sachit Murarka <co...@gmail.com>
wrote:
> Thanks @Jose Manuel Vega Monroy <jo...@williamhill.com> for reply.
>
> I think some issue is there with CN name. I am actually doing it in
> docker.
>
> Kind Regards,
> Sachit Murarka
>
>
> On Mon, Jan 18, 2021 at 6:11 PM Jose Manuel Vega Monroy <
> jose.monroy@williamhill.com> wrote:
>
>> @Sachit
>>
>>
>>
>> You can use this in your client to see details of SSL connection and
>> handshake.
>>
>>
>>
>> -Djavax.net.debug=ssl,handshake
>>
>>
>>
>> Ensure your certificate is valid, signed and imported properly in your
>> keystore, and having root CA certificate into your truststore.
>>
>>
>>
>> Additionally, review SSL config in your client is right one, for example
>> SSL protocol version or SSL auth if you using it, plus pointing to
>> trustsore and keystore files path.
>>
>>
>>
>> Cheers,
>>
>>
>>
>> [image:
>> https://www.williamhillplc.com/content/signature/WHlogo.gif?width=180]
>> <http://www.williamhill.com/>
>>
>> [image:
>> https://www.williamhillplc.com/content/signature/senet.gif?width=180]
>> <http://www.whenthefunstops.co.uk/>
>>
>> *Jose Manuel Vega Monroy *
>> *Java Developer / Software Developer Engineer in Test*
>>
>> Direct: +*0035 0 2008038 (Ext. 8038)*
>> Email: jose.monroy@williamhill.com
>>
>> William Hill | 6/1 Waterport Place | Gibraltar | GX11 1AA
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *From: *Sachit Murarka <co...@gmail.com>
>> *Date: *Monday, 18 January 2021 at 13:32
>> *To: *Jose Manuel Vega Monroy <jo...@williamhill.com>
>> *Cc: *"users@kafka.apache.org" <us...@kafka.apache.org>
>> *Subject: *Re: [EXTERNAL] Unable to connect to SSL enabled kafka
>>
>>
>>
>> Hey Jose,
>>
>>
>> Used these sets of commands for SSL config.
>>
>>
>>
>> keytool -keystore client.truststore.jks -storepass pass -alias CARoot
>> -import -file root.crt -noprompt
>> keytool -keystore client.keystore.jks -storepass pass -alias client
>> -validity 365 -keyalg RSA -genkey -keypass pass -dname
>> "CN=client,OU=xyz,O=abc,L=BLR,ST=ka,C=IN"
>> keytool -keystore client.keystore.jks -storepass pass -alias client
>> -certreq -file client.unsigned.crt
>> openssl x509 -req -CA root.crt -CAkey root.key -in client.unsigned.crt
>> -out client.signed.crt -days 365 -CAcreateserial -passin pass:pass
>> -extensions SAN -extfile <(printf
>> "\n[SAN]\nsubjectAltName=DNS:client,DNS:localhost")
>> keytool -keystore client.keystore.jks -storepass pass -alias CARoot
>> -import -file root.crt -noprompt
>> keytool -keystore client.keystore.jks -storepass pass -alias client
>> -import -file client.signed.crt
>>
>> Not sure what is causing the issue exactly.
>>
>>
>>
>> Kind Regards,
>> Sachit Murarka
>>
>>
>>
>>
>>
>> On Mon, Jan 18, 2021 at 5:49 PM Jose Manuel Vega Monroy <
>> jose.monroy@williamhill.com> wrote:
>>
>> @Sachit
>>
>> Review your SSL client config.
>>
>> Cheers,
>>
>> <http://www.williamhill.com/>
>> <http://www.whenthefunstops.co.uk/
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.whenthefunstops.co.uk_&d=DwMFaQ&c=pWn2jKJ-j-AhxLuiRFe-Qw&r=i5Pk4pirVCmwsmddZqplM1jyQtVWeoOOb-vkuqku5P8&m=qVT7wcON5mp40KH_xQ8EYLqCjpSmpEwpY1vf4EOvqwk&s=h3nzeeeSku1rOVllWxdpg11-1tKZ96zi6QB1MsDH8cw&e=>
>> >
>> Jose Manuel Vega Monroy
>> Java Developer / Software Developer Engineer in Test
>> Direct: +0035 0 2008038 (Ext. 8038)
>> Email: jose.monroy@williamhill.com
>> William Hill | 6/1 Waterport Place | Gibraltar | GX11 1AA
>>
>>
>>
>>
>> On 18/01/2021, 12:47, "Sachit Murarka" <co...@gmail.com> wrote:
>>
>> Hey Users,
>>
>> I am getting the following error. Can anyone suggest?
>>
>> Error in attempt 3 getting Kafka offsets:
>> org.apache.kafka.common.errors.SslAuthenticationException: SSL
>> handshake
>> failed
>> Caused by: javax.net.ssl.SSLProtocolException: Unexpected handshake
>> message: server_hello
>> at sun.security.ssl.Alert.createSSLException(Alert.java:129)
>> at sun.security.ssl.Alert.createSSLException(Alert.java:117)
>> at
>> sun.security.ssl.TransportContext.fatal(TransportContext.java:314)
>> at
>> sun.security.ssl.TransportContext.fatal(TransportContext.java:270)
>> at
>> sun.security.ssl.TransportContext.fatal(TransportContext.java:261)
>> at
>> sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
>> at
>>
>> sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:987)
>> at
>>
>> sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:974)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at
>>
>> sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:921)
>> at
>>
>> org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:402)
>> at
>>
>> org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:484)
>> at
>>
>> org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
>> at
>>
>> org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:265)
>> at
>>
>> org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:170)
>> at
>>
>> org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:547)
>> at
>> org.apache.kafka.common.network.Selector.poll(Selector.java:483)
>> at
>> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:547)
>> at
>>
>> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:262)
>> at
>>
>> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233)
>> at
>>
>> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:212)
>> at
>>
>> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:230)
>> at
>>
>> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:444)
>> at
>>
>> org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1267)
>> at
>>
>> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1235)
>> at
>>
>> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1168)
>> at
>>
>> org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$partitionsAssignedToConsumer$2(KafkaOffsetReader.scala:538)
>> at
>>
>> org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$withRetriesWithoutInterrupt$1(KafkaOffsetReader.scala:600)
>> at
>> scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
>> at
>>
>> org.apache.spark.util.UninterruptibleThread.runUninterruptibly(UninterruptibleThread.scala:77)
>> at
>>
>> org.apache.spark.sql.kafka010.KafkaOffsetReader.withRetriesWithoutInterrupt(KafkaOffsetReader.scala:599)
>> at
>>
>> org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$partitionsAssignedToConsumer$1(KafkaOffsetReader.scala:536)
>> at
>>
>> org.apache.spark.sql.kafka010.KafkaOffsetReader.runUninterruptibly(KafkaOffsetReader.scala:567)
>> at
>>
>> org.apache.spark.sql.kafka010.KafkaOffsetReader.partitionsAssignedToConsumer(KafkaOffsetReader.scala:536)
>> at
>>
>> org.apache.spark.sql.kafka010.KafkaOffsetReader.fetchEarliestOffsets(KafkaOffsetReader.scala:298)
>> at
>>
>> org.apache.spark.sql.kafka010.KafkaMicroBatchStream.$anonfun$getOrCreateInitialPartitionOffsets$1(KafkaMicroBatchStream.scala:151)
>> at scala.Option.getOrElse(Option.scala:189)
>> at
>>
>> org.apache.spark.sql.kafka010.KafkaMicroBatchStream.getOrCreateInitialPartitionOffsets(KafkaMicroBatchStream.scala:148)
>> at
>>
>> org.apache.spark.sql.kafka010.KafkaMicroBatchStream.initialOffset(KafkaMicroBatchStream.scala:76)
>> at
>>
>> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$5(MicroBatchExecution.scala:378)
>> at scala.Option.getOrElse(Option.scala:189)
>> at
>>
>> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$3(MicroBatchExecution.scala:378)
>> at
>>
>> org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken(ProgressReporter.scala:352)
>> at
>>
>> org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken$(ProgressReporter.scala:350)
>> at
>>
>> org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:69)
>> at
>>
>> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$2(MicroBatchExecution.scala:371)
>> at
>>
>> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
>> at scala.collection.immutable.Map$Map1.foreach(Map.scala:128)
>> at
>> scala.collection.TraversableLike.map(TraversableLike.scala:238)
>> at
>> scala.collection.TraversableLike.map$(TraversableLike.scala:231)
>> at
>> scala.collection.AbstractTraversable.map(Traversable.scala:108)
>> at
>>
>> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$1(MicroBatchExecution.scala:368)
>> at
>> scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
>> at
>>
>> org.apache.spark.sql.execution.streaming.MicroBatchExecution.withProgressLocked(MicroBatchExecution.scala:597)
>> at
>>
>> org.apache.spark.sql.execution.streaming.MicroBatchExecution.constructNextBatch(MicroBatchExecution.scala:364)
>> at
>>
>> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$2(MicroBatchExecution.scala:208)
>> at
>> scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
>> at
>>
>> org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken(ProgressReporter.scala:352)
>> at
>>
>> org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken$(ProgressReporter.scala:350)
>> at
>>
>> org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:69)
>> at
>>
>> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$1(MicroBatchExecution.scala:191)
>> at
>>
>> org.apache.spark.sql.execution.streaming.ProcessingTimeExecutor.execute(TriggerExecutor.scala:57)
>> at
>>
>> org.apache.spark.sql.execution.streaming.MicroBatchExecution.runActivatedStream(MicroBatchExecution.scala:185)
>> at
>> org.apache.spark.sql.execution.streaming.StreamExecution.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__org.apache.spark.sql.execution.streaming.StreamExecution.org&d=DwMFaQ&c=pWn2jKJ-j-AhxLuiRFe-Qw&r=i5Pk4pirVCmwsmddZqplM1jyQtVWeoOOb-vkuqku5P8&m=qVT7wcON5mp40KH_xQ8EYLqCjpSmpEwpY1vf4EOvqwk&s=hD4fhOqOrPZVIjjflBeqtXJDA6mFkMDhPyjiusCthOI&e=>
>>
>> $apache$spark$sql$execution$streaming$StreamExecution$$runStream(StreamExecution.sca
>> la:334)
>>
>>
>>
>> Kind Regards,
>> Sachit Murarka
>>
>>
>> Confidentiality: The contents of this e-mail and any attachments
>> transmitted with it are intended to be confidential to the intended
>> recipient; and may be privileged or otherwise protected from disclosure. If
>> you are not an intended recipient of this e-mail, do not duplicate or
>> redistribute it by any means. Please delete it and any attachments and
>> notify the sender that you have received it in error. This e-mail is sent
>> by a William Hill PLC group company. The William Hill group companies
>> include, among others, William Hill PLC (registered number 4212563),
>> William Hill Organization Limited (registered number 278208), William Hill
>> US HoldCo Inc, WHG (International) Limited (registered number 99191) and Mr
>> Green Limited (registered number C43260). Each of William Hill PLC and
>> William Hill Organization Limited is registered in England and Wales and
>> has its registered office at 1 Bedford Avenue, London, WC1B 3AU, UK.
>> William Hill U.S. HoldCo, Inc. is registered in Delaware and has its
>> registered office at 1007 N. Orange Street, 9 Floor, Wilmington, New Castle
>> County DE 19801 Delaware, United States of America. WHG (International)
>> Limited is registered in Gibraltar and has its registered office at 6/1
>> Waterport Place, Gibraltar. Mr Green Limited is registered in Malta and has
>> its registered office at Tagliaferro Business Centre, Level 7, 14 High
>> Street, Sliema SLM 1549, Malta. Unless specifically indicated otherwise,
>> the contents of this e-mail are subject to contract; and are not an
>> official statement, and do not necessarily represent the views, of William
>> Hill PLC, its subsidiaries or affiliated companies. Please note that
>> neither William Hill PLC, nor its subsidiaries and affiliated companies can
>> accept any responsibility for any viruses contained within this e-mail and
>> it is your responsibility to scan any emails and their attachments. William
>> Hill PLC, its subsidiaries and affiliated companies may monitor e-mail
>> traffic data and also the content of e-mails for effective operation of the
>> e-mail system, or for security, purposes.
>>
>> Confidentiality: The contents of this e-mail and any attachments
>> transmitted with it are intended to be confidential to the intended
>> recipient; and may be privileged or otherwise protected from disclosure. If
>> you are not an intended recipient of this e-mail, do not duplicate or
>> redistribute it by any means. Please delete it and any attachments and
>> notify the sender that you have received it in error. This e-mail is sent
>> by a William Hill PLC group company. The William Hill group companies
>> include, among others, William Hill PLC (registered number 4212563),
>> William Hill Organization Limited (registered number 278208), William Hill
>> US HoldCo Inc, WHG (International) Limited (registered number 99191) and Mr
>> Green Limited (registered number C43260). Each of William Hill PLC and
>> William Hill Organization Limited is registered in England and Wales and
>> has its registered office at 1 Bedford Avenue, London, WC1B 3AU, UK.
>> William Hill U.S. HoldCo, Inc. is registered in Delaware and has its
>> registered office at 1007 N. Orange Street, 9 Floor, Wilmington, New Castle
>> County DE 19801 Delaware, United States of America. WHG (International)
>> Limited is registered in Gibraltar and has its registered office at 6/1
>> Waterport Place, Gibraltar. Mr Green Limited is registered in Malta and has
>> its registered office at Tagliaferro Business Centre, Level 7, 14 High
>> Street, Sliema SLM 1549, Malta. Unless specifically indicated otherwise,
>> the contents of this e-mail are subject to contract; and are not an
>> official statement, and do not necessarily represent the views, of William
>> Hill PLC, its subsidiaries or affiliated companies. Please note that
>> neither William Hill PLC, nor its subsidiaries and affiliated companies can
>> accept any responsibility for any viruses contained within this e-mail and
>> it is your responsibility to scan any emails and their attachments. William
>> Hill PLC, its subsidiaries and affiliated companies may monitor e-mail
>> traffic data and also the content of e-mails for effective operation of the
>> e-mail system, or for security, purposes.
>>
>
Re: [EXTERNAL] Unable to connect to SSL enabled kafka
Posted by Sachit Murarka <co...@gmail.com>.
Thanks @Jose Manuel Vega Monroy <jo...@williamhill.com> for reply.
I think some issue is there with CN name. I am actually doing it in docker.
Kind Regards,
Sachit Murarka
On Mon, Jan 18, 2021 at 6:11 PM Jose Manuel Vega Monroy <
jose.monroy@williamhill.com> wrote:
> @Sachit
>
>
>
> You can use this in your client to see details of SSL connection and
> handshake.
>
>
>
> -Djavax.net.debug=ssl,handshake
>
>
>
> Ensure your certificate is valid, signed and imported properly in your
> keystore, and having root CA certificate into your truststore.
>
>
>
> Additionally, review SSL config in your client is right one, for example
> SSL protocol version or SSL auth if you using it, plus pointing to
> trustsore and keystore files path.
>
>
>
> Cheers,
>
>
>
> [image:
> https://www.williamhillplc.com/content/signature/WHlogo.gif?width=180]
> <http://www.williamhill.com/>
>
> [image:
> https://www.williamhillplc.com/content/signature/senet.gif?width=180]
> <http://www.whenthefunstops.co.uk/>
>
> *Jose Manuel Vega Monroy *
> *Java Developer / Software Developer Engineer in Test*
>
> Direct: +*0035 0 2008038 (Ext. 8038)*
> Email: jose.monroy@williamhill.com
>
> William Hill | 6/1 Waterport Place | Gibraltar | GX11 1AA
>
>
>
>
>
>
>
>
>
> *From: *Sachit Murarka <co...@gmail.com>
> *Date: *Monday, 18 January 2021 at 13:32
> *To: *Jose Manuel Vega Monroy <jo...@williamhill.com>
> *Cc: *"users@kafka.apache.org" <us...@kafka.apache.org>
> *Subject: *Re: [EXTERNAL] Unable to connect to SSL enabled kafka
>
>
>
> Hey Jose,
>
>
> Used these sets of commands for SSL config.
>
>
>
> keytool -keystore client.truststore.jks -storepass pass -alias CARoot
> -import -file root.crt -noprompt
> keytool -keystore client.keystore.jks -storepass pass -alias client
> -validity 365 -keyalg RSA -genkey -keypass pass -dname
> "CN=client,OU=xyz,O=abc,L=BLR,ST=ka,C=IN"
> keytool -keystore client.keystore.jks -storepass pass -alias client
> -certreq -file client.unsigned.crt
> openssl x509 -req -CA root.crt -CAkey root.key -in client.unsigned.crt
> -out client.signed.crt -days 365 -CAcreateserial -passin pass:pass
> -extensions SAN -extfile <(printf
> "\n[SAN]\nsubjectAltName=DNS:client,DNS:localhost")
> keytool -keystore client.keystore.jks -storepass pass -alias CARoot
> -import -file root.crt -noprompt
> keytool -keystore client.keystore.jks -storepass pass -alias client
> -import -file client.signed.crt
>
> Not sure what is causing the issue exactly.
>
>
>
> Kind Regards,
> Sachit Murarka
>
>
>
>
>
> On Mon, Jan 18, 2021 at 5:49 PM Jose Manuel Vega Monroy <
> jose.monroy@williamhill.com> wrote:
>
> @Sachit
>
> Review your SSL client config.
>
> Cheers,
>
> <http://www.williamhill.com/>
> <http://www.whenthefunstops.co.uk/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.whenthefunstops.co.uk_&d=DwMFaQ&c=pWn2jKJ-j-AhxLuiRFe-Qw&r=i5Pk4pirVCmwsmddZqplM1jyQtVWeoOOb-vkuqku5P8&m=qVT7wcON5mp40KH_xQ8EYLqCjpSmpEwpY1vf4EOvqwk&s=h3nzeeeSku1rOVllWxdpg11-1tKZ96zi6QB1MsDH8cw&e=>
> >
> Jose Manuel Vega Monroy
> Java Developer / Software Developer Engineer in Test
> Direct: +0035 0 2008038 (Ext. 8038)
> Email: jose.monroy@williamhill.com
> William Hill | 6/1 Waterport Place | Gibraltar | GX11 1AA
>
>
>
>
> On 18/01/2021, 12:47, "Sachit Murarka" <co...@gmail.com> wrote:
>
> Hey Users,
>
> I am getting the following error. Can anyone suggest?
>
> Error in attempt 3 getting Kafka offsets:
> org.apache.kafka.common.errors.SslAuthenticationException: SSL
> handshake
> failed
> Caused by: javax.net.ssl.SSLProtocolException: Unexpected handshake
> message: server_hello
> at sun.security.ssl.Alert.createSSLException(Alert.java:129)
> at sun.security.ssl.Alert.createSSLException(Alert.java:117)
> at
> sun.security.ssl.TransportContext.fatal(TransportContext.java:314)
> at
> sun.security.ssl.TransportContext.fatal(TransportContext.java:270)
> at
> sun.security.ssl.TransportContext.fatal(TransportContext.java:261)
> at
> sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
> at
>
> sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:987)
> at
>
> sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:974)
> at java.security.AccessController.doPrivileged(Native Method)
> at
>
> sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:921)
> at
>
> org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:402)
> at
>
> org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:484)
> at
>
> org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
> at
>
> org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:265)
> at
>
> org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:170)
> at
>
> org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:547)
> at
> org.apache.kafka.common.network.Selector.poll(Selector.java:483)
> at
> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:547)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:262)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:212)
> at
>
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:230)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:444)
> at
>
> org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1267)
> at
>
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1235)
> at
>
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1168)
> at
>
> org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$partitionsAssignedToConsumer$2(KafkaOffsetReader.scala:538)
> at
>
> org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$withRetriesWithoutInterrupt$1(KafkaOffsetReader.scala:600)
> at
> scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
> at
>
> org.apache.spark.util.UninterruptibleThread.runUninterruptibly(UninterruptibleThread.scala:77)
> at
>
> org.apache.spark.sql.kafka010.KafkaOffsetReader.withRetriesWithoutInterrupt(KafkaOffsetReader.scala:599)
> at
>
> org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$partitionsAssignedToConsumer$1(KafkaOffsetReader.scala:536)
> at
>
> org.apache.spark.sql.kafka010.KafkaOffsetReader.runUninterruptibly(KafkaOffsetReader.scala:567)
> at
>
> org.apache.spark.sql.kafka010.KafkaOffsetReader.partitionsAssignedToConsumer(KafkaOffsetReader.scala:536)
> at
>
> org.apache.spark.sql.kafka010.KafkaOffsetReader.fetchEarliestOffsets(KafkaOffsetReader.scala:298)
> at
>
> org.apache.spark.sql.kafka010.KafkaMicroBatchStream.$anonfun$getOrCreateInitialPartitionOffsets$1(KafkaMicroBatchStream.scala:151)
> at scala.Option.getOrElse(Option.scala:189)
> at
>
> org.apache.spark.sql.kafka010.KafkaMicroBatchStream.getOrCreateInitialPartitionOffsets(KafkaMicroBatchStream.scala:148)
> at
>
> org.apache.spark.sql.kafka010.KafkaMicroBatchStream.initialOffset(KafkaMicroBatchStream.scala:76)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$5(MicroBatchExecution.scala:378)
> at scala.Option.getOrElse(Option.scala:189)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$3(MicroBatchExecution.scala:378)
> at
>
> org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken(ProgressReporter.scala:352)
> at
>
> org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken$(ProgressReporter.scala:350)
> at
>
> org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:69)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$2(MicroBatchExecution.scala:371)
> at
>
> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
> at scala.collection.immutable.Map$Map1.foreach(Map.scala:128)
> at
> scala.collection.TraversableLike.map(TraversableLike.scala:238)
> at
> scala.collection.TraversableLike.map$(TraversableLike.scala:231)
> at
> scala.collection.AbstractTraversable.map(Traversable.scala:108)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$1(MicroBatchExecution.scala:368)
> at
> scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.withProgressLocked(MicroBatchExecution.scala:597)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.constructNextBatch(MicroBatchExecution.scala:364)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$2(MicroBatchExecution.scala:208)
> at
> scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
> at
>
> org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken(ProgressReporter.scala:352)
> at
>
> org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken$(ProgressReporter.scala:350)
> at
>
> org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:69)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$1(MicroBatchExecution.scala:191)
> at
>
> org.apache.spark.sql.execution.streaming.ProcessingTimeExecutor.execute(TriggerExecutor.scala:57)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.runActivatedStream(MicroBatchExecution.scala:185)
> at
> org.apache.spark.sql.execution.streaming.StreamExecution.org
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__org.apache.spark.sql.execution.streaming.StreamExecution.org&d=DwMFaQ&c=pWn2jKJ-j-AhxLuiRFe-Qw&r=i5Pk4pirVCmwsmddZqplM1jyQtVWeoOOb-vkuqku5P8&m=qVT7wcON5mp40KH_xQ8EYLqCjpSmpEwpY1vf4EOvqwk&s=hD4fhOqOrPZVIjjflBeqtXJDA6mFkMDhPyjiusCthOI&e=>
>
> $apache$spark$sql$execution$streaming$StreamExecution$$runStream(StreamExecution.sca
> la:334)
>
>
>
> Kind Regards,
> Sachit Murarka
>
>
> Confidentiality: The contents of this e-mail and any attachments
> transmitted with it are intended to be confidential to the intended
> recipient; and may be privileged or otherwise protected from disclosure. If
> you are not an intended recipient of this e-mail, do not duplicate or
> redistribute it by any means. Please delete it and any attachments and
> notify the sender that you have received it in error. This e-mail is sent
> by a William Hill PLC group company. The William Hill group companies
> include, among others, William Hill PLC (registered number 4212563),
> William Hill Organization Limited (registered number 278208), William Hill
> US HoldCo Inc, WHG (International) Limited (registered number 99191) and Mr
> Green Limited (registered number C43260). Each of William Hill PLC and
> William Hill Organization Limited is registered in England and Wales and
> has its registered office at 1 Bedford Avenue, London, WC1B 3AU, UK.
> William Hill U.S. HoldCo, Inc. is registered in Delaware and has its
> registered office at 1007 N. Orange Street, 9 Floor, Wilmington, New Castle
> County DE 19801 Delaware, United States of America. WHG (International)
> Limited is registered in Gibraltar and has its registered office at 6/1
> Waterport Place, Gibraltar. Mr Green Limited is registered in Malta and has
> its registered office at Tagliaferro Business Centre, Level 7, 14 High
> Street, Sliema SLM 1549, Malta. Unless specifically indicated otherwise,
> the contents of this e-mail are subject to contract; and are not an
> official statement, and do not necessarily represent the views, of William
> Hill PLC, its subsidiaries or affiliated companies. Please note that
> neither William Hill PLC, nor its subsidiaries and affiliated companies can
> accept any responsibility for any viruses contained within this e-mail and
> it is your responsibility to scan any emails and their attachments. William
> Hill PLC, its subsidiaries and affiliated companies may monitor e-mail
> traffic data and also the content of e-mails for effective operation of the
> e-mail system, or for security, purposes.
>
> Confidentiality: The contents of this e-mail and any attachments
> transmitted with it are intended to be confidential to the intended
> recipient; and may be privileged or otherwise protected from disclosure. If
> you are not an intended recipient of this e-mail, do not duplicate or
> redistribute it by any means. Please delete it and any attachments and
> notify the sender that you have received it in error. This e-mail is sent
> by a William Hill PLC group company. The William Hill group companies
> include, among others, William Hill PLC (registered number 4212563),
> William Hill Organization Limited (registered number 278208), William Hill
> US HoldCo Inc, WHG (International) Limited (registered number 99191) and Mr
> Green Limited (registered number C43260). Each of William Hill PLC and
> William Hill Organization Limited is registered in England and Wales and
> has its registered office at 1 Bedford Avenue, London, WC1B 3AU, UK.
> William Hill U.S. HoldCo, Inc. is registered in Delaware and has its
> registered office at 1007 N. Orange Street, 9 Floor, Wilmington, New Castle
> County DE 19801 Delaware, United States of America. WHG (International)
> Limited is registered in Gibraltar and has its registered office at 6/1
> Waterport Place, Gibraltar. Mr Green Limited is registered in Malta and has
> its registered office at Tagliaferro Business Centre, Level 7, 14 High
> Street, Sliema SLM 1549, Malta. Unless specifically indicated otherwise,
> the contents of this e-mail are subject to contract; and are not an
> official statement, and do not necessarily represent the views, of William
> Hill PLC, its subsidiaries or affiliated companies. Please note that
> neither William Hill PLC, nor its subsidiaries and affiliated companies can
> accept any responsibility for any viruses contained within this e-mail and
> it is your responsibility to scan any emails and their attachments. William
> Hill PLC, its subsidiaries and affiliated companies may monitor e-mail
> traffic data and also the content of e-mails for effective operation of the
> e-mail system, or for security, purposes.
>
Re: [EXTERNAL] Unable to connect to SSL enabled kafka
Posted by Jose Manuel Vega Monroy <jo...@williamhill.com>.
@Sachit
You can use this in your client to see details of SSL connection and handshake.
-Djavax.net.debug=ssl,handshake
Ensure your certificate is valid, signed and imported properly in your keystore, and having root CA certificate into your truststore.
Additionally, review SSL config in your client is right one, for example SSL protocol version or SSL auth if you using it, plus pointing to trustsore and keystore files path.
Cheers,
[https://www.williamhillplc.com/content/signature/WHlogo.gif?width=180]<http://www.williamhill.com/>
[https://www.williamhillplc.com/content/signature/senet.gif?width=180]<http://www.whenthefunstops.co.uk/>
Jose Manuel Vega Monroy
Java Developer / Software Developer Engineer in Test
Direct: +0035 0 2008038 (Ext. 8038)
Email: jose.monroy@williamhill.com<ma...@williamhill.com>
William Hill | 6/1 Waterport Place | Gibraltar | GX11 1AA
From: Sachit Murarka <co...@gmail.com>
Date: Monday, 18 January 2021 at 13:32
To: Jose Manuel Vega Monroy <jo...@williamhill.com>
Cc: "users@kafka.apache.org" <us...@kafka.apache.org>
Subject: Re: [EXTERNAL] Unable to connect to SSL enabled kafka
Hey Jose,
Used these sets of commands for SSL config.
keytool -keystore client.truststore.jks -storepass pass -alias CARoot -import -file root.crt -noprompt
keytool -keystore client.keystore.jks -storepass pass -alias client -validity 365 -keyalg RSA -genkey -keypass pass -dname "CN=client,OU=xyz,O=abc,L=BLR,ST=ka,C=IN"
keytool -keystore client.keystore.jks -storepass pass -alias client -certreq -file client.unsigned.crt
openssl x509 -req -CA root.crt -CAkey root.key -in client.unsigned.crt -out client.signed.crt -days 365 -CAcreateserial -passin pass:pass -extensions SAN -extfile <(printf "\n[SAN]\nsubjectAltName=DNS:client,DNS:localhost")
keytool -keystore client.keystore.jks -storepass pass -alias CARoot -import -file root.crt -noprompt
keytool -keystore client.keystore.jks -storepass pass -alias client -import -file client.signed.crt
Not sure what is causing the issue exactly.
Kind Regards,
Sachit Murarka
On Mon, Jan 18, 2021 at 5:49 PM Jose Manuel Vega Monroy <jo...@williamhill.com>> wrote:
@Sachit
Review your SSL client config.
Cheers,
<http://www.williamhill.com/>
<http://www.whenthefunstops.co.uk/<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.whenthefunstops.co.uk_&d=DwMFaQ&c=pWn2jKJ-j-AhxLuiRFe-Qw&r=i5Pk4pirVCmwsmddZqplM1jyQtVWeoOOb-vkuqku5P8&m=qVT7wcON5mp40KH_xQ8EYLqCjpSmpEwpY1vf4EOvqwk&s=h3nzeeeSku1rOVllWxdpg11-1tKZ96zi6QB1MsDH8cw&e=>>
Jose Manuel Vega Monroy
Java Developer / Software Developer Engineer in Test
Direct: +0035 0 2008038 (Ext. 8038)
Email: jose.monroy@williamhill.com<ma...@williamhill.com>
William Hill | 6/1 Waterport Place | Gibraltar | GX11 1AA
On 18/01/2021, 12:47, "Sachit Murarka" <co...@gmail.com>> wrote:
Hey Users,
I am getting the following error. Can anyone suggest?
Error in attempt 3 getting Kafka offsets:
org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake
failed
Caused by: javax.net.ssl.SSLProtocolException: Unexpected handshake
message: server_hello
at sun.security.ssl.Alert.createSSLException(Alert.java:129)
at sun.security.ssl.Alert.createSSLException(Alert.java:117)
at
sun.security.ssl.TransportContext.fatal(TransportContext.java:314)
at
sun.security.ssl.TransportContext.fatal(TransportContext.java:270)
at
sun.security.ssl.TransportContext.fatal(TransportContext.java:261)
at
sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
at
sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:987)
at
sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:974)
at java.security.AccessController.doPrivileged(Native Method)
at
sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:921)
at
org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:402)
at
org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:484)
at
org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
at
org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:265)
at
org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:170)
at
org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:547)
at org.apache.kafka.common.network.Selector.poll(Selector.java:483)
at
org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:547)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:262)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:212)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:230)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:444)
at
org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1267)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1235)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1168)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$partitionsAssignedToConsumer$2(KafkaOffsetReader.scala:538)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$withRetriesWithoutInterrupt$1(KafkaOffsetReader.scala:600)
at
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.util.UninterruptibleThread.runUninterruptibly(UninterruptibleThread.scala:77)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.withRetriesWithoutInterrupt(KafkaOffsetReader.scala:599)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$partitionsAssignedToConsumer$1(KafkaOffsetReader.scala:536)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.runUninterruptibly(KafkaOffsetReader.scala:567)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.partitionsAssignedToConsumer(KafkaOffsetReader.scala:536)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.fetchEarliestOffsets(KafkaOffsetReader.scala:298)
at
org.apache.spark.sql.kafka010.KafkaMicroBatchStream.$anonfun$getOrCreateInitialPartitionOffsets$1(KafkaMicroBatchStream.scala:151)
at scala.Option.getOrElse(Option.scala:189)
at
org.apache.spark.sql.kafka010.KafkaMicroBatchStream.getOrCreateInitialPartitionOffsets(KafkaMicroBatchStream.scala:148)
at
org.apache.spark.sql.kafka010.KafkaMicroBatchStream.initialOffset(KafkaMicroBatchStream.scala:76)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$5(MicroBatchExecution.scala:378)
at scala.Option.getOrElse(Option.scala:189)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$3(MicroBatchExecution.scala:378)
at
org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken(ProgressReporter.scala:352)
at
org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken$(ProgressReporter.scala:350)
at
org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:69)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$2(MicroBatchExecution.scala:371)
at
scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
at scala.collection.immutable.Map$Map1.foreach(Map.scala:128)
at scala.collection.TraversableLike.map(TraversableLike.scala:238)
at scala.collection.TraversableLike.map$(TraversableLike.scala:231)
at scala.collection.AbstractTraversable.map(Traversable.scala:108)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$1(MicroBatchExecution.scala:368)
at
scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.withProgressLocked(MicroBatchExecution.scala:597)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.constructNextBatch(MicroBatchExecution.scala:364)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$2(MicroBatchExecution.scala:208)
at
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken(ProgressReporter.scala:352)
at
org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken$(ProgressReporter.scala:350)
at
org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:69)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$1(MicroBatchExecution.scala:191)
at
org.apache.spark.sql.execution.streaming.ProcessingTimeExecutor.execute(TriggerExecutor.scala:57)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.runActivatedStream(MicroBatchExecution.scala:185)
at org.apache.spark.sql.execution.streaming.StreamExecution.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__org.apache.spark.sql.execution.streaming.StreamExecution.org&d=DwMFaQ&c=pWn2jKJ-j-AhxLuiRFe-Qw&r=i5Pk4pirVCmwsmddZqplM1jyQtVWeoOOb-vkuqku5P8&m=qVT7wcON5mp40KH_xQ8EYLqCjpSmpEwpY1vf4EOvqwk&s=hD4fhOqOrPZVIjjflBeqtXJDA6mFkMDhPyjiusCthOI&e=>
$apache$spark$sql$execution$streaming$StreamExecution$$runStream(StreamExecution.sca
la:334)
Kind Regards,
Sachit Murarka
Confidentiality: The contents of this e-mail and any attachments transmitted with it are intended to be confidential to the intended recipient; and may be privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. This e-mail is sent by a William Hill PLC group company. The William Hill group companies include, among others, William Hill PLC (registered number 4212563), William Hill Organization Limited (registered number 278208), William Hill US HoldCo Inc, WHG (International) Limited (registered number 99191) and Mr Green Limited (registered number C43260). Each of William Hill PLC and William Hill Organization Limited is registered in England and Wales and has its registered office at 1 Bedford Avenue, London, WC1B 3AU, UK. William Hill U.S. HoldCo, Inc. is registered in Delaware and has its registered office at 1007 N. Orange Street, 9 Floor, Wilmington, New Castle County DE 19801 Delaware, United States of America. WHG (International) Limited is registered in Gibraltar and has its registered office at 6/1 Waterport Place, Gibraltar. Mr Green Limited is registered in Malta and has its registered office at Tagliaferro Business Centre, Level 7, 14 High Street, Sliema SLM 1549, Malta. Unless specifically indicated otherwise, the contents of this e-mail are subject to contract; and are not an official statement, and do not necessarily represent the views, of William Hill PLC, its subsidiaries or affiliated companies. Please note that neither William Hill PLC, nor its subsidiaries and affiliated companies can accept any responsibility for any viruses contained within this e-mail and it is your responsibility to scan any emails and their attachments. William Hill PLC, its subsidiaries and affiliated companies may monitor e-mail traffic data and also the content of e-mails for effective operation of the e-mail system, or for security, purposes.
Confidentiality: The contents of this e-mail and any attachments transmitted with it are intended to be confidential to the intended recipient; and may be privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. This e-mail is sent by a William Hill PLC group company. The William Hill group companies include, among others, William Hill PLC (registered number 4212563), William Hill Organization Limited (registered number 278208), William Hill US HoldCo Inc, WHG (International) Limited (registered number 99191) and Mr Green Limited (registered number C43260). Each of William Hill PLC and William Hill Organization Limited is registered in England and Wales and has its registered office at 1 Bedford Avenue, London, WC1B 3AU, UK. William Hill U.S. HoldCo, Inc. is registered in Delaware and has its registered office at 1007 N. Orange Street, 9 Floor, Wilmington, New Castle County DE 19801 Delaware, United States of America. WHG (International) Limited is registered in Gibraltar and has its registered office at 6/1 Waterport Place, Gibraltar. Mr Green Limited is registered in Malta and has its registered office at Tagliaferro Business Centre, Level 7, 14 High Street, Sliema SLM 1549, Malta. Unless specifically indicated otherwise, the contents of this e-mail are subject to contract; and are not an official statement, and do not necessarily represent the views, of William Hill PLC, its subsidiaries or affiliated companies. Please note that neither William Hill PLC, nor its subsidiaries and affiliated companies can accept any responsibility for any viruses contained within this e-mail and it is your responsibility to scan any emails and their attachments. William Hill PLC, its subsidiaries and affiliated companies may monitor e-mail traffic data and also the content of e-mails for effective operation of the e-mail system, or for security, purposes.
Re: [EXTERNAL] Unable to connect to SSL enabled kafka
Posted by Sachit Murarka <co...@gmail.com>.
Hey Jose,
Used these sets of commands for SSL config.
keytool -keystore client.truststore.jks -storepass pass -alias CARoot
-import -file root.crt -noprompt
keytool -keystore client.keystore.jks -storepass pass -alias client
-validity 365 -keyalg RSA -genkey -keypass pass -dname
"CN=client,OU=xyz,O=abc,L=BLR,ST=ka,C=IN"
keytool -keystore client.keystore.jks -storepass pass -alias client
-certreq -file client.unsigned.crt
openssl x509 -req -CA root.crt -CAkey root.key -in client.unsigned.crt -out
client.signed.crt -days 365 -CAcreateserial -passin pass:pass -extensions
SAN -extfile <(printf "\n[SAN]\nsubjectAltName=DNS:client,DNS:localhost")
keytool -keystore client.keystore.jks -storepass pass -alias CARoot -import
-file root.crt -noprompt
keytool -keystore client.keystore.jks -storepass pass -alias client -import
-file client.signed.crt
Not sure what is causing the issue exactly.
Kind Regards,
Sachit Murarka
On Mon, Jan 18, 2021 at 5:49 PM Jose Manuel Vega Monroy <
jose.monroy@williamhill.com> wrote:
> @Sachit
>
> Review your SSL client config.
>
> Cheers,
>
> <http://www.williamhill.com/>
> <http://www.whenthefunstops.co.uk/>
> Jose Manuel Vega Monroy
> Java Developer / Software Developer Engineer in Test
> Direct: +0035 0 2008038 (Ext. 8038)
> Email: jose.monroy@williamhill.com
> William Hill | 6/1 Waterport Place | Gibraltar | GX11 1AA
>
>
>
>
> On 18/01/2021, 12:47, "Sachit Murarka" <co...@gmail.com> wrote:
>
> Hey Users,
>
> I am getting the following error. Can anyone suggest?
>
> Error in attempt 3 getting Kafka offsets:
> org.apache.kafka.common.errors.SslAuthenticationException: SSL
> handshake
> failed
> Caused by: javax.net.ssl.SSLProtocolException: Unexpected handshake
> message: server_hello
> at sun.security.ssl.Alert.createSSLException(Alert.java:129)
> at sun.security.ssl.Alert.createSSLException(Alert.java:117)
> at
> sun.security.ssl.TransportContext.fatal(TransportContext.java:314)
> at
> sun.security.ssl.TransportContext.fatal(TransportContext.java:270)
> at
> sun.security.ssl.TransportContext.fatal(TransportContext.java:261)
> at
> sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
> at
>
> sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:987)
> at
>
> sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:974)
> at java.security.AccessController.doPrivileged(Native Method)
> at
>
> sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:921)
> at
>
> org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:402)
> at
>
> org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:484)
> at
>
> org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
> at
>
> org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:265)
> at
>
> org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:170)
> at
>
> org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:547)
> at
> org.apache.kafka.common.network.Selector.poll(Selector.java:483)
> at
> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:547)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:262)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:212)
> at
>
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:230)
> at
>
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:444)
> at
>
> org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1267)
> at
>
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1235)
> at
>
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1168)
> at
>
> org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$partitionsAssignedToConsumer$2(KafkaOffsetReader.scala:538)
> at
>
> org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$withRetriesWithoutInterrupt$1(KafkaOffsetReader.scala:600)
> at
> scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
> at
>
> org.apache.spark.util.UninterruptibleThread.runUninterruptibly(UninterruptibleThread.scala:77)
> at
>
> org.apache.spark.sql.kafka010.KafkaOffsetReader.withRetriesWithoutInterrupt(KafkaOffsetReader.scala:599)
> at
>
> org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$partitionsAssignedToConsumer$1(KafkaOffsetReader.scala:536)
> at
>
> org.apache.spark.sql.kafka010.KafkaOffsetReader.runUninterruptibly(KafkaOffsetReader.scala:567)
> at
>
> org.apache.spark.sql.kafka010.KafkaOffsetReader.partitionsAssignedToConsumer(KafkaOffsetReader.scala:536)
> at
>
> org.apache.spark.sql.kafka010.KafkaOffsetReader.fetchEarliestOffsets(KafkaOffsetReader.scala:298)
> at
>
> org.apache.spark.sql.kafka010.KafkaMicroBatchStream.$anonfun$getOrCreateInitialPartitionOffsets$1(KafkaMicroBatchStream.scala:151)
> at scala.Option.getOrElse(Option.scala:189)
> at
>
> org.apache.spark.sql.kafka010.KafkaMicroBatchStream.getOrCreateInitialPartitionOffsets(KafkaMicroBatchStream.scala:148)
> at
>
> org.apache.spark.sql.kafka010.KafkaMicroBatchStream.initialOffset(KafkaMicroBatchStream.scala:76)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$5(MicroBatchExecution.scala:378)
> at scala.Option.getOrElse(Option.scala:189)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$3(MicroBatchExecution.scala:378)
> at
>
> org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken(ProgressReporter.scala:352)
> at
>
> org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken$(ProgressReporter.scala:350)
> at
>
> org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:69)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$2(MicroBatchExecution.scala:371)
> at
>
> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
> at scala.collection.immutable.Map$Map1.foreach(Map.scala:128)
> at
> scala.collection.TraversableLike.map(TraversableLike.scala:238)
> at
> scala.collection.TraversableLike.map$(TraversableLike.scala:231)
> at
> scala.collection.AbstractTraversable.map(Traversable.scala:108)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$1(MicroBatchExecution.scala:368)
> at
> scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.withProgressLocked(MicroBatchExecution.scala:597)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.constructNextBatch(MicroBatchExecution.scala:364)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$2(MicroBatchExecution.scala:208)
> at
> scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
> at
>
> org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken(ProgressReporter.scala:352)
> at
>
> org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken$(ProgressReporter.scala:350)
> at
>
> org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:69)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$1(MicroBatchExecution.scala:191)
> at
>
> org.apache.spark.sql.execution.streaming.ProcessingTimeExecutor.execute(TriggerExecutor.scala:57)
> at
>
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.runActivatedStream(MicroBatchExecution.scala:185)
> at
> org.apache.spark.sql.execution.streaming.StreamExecution.org
>
> $apache$spark$sql$execution$streaming$StreamExecution$$runStream(StreamExecution.sca
> la:334)
>
>
>
> Kind Regards,
> Sachit Murarka
>
>
> Confidentiality: The contents of this e-mail and any attachments
> transmitted with it are intended to be confidential to the intended
> recipient; and may be privileged or otherwise protected from disclosure. If
> you are not an intended recipient of this e-mail, do not duplicate or
> redistribute it by any means. Please delete it and any attachments and
> notify the sender that you have received it in error. This e-mail is sent
> by a William Hill PLC group company. The William Hill group companies
> include, among others, William Hill PLC (registered number 4212563),
> William Hill Organization Limited (registered number 278208), William Hill
> US HoldCo Inc, WHG (International) Limited (registered number 99191) and Mr
> Green Limited (registered number C43260). Each of William Hill PLC and
> William Hill Organization Limited is registered in England and Wales and
> has its registered office at 1 Bedford Avenue, London, WC1B 3AU, UK.
> William Hill U.S. HoldCo, Inc. is registered in Delaware and has its
> registered office at 1007 N. Orange Street, 9 Floor, Wilmington, New Castle
> County DE 19801 Delaware, United States of America. WHG (International)
> Limited is registered in Gibraltar and has its registered office at 6/1
> Waterport Place, Gibraltar. Mr Green Limited is registered in Malta and has
> its registered office at Tagliaferro Business Centre, Level 7, 14 High
> Street, Sliema SLM 1549, Malta. Unless specifically indicated otherwise,
> the contents of this e-mail are subject to contract; and are not an
> official statement, and do not necessarily represent the views, of William
> Hill PLC, its subsidiaries or affiliated companies. Please note that
> neither William Hill PLC, nor its subsidiaries and affiliated companies can
> accept any responsibility for any viruses contained within this e-mail and
> it is your responsibility to scan any emails and their attachments. William
> Hill PLC, its subsidiaries and affiliated companies may monitor e-mail
> traffic data and also the content of e-mails for effective operation of the
> e-mail system, or for security, purposes.
>
Re: [EXTERNAL] Unable to connect to SSL enabled kafka
Posted by Jose Manuel Vega Monroy <jo...@williamhill.com>.
@Sachit
Review your SSL client config.
Cheers,
<http://www.williamhill.com/>
<http://www.whenthefunstops.co.uk/>
Jose Manuel Vega Monroy
Java Developer / Software Developer Engineer in Test
Direct: +0035 0 2008038 (Ext. 8038)
Email: jose.monroy@williamhill.com
William Hill | 6/1 Waterport Place | Gibraltar | GX11 1AA
On 18/01/2021, 12:47, "Sachit Murarka" <co...@gmail.com> wrote:
Hey Users,
I am getting the following error. Can anyone suggest?
Error in attempt 3 getting Kafka offsets:
org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake
failed
Caused by: javax.net.ssl.SSLProtocolException: Unexpected handshake
message: server_hello
at sun.security.ssl.Alert.createSSLException(Alert.java:129)
at sun.security.ssl.Alert.createSSLException(Alert.java:117)
at
sun.security.ssl.TransportContext.fatal(TransportContext.java:314)
at
sun.security.ssl.TransportContext.fatal(TransportContext.java:270)
at
sun.security.ssl.TransportContext.fatal(TransportContext.java:261)
at
sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
at
sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:987)
at
sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:974)
at java.security.AccessController.doPrivileged(Native Method)
at
sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:921)
at
org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:402)
at
org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:484)
at
org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
at
org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:265)
at
org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:170)
at
org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:547)
at org.apache.kafka.common.network.Selector.poll(Selector.java:483)
at
org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:547)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:262)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:212)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:230)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:444)
at
org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1267)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1235)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1168)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$partitionsAssignedToConsumer$2(KafkaOffsetReader.scala:538)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$withRetriesWithoutInterrupt$1(KafkaOffsetReader.scala:600)
at
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.util.UninterruptibleThread.runUninterruptibly(UninterruptibleThread.scala:77)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.withRetriesWithoutInterrupt(KafkaOffsetReader.scala:599)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.$anonfun$partitionsAssignedToConsumer$1(KafkaOffsetReader.scala:536)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.runUninterruptibly(KafkaOffsetReader.scala:567)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.partitionsAssignedToConsumer(KafkaOffsetReader.scala:536)
at
org.apache.spark.sql.kafka010.KafkaOffsetReader.fetchEarliestOffsets(KafkaOffsetReader.scala:298)
at
org.apache.spark.sql.kafka010.KafkaMicroBatchStream.$anonfun$getOrCreateInitialPartitionOffsets$1(KafkaMicroBatchStream.scala:151)
at scala.Option.getOrElse(Option.scala:189)
at
org.apache.spark.sql.kafka010.KafkaMicroBatchStream.getOrCreateInitialPartitionOffsets(KafkaMicroBatchStream.scala:148)
at
org.apache.spark.sql.kafka010.KafkaMicroBatchStream.initialOffset(KafkaMicroBatchStream.scala:76)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$5(MicroBatchExecution.scala:378)
at scala.Option.getOrElse(Option.scala:189)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$3(MicroBatchExecution.scala:378)
at
org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken(ProgressReporter.scala:352)
at
org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken$(ProgressReporter.scala:350)
at
org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:69)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$2(MicroBatchExecution.scala:371)
at
scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
at scala.collection.immutable.Map$Map1.foreach(Map.scala:128)
at scala.collection.TraversableLike.map(TraversableLike.scala:238)
at scala.collection.TraversableLike.map$(TraversableLike.scala:231)
at scala.collection.AbstractTraversable.map(Traversable.scala:108)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$constructNextBatch$1(MicroBatchExecution.scala:368)
at
scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.withProgressLocked(MicroBatchExecution.scala:597)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.constructNextBatch(MicroBatchExecution.scala:364)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$2(MicroBatchExecution.scala:208)
at
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken(ProgressReporter.scala:352)
at
org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken$(ProgressReporter.scala:350)
at
org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:69)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$1(MicroBatchExecution.scala:191)
at
org.apache.spark.sql.execution.streaming.ProcessingTimeExecutor.execute(TriggerExecutor.scala:57)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution.runActivatedStream(MicroBatchExecution.scala:185)
at org.apache.spark.sql.execution.streaming.StreamExecution.org
$apache$spark$sql$execution$streaming$StreamExecution$$runStream(StreamExecution.sca
la:334)
Kind Regards,
Sachit Murarka
Confidentiality: The contents of this e-mail and any attachments transmitted with it are intended to be confidential to the intended recipient; and may be privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. This e-mail is sent by a William Hill PLC group company. The William Hill group companies include, among others, William Hill PLC (registered number 4212563), William Hill Organization Limited (registered number 278208), William Hill US HoldCo Inc, WHG (International) Limited (registered number 99191) and Mr Green Limited (registered number C43260). Each of William Hill PLC and William Hill Organization Limited is registered in England and Wales and has its registered office at 1 Bedford Avenue, London, WC1B 3AU, UK. William Hill U.S. HoldCo, Inc. is registered in Delaware and has its registered office at 1007 N. Orange Street, 9 Floor, Wilmington, New Castle County DE 19801 Delaware, United States of America. WHG (International) Limited is registered in Gibraltar and has its registered office at 6/1 Waterport Place, Gibraltar. Mr Green Limited is registered in Malta and has its registered office at Tagliaferro Business Centre, Level 7, 14 High Street, Sliema SLM 1549, Malta. Unless specifically indicated otherwise, the contents of this e-mail are subject to contract; and are not an official statement, and do not necessarily represent the views, of William Hill PLC, its subsidiaries or affiliated companies. Please note that neither William Hill PLC, nor its subsidiaries and affiliated companies can accept any responsibility for any viruses contained within this e-mail and it is your responsibility to scan any emails and their attachments. William Hill PLC, its subsidiaries and affiliated companies may monitor e-mail traffic data and also the content of e-mails for effective operation of the e-mail system, or for security, purposes.