You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Sagar Sumit (Jira)" <ji...@apache.org> on 2023/01/25 13:36:00 UTC
[jira] [Closed] (HUDI-4991) Make sure DeltaStreamer passes SSL key/truststore configs connecting to Schema Registry
[ https://issues.apache.org/jira/browse/HUDI-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sagar Sumit closed HUDI-4991.
-----------------------------
Resolution: Fixed
> Make sure DeltaStreamer passes SSL key/truststore configs connecting to Schema Registry
> ---------------------------------------------------------------------------------------
>
> Key: HUDI-4991
> URL: https://issues.apache.org/jira/browse/HUDI-4991
> Project: Apache Hudi
> Issue Type: Bug
> Components: deltastreamer
> Reporter: Alexey Kudinkin
> Assignee: Jonathan Vexler
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.13.0
>
>
> Originally reported at:
> [https://github.com/apache/hudi/issues/6842]
>
> Whenever Schema Registry is used requiring passing keystore/truststore params to access SSL certificates (like below) DeltaStreamer fails:
> {code:java}
> mode.hoodie.deltastreamer.schemaprovider.registry.url=https://schemaregistry.com
> schema.registry.ssl.keystore.location=/artifacts/topics/certs/keystore.jks
> schema.registry.ssl.keystore.password=****
> schema.registry.ssl.truststore.location=/artifacts/topics/certs/truststore.jks
> schema.registry.ssl.truststore.password=****
> schema.registry.ssl.key.password=**** {code}
> {code:java}
> at org.apache.hudi.utilities.schema.SchemaRegistryProvider.getSourceSchema(SchemaRegistryProvider.java:109)
> at org.apache.hudi.utilities.schema.SchemaProviderWithPostProcessor.lambda$getSourceSchema$0(SchemaProviderWithPostProcessor.java:41)
> at org.apache.hudi.common.util.Option.map(Option.java:108)
> at org.apache.hudi.utilities.schema.SchemaProviderWithPostProcessor.getSourceSchema(SchemaProviderWithPostProcessor.java:41)
> at org.apache.hudi.utilities.deltastreamer.DeltaSync.registerAvroSchemas(DeltaSync.java:839)
> at org.apache.hudi.utilities.deltastreamer.DeltaSync.<init>(DeltaSync.java:233)
> at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.<init>(HoodieDeltaStreamer.java:646)
> at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.<init>(HoodieDeltaStreamer.java:142)
> at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.<init>(HoodieDeltaStreamer.java:115)
> at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.main(HoodieDeltaStreamer.java:549)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1000)
> at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
> at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1089)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1098)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
> at sun.security.ssl.Alert.createSSLException(Alert.java:131)
> at sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
> at sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
> at sun.security.ssl.TransportContext.fatal(TransportContext.java:262)
> at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:654)
> at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:473)
> at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:369)
> at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:377)
> at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
> at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:422)
> at sun.security.ssl.TransportContext.dispatch(TransportContext.java:182)
> at sun.security.ssl.SSLTransport.decode(SSLTransport.java:152)
> at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1397)
> at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1305)
> at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:440)
> at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
> at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:197)
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1572)
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1500)
> at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:268)
> at org.apache.hudi.utilities.schema.SchemaRegistryProvider.getStream(SchemaRegistryProvider.java:91)
> at org.apache.hudi.utilities.schema.SchemaRegistryProvider.fetchSchemaFromRegistry(SchemaRegistryProvider.java:81)
> at org.apache.hudi.utilities.schema.SchemaRegistryProvider.getSchema(SchemaRegistryProvider.java:100)
> at org.apache.hudi.utilities.schema.SchemaRegistryProvider.getSourceSchema(SchemaRegistryProvider.java:107)
> ... 21 more
> Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
> at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:456)
> at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:323)
> at sun.security.validator.Validator.validate(Validator.java:271)
> at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:315)
> at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:223)
> at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129)
> at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:638)
> ... 40 more
> Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
> at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
> at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
> at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
> at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:451)
> ... 46 more
> 22/09/27 18:02:25 INFO ShutdownHookManager: Shutdown hook called
> 22/09/27 18:02:25 INFO ShutdownHookManager: Deleting directory /mnt/tmp/spark-c6361b3d-e191-4cd5-906e-b6e9235aa5b5
> 22/09/27 18:02:25 INFO ShutdownHookManager: Deleting directory /mnt/tmp/spark-fdca9bfd-a552-4ea3-b8b6-a7481f873440
> Command exiting with ret '1'``` {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)