You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "SHOBHIT SHUKLA (Jira)" <ji...@apache.org> on 2023/02/21 13:20:00 UTC
[jira] [Comment Edited] (SPARK-42511) Spark MasterWebUI and WorkerWebUI fail to start when NSSDB used as keystore.
[ https://issues.apache.org/jira/browse/SPARK-42511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691599#comment-17691599 ]
SHOBHIT SHUKLA edited comment on SPARK-42511 at 2/21/23 1:19 PM:
-----------------------------------------------------------------
setting up below options as well in spark submit command.
{quote}--conf spark.driver.extraJavaOptions= -Dderby.system.home=/.local/share/jupyter/runtime/kernel-21f4c67c-bcf6-4600-96dc-e59510692c6a-20230221_122939 -Dlog4j.logFile=/home/spark/shared/logs/kernel-python3.10-python3.10-20230221_122939.log -Dlog4j.configuration=file:/opt/ibm/jkg/log4j/log4j.properties -Dsemeru.fips=true -Djavax.net.ssl.keyStore=/opt/ibm/jdk/conf/security/nss.fips.cfg -Djavax.net.ssl.keyStorePassword=changeit -Djavax.net.ssl.keyStoreType=PKCS11 -Dfile.encoding=UTF-8
--conf spark.executor.extraJavaOptions= -Dderby.system.home=/.local/share/jupyter/runtime/kernel-21f4c67c-bcf6-4600-96dc-e59510692c6a-20230221_122939 -Dlog4j.logFile=/home/spark/shared/logs/kernel-python3.10-python3.10-20230221_122939.log -Dlog4j.configuration=file:/opt/ibm/jkg/log4j/log4j.properties -Dsemeru.fips=true -Djavax.net.ssl.keyStore=/opt/ibm/jdk/conf/security/nss.fips.cfg -Djavax.net.ssl.keyStorePassword=changeit -Djavax.net.ssl.keyStoreType=PKCS11 -Dfile.encoding=UTF-8
{quote}
was (Author: sshukla05):
setting up below options as well.
{quote}--conf spark.driver.extraJavaOptions= -Dderby.system.home=/.local/share/jupyter/runtime/kernel-21f4c67c-bcf6-4600-96dc-e59510692c6a-20230221_122939 -Dlog4j.logFile=/home/spark/shared/logs/kernel-python3.10-python3.10-20230221_122939.log -Dlog4j.configuration=file:/opt/ibm/jkg/log4j/log4j.properties -Dsemeru.fips=true -Djavax.net.ssl.keyStore=/opt/ibm/jdk/conf/security/nss.fips.cfg -Djavax.net.ssl.keyStorePassword=changeit -Djavax.net.ssl.keyStoreType=PKCS11 -Dfile.encoding=UTF-8
--conf spark.executor.extraJavaOptions= -Dderby.system.home=/.local/share/jupyter/runtime/kernel-21f4c67c-bcf6-4600-96dc-e59510692c6a-20230221_122939 -Dlog4j.logFile=/home/spark/shared/logs/kernel-python3.10-python3.10-20230221_122939.log -Dlog4j.configuration=file:/opt/ibm/jkg/log4j/log4j.properties -Dsemeru.fips=true -Djavax.net.ssl.keyStore=/opt/ibm/jdk/conf/security/nss.fips.cfg -Djavax.net.ssl.keyStorePassword=changeit -Djavax.net.ssl.keyStoreType=PKCS11 -Dfile.encoding=UTF-8
{quote}
> Spark MasterWebUI and WorkerWebUI fail to start when NSSDB used as keystore.
> ----------------------------------------------------------------------------
>
> Key: SPARK-42511
> URL: https://issues.apache.org/jira/browse/SPARK-42511
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, Spark Submit
> Affects Versions: 3.3.0, 3.3.1, 3.3.2
> Reporter: SHOBHIT SHUKLA
> Priority: Major
>
> When we are running Spark by setting up below SSL configurations, Spark masterwebui and workerwebui is fail to start.
> {quote}
> "spark.ssl.enabled":"true",
> "spark.ssl.keyStore":"/opt/ibm/jdk/conf/security/nss.fips.cfg",
> "spark.ssl.keyStorePassword":"<keystore passwd>",
> "spark.ssl.keyStoreType":"PKCS11"
> {quote}
> *Errors :*
> {quote}23/02/21 12:29:43 INFO Master: Running Spark version 3.3.1
> 23/02/21 12:29:43 ERROR MasterWebUI: Failed to bind MasterWebUI
> java.security.KeyStoreException: PKCS11 not found
> at java.base/java.security.KeyStore.getInstance(KeyStore.java:878)
> at org.sparkproject.jetty.util.security.CertificateUtils.getKeyStore(CertificateUtils.java:46)
> at org.sparkproject.jetty.util.ssl.SslContextFactory.loadKeyStore(SslContextFactory.java:1203)
> at org.sparkproject.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:322)
> at org.sparkproject.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:244)
> at org.sparkproject.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at org.sparkproject.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
> at org.sparkproject.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
> at org.sparkproject.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:97)
> at org.sparkproject.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at org.sparkproject.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
> at org.sparkproject.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
> at org.sparkproject.jetty.server.AbstractConnector.doStart(AbstractConnector.java:323)
> at org.sparkproject.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81)
> at org.sparkproject.jetty.server.ServerConnector.doStart(ServerConnector.java:234)
> at org.sparkproject.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at org.apache.spark.ui.JettyUtils$.newConnector$1(JettyUtils.scala:303)
> at org.apache.spark.ui.JettyUtils$.sslConnect$1(JettyUtils.scala:322)
> at org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$4(JettyUtils.scala:326)
> at org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$4$adapted(JettyUtils.scala:326)
> at org.apache.spark.util.Utils$.$anonfun$startServiceOnPort$2(Utils.scala:2401)
> at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:158)
> at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:2393)
> at org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$2(JettyUtils.scala:326)
> at org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$2$adapted(JettyUtils.scala:315)
> at scala.Option.map(Option.scala:230)
> at org.apache.spark.ui.JettyUtils$.startJettyServer(JettyUtils.scala:315)
> at org.apache.spark.ui.WebUI.initServer(WebUI.scala:144)
> at org.apache.spark.ui.WebUI.bind(WebUI.scala:153)
> at org.apache.spark.deploy.master.Master.onStart(Master.scala:138)
> at org.apache.spark.rpc.netty.Inbox.$anonfun$process$1(Inbox.scala:120)
> at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:213)
> at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:100)
> at org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:75)
> at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:41)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:839)
> Caused by: java.security.NoSuchAlgorithmException: PKCS11 KeyStore not available
> at java.base/sun.security.jca.GetInstance.getInstance(GetInstance.java:159)
> at java.base/java.security.Security.getImpl(Security.java:719)
> at java.base/java.security.KeyStore.getInstance(KeyStore.java:875)
> ... 37 more
> {quote}
> content of nss fips config file.
> {quote}# ===========================================================================
> # (c) Copyright IBM Corp. 2022, 2022 All Rights Reserved
> # ===========================================================================
> # This code is free software; you can redistribute it and/or modify it
> # under the terms of the GNU General Public License version 2 only, as
> # published by the Free Software Foundation.
> #
> # IBM designates this particular file as subject to the "Classpath" exception
> # as provided by IBM in the LICENSE file that accompanied this code.
> #
> # This code is distributed in the hope that it will be useful, but WITHOUT
> # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> # version 2 for more details (a copy is included in the LICENSE file that
> # accompanied this code).
> #
> # You should have received a copy of the GNU General Public License version
> # 2 along with this work; if not, see <http://www.gnu.org/licenses/>.
> # ===========================================================================
> name = NSS-FIPS
> nssLibraryDirectory = /usr/lib64
> nssSecmodDirectory = /etc/pki/nssdb
> nssDbMode = readOnly
> nssModule = fips
> attributes(*,CKO_SECRET_KEY,CKK_GENERIC_SECRET)={ CKA_SIGN=true }{quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org