You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/07/21 00:40:03 UTC
[GitHub] [spark] ramrock2008 opened a new pull request #29171: Spark works despite SSL certificate in keystore has expired
ramrock2008 opened a new pull request #29171:
URL: https://github.com/apache/spark/pull/29171
Hello Spark developers,
I'm running the following basic spark job on YARN with SSL enabled:
```
spark-submit --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode client /usr/lib/spark/examples/jars/spark-examples.jar 3
```
On my cluster, SSL was enabled for spark and it contains the following configurations:
```
$ cat /etc/spark/conf/spark-defaults.conf
spark.network.crypto.enabled true
spark.network.crypto.keyFactoryAlgorithm PBKDF2WithHmacSHA256
spark.ssl.protocol TLSv1.2
spark.ssl.keyStorePassword *********
spark.ssl.keyStore /tmp/abc/keystore.jks
spark.authenticate true
spark.network.crypto.keyLength 256
spark.network.crypto.saslFallback true
spark.ssl.keyPassword *********
spark.ssl.enabled true
spark.ssl.enabledAlgorithms TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA
spark.ssl.trustStore /tmp/abc/truststore.jks
spark.ssl.trustStorePassword **********
```
Although the keystores specified in the above configuration contain expired certificate, spark job still runs fine without throwing any error.
Any reason why spark job is not verifying the expiry date on certificates.
```
hadoop@ip-172-31-13-56 ~]$ keytool -list -v -keystore /tmp/abc/truststore.jks
Enter keystore password:
***************** WARNING WARNING WARNING *****************
* The integrity of the information stored in your keystore *
* has NOT been verified! In order to verify its integrity, *
* you must provide your keystore password. *
***************** WARNING WARNING WARNING *****************
Keystore type: jks
Keystore provider: SUN
Your keystore contains 1 entry
Alias name: test
Creation date: Jul 16, 2020
Entry type: trustedCertEntry
Owner: CN=Unknown, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown
Issuer: CN=Unknown, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown
Serial number: 42271acc
Valid from: Wed Jul 15 01:22:07 UTC 2020 until: Thu Jul 16 01:22:07 UTC 2020
Certificate fingerprints:
MD5: EB:B0:1E:C7:2A:81:1C:EB:25:DC:FD:47:FC:5D:9B:F6
SHA1: 15:C5:3F:E4:37:D0:F7:1B:7F:4D:13:B0:03:C6:18:FF:F1:6E:20:1A
SHA256: 08:0F:63:4B:29:B5:54:E7:24:46:C1:A3:9F:A9:45:D8:50:BF:49:38:6E:EA:8C:F7:2D:1C:30:B2:F5:72:9B:51
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3
Extensions:
#1: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 8D 2B E1 AC 3C B1 81 9D 79 FE 44 D8 EB BC E8 7F .+..<...y.D.....
0010: F7 0B F7 3D ...=
]
]
*******************************************
*******************************************
[hadoop@ip-172-31-13-56 ~]$ keytool -list -v -keystore /tmp/abc/keystore.jks
Enter keystore password:
***************** WARNING WARNING WARNING *****************
* The integrity of the information stored in your keystore *
* has NOT been verified! In order to verify its integrity, *
* you must provide your keystore password. *
***************** WARNING WARNING WARNING *****************
Keystore type: jks
Keystore provider: SUN
Your keystore contains 1 entry
Alias name: test
Creation date: Jul 15, 2020
Entry type: PrivateKeyEntry
Certificate chain length: 1
Certificate[1]:
Owner: CN=Unknown, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown
Issuer: CN=Unknown, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown
Serial number: 42271acc
Valid from: Wed Jul 15 01:22:07 UTC 2020 until: Thu Jul 16 01:22:07 UTC 2020
Certificate fingerprints:
MD5: EB:B0:1E:C7:2A:81:1C:EB:25:DC:FD:47:FC:5D:9B:F6
SHA1: 15:C5:3F:E4:37:D0:F7:1B:7F:4D:13:B0:03:C6:18:FF:F1:6E:20:1A
SHA256: 08:0F:63:4B:29:B5:54:E7:24:46:C1:A3:9F:A9:45:D8:50:BF:49:38:6E:EA:8C:F7:2D:1C:30:B2:F5:72:9B:51
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3
Extensions:
#1: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 8D 2B E1 AC 3C B1 81 9D 79 FE 44 D8 EB BC E8 7F .+..<...y.D.....
0010: F7 0B F7 3D ...=
]
]
*******************************************
*******************************************
Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /tmp/abc/keystore.jks -destkeystore /tmp/abc/keystore.jks -deststoretype pkcs12".
```
Is there something I'm missing here or is that the default behaviour of spark. Is there a way to make this authentication strict if possible
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #29171: Spark works despite SSL certificate in keystore has expired
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #29171:
URL: https://github.com/apache/spark/pull/29171#issuecomment-661614936
Please file an issue in JIRA, and read http://spark.apache.org/contributing.html
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29171: Spark works despite SSL certificate in keystore has expired
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29171:
URL: https://github.com/apache/spark/pull/29171#issuecomment-661497286
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29171: Spark works despite SSL certificate in keystore has expired
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29171:
URL: https://github.com/apache/spark/pull/29171#issuecomment-661497286
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #29171: Spark works despite SSL certificate in keystore has expired
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #29171:
URL: https://github.com/apache/spark/pull/29171
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29171: Spark works despite SSL certificate in keystore has expired
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29171:
URL: https://github.com/apache/spark/pull/29171#issuecomment-661498315
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org