You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ambari.apache.org by Greg Hill <gr...@RACKSPACE.COM> on 2016/01/22 16:01:47 UTC
openjdk update breaks ambari-agent 2-way ssl
We discovered a bug last night when our centos mirror updated openjdk and caused cluster builds to start failing. This is in Ambari 2.1.1 but I didn't see anything in github to indicate that this code has since changed. We tracked it down to the removal of the md5 algorithm from the list of supported algorithms in openjdk:
https://rhn.redhat.com/errata/RHSA-2016-0049.html
The ambari-server log (in DEBUG mode):
sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Algorithm constraints check failed: MD5withRSA
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:352)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:249)
at sun.security.validator.Validator.validate(Validator.java:260)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:279)
at sun.security.ssl.X509TrustManagerImpl.checkClientTrusted(X509TrustManagerImpl.java:130)
at sun.security.ssl.ServerHandshaker.clientCertificate(ServerHandshaker.java:1896)
... 13 more
Caused by: java.security.cert.CertPathValidatorException: Algorithm constraints check failed: MD5withRSA
at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:135)
at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:219)
at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:140)
at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:79)
at java.security.cert.CertPathValidator.validate(CertPathValidator.java:292)
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:347)
I looked at the agent code to see how it generates the cert, and it doesn't appear to be using md5:
https://github.com/apache/ambari/blob/trunk/ambari-agent/src/main/python/ambari_agent/security.py#L35
The openssl default *is* md5 but CentOS resets the default to sha256 in /etc/pki/tls/openssl.cnf:
[ req ]
default_bits = 2048
default_md = sha256
default_keyfile = privkey.pem
distinguished_name = req_distinguished_name
attributes = req_attributes
x509_extensions = v3_ca # The extentions to add to the self signed cert
I'm not sure where to look next. I think this is an Ambari bug, but I'm not exactly sure how to fix it or if we can fix it via configuration somehow.
Anyone know this stuff well and care to chime in? Or pull someone else in who does?
Greg
Re: openjdk update breaks ambari-agent 2-way ssl
Posted by Robert Levas <rl...@hortonworks.com>.
Greg…
I filed https://issues.apache.org/jira/browse/AMBARI-14778 for this issue.
Rob
From: Robert Levas <rl...@hortonworks.com>>
Reply-To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Date: Friday, January 22, 2016 at 1:09 PM
To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Subject: Re: openjdk update breaks ambari-agent 2-way ssl
Hi Greg.
Can you check the details about the agent-side certificate.
openssl x509 -in /var/lib/ambari-agent/keys/HIOSTNAME.crt -text –noout
I assume the signature algorithm is md5WithRSAEncryption:
Signature Algorithm: md5WithRSAEncryption
Ambari is generating this cert using a custom cnf file.
So to fix your issue, you need to edit /var/lib/ambari-server/keys/ca.config and change
default_md = md5
To
default_md = sha1
Then on each of your hosts, remove the cert files and restart the agent:
rm /var/lib/ambari-agent/keys/HOSTAME.*
ambari-agent restart
I think that this should be permanently changed in Ambari since md5 is no longer trusted. Then again sha1 isn’t either, so maybe the default needs to be sha256.
I hope this helps,
Rob
From: Greg Hill <gr...@RACKSPACE.COM>>
Reply-To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Date: Friday, January 22, 2016 at 10:01 AM
To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Subject: openjdk update breaks ambari-agent 2-way ssl
We discovered a bug last night when our centos mirror updated openjdk and caused cluster builds to start failing. This is in Ambari 2.1.1 but I didn't see anything in github to indicate that this code has since changed. We tracked it down to the removal of the md5 algorithm from the list of supported algorithms in openjdk:
https://rhn.redhat.com/errata/RHSA-2016-0049.html
The ambari-server log (in DEBUG mode):
sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Algorithm constraints check failed: MD5withRSA
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:352)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:249)
at sun.security.validator.Validator.validate(Validator.java:260)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:279)
at sun.security.ssl.X509TrustManagerImpl.checkClientTrusted(X509TrustManagerImpl.java:130)
at sun.security.ssl.ServerHandshaker.clientCertificate(ServerHandshaker.java:1896)
... 13 more
Caused by: java.security.cert.CertPathValidatorException: Algorithm constraints check failed: MD5withRSA
at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:135)
at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:219)
at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:140)
at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:79)
at java.security.cert.CertPathValidator.validate(CertPathValidator.java:292)
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:347)
I looked at the agent code to see how it generates the cert, and it doesn't appear to be using md5:
https://github.com/apache/ambari/blob/trunk/ambari-agent/src/main/python/ambari_agent/security.py#L35
The openssl default *is* md5 but CentOS resets the default to sha256 in /etc/pki/tls/openssl.cnf:
[ req ]
default_bits = 2048
default_md = sha256
default_keyfile = privkey.pem
distinguished_name = req_distinguished_name
attributes = req_attributes
x509_extensions = v3_ca # The extentions to add to the self signed cert
I'm not sure where to look next. I think this is an Ambari bug, but I'm not exactly sure how to fix it or if we can fix it via configuration somehow.
Anyone know this stuff well and care to chime in? Or pull someone else in who does?
Greg
Re: openjdk update breaks ambari-agent 2-way ssl
Posted by Greg Hill <gr...@RACKSPACE.COM>.
Oh, ambari uses its own config file, I didn't notice that. I'll test this out and report back next week probably.
Greg
From: Robert Levas <rl...@hortonworks.com>>
Reply-To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Date: Friday, January 22, 2016 at 12:09 PM
To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Subject: Re: openjdk update breaks ambari-agent 2-way ssl
Hi Greg.
Can you check the details about the agent-side certificate.
openssl x509 -in /var/lib/ambari-agent/keys/HIOSTNAME.crt -text -noout
I assume the signature algorithm is md5WithRSAEncryption:
Signature Algorithm: md5WithRSAEncryption
Ambari is generating this cert using a custom cnf file.
So to fix your issue, you need to edit /var/lib/ambari-server/keys/ca.config and change
default_md = md5
To
default_md = sha1
Then on each of your hosts, remove the cert files and restart the agent:
rm /var/lib/ambari-agent/keys/HOSTAME.*
ambari-agent restart
I think that this should be permanently changed in Ambari since md5 is no longer trusted. Then again sha1 isn't either, so maybe the default needs to be sha256.
I hope this helps,
Rob
From: Greg Hill <gr...@RACKSPACE.COM>>
Reply-To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Date: Friday, January 22, 2016 at 10:01 AM
To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Subject: openjdk update breaks ambari-agent 2-way ssl
We discovered a bug last night when our centos mirror updated openjdk and caused cluster builds to start failing. This is in Ambari 2.1.1 but I didn't see anything in github to indicate that this code has since changed. We tracked it down to the removal of the md5 algorithm from the list of supported algorithms in openjdk:
https://rhn.redhat.com/errata/RHSA-2016-0049.html
The ambari-server log (in DEBUG mode):
sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Algorithm constraints check failed: MD5withRSA
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:352)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:249)
at sun.security.validator.Validator.validate(Validator.java:260)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:279)
at sun.security.ssl.X509TrustManagerImpl.checkClientTrusted(X509TrustManagerImpl.java:130)
at sun.security.ssl.ServerHandshaker.clientCertificate(ServerHandshaker.java:1896)
... 13 more
Caused by: java.security.cert.CertPathValidatorException: Algorithm constraints check failed: MD5withRSA
at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:135)
at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:219)
at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:140)
at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:79)
at java.security.cert.CertPathValidator.validate(CertPathValidator.java:292)
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:347)
I looked at the agent code to see how it generates the cert, and it doesn't appear to be using md5:
https://github.com/apache/ambari/blob/trunk/ambari-agent/src/main/python/ambari_agent/security.py#L35
The openssl default *is* md5 but CentOS resets the default to sha256 in /etc/pki/tls/openssl.cnf:
[ req ]
default_bits = 2048
default_md = sha256
default_keyfile = privkey.pem
distinguished_name = req_distinguished_name
attributes = req_attributes
x509_extensions = v3_ca # The extentions to add to the self signed cert
I'm not sure where to look next. I think this is an Ambari bug, but I'm not exactly sure how to fix it or if we can fix it via configuration somehow.
Anyone know this stuff well and care to chime in? Or pull someone else in who does?
Greg
Re: openjdk update breaks ambari-agent 2-way ssl
Posted by Robert Levas <rl...@hortonworks.com>.
Hi Greg.
Can you check the details about the agent-side certificate.
openssl x509 -in /var/lib/ambari-agent/keys/HIOSTNAME.crt -text –noout
I assume the signature algorithm is md5WithRSAEncryption:
Signature Algorithm: md5WithRSAEncryption
Ambari is generating this cert using a custom cnf file.
So to fix your issue, you need to edit /var/lib/ambari-server/keys/ca.config and change
default_md = md5
To
default_md = sha1
Then on each of your hosts, remove the cert files and restart the agent:
rm /var/lib/ambari-agent/keys/HOSTAME.*
ambari-agent restart
I think that this should be permanently changed in Ambari since md5 is no longer trusted. Then again sha1 isn’t either, so maybe the default needs to be sha256.
I hope this helps,
Rob
From: Greg Hill <gr...@RACKSPACE.COM>>
Reply-To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Date: Friday, January 22, 2016 at 10:01 AM
To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Subject: openjdk update breaks ambari-agent 2-way ssl
We discovered a bug last night when our centos mirror updated openjdk and caused cluster builds to start failing. This is in Ambari 2.1.1 but I didn't see anything in github to indicate that this code has since changed. We tracked it down to the removal of the md5 algorithm from the list of supported algorithms in openjdk:
https://rhn.redhat.com/errata/RHSA-2016-0049.html
The ambari-server log (in DEBUG mode):
sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Algorithm constraints check failed: MD5withRSA
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:352)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:249)
at sun.security.validator.Validator.validate(Validator.java:260)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:279)
at sun.security.ssl.X509TrustManagerImpl.checkClientTrusted(X509TrustManagerImpl.java:130)
at sun.security.ssl.ServerHandshaker.clientCertificate(ServerHandshaker.java:1896)
... 13 more
Caused by: java.security.cert.CertPathValidatorException: Algorithm constraints check failed: MD5withRSA
at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:135)
at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:219)
at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:140)
at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:79)
at java.security.cert.CertPathValidator.validate(CertPathValidator.java:292)
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:347)
I looked at the agent code to see how it generates the cert, and it doesn't appear to be using md5:
https://github.com/apache/ambari/blob/trunk/ambari-agent/src/main/python/ambari_agent/security.py#L35
The openssl default *is* md5 but CentOS resets the default to sha256 in /etc/pki/tls/openssl.cnf:
[ req ]
default_bits = 2048
default_md = sha256
default_keyfile = privkey.pem
distinguished_name = req_distinguished_name
attributes = req_attributes
x509_extensions = v3_ca # The extentions to add to the self signed cert
I'm not sure where to look next. I think this is an Ambari bug, but I'm not exactly sure how to fix it or if we can fix it via configuration somehow.
Anyone know this stuff well and care to chime in? Or pull someone else in who does?
Greg