You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Joel Bernstein <jo...@gmail.com> on 2018/04/04 15:54:57 UTC
TestSSLRandomization is failing everytime
TestSSLRandomization is failing 100% of the time. Anybody make changes
recently to this code?
Re: TestSSLRandomization is failing everytime
Posted by Kevin Risden <kr...@apache.org>.
What Java version are you using? It's almost like it can't find the Java
cipher?
Kevin Risden
On Wed, Apr 4, 2018, 11:14 Joel Bernstein <jo...@gmail.com> wrote:
> I've looked through the recent commits but don't see anything that looks
> like it might have caused this. I don't see jenkins errors yet either.
>
> I'm running my tests on a fresh clone so I don't think this is due to an
> issue with my local repo.
>
> Anybody else seeing this problem locally?
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Wed, Apr 4, 2018 at 11:59 AM, Joel Bernstein <jo...@gmail.com>
> wrote:
>
>> When I run locally I get this stack trace:
>>
>> ERROR 0.02s | TestSSLRandomization.testRandomizedSslAndClientAuth <<<
>>
>> [junit4] > Throwable #1: java.lang.ExceptionInInitializerError
>>
>> [junit4] > at
>> __randomizedtesting.SeedInfo.seed([59B26B23CF90404E:D2E6DA446D0F0132]:0)
>>
>> [junit4] > at
>> org.apache.solr.cloud.TestSSLRandomization.testRandomizedSslAndClientAuth(TestSSLRandomization.java:50)
>>
>> [junit4] > at java.lang.Thread.run(Thread.java:745)
>>
>> [junit4] > Caused by: java.lang.RuntimeException: Unable to
>> initialize 'Default' SSLContext Algorithm, JVM is borked
>>
>> [junit4] > at
>> org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.<clinit>(TestMiniSolrCloudClusterSSL.java:67)
>>
>> [junit4] > ... 40 more
>>
>> [junit4] > Caused by: java.security.NoSuchAlgorithmException:
>> Error constructing implementation (algorithm: Default, provider: SunJSSE,
>> class: sun.security.ssl.SSLContextImpl$DefaultSSLContext)
>>
>> [junit4] > at
>> java.security.Provider$Service.newInstance(Provider.java:1617)
>>
>> [junit4] > at
>> sun.security.jca.GetInstance.getInstance(GetInstance.java:236)
>>
>> [junit4] > at
>> sun.security.jca.GetInstance.getInstance(GetInstance.java:164)
>>
>> [junit4] > at
>> javax.net.ssl.SSLContext.getInstance(SSLContext.java:156)
>>
>> [junit4] > at
>> javax.net.ssl.SSLContext.getDefault(SSLContext.java:96)
>>
>> [junit4] > at
>> org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.<clinit>(TestMiniSolrCloudClusterSSL.java:64)
>>
>> [junit4] > ... 40 more
>>
>> [junit4] > Caused by: java.io.IOException: Keystore was tampered
>> with, or password was incorrect
>>
>> [junit4] > at
>> sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:772)
>>
>> [junit4] > at
>> sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:55)
>>
>> [junit4] > at java.security.KeyStore.load(KeyStore.java:1445)
>>
>> [junit4] > at
>> sun.security.ssl.TrustManagerFactoryImpl.getCacertsKeyStore(TrustManagerFactoryImpl.java:226)
>>
>> [junit4] > at
>> sun.security.ssl.SSLContextImpl$DefaultSSLContext.getDefaultTrustManager(SSLContextImpl.java:767)
>>
>> [junit4] > at
>> sun.security.ssl.SSLContextImpl$DefaultSSLContext.<init>(SSLContextImpl.java:733)
>>
>> [junit4] > at
>> java.lang.reflect.Constructor.newInstance(Constructor.java:422)
>>
>> [junit4] > at
>> java.security.Provider$Service.newInstance(Provider.java:1595)
>>
>> [junit4] > ... 45 more
>>
>> [junit4] > Caused by: java.security.UnrecoverableKeyException:
>> Password verification failed
>>
>> [junit4] > at
>> sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:770)
>>
>> [junit4] > ... 55 more
>>
>>
>> Joel Bernstein
>> http://joelsolr.blogspot.com/
>>
>> On Wed, Apr 4, 2018 at 11:54 AM, Joel Bernstein <jo...@gmail.com>
>> wrote:
>>
>>> TestSSLRandomization is failing 100% of the time. Anybody make changes
>>> recently to this code?
>>>
>>
>>
>
Re: TestSSLRandomization is failing everytime
Posted by Joel Bernstein <jo...@gmail.com>.
I've looked through the recent commits but don't see anything that looks
like it might have caused this. I don't see jenkins errors yet either.
I'm running my tests on a fresh clone so I don't think this is due to an
issue with my local repo.
Anybody else seeing this problem locally?
Joel Bernstein
http://joelsolr.blogspot.com/
On Wed, Apr 4, 2018 at 11:59 AM, Joel Bernstein <jo...@gmail.com> wrote:
> When I run locally I get this stack trace:
>
> ERROR 0.02s | TestSSLRandomization.testRandomizedSslAndClientAuth <<<
>
> [junit4] > Throwable #1: java.lang.ExceptionInInitializerError
>
> [junit4] > at __randomizedtesting.SeedInfo.seed([59B26B23CF90404E:
> D2E6DA446D0F0132]:0)
>
> [junit4] > at org.apache.solr.cloud.TestSSLRandomization.
> testRandomizedSslAndClientAuth(TestSSLRandomization.java:50)
>
> [junit4] > at java.lang.Thread.run(Thread.java:745)
>
> [junit4] > Caused by: java.lang.RuntimeException: Unable to
> initialize 'Default' SSLContext Algorithm, JVM is borked
>
> [junit4] > at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.<
> clinit>(TestMiniSolrCloudClusterSSL.java:67)
>
> [junit4] > ... 40 more
>
> [junit4] > Caused by: java.security.NoSuchAlgorithmException: Error
> constructing implementation (algorithm: Default, provider: SunJSSE, class:
> sun.security.ssl.SSLContextImpl$DefaultSSLContext)
>
> [junit4] > at java.security.Provider$Service.newInstance(Provider.
> java:1617)
>
> [junit4] > at sun.security.jca.GetInstance.
> getInstance(GetInstance.java:236)
>
> [junit4] > at sun.security.jca.GetInstance.
> getInstance(GetInstance.java:164)
>
> [junit4] > at javax.net.ssl.SSLContext.getInstance(SSLContext.java:
> 156)
>
> [junit4] > at javax.net.ssl.SSLContext.
> getDefault(SSLContext.java:96)
>
> [junit4] > at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.<
> clinit>(TestMiniSolrCloudClusterSSL.java:64)
>
> [junit4] > ... 40 more
>
> [junit4] > Caused by: java.io.IOException: Keystore was tampered
> with, or password was incorrect
>
> [junit4] > at sun.security.provider.JavaKeyStore.engineLoad(
> JavaKeyStore.java:772)
>
> [junit4] > at sun.security.provider.JavaKeyStore$JKS.engineLoad(
> JavaKeyStore.java:55)
>
> [junit4] > at java.security.KeyStore.load(KeyStore.java:1445)
>
> [junit4] > at sun.security.ssl.TrustManagerFactoryImpl.
> getCacertsKeyStore(TrustManagerFactoryImpl.java:226)
>
> [junit4] > at sun.security.ssl.SSLContextImpl$DefaultSSLContext.
> getDefaultTrustManager(SSLContextImpl.java:767)
>
> [junit4] > at sun.security.ssl.SSLContextImpl$
> DefaultSSLContext.<init>(SSLContextImpl.java:733)
>
> [junit4] > at java.lang.reflect.Constructor.
> newInstance(Constructor.java:422)
>
> [junit4] > at java.security.Provider$Service.newInstance(Provider.
> java:1595)
>
> [junit4] > ... 45 more
>
> [junit4] > Caused by: java.security.UnrecoverableKeyException:
> Password verification failed
>
> [junit4] > at sun.security.provider.JavaKeyStore.engineLoad(
> JavaKeyStore.java:770)
>
> [junit4] > ... 55 more
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Wed, Apr 4, 2018 at 11:54 AM, Joel Bernstein <jo...@gmail.com>
> wrote:
>
>> TestSSLRandomization is failing 100% of the time. Anybody make changes
>> recently to this code?
>>
>
>
Re: TestSSLRandomization is failing everytime
Posted by Joel Bernstein <jo...@gmail.com>.
The code:
public static void main(String[] args) throws Exception {
System.out.println(javax.net.ssl.SSLContext.getDefault().get
Provider());
System.out.println(javax.net.ssl.SSLContext.getDefault().get
Protocol());
}
returns this from the command line:
SunJSSE version 1.8
Default
I believe this won't trip the issue I'm seeing because the error coming
when from the password check, which is not occurring from the command line.
The SSL test cases do cause a password check and that's where the error is
generated when comparing the digests.
Joel Bernstein
http://joelsolr.blogspot.com/
On Wed, Apr 4, 2018 at 5:26 PM, Martin Gainty <mg...@hotmail.com> wrote:
> the other way to determine your default provider is to dump java.security:
>
> cat $JAVA_HOME/jre/lib/security/java.security
>
>
> hoss suggests you determine the default provider and default protocol with
> this simple test:
>
> public static void main(String[] args) throws Exception {
> System.out.println(javax.net.ssl.SSLContext.getDefault().get
> Provider());
> System.out.println(javax.net.ssl.SSLContext.getDefault().get
> Protocol());
> }
>
> what do you see for either test?
>
>
> Martin
> ______________________________________________
>
>
>
>
> ------------------------------
> *From:* Joel Bernstein <jo...@gmail.com>
> *Sent:* Wednesday, April 4, 2018 3:35 PM
> *To:* lucene dev
> *Subject:* Re: TestSSLRandomization is failing everytime
>
> The code below ran fine from the command line and from a basic test case:
> System.out.println(javax.net.
> javax.net - This domain may be for sale! <http://javax.net/>
> javax.net
> {domain} has been informing visitors about topics such as {related1}. Join
> thousands of satisfied visitors who discovered {related2}. This domain may
> be for sale!
>
> ssl.SSLContext.getDefault().getProvider());
> System.out.println(javax.net.ssl.SSLContext.getDefault().getProtocol());
>
> The source code that throws the exception in JavaKeyStore.engineLoad is:
>
> if (password != null) {
> byte computed[], actual[];
> computed = md.digest();
> actual = new byte[computed.length];
> dis.readFully(actual);
> for (int i = 0; i < computed.length; i++) {
> if (computed[i] != actual[i]) {
> Throwable t = new UnrecoverableKeyException
> ("Password verification failed");
> throw (IOException)new IOException
> ("Keystore was tampered with, or "
> + "password was incorrect").initCause(t);
> }
> }
> }
>
>
> Notice that it's simply comparing the bytes from two digests.
>
> The digests are prepared using a SHA digest, notice that it just specifies
> SHA, which must choose the default SHA digest for the system it's on. If
> it choses a different SHA digest the password would not match. My best bet
> right now is that I've changed my default SHA digest to be something
> other then what was used to create the passwords for the test framework:
>
>
> /**
> * To guard against tampering with the keystore, we append a keyed
> * hash with a bit of whitener.
> */
> private MessageDigest getPreKeyedHash(char[] password)
> throws NoSuchAlgorithmException, UnsupportedEncodingException
> {
> int i, j;
>
> MessageDigest md = MessageDigest.getInstance("SHA");
> byte[] passwdBytes = new byte[password.length * 2];
> for (i=0, j=0; i<password.length; i++) {
> passwdBytes[j++] = (byte)(password[i] >> 8);
> passwdBytes[j++] = (byte)password[i];
> }
> md.update(passwdBytes);
> for (i=0; i<passwdBytes.length; i++)
> passwdBytes[i] = 0;
> md.update("Mighty Aphrodite".getBytes("UTF8"));
> return md;
> }
>
>
>
>
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Wed, Apr 4, 2018 at 2:11 PM, Joel Bernstein <jo...@gmail.com> wrote:
>
> Thanks Hoss, I will give this a try.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Wed, Apr 4, 2018 at 1:42 PM, Chris Hostetter <ho...@fucit.org>
> wrote:
>
>
> : I suspect I hosed something to do with my root certs on my local machine.
> : Fairly recently I was playing around with these certs while doing some
> SSL
> : work for Alfresco. This should be fun to fix...
>
> if that's your suspicion, i would start by testing out a simple java app
> that does nothing but...
>
> public static void main(String[] args) throws Exception {
> System.out.println(javax.net.ssl.SSLContext.getDefault().get
> Provider());
> System.out.println(javax.net.ssl.SSLContext.getDefault().get
> Protocol());
> }
>
> ...if thta fails on the commandline, then ou definitely hozed your
> machine.
>
> If that *does* work on the commandline, then try the same code in a
> trivial Junit test that just subclasses LuceneTestCase -- NOT
> SolrTestCaseJ4 -- to see if the problem is somewhere in our ant/lucene
> build setup, independent of any Solr SSL randomization.
>
> :
> : Joel Bernstein
> : http://joelsolr.blogspot.com/
> :
> : On Wed, Apr 4, 2018 at 12:29 PM, Joel Bernstein <jo...@gmail.com>
> wrote:
> :
> : > Ok, so it does sounds like a local problem then. Nothing much has
> changed
> : > locally. I'm still using the same Mac and Java version:
> : >
> : > defaultuildsMBP:clone2 joelbernstein$ java -version
> : >
> : > java version "1.8.0_40"
> : >
> : > Java(TM) SE Runtime Environment (build 1.8.0_40-b27)
> : >
> : > Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)
> : >
> : > I'll try running on a newer version of Java.
> : >
> : >
> : >
> : > Joel Bernstein
> : > http://joelsolr.blogspot.com/
> : >
> : > On Wed, Apr 4, 2018 at 12:19 PM, Chris Hostetter <
> hossman_lucene@fucit.org
> : > > wrote:
> : >
> : >>
> : >> : Subject: Re: TestSSLRandomization is failing everytime
> : >>
> : >> : When I run locally I get this stack trace:
> : >>
> : >> would be helpful to konw the branch, and the GIT SHA ... and if you
> can
> : >> reproduce if you checkout an older branch/SHA where you know you
> didn't
> : >> see this failure in the past (ex: the last SHA you committed, where
> you
> : >> should have run all tests to be certain you didn't break anything)
> : >>
> : >> Personally I can't reproduce on master/8e276b90f520d ...
> : >>
> : >> Let's look at the exception...
> : >>
> : >> : [junit4] > Caused by: java.lang.RuntimeException: Unable to
> : >> : initialize 'Default' SSLContext Algorithm, JVM is borked
> : >> :
> : >> : [junit4] > at
> : >> : org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.<clinit>(T
> : >> estMiniSolrCloudClusterSSL.java:67)
> : >> :
> : >> : [junit4] > ... 40 more
> : >> :
> : >> : [junit4] > Caused by: java.security.NoSuchAlgorithmException:
> : >> Error
> : >> : constructing implementation (algorithm: Default, provider: SunJSSE,
> : >> class:
> : >> : sun.security.ssl.SSLContextImpl$DefaultSSLContext)
> : >>
> : >> At first glance, it sounds like your JVM is completley jacked and
> doesn't
> : >> have any SSL support?
> : >>
> : >> The code throwing that exception is litterally...
> : >>
> : >> DEFAULT_SSL_CONTEXT = SSLContext.getDefault();
> : >>
> : >> ...ie: your JVM is saying the *default* SSL Algorithm, as choosen by
> the
> : >> JVM config, doens't exist ... but if we look farther down...
> : >>
> : >> : [junit4] > Caused by: java.io.IOException: Keystore was
> tampered
> : >> : with, or password was incorrect
> : >> ...
> : >> : [junit4] > Caused by: java.security.UnrecoverableKey
> Exception:
> : >> : Password verification failed
> : >>
> : >> ...well that's interesting. We do provide our own keystore when using
> : >> SSLTestConfig, but I honestly don't know off the top of my head if
> that's
> : >> even in use when this code is running?
> : >>
> : >> Can you tell us *ANYTHING* about the machine/jvm where you are running
> : >> this, and or what might have changed on your end since hte last time
> you
> : >> ran all tests w/o failure? what OS? new laptop? new java install? if
> you
> : >> "git co releases/lucene-solr/7.2.0" does this test pass? if so can you
> : >> "git bisect" to track down when it starts failing? etc...
> : >>
> : >>
> : >>
> : >> -Hoss
> : >> http://www.lucidworks.com/
> : >>
> : >> ---------------------------------------------------------------------
> : >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> : >> For additional commands, e-mail: dev-help@lucene.apache.org
> : >>
> : >>
> : >
> :
>
> -Hoss
> http://www.lucidworks.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
>
>
Re: TestSSLRandomization is failing everytime
Posted by Martin Gainty <mg...@hotmail.com>.
the other way to determine your default provider is to dump java.security:
cat $JAVA_HOME/jre/lib/security/java.security
hoss suggests you determine the default provider and default protocol with this simple test:
public static void main(String[] args) throws Exception {
System.out.println(javax.net<http://javax.net/>.ssl.SSLContext.getDefault().getProvider());
System.out.println(javax.net<http://javax.net/>.ssl.SSLContext.getDefault().getProtocol());
}
what do you see for either test?
Martin
______________________________________________
________________________________
From: Joel Bernstein <jo...@gmail.com>
Sent: Wednesday, April 4, 2018 3:35 PM
To: lucene dev
Subject: Re: TestSSLRandomization is failing everytime
The code below ran fine from the command line and from a basic test case:
System.out.println(javax.net<http://javax.net>.
javax.net - This domain may be for sale!<http://javax.net/>
javax.net
{domain} has been informing visitors about topics such as {related1}. Join thousands of satisfied visitors who discovered {related2}. This domain may be for sale!
ssl.SSLContext.getDefault().getProvider());
System.out.println(javax.net<http://javax.net>.ssl.SSLContext.getDefault().getProtocol());
The source code that throws the exception in JavaKeyStore.engineLoad is:
if (password != null) {
byte computed[], actual[];
computed = md.digest();
actual = new byte[computed.length];
dis.readFully(actual);
for (int i = 0; i < computed.length; i++) {
if (computed[i] != actual[i]) {
Throwable t = new UnrecoverableKeyException
("Password verification failed");
throw (IOException)new IOException
("Keystore was tampered with, or "
+ "password was incorrect").initCause(t);
}
}
}
Notice that it's simply comparing the bytes from two digests.
The digests are prepared using a SHA digest, notice that it just specifies SHA, which must choose the default SHA digest for the system it's on. If
it choses a different SHA digest the password would not match. My best bet right now is that I've changed my default SHA digest to be something
other then what was used to create the passwords for the test framework:
/**
* To guard against tampering with the keystore, we append a keyed
* hash with a bit of whitener.
*/
private MessageDigest getPreKeyedHash(char[] password)
throws NoSuchAlgorithmException, UnsupportedEncodingException
{
int i, j;
MessageDigest md = MessageDigest.getInstance("SHA");
byte[] passwdBytes = new byte[password.length * 2];
for (i=0, j=0; i<password.length; i++) {
passwdBytes[j++] = (byte)(password[i] >> 8);
passwdBytes[j++] = (byte)password[i];
}
md.update(passwdBytes);
for (i=0; i<passwdBytes.length; i++)
passwdBytes[i] = 0;
md.update("Mighty Aphrodite".getBytes("UTF8"));
return md;
}
Joel Bernstein
http://joelsolr.blogspot.com/
On Wed, Apr 4, 2018 at 2:11 PM, Joel Bernstein <jo...@gmail.com>> wrote:
Thanks Hoss, I will give this a try.
Joel Bernstein
http://joelsolr.blogspot.com/
On Wed, Apr 4, 2018 at 1:42 PM, Chris Hostetter <ho...@fucit.org>> wrote:
: I suspect I hosed something to do with my root certs on my local machine.
: Fairly recently I was playing around with these certs while doing some SSL
: work for Alfresco. This should be fun to fix...
if that's your suspicion, i would start by testing out a simple java app
that does nothing but...
public static void main(String[] args) throws Exception {
System.out.println(javax.net<http://javax.net>.ssl.SSLContext.getDefault().getProvider());
System.out.println(javax.net<http://javax.net>.ssl.SSLContext.getDefault().getProtocol());
}
...if thta fails on the commandline, then ou definitely hozed your
machine.
If that *does* work on the commandline, then try the same code in a
trivial Junit test that just subclasses LuceneTestCase -- NOT
SolrTestCaseJ4 -- to see if the problem is somewhere in our ant/lucene
build setup, independent of any Solr SSL randomization.
:
: Joel Bernstein
: http://joelsolr.blogspot.com/
:
: On Wed, Apr 4, 2018 at 12:29 PM, Joel Bernstein <jo...@gmail.com>> wrote:
:
: > Ok, so it does sounds like a local problem then. Nothing much has changed
: > locally. I'm still using the same Mac and Java version:
: >
: > defaultuildsMBP:clone2 joelbernstein$ java -version
: >
: > java version "1.8.0_40"
: >
: > Java(TM) SE Runtime Environment (build 1.8.0_40-b27)
: >
: > Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)
: >
: > I'll try running on a newer version of Java.
: >
: >
: >
: > Joel Bernstein
: > http://joelsolr.blogspot.com/
: >
: > On Wed, Apr 4, 2018 at 12:19 PM, Chris Hostetter <ho...@fucit.org>
: > > wrote:
: >
: >>
: >> : Subject: Re: TestSSLRandomization is failing everytime
: >>
: >> : When I run locally I get this stack trace:
: >>
: >> would be helpful to konw the branch, and the GIT SHA ... and if you can
: >> reproduce if you checkout an older branch/SHA where you know you didn't
: >> see this failure in the past (ex: the last SHA you committed, where you
: >> should have run all tests to be certain you didn't break anything)
: >>
: >> Personally I can't reproduce on master/8e276b90f520d ...
: >>
: >> Let's look at the exception...
: >>
: >> : [junit4] > Caused by: java.lang.RuntimeException: Unable to
: >> : initialize 'Default' SSLContext Algorithm, JVM is borked
: >> :
: >> : [junit4] > at
: >> : org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.<clinit>(T
: >> estMiniSolrCloudClusterSSL.java:67)
: >> :
: >> : [junit4] > ... 40 more
: >> :
: >> : [junit4] > Caused by: java.security.NoSuchAlgorithmException:
: >> Error
: >> : constructing implementation (algorithm: Default, provider: SunJSSE,
: >> class:
: >> : sun.security.ssl.SSLContextImpl$DefaultSSLContext)
: >>
: >> At first glance, it sounds like your JVM is completley jacked and doesn't
: >> have any SSL support?
: >>
: >> The code throwing that exception is litterally...
: >>
: >> DEFAULT_SSL_CONTEXT = SSLContext.getDefault();
: >>
: >> ...ie: your JVM is saying the *default* SSL Algorithm, as choosen by the
: >> JVM config, doens't exist ... but if we look farther down...
: >>
: >> : [junit4] > Caused by: java.io.IOException: Keystore was tampered
: >> : with, or password was incorrect
: >> ...
: >> : [junit4] > Caused by: java.security.UnrecoverableKeyException:
: >> : Password verification failed
: >>
: >> ...well that's interesting. We do provide our own keystore when using
: >> SSLTestConfig, but I honestly don't know off the top of my head if that's
: >> even in use when this code is running?
: >>
: >> Can you tell us *ANYTHING* about the machine/jvm where you are running
: >> this, and or what might have changed on your end since hte last time you
: >> ran all tests w/o failure? what OS? new laptop? new java install? if you
: >> "git co releases/lucene-solr/7.2.0" does this test pass? if so can you
: >> "git bisect" to track down when it starts failing? etc...
: >>
: >>
: >>
: >> -Hoss
: >> http://www.lucidworks.com/
: >>
: >> ---------------------------------------------------------------------
: >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org<ma...@lucene.apache.org>
: >> For additional commands, e-mail: dev-help@lucene.apache.org<ma...@lucene.apache.org>
: >>
: >>
: >
:
-Hoss
http://www.lucidworks.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org<ma...@lucene.apache.org>
For additional commands, e-mail: dev-help@lucene.apache.org<ma...@lucene.apache.org>
Re: TestSSLRandomization is failing everytime
Posted by Joel Bernstein <jo...@gmail.com>.
Nice little program. I got the same result, so the digests are not the
issue:
SHA == SHA
SHA == SHA-1
SHA != SHA-256
SHA != SHA-512
SHA-1 == SHA
SHA-1 == SHA-1
SHA-1 != SHA-256
SHA-1 != SHA-512
SHA-256 != SHA
SHA-256 != SHA-1
SHA-256 == SHA-256
SHA-256 != SHA-512
SHA-512 != SHA
SHA-512 != SHA-1
SHA-512 != SHA-256
SHA-512 == SHA-512
So, I got to thinking if it's not the digest, then the password must be the
problem. So I checked my env and what do you know:
SOLR_SSL_KEY_STORE_PASSWORD=joelbern
I unset this and the test passes. So the issue is if you are using the
environment to test various SSL parameters on a live system, it leaks over
to the test cases. Perhaps we should stamp this out.
Joel Bernstein
http://joelsolr.blogspot.com/
On Wed, Apr 4, 2018 at 7:10 PM, Chris Hostetter <ho...@fucit.org>
wrote:
>
> : I have not been able to get the actual SHA implementation (SHA-1,
> : SHA-256...) from the MessageDigest instance. If we could get that, I
> : suspect it would be different on my machine then yours.
>
> How about this...
>
> import java.security.MessageDigest;
> public final class Temp {
> public static void main(String[] args) throws Exception {
> final byte[] INPUT = "How now brown Cow?".getBytes("UTF-8");
> final String[] ALGOS = new String[]{"SHA", "SHA-1", "SHA-256",
> "SHA-512"};
> final byte[][] OUTPUT = new byte[ALGOS.length][];
> for (int a = 0; a < ALGOS.length; a++) {
> final MessageDigest d = MessageDigest.getInstance(ALGOS[a]);
> d.update(INPUT);
> OUTPUT[a] = d.digest();
> }
> for (int x = 0; x < ALGOS.length; x++) {
> for (int y = 0; y < ALGOS.length; y++) {
> System.out.println(ALGOS[x] +
> (MessageDigest.isEqual(OUTPUT[x], OUTPUT[y]) ?
> " == " : " != ") +
> ALGOS[y]);
> }
> }
> }
> }
>
>
> hossman@tray:~/tmp$ javac Temp.java
> hossman@tray:~/tmp$ java -ea Temp
> SHA == SHA
> SHA == SHA-1
> SHA != SHA-256
> SHA != SHA-512
> SHA-1 == SHA
> SHA-1 == SHA-1
> SHA-1 != SHA-256
> SHA-1 != SHA-512
> SHA-256 != SHA
> SHA-256 != SHA-1
> SHA-256 == SHA-256
> SHA-256 != SHA-512
> SHA-512 != SHA
> SHA-512 != SHA-1
> SHA-512 != SHA-256
> SHA-512 == SHA-512
>
>
>
>
>
> -Hoss
> http://www.lucidworks.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: TestSSLRandomization is failing everytime
Posted by Chris Hostetter <ho...@fucit.org>.
: I have not been able to get the actual SHA implementation (SHA-1,
: SHA-256...) from the MessageDigest instance. If we could get that, I
: suspect it would be different on my machine then yours.
How about this...
import java.security.MessageDigest;
public final class Temp {
public static void main(String[] args) throws Exception {
final byte[] INPUT = "How now brown Cow?".getBytes("UTF-8");
final String[] ALGOS = new String[]{"SHA", "SHA-1", "SHA-256",
"SHA-512"};
final byte[][] OUTPUT = new byte[ALGOS.length][];
for (int a = 0; a < ALGOS.length; a++) {
final MessageDigest d = MessageDigest.getInstance(ALGOS[a]);
d.update(INPUT);
OUTPUT[a] = d.digest();
}
for (int x = 0; x < ALGOS.length; x++) {
for (int y = 0; y < ALGOS.length; y++) {
System.out.println(ALGOS[x] +
(MessageDigest.isEqual(OUTPUT[x], OUTPUT[y]) ? " == " : " != ") +
ALGOS[y]);
}
}
}
}
hossman@tray:~/tmp$ javac Temp.java
hossman@tray:~/tmp$ java -ea Temp
SHA == SHA
SHA == SHA-1
SHA != SHA-256
SHA != SHA-512
SHA-1 == SHA
SHA-1 == SHA-1
SHA-1 != SHA-256
SHA-1 != SHA-512
SHA-256 != SHA
SHA-256 != SHA-1
SHA-256 == SHA-256
SHA-256 != SHA-512
SHA-512 != SHA
SHA-512 != SHA-1
SHA-512 != SHA-256
SHA-512 == SHA-512
-Hoss
http://www.lucidworks.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: TestSSLRandomization is failing everytime
Posted by Joel Bernstein <jo...@gmail.com>.
I get the same result:
SHA Message Digest from SUN, <initialized>
SHA
SUN version 1.8
I have not been able to get the actual SHA implementation (SHA-1,
SHA-256...) from the MessageDigest instance. If we could get that, I
suspect it would be different on my machine then yours.
Joel Bernstein
http://joelsolr.blogspot.com/
On Wed, Apr 4, 2018 at 6:36 PM, Chris Hostetter <ho...@fucit.org>
wrote:
> : http://grepcode.com/file_/repository.grepcode.com/java/
> root/jdk/openjdk/8u40-b25/sun/security/provider/
> JavaKeyStore.java/?v=source
> :
> : The getPreKeyedHash method is where MessageDigest.getInstance("SHA") is
> : called. From everything I've read this code is incorrect because SHA is
> not
> : a valid algorithm.
>
> Interesting... what exactly does your JVM produce if you run this code...
>
> public static void main(String[] args) throws Exception {
> java.security.MessageDigest x = java.security.MessageDigest.
> getInstance("SHA");
> System.out.println(x.toString());
> System.out.println(x.getAlgorithm());
> System.out.println(x.getProvider().toString());
> }
>
> On my system i get...
>
> ---
> hossman@tray:~/tmp$ java -ea Temp
> SHA Message Digest from SUN, <initialized>
>
> SHA
> SUN version 1.8
> ---
>
> It perplexes me that in the javadocs for MessageDigest the sample usage
> code shows 'MessageDigest.getInstance("SHA");' as the very first line, but
> then lower down it says...
>
> >> Every implementation of the Java platform is required to support the
> >> following standard MessageDigest algorithms:
> >>
> >> * MD5
> >> * SHA-1
> >> * SHA-256
>
> ...and the linked to "Java Cryptography Architecture Standard Algorithm
> Name Documentation" doesn't mention "SHA" but does mention the various
> "SHA-n" specific impls...
>
> https://docs.oracle.com/javase/8/docs/api/java/security/MessageDigest.html
> https://docs.oracle.com/javase/8/docs/technotes/
> guides/security/StandardNames.html#MessageDigest
>
>
>
> -Hoss
> http://www.lucidworks.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: TestSSLRandomization is failing everytime
Posted by Chris Hostetter <ho...@fucit.org>.
: http://grepcode.com/file_/repository.grepcode.com/java/root/jdk/openjdk/8u40-b25/sun/security/provider/JavaKeyStore.java/?v=source
:
: The getPreKeyedHash method is where MessageDigest.getInstance("SHA") is
: called. From everything I've read this code is incorrect because SHA is not
: a valid algorithm.
Interesting... what exactly does your JVM produce if you run this code...
public static void main(String[] args) throws Exception {
java.security.MessageDigest x = java.security.MessageDigest.getInstance("SHA");
System.out.println(x.toString());
System.out.println(x.getAlgorithm());
System.out.println(x.getProvider().toString());
}
On my system i get...
---
hossman@tray:~/tmp$ java -ea Temp
SHA Message Digest from SUN, <initialized>
SHA
SUN version 1.8
---
It perplexes me that in the javadocs for MessageDigest the sample usage
code shows 'MessageDigest.getInstance("SHA");' as the very first line, but
then lower down it says...
>> Every implementation of the Java platform is required to support the
>> following standard MessageDigest algorithms:
>>
>> * MD5
>> * SHA-1
>> * SHA-256
...and the linked to "Java Cryptography Architecture Standard Algorithm
Name Documentation" doesn't mention "SHA" but does mention the various
"SHA-n" specific impls...
https://docs.oracle.com/javase/8/docs/api/java/security/MessageDigest.html
https://docs.oracle.com/javase/8/docs/technotes/guides/security/StandardNames.html#MessageDigest
-Hoss
http://www.lucidworks.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: TestSSLRandomization is failing everytime
Posted by Joel Bernstein <jo...@gmail.com>.
This looks a bug in JavaKeyStore. The source code is here:
http://grepcode.com/file_/repository.grepcode.com/java/root/jdk/openjdk/8u40-b25/sun/security/provider/JavaKeyStore.java/?v=source
The getPreKeyedHash method is where MessageDigest.getInstance("SHA") is
called. From everything I've read this code is incorrect because SHA is not
a valid algorithm.
Joel Bernstein
http://joelsolr.blogspot.com/
On Wed, Apr 4, 2018 at 3:35 PM, Joel Bernstein <jo...@gmail.com> wrote:
> The code below ran fine from the command line and from a basic test case:
> System.out.println(javax.net.ssl.SSLContext.getDefault().getProvider());
> System.out.println(javax.net.ssl.SSLContext.getDefault().getProtocol());
>
> The source code that throws the exception in JavaKeyStore.engineLoad is:
>
> if (password != null) {
> byte computed[], actual[];
> computed = md.digest();
> actual = new byte[computed.length];
> dis.readFully(actual);
> for (int i = 0; i < computed.length; i++) {
> if (computed[i] != actual[i]) {
> Throwable t = new UnrecoverableKeyException
> ("Password verification failed");
> throw (IOException)new IOException
> ("Keystore was tampered with, or "
> + "password was incorrect").initCause(t);
> }
> }
> }
>
>
> Notice that it's simply comparing the bytes from two digests.
>
> The digests are prepared using a SHA digest, notice that it just specifies
> SHA, which must choose the default SHA digest for the system it's on. If
> it choses a different SHA digest the password would not match. My best bet
> right now is that I've changed my default SHA digest to be something
> other then what was used to create the passwords for the test framework:
>
>
> /**
> * To guard against tampering with the keystore, we append a keyed
> * hash with a bit of whitener.
> */
> private MessageDigest getPreKeyedHash(char[] password)
> throws NoSuchAlgorithmException, UnsupportedEncodingException
> {
> int i, j;
>
> MessageDigest md = MessageDigest.getInstance("SHA");
> byte[] passwdBytes = new byte[password.length * 2];
> for (i=0, j=0; i<password.length; i++) {
> passwdBytes[j++] = (byte)(password[i] >> 8);
> passwdBytes[j++] = (byte)password[i];
> }
> md.update(passwdBytes);
> for (i=0; i<passwdBytes.length; i++)
> passwdBytes[i] = 0;
> md.update("Mighty Aphrodite".getBytes("UTF8"));
> return md;
> }
>
>
>
>
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Wed, Apr 4, 2018 at 2:11 PM, Joel Bernstein <jo...@gmail.com> wrote:
>
>> Thanks Hoss, I will give this a try.
>>
>> Joel Bernstein
>> http://joelsolr.blogspot.com/
>>
>> On Wed, Apr 4, 2018 at 1:42 PM, Chris Hostetter <hossman_lucene@fucit.org
>> > wrote:
>>
>>>
>>> : I suspect I hosed something to do with my root certs on my local
>>> machine.
>>> : Fairly recently I was playing around with these certs while doing some
>>> SSL
>>> : work for Alfresco. This should be fun to fix...
>>>
>>> if that's your suspicion, i would start by testing out a simple java app
>>> that does nothing but...
>>>
>>> public static void main(String[] args) throws Exception {
>>> System.out.println(javax.net.ssl.SSLContext.getDefault().get
>>> Provider());
>>> System.out.println(javax.net.ssl.SSLContext.getDefault().get
>>> Protocol());
>>> }
>>>
>>> ...if thta fails on the commandline, then ou definitely hozed your
>>> machine.
>>>
>>> If that *does* work on the commandline, then try the same code in a
>>> trivial Junit test that just subclasses LuceneTestCase -- NOT
>>> SolrTestCaseJ4 -- to see if the problem is somewhere in our ant/lucene
>>> build setup, independent of any Solr SSL randomization.
>>>
>>> :
>>> : Joel Bernstein
>>> : http://joelsolr.blogspot.com/
>>> :
>>> : On Wed, Apr 4, 2018 at 12:29 PM, Joel Bernstein <jo...@gmail.com>
>>> wrote:
>>> :
>>> : > Ok, so it does sounds like a local problem then. Nothing much has
>>> changed
>>> : > locally. I'm still using the same Mac and Java version:
>>> : >
>>> : > defaultuildsMBP:clone2 joelbernstein$ java -version
>>> : >
>>> : > java version "1.8.0_40"
>>> : >
>>> : > Java(TM) SE Runtime Environment (build 1.8.0_40-b27)
>>> : >
>>> : > Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)
>>> : >
>>> : > I'll try running on a newer version of Java.
>>> : >
>>> : >
>>> : >
>>> : > Joel Bernstein
>>> : > http://joelsolr.blogspot.com/
>>> : >
>>> : > On Wed, Apr 4, 2018 at 12:19 PM, Chris Hostetter <
>>> hossman_lucene@fucit.org
>>> : > > wrote:
>>> : >
>>> : >>
>>> : >> : Subject: Re: TestSSLRandomization is failing everytime
>>> : >>
>>> : >> : When I run locally I get this stack trace:
>>> : >>
>>> : >> would be helpful to konw the branch, and the GIT SHA ... and if you
>>> can
>>> : >> reproduce if you checkout an older branch/SHA where you know you
>>> didn't
>>> : >> see this failure in the past (ex: the last SHA you committed, where
>>> you
>>> : >> should have run all tests to be certain you didn't break anything)
>>> : >>
>>> : >> Personally I can't reproduce on master/8e276b90f520d ...
>>> : >>
>>> : >> Let's look at the exception...
>>> : >>
>>> : >> : [junit4] > Caused by: java.lang.RuntimeException: Unable to
>>> : >> : initialize 'Default' SSLContext Algorithm, JVM is borked
>>> : >> :
>>> : >> : [junit4] > at
>>> : >> : org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.<clinit>(T
>>> : >> estMiniSolrCloudClusterSSL.java:67)
>>> : >> :
>>> : >> : [junit4] > ... 40 more
>>> : >> :
>>> : >> : [junit4] > Caused by: java.security.NoSuchAlgorithmE
>>> xception:
>>> : >> Error
>>> : >> : constructing implementation (algorithm: Default, provider:
>>> SunJSSE,
>>> : >> class:
>>> : >> : sun.security.ssl.SSLContextImpl$DefaultSSLContext)
>>> : >>
>>> : >> At first glance, it sounds like your JVM is completley jacked and
>>> doesn't
>>> : >> have any SSL support?
>>> : >>
>>> : >> The code throwing that exception is litterally...
>>> : >>
>>> : >> DEFAULT_SSL_CONTEXT = SSLContext.getDefault();
>>> : >>
>>> : >> ...ie: your JVM is saying the *default* SSL Algorithm, as choosen
>>> by the
>>> : >> JVM config, doens't exist ... but if we look farther down...
>>> : >>
>>> : >> : [junit4] > Caused by: java.io.IOException: Keystore was
>>> tampered
>>> : >> : with, or password was incorrect
>>> : >> ...
>>> : >> : [junit4] > Caused by: java.security.UnrecoverableKey
>>> Exception:
>>> : >> : Password verification failed
>>> : >>
>>> : >> ...well that's interesting. We do provide our own keystore when
>>> using
>>> : >> SSLTestConfig, but I honestly don't know off the top of my head if
>>> that's
>>> : >> even in use when this code is running?
>>> : >>
>>> : >> Can you tell us *ANYTHING* about the machine/jvm where you are
>>> running
>>> : >> this, and or what might have changed on your end since hte last
>>> time you
>>> : >> ran all tests w/o failure? what OS? new laptop? new java install?
>>> if you
>>> : >> "git co releases/lucene-solr/7.2.0" does this test pass? if so can
>>> you
>>> : >> "git bisect" to track down when it starts failing? etc...
>>> : >>
>>> : >>
>>> : >>
>>> : >> -Hoss
>>> : >> http://www.lucidworks.com/
>>> : >>
>>> : >> ------------------------------------------------------------
>>> ---------
>>> : >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> : >> For additional commands, e-mail: dev-help@lucene.apache.org
>>> : >>
>>> : >>
>>> : >
>>> :
>>>
>>> -Hoss
>>> http://www.lucidworks.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>>
>>
>
Re: TestSSLRandomization is failing everytime
Posted by Joel Bernstein <jo...@gmail.com>.
The code below ran fine from the command line and from a basic test case:
System.out.println(javax.net.ssl.SSLContext.getDefault().getProvider());
System.out.println(javax.net.ssl.SSLContext.getDefault().getProtocol());
The source code that throws the exception in JavaKeyStore.engineLoad is:
if (password != null) {
byte computed[], actual[];
computed = md.digest();
actual = new byte[computed.length];
dis.readFully(actual);
for (int i = 0; i < computed.length; i++) {
if (computed[i] != actual[i]) {
Throwable t = new UnrecoverableKeyException
("Password verification failed");
throw (IOException)new IOException
("Keystore was tampered with, or "
+ "password was incorrect").initCause(t);
}
}
}
Notice that it's simply comparing the bytes from two digests.
The digests are prepared using a SHA digest, notice that it just specifies
SHA, which must choose the default SHA digest for the system it's on. If
it choses a different SHA digest the password would not match. My best bet
right now is that I've changed my default SHA digest to be something
other then what was used to create the passwords for the test framework:
/**
* To guard against tampering with the keystore, we append a keyed
* hash with a bit of whitener.
*/
private MessageDigest getPreKeyedHash(char[] password)
throws NoSuchAlgorithmException, UnsupportedEncodingException
{
int i, j;
MessageDigest md = MessageDigest.getInstance("SHA");
byte[] passwdBytes = new byte[password.length * 2];
for (i=0, j=0; i<password.length; i++) {
passwdBytes[j++] = (byte)(password[i] >> 8);
passwdBytes[j++] = (byte)password[i];
}
md.update(passwdBytes);
for (i=0; i<passwdBytes.length; i++)
passwdBytes[i] = 0;
md.update("Mighty Aphrodite".getBytes("UTF8"));
return md;
}
Joel Bernstein
http://joelsolr.blogspot.com/
On Wed, Apr 4, 2018 at 2:11 PM, Joel Bernstein <jo...@gmail.com> wrote:
> Thanks Hoss, I will give this a try.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Wed, Apr 4, 2018 at 1:42 PM, Chris Hostetter <ho...@fucit.org>
> wrote:
>
>>
>> : I suspect I hosed something to do with my root certs on my local
>> machine.
>> : Fairly recently I was playing around with these certs while doing some
>> SSL
>> : work for Alfresco. This should be fun to fix...
>>
>> if that's your suspicion, i would start by testing out a simple java app
>> that does nothing but...
>>
>> public static void main(String[] args) throws Exception {
>> System.out.println(javax.net.ssl.SSLContext.getDefault().get
>> Provider());
>> System.out.println(javax.net.ssl.SSLContext.getDefault().get
>> Protocol());
>> }
>>
>> ...if thta fails on the commandline, then ou definitely hozed your
>> machine.
>>
>> If that *does* work on the commandline, then try the same code in a
>> trivial Junit test that just subclasses LuceneTestCase -- NOT
>> SolrTestCaseJ4 -- to see if the problem is somewhere in our ant/lucene
>> build setup, independent of any Solr SSL randomization.
>>
>> :
>> : Joel Bernstein
>> : http://joelsolr.blogspot.com/
>> :
>> : On Wed, Apr 4, 2018 at 12:29 PM, Joel Bernstein <jo...@gmail.com>
>> wrote:
>> :
>> : > Ok, so it does sounds like a local problem then. Nothing much has
>> changed
>> : > locally. I'm still using the same Mac and Java version:
>> : >
>> : > defaultuildsMBP:clone2 joelbernstein$ java -version
>> : >
>> : > java version "1.8.0_40"
>> : >
>> : > Java(TM) SE Runtime Environment (build 1.8.0_40-b27)
>> : >
>> : > Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)
>> : >
>> : > I'll try running on a newer version of Java.
>> : >
>> : >
>> : >
>> : > Joel Bernstein
>> : > http://joelsolr.blogspot.com/
>> : >
>> : > On Wed, Apr 4, 2018 at 12:19 PM, Chris Hostetter <
>> hossman_lucene@fucit.org
>> : > > wrote:
>> : >
>> : >>
>> : >> : Subject: Re: TestSSLRandomization is failing everytime
>> : >>
>> : >> : When I run locally I get this stack trace:
>> : >>
>> : >> would be helpful to konw the branch, and the GIT SHA ... and if you
>> can
>> : >> reproduce if you checkout an older branch/SHA where you know you
>> didn't
>> : >> see this failure in the past (ex: the last SHA you committed, where
>> you
>> : >> should have run all tests to be certain you didn't break anything)
>> : >>
>> : >> Personally I can't reproduce on master/8e276b90f520d ...
>> : >>
>> : >> Let's look at the exception...
>> : >>
>> : >> : [junit4] > Caused by: java.lang.RuntimeException: Unable to
>> : >> : initialize 'Default' SSLContext Algorithm, JVM is borked
>> : >> :
>> : >> : [junit4] > at
>> : >> : org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.<clinit>(T
>> : >> estMiniSolrCloudClusterSSL.java:67)
>> : >> :
>> : >> : [junit4] > ... 40 more
>> : >> :
>> : >> : [junit4] > Caused by: java.security.NoSuchAlgorithmE
>> xception:
>> : >> Error
>> : >> : constructing implementation (algorithm: Default, provider: SunJSSE,
>> : >> class:
>> : >> : sun.security.ssl.SSLContextImpl$DefaultSSLContext)
>> : >>
>> : >> At first glance, it sounds like your JVM is completley jacked and
>> doesn't
>> : >> have any SSL support?
>> : >>
>> : >> The code throwing that exception is litterally...
>> : >>
>> : >> DEFAULT_SSL_CONTEXT = SSLContext.getDefault();
>> : >>
>> : >> ...ie: your JVM is saying the *default* SSL Algorithm, as choosen by
>> the
>> : >> JVM config, doens't exist ... but if we look farther down...
>> : >>
>> : >> : [junit4] > Caused by: java.io.IOException: Keystore was
>> tampered
>> : >> : with, or password was incorrect
>> : >> ...
>> : >> : [junit4] > Caused by: java.security.UnrecoverableKey
>> Exception:
>> : >> : Password verification failed
>> : >>
>> : >> ...well that's interesting. We do provide our own keystore when
>> using
>> : >> SSLTestConfig, but I honestly don't know off the top of my head if
>> that's
>> : >> even in use when this code is running?
>> : >>
>> : >> Can you tell us *ANYTHING* about the machine/jvm where you are
>> running
>> : >> this, and or what might have changed on your end since hte last time
>> you
>> : >> ran all tests w/o failure? what OS? new laptop? new java install?
>> if you
>> : >> "git co releases/lucene-solr/7.2.0" does this test pass? if so can
>> you
>> : >> "git bisect" to track down when it starts failing? etc...
>> : >>
>> : >>
>> : >>
>> : >> -Hoss
>> : >> http://www.lucidworks.com/
>> : >>
>> : >> ------------------------------------------------------------
>> ---------
>> : >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> : >> For additional commands, e-mail: dev-help@lucene.apache.org
>> : >>
>> : >>
>> : >
>> :
>>
>> -Hoss
>> http://www.lucidworks.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>
Re: TestSSLRandomization is failing everytime
Posted by Joel Bernstein <jo...@gmail.com>.
Thanks Hoss, I will give this a try.
Joel Bernstein
http://joelsolr.blogspot.com/
On Wed, Apr 4, 2018 at 1:42 PM, Chris Hostetter <ho...@fucit.org>
wrote:
>
> : I suspect I hosed something to do with my root certs on my local machine.
> : Fairly recently I was playing around with these certs while doing some
> SSL
> : work for Alfresco. This should be fun to fix...
>
> if that's your suspicion, i would start by testing out a simple java app
> that does nothing but...
>
> public static void main(String[] args) throws Exception {
> System.out.println(javax.net.ssl.SSLContext.getDefault().
> getProvider());
> System.out.println(javax.net.ssl.SSLContext.getDefault().
> getProtocol());
> }
>
> ...if thta fails on the commandline, then ou definitely hozed your
> machine.
>
> If that *does* work on the commandline, then try the same code in a
> trivial Junit test that just subclasses LuceneTestCase -- NOT
> SolrTestCaseJ4 -- to see if the problem is somewhere in our ant/lucene
> build setup, independent of any Solr SSL randomization.
>
> :
> : Joel Bernstein
> : http://joelsolr.blogspot.com/
> :
> : On Wed, Apr 4, 2018 at 12:29 PM, Joel Bernstein <jo...@gmail.com>
> wrote:
> :
> : > Ok, so it does sounds like a local problem then. Nothing much has
> changed
> : > locally. I'm still using the same Mac and Java version:
> : >
> : > defaultuildsMBP:clone2 joelbernstein$ java -version
> : >
> : > java version "1.8.0_40"
> : >
> : > Java(TM) SE Runtime Environment (build 1.8.0_40-b27)
> : >
> : > Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)
> : >
> : > I'll try running on a newer version of Java.
> : >
> : >
> : >
> : > Joel Bernstein
> : > http://joelsolr.blogspot.com/
> : >
> : > On Wed, Apr 4, 2018 at 12:19 PM, Chris Hostetter <
> hossman_lucene@fucit.org
> : > > wrote:
> : >
> : >>
> : >> : Subject: Re: TestSSLRandomization is failing everytime
> : >>
> : >> : When I run locally I get this stack trace:
> : >>
> : >> would be helpful to konw the branch, and the GIT SHA ... and if you
> can
> : >> reproduce if you checkout an older branch/SHA where you know you
> didn't
> : >> see this failure in the past (ex: the last SHA you committed, where
> you
> : >> should have run all tests to be certain you didn't break anything)
> : >>
> : >> Personally I can't reproduce on master/8e276b90f520d ...
> : >>
> : >> Let's look at the exception...
> : >>
> : >> : [junit4] > Caused by: java.lang.RuntimeException: Unable to
> : >> : initialize 'Default' SSLContext Algorithm, JVM is borked
> : >> :
> : >> : [junit4] > at
> : >> : org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.<clinit>(T
> : >> estMiniSolrCloudClusterSSL.java:67)
> : >> :
> : >> : [junit4] > ... 40 more
> : >> :
> : >> : [junit4] > Caused by: java.security.NoSuchAlgorithmException:
> : >> Error
> : >> : constructing implementation (algorithm: Default, provider: SunJSSE,
> : >> class:
> : >> : sun.security.ssl.SSLContextImpl$DefaultSSLContext)
> : >>
> : >> At first glance, it sounds like your JVM is completley jacked and
> doesn't
> : >> have any SSL support?
> : >>
> : >> The code throwing that exception is litterally...
> : >>
> : >> DEFAULT_SSL_CONTEXT = SSLContext.getDefault();
> : >>
> : >> ...ie: your JVM is saying the *default* SSL Algorithm, as choosen by
> the
> : >> JVM config, doens't exist ... but if we look farther down...
> : >>
> : >> : [junit4] > Caused by: java.io.IOException: Keystore was
> tampered
> : >> : with, or password was incorrect
> : >> ...
> : >> : [junit4] > Caused by: java.security.
> UnrecoverableKeyException:
> : >> : Password verification failed
> : >>
> : >> ...well that's interesting. We do provide our own keystore when using
> : >> SSLTestConfig, but I honestly don't know off the top of my head if
> that's
> : >> even in use when this code is running?
> : >>
> : >> Can you tell us *ANYTHING* about the machine/jvm where you are running
> : >> this, and or what might have changed on your end since hte last time
> you
> : >> ran all tests w/o failure? what OS? new laptop? new java install? if
> you
> : >> "git co releases/lucene-solr/7.2.0" does this test pass? if so can you
> : >> "git bisect" to track down when it starts failing? etc...
> : >>
> : >>
> : >>
> : >> -Hoss
> : >> http://www.lucidworks.com/
> : >>
> : >> ---------------------------------------------------------------------
> : >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> : >> For additional commands, e-mail: dev-help@lucene.apache.org
> : >>
> : >>
> : >
> :
>
> -Hoss
> http://www.lucidworks.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: TestSSLRandomization is failing everytime
Posted by Chris Hostetter <ho...@fucit.org>.
: I suspect I hosed something to do with my root certs on my local machine.
: Fairly recently I was playing around with these certs while doing some SSL
: work for Alfresco. This should be fun to fix...
if that's your suspicion, i would start by testing out a simple java app
that does nothing but...
public static void main(String[] args) throws Exception {
System.out.println(javax.net.ssl.SSLContext.getDefault().getProvider());
System.out.println(javax.net.ssl.SSLContext.getDefault().getProtocol());
}
...if thta fails on the commandline, then ou definitely hozed your
machine.
If that *does* work on the commandline, then try the same code in a
trivial Junit test that just subclasses LuceneTestCase -- NOT
SolrTestCaseJ4 -- to see if the problem is somewhere in our ant/lucene
build setup, independent of any Solr SSL randomization.
:
: Joel Bernstein
: http://joelsolr.blogspot.com/
:
: On Wed, Apr 4, 2018 at 12:29 PM, Joel Bernstein <jo...@gmail.com> wrote:
:
: > Ok, so it does sounds like a local problem then. Nothing much has changed
: > locally. I'm still using the same Mac and Java version:
: >
: > defaultuildsMBP:clone2 joelbernstein$ java -version
: >
: > java version "1.8.0_40"
: >
: > Java(TM) SE Runtime Environment (build 1.8.0_40-b27)
: >
: > Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)
: >
: > I'll try running on a newer version of Java.
: >
: >
: >
: > Joel Bernstein
: > http://joelsolr.blogspot.com/
: >
: > On Wed, Apr 4, 2018 at 12:19 PM, Chris Hostetter <hossman_lucene@fucit.org
: > > wrote:
: >
: >>
: >> : Subject: Re: TestSSLRandomization is failing everytime
: >>
: >> : When I run locally I get this stack trace:
: >>
: >> would be helpful to konw the branch, and the GIT SHA ... and if you can
: >> reproduce if you checkout an older branch/SHA where you know you didn't
: >> see this failure in the past (ex: the last SHA you committed, where you
: >> should have run all tests to be certain you didn't break anything)
: >>
: >> Personally I can't reproduce on master/8e276b90f520d ...
: >>
: >> Let's look at the exception...
: >>
: >> : [junit4] > Caused by: java.lang.RuntimeException: Unable to
: >> : initialize 'Default' SSLContext Algorithm, JVM is borked
: >> :
: >> : [junit4] > at
: >> : org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.<clinit>(T
: >> estMiniSolrCloudClusterSSL.java:67)
: >> :
: >> : [junit4] > ... 40 more
: >> :
: >> : [junit4] > Caused by: java.security.NoSuchAlgorithmException:
: >> Error
: >> : constructing implementation (algorithm: Default, provider: SunJSSE,
: >> class:
: >> : sun.security.ssl.SSLContextImpl$DefaultSSLContext)
: >>
: >> At first glance, it sounds like your JVM is completley jacked and doesn't
: >> have any SSL support?
: >>
: >> The code throwing that exception is litterally...
: >>
: >> DEFAULT_SSL_CONTEXT = SSLContext.getDefault();
: >>
: >> ...ie: your JVM is saying the *default* SSL Algorithm, as choosen by the
: >> JVM config, doens't exist ... but if we look farther down...
: >>
: >> : [junit4] > Caused by: java.io.IOException: Keystore was tampered
: >> : with, or password was incorrect
: >> ...
: >> : [junit4] > Caused by: java.security.UnrecoverableKeyException:
: >> : Password verification failed
: >>
: >> ...well that's interesting. We do provide our own keystore when using
: >> SSLTestConfig, but I honestly don't know off the top of my head if that's
: >> even in use when this code is running?
: >>
: >> Can you tell us *ANYTHING* about the machine/jvm where you are running
: >> this, and or what might have changed on your end since hte last time you
: >> ran all tests w/o failure? what OS? new laptop? new java install? if you
: >> "git co releases/lucene-solr/7.2.0" does this test pass? if so can you
: >> "git bisect" to track down when it starts failing? etc...
: >>
: >>
: >>
: >> -Hoss
: >> http://www.lucidworks.com/
: >>
: >> ---------------------------------------------------------------------
: >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: >> For additional commands, e-mail: dev-help@lucene.apache.org
: >>
: >>
: >
:
-Hoss
http://www.lucidworks.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: TestSSLRandomization is failing everytime
Posted by Joel Bernstein <jo...@gmail.com>.
I suspect I hosed something to do with my root certs on my local machine.
Fairly recently I was playing around with these certs while doing some SSL
work for Alfresco. This should be fun to fix...
Joel Bernstein
http://joelsolr.blogspot.com/
On Wed, Apr 4, 2018 at 12:29 PM, Joel Bernstein <jo...@gmail.com> wrote:
> Ok, so it does sounds like a local problem then. Nothing much has changed
> locally. I'm still using the same Mac and Java version:
>
> defaultuildsMBP:clone2 joelbernstein$ java -version
>
> java version "1.8.0_40"
>
> Java(TM) SE Runtime Environment (build 1.8.0_40-b27)
>
> Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)
>
> I'll try running on a newer version of Java.
>
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Wed, Apr 4, 2018 at 12:19 PM, Chris Hostetter <hossman_lucene@fucit.org
> > wrote:
>
>>
>> : Subject: Re: TestSSLRandomization is failing everytime
>>
>> : When I run locally I get this stack trace:
>>
>> would be helpful to konw the branch, and the GIT SHA ... and if you can
>> reproduce if you checkout an older branch/SHA where you know you didn't
>> see this failure in the past (ex: the last SHA you committed, where you
>> should have run all tests to be certain you didn't break anything)
>>
>> Personally I can't reproduce on master/8e276b90f520d ...
>>
>> Let's look at the exception...
>>
>> : [junit4] > Caused by: java.lang.RuntimeException: Unable to
>> : initialize 'Default' SSLContext Algorithm, JVM is borked
>> :
>> : [junit4] > at
>> : org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.<clinit>(T
>> estMiniSolrCloudClusterSSL.java:67)
>> :
>> : [junit4] > ... 40 more
>> :
>> : [junit4] > Caused by: java.security.NoSuchAlgorithmException:
>> Error
>> : constructing implementation (algorithm: Default, provider: SunJSSE,
>> class:
>> : sun.security.ssl.SSLContextImpl$DefaultSSLContext)
>>
>> At first glance, it sounds like your JVM is completley jacked and doesn't
>> have any SSL support?
>>
>> The code throwing that exception is litterally...
>>
>> DEFAULT_SSL_CONTEXT = SSLContext.getDefault();
>>
>> ...ie: your JVM is saying the *default* SSL Algorithm, as choosen by the
>> JVM config, doens't exist ... but if we look farther down...
>>
>> : [junit4] > Caused by: java.io.IOException: Keystore was tampered
>> : with, or password was incorrect
>> ...
>> : [junit4] > Caused by: java.security.UnrecoverableKeyException:
>> : Password verification failed
>>
>> ...well that's interesting. We do provide our own keystore when using
>> SSLTestConfig, but I honestly don't know off the top of my head if that's
>> even in use when this code is running?
>>
>> Can you tell us *ANYTHING* about the machine/jvm where you are running
>> this, and or what might have changed on your end since hte last time you
>> ran all tests w/o failure? what OS? new laptop? new java install? if you
>> "git co releases/lucene-solr/7.2.0" does this test pass? if so can you
>> "git bisect" to track down when it starts failing? etc...
>>
>>
>>
>> -Hoss
>> http://www.lucidworks.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>
Re: TestSSLRandomization is failing everytime
Posted by Joel Bernstein <jo...@gmail.com>.
Ok, so it does sounds like a local problem then. Nothing much has changed
locally. I'm still using the same Mac and Java version:
defaultuildsMBP:clone2 joelbernstein$ java -version
java version "1.8.0_40"
Java(TM) SE Runtime Environment (build 1.8.0_40-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)
I'll try running on a newer version of Java.
Joel Bernstein
http://joelsolr.blogspot.com/
On Wed, Apr 4, 2018 at 12:19 PM, Chris Hostetter <ho...@fucit.org>
wrote:
>
> : Subject: Re: TestSSLRandomization is failing everytime
>
> : When I run locally I get this stack trace:
>
> would be helpful to konw the branch, and the GIT SHA ... and if you can
> reproduce if you checkout an older branch/SHA where you know you didn't
> see this failure in the past (ex: the last SHA you committed, where you
> should have run all tests to be certain you didn't break anything)
>
> Personally I can't reproduce on master/8e276b90f520d ...
>
> Let's look at the exception...
>
> : [junit4] > Caused by: java.lang.RuntimeException: Unable to
> : initialize 'Default' SSLContext Algorithm, JVM is borked
> :
> : [junit4] > at
> : org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.<clinit>(
> TestMiniSolrCloudClusterSSL.java:67)
> :
> : [junit4] > ... 40 more
> :
> : [junit4] > Caused by: java.security.NoSuchAlgorithmException:
> Error
> : constructing implementation (algorithm: Default, provider: SunJSSE,
> class:
> : sun.security.ssl.SSLContextImpl$DefaultSSLContext)
>
> At first glance, it sounds like your JVM is completley jacked and doesn't
> have any SSL support?
>
> The code throwing that exception is litterally...
>
> DEFAULT_SSL_CONTEXT = SSLContext.getDefault();
>
> ...ie: your JVM is saying the *default* SSL Algorithm, as choosen by the
> JVM config, doens't exist ... but if we look farther down...
>
> : [junit4] > Caused by: java.io.IOException: Keystore was tampered
> : with, or password was incorrect
> ...
> : [junit4] > Caused by: java.security.UnrecoverableKeyException:
> : Password verification failed
>
> ...well that's interesting. We do provide our own keystore when using
> SSLTestConfig, but I honestly don't know off the top of my head if that's
> even in use when this code is running?
>
> Can you tell us *ANYTHING* about the machine/jvm where you are running
> this, and or what might have changed on your end since hte last time you
> ran all tests w/o failure? what OS? new laptop? new java install? if you
> "git co releases/lucene-solr/7.2.0" does this test pass? if so can you
> "git bisect" to track down when it starts failing? etc...
>
>
>
> -Hoss
> http://www.lucidworks.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: TestSSLRandomization is failing everytime
Posted by Chris Hostetter <ho...@fucit.org>.
: Subject: Re: TestSSLRandomization is failing everytime
: When I run locally I get this stack trace:
would be helpful to konw the branch, and the GIT SHA ... and if you can
reproduce if you checkout an older branch/SHA where you know you didn't
see this failure in the past (ex: the last SHA you committed, where you
should have run all tests to be certain you didn't break anything)
Personally I can't reproduce on master/8e276b90f520d ...
Let's look at the exception...
: [junit4] > Caused by: java.lang.RuntimeException: Unable to
: initialize 'Default' SSLContext Algorithm, JVM is borked
:
: [junit4] > at
: org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.<clinit>(TestMiniSolrCloudClusterSSL.java:67)
:
: [junit4] > ... 40 more
:
: [junit4] > Caused by: java.security.NoSuchAlgorithmException: Error
: constructing implementation (algorithm: Default, provider: SunJSSE, class:
: sun.security.ssl.SSLContextImpl$DefaultSSLContext)
At first glance, it sounds like your JVM is completley jacked and doesn't
have any SSL support?
The code throwing that exception is litterally...
DEFAULT_SSL_CONTEXT = SSLContext.getDefault();
...ie: your JVM is saying the *default* SSL Algorithm, as choosen by the
JVM config, doens't exist ... but if we look farther down...
: [junit4] > Caused by: java.io.IOException: Keystore was tampered
: with, or password was incorrect
...
: [junit4] > Caused by: java.security.UnrecoverableKeyException:
: Password verification failed
...well that's interesting. We do provide our own keystore when using
SSLTestConfig, but I honestly don't know off the top of my head if that's
even in use when this code is running?
Can you tell us *ANYTHING* about the machine/jvm where you are running
this, and or what might have changed on your end since hte last time you
ran all tests w/o failure? what OS? new laptop? new java install? if you
"git co releases/lucene-solr/7.2.0" does this test pass? if so can you
"git bisect" to track down when it starts failing? etc...
-Hoss
http://www.lucidworks.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: TestSSLRandomization is failing everytime
Posted by Joel Bernstein <jo...@gmail.com>.
When I run locally I get this stack trace:
ERROR 0.02s | TestSSLRandomization.testRandomizedSslAndClientAuth <<<
[junit4] > Throwable #1: java.lang.ExceptionInInitializerError
[junit4] > at
__randomizedtesting.SeedInfo.seed([59B26B23CF90404E:D2E6DA446D0F0132]:0)
[junit4] > at
org.apache.solr.cloud.TestSSLRandomization.testRandomizedSslAndClientAuth(TestSSLRandomization.java:50)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > Caused by: java.lang.RuntimeException: Unable to
initialize 'Default' SSLContext Algorithm, JVM is borked
[junit4] > at
org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.<clinit>(TestMiniSolrCloudClusterSSL.java:67)
[junit4] > ... 40 more
[junit4] > Caused by: java.security.NoSuchAlgorithmException: Error
constructing implementation (algorithm: Default, provider: SunJSSE, class:
sun.security.ssl.SSLContextImpl$DefaultSSLContext)
[junit4] > at
java.security.Provider$Service.newInstance(Provider.java:1617)
[junit4] > at
sun.security.jca.GetInstance.getInstance(GetInstance.java:236)
[junit4] > at
sun.security.jca.GetInstance.getInstance(GetInstance.java:164)
[junit4] > at
javax.net.ssl.SSLContext.getInstance(SSLContext.java:156)
[junit4] > at javax.net.ssl.SSLContext.getDefault(SSLContext.java:96)
[junit4] > at
org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.<clinit>(TestMiniSolrCloudClusterSSL.java:64)
[junit4] > ... 40 more
[junit4] > Caused by: java.io.IOException: Keystore was tampered
with, or password was incorrect
[junit4] > at
sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:772)
[junit4] > at
sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:55)
[junit4] > at java.security.KeyStore.load(KeyStore.java:1445)
[junit4] > at
sun.security.ssl.TrustManagerFactoryImpl.getCacertsKeyStore(TrustManagerFactoryImpl.java:226)
[junit4] > at
sun.security.ssl.SSLContextImpl$DefaultSSLContext.getDefaultTrustManager(SSLContextImpl.java:767)
[junit4] > at
sun.security.ssl.SSLContextImpl$DefaultSSLContext.<init>(SSLContextImpl.java:733)
[junit4] > at
java.lang.reflect.Constructor.newInstance(Constructor.java:422)
[junit4] > at
java.security.Provider$Service.newInstance(Provider.java:1595)
[junit4] > ... 45 more
[junit4] > Caused by: java.security.UnrecoverableKeyException:
Password verification failed
[junit4] > at
sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:770)
[junit4] > ... 55 more
Joel Bernstein
http://joelsolr.blogspot.com/
On Wed, Apr 4, 2018 at 11:54 AM, Joel Bernstein <jo...@gmail.com> wrote:
> TestSSLRandomization is failing 100% of the time. Anybody make changes
> recently to this code?
>