You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Kelly Rusk <Ke...@rackspace.com> on 2018/06/01 20:01:54 UTC
Self Signed Certificate for Load Balancer and Solr Nodes
Hello all,
We are using self-signed certificates for our two servers in an HTTPS Master/Slave configuration running on Windows (please no discussions about the merits of Linux vs. Windows for Solr, it's a requirement).
We have solr1.com and solr2.com self-signed certs that correspond to the two servers. We also have a load balancer with an address named solrlb.com. When we hit the load balancer it gives us an SSL error, as it is passing us back to either solr1.com or solr2.com, but since these two Solr servers only have each other's self-signed cert installed in their Keystore, it doesn't resolve when it comes in through the load balanced address of solrlb.com.
We tried a san certificate that has all 3 addresses, but when we do this, we get the following error:
This page can't be displayed
Turn on TLS 1.0, TLS 1.1, and TLS 1.2 in Advanced settings and try connecting to https://b-win-solr-01.azure-dfa.com:8983 again. If this error persists, it is possible that this site uses an unsupported protocol or cipher suite such as RC4 (link for the details), which is not considered secure. Please contact your site administrator.
What is the correct method of using a self-signed certificate or certificates so that the pass thru Load Balancer address of solrlb.com that will either route to solr1 or solr2 works?
Regards,
Kelly
RE: Self Signed Certificate for Load Balancer and Solr Nodes
Posted by Kelly Rusk <Ke...@rackspace.com>.
Thank you Chris,
Per the sans, we had attempted that, but had already generated some certificates. I will see if we can back out of that with a fresh install using sans. I will give the first option a try, and appreciate the assistance.
Regards,
Kelly
-----Original Message-----
From: Christopher Schultz <ch...@christopherschultz.net>
Sent: Friday, June 1, 2018 5:59 PM
To: solr-user@lucene.apache.org
Subject: Re: Self Signed Certificate for Load Balancer and Solr Nodes
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Kelly,
On 6/1/18 5:41 PM, Kelly Rusk wrote:
> I can directly connect to either node without issue, it is only when
> the Load Balancer routes to either solr1 or solr2 that I get the
> security error (ex. https://solrlb.com:8983/solr). The Load Balancer
> is not managing HTTPS but just acting as a pure TCP proxy.
> Nothing more complex than sending traffic to either solr1 or solr2...
> however, the URL will be displayed as solrlb.com as it hides the real
> address of what is being routed to.
>
> In this case, do we need a certificate for solrlb.com installed on
> both solr1 and solr2?
That's exactly what you need. It would be best to:
1. Create a certificate for solrlb.com
2. Install the same key + certificate on both Solr nodes 3. Always use solrlb.com for any links and redirects you generate
Optionally, you could add SANs for that certificate for both solr1 and
solr2 just in case you want to be able to connect directly to either back-end node without getting hostname mismatch complaints.
> In our previous environments we used the same load balancer setup,
> but that worked since the Solr nodes were serving over http and not
> https.
You probably never noticed that redirects were occurring that were sending users to a particular node instead of always using the lb's hostname because there was never anything double-checking the hostname.
In your previous message, you mentioned that you got an error message including the hostname "b-win-solr-01.azure-dfa.com" which probably isn't your load-balancer's hostname. That suggests to me that some kind of redirect (or similar) is occurring and that the redirect doesn't understand that there is a reverse-proxy/lb out in front of the node.
Hope that helps,
- -chris
> -----Original Message----- From: Shawn Heisey <ap...@elyograg.org>
> Sent: Friday, June 1, 2018 5:25 PM To:
> solr-user@lucene.apache.org Subject: Re: Self Signed Certificate for
> Load Balancer and Solr Nodes
>
> On 6/1/2018 2:01 PM, Kelly Rusk wrote:
>> We have solr1.com and solr2.com self-signed certs that correspond to
>> the two servers. We also have a load balancer with an address named
>> solrlb.com. When we hit the load balancer it gives us an SSL error,
>> as it is passing us back to either solr1.com or solr2.com, but since
>> these two Solr servers only have each other's self-signed cert
>> installed in their Keystore, it doesn't resolve when it comes in
>> through the load balanced address of solrlb.com.
>>
>> We tried a san certificate that has all 3 addresses, but when we do
>> this, we get the following error:
>>
>> This page can't be displayed Turn on TLS 1.0, TLS 1.1, and TLS
>> 1.2 in Advanced settings and try connecting to
>> https://b-win-solr-01.azure-dfa.com:8983 again. If this error
>> persists, it is possible that this site uses an unsupported protocol
>> or cipher suite such as RC4 (link for the details), which is not
>> considered secure. Please contact your site administrator.
>
> One really important question is whether the load balancer acts as a
> pure TCP proxy, or whether the load balancer is configured with a
> certificate and handles HTTPS itself.
>
> If the load balancer is handling HTTPS, it's very likely that the load
> balancer either cannot use modern TLS protocols and/or ciphers, or
> that it has the modern protocols/ciphers turned off.
> There's probably nothing that we can do to help you in this situation.
> You will need to find support for your load balancer.
>
> If the load balancer is just a TCP proxy and lets the back end server
> handle HTTPS, then you may need to ensure that you're running a very
> recent version of Java 8. You may also need to install the JCE policy
> files for unlimited strength encryption into your Java. I see from
> other messages on the list that you're running Solr 6.6.2, so it would
> not be a good idea for you to use Java 9 or Java 10. If you need
> them, the JCE policy files for Java
> 8 can be found here:
>
> http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-
2133166.html
>
> One thing you didn't explicitly mention is whether the connection
> works when talking directly to one of the Solr servers instead of the
> load balancer. If that works, then your Java version is probably
> fine, and it's even more evidence that the problem is on the load
> balancer.
>
> Thanks, Shawn
>
>
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlsRwYwACgkQHPApP6U8
pFh5LA/+MWkaeylVbsXwL9TxU/qe6fyW82/OVznqDNVKF2KwvtloMjKIyo90ZdqB
N2fqRfczyqN2NporI7dZtj68Qcb7JiOkzfKUQJX/4Ecgfl6WhcrcnzC6jt9B6oQR
c0W02QGGKREz2l719ZI4wohgGPX7HD+u+GXlUdz+v1Bw+4vZlG9LzDJ7YC9XDgXX
1hUDfdmBHS2krMnp5/1bsIvg9Xr58Orrwz20EKyumzUZ/P9WekoUw7WeqJSuuQoN
n3+yM8BMPp/AUy7+5gcvaKtd9mB6J4oUyQQAfj+cNOg/eOiY2t+EFr8b+pVBDG+z
yQ+9K9g0Z9b1GTjZsHtGymdPWjwNWbhnNKKh6fvQSY5o/YBQUD0tQL/bUPEnoMW2
S0vflC10ptJ0d/NqAio2VS9bPE3zMsYNDLlt3zUw+OPugT1IOwPLOLENArbYsrSk
jcXP+f70MMPSMYx7FwNGPwE2X/izFfv+Bal7wuPqusI4SV1b8D9tO236K/bg7/g4
OjfUEXwPaWrVipFQWHm+d0uMr4O/DAQqwN09tzmhF3sqcatmLyHOXaGhO2ZS6O8i
v1XJ52ZieXOO71nIr8LsP+9VKON2kS9x+n4rX3eqGBCEdkysb/Z5IBkddYtydtDk
L6I0VuHm78CrJrkKuqq4SvFUGpXTtvKQYDZp1DZOJPCiGiGn9/o=
=Fdyp
-----END PGP SIGNATURE-----
Re: Self Signed Certificate for Load Balancer and Solr Nodes
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Kelly,
On 6/1/18 5:41 PM, Kelly Rusk wrote:
> I can directly connect to either node without issue, it is only
> when the Load Balancer routes to either solr1 or solr2 that I get
> the security error (ex. https://solrlb.com:8983/solr). The Load
> Balancer is not managing HTTPS but just acting as a pure TCP proxy.
> Nothing more complex than sending traffic to either solr1 or
> solr2... however, the URL will be displayed as solrlb.com as it
> hides the real address of what is being routed to.
>
> In this case, do we need a certificate for solrlb.com installed on
> both solr1 and solr2?
That's exactly what you need. It would be best to:
1. Create a certificate for solrlb.com
2. Install the same key + certificate on both Solr nodes
3. Always use solrlb.com for any links and redirects you generate
Optionally, you could add SANs for that certificate for both solr1 and
solr2 just in case you want to be able to connect directly to either
back-end node without getting hostname mismatch complaints.
> In our previous environments we used the same load balancer setup,
> but that worked since the Solr nodes were serving over http and
> not https.
You probably never noticed that redirects were occurring that were
sending users to a particular node instead of always using the lb's
hostname because there was never anything double-checking the hostname.
In your previous message, you mentioned that you got an error message
including the hostname "b-win-solr-01.azure-dfa.com" which probably
isn't your load-balancer's hostname. That suggests to me that some
kind of redirect (or similar) is occurring and that the redirect
doesn't understand that there is a reverse-proxy/lb out in front of
the node.
Hope that helps,
- -chris
> -----Original Message----- From: Shawn Heisey <ap...@elyograg.org>
> Sent: Friday, June 1, 2018 5:25 PM To:
> solr-user@lucene.apache.org Subject: Re: Self Signed Certificate
> for Load Balancer and Solr Nodes
>
> On 6/1/2018 2:01 PM, Kelly Rusk wrote:
>> We have solr1.com and solr2.com self-signed certs that correspond
>> to the two servers. We also have a load balancer with an address
>> named solrlb.com. When we hit the load balancer it gives us an
>> SSL error, as it is passing us back to either solr1.com or
>> solr2.com, but since these two Solr servers only have each
>> other's self-signed cert installed in their Keystore, it doesn't
>> resolve when it comes in through the load balanced address of
>> solrlb.com.
>>
>> We tried a san certificate that has all 3 addresses, but when we
>> do this, we get the following error:
>>
>> This page can't be displayed Turn on TLS 1.0, TLS 1.1, and TLS
>> 1.2 in Advanced settings and try connecting to
>> https://b-win-solr-01.azure-dfa.com:8983 again. If this error
>> persists, it is possible that this site uses an unsupported
>> protocol or cipher suite such as RC4 (link for the details),
>> which is not considered secure. Please contact your site
>> administrator.
>
> One really important question is whether the load balancer acts as
> a pure TCP proxy, or whether the load balancer is configured with a
> certificate and handles HTTPS itself.
>
> If the load balancer is handling HTTPS, it's very likely that the
> load balancer either cannot use modern TLS protocols and/or
> ciphers, or that it has the modern protocols/ciphers turned off.
> There's probably nothing that we can do to help you in this
> situation. You will need to find support for your load balancer.
>
> If the load balancer is just a TCP proxy and lets the back end
> server handle HTTPS, then you may need to ensure that you're
> running a very recent version of Java 8. You may also need to
> install the JCE policy files for unlimited strength encryption into
> your Java. I see from other messages on the list that you're
> running Solr 6.6.2, so it would not be a good idea for you to use
> Java 9 or Java 10. If you need them, the JCE policy files for Java
> 8 can be found here:
>
> http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-
2133166.html
>
> One thing you didn't explicitly mention is whether the connection
> works when talking directly to one of the Solr servers instead of
> the load balancer. If that works, then your Java version is
> probably fine, and it's even more evidence that the problem is on
> the load balancer.
>
> Thanks, Shawn
>
>
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlsRwYwACgkQHPApP6U8
pFh5LA/+MWkaeylVbsXwL9TxU/qe6fyW82/OVznqDNVKF2KwvtloMjKIyo90ZdqB
N2fqRfczyqN2NporI7dZtj68Qcb7JiOkzfKUQJX/4Ecgfl6WhcrcnzC6jt9B6oQR
c0W02QGGKREz2l719ZI4wohgGPX7HD+u+GXlUdz+v1Bw+4vZlG9LzDJ7YC9XDgXX
1hUDfdmBHS2krMnp5/1bsIvg9Xr58Orrwz20EKyumzUZ/P9WekoUw7WeqJSuuQoN
n3+yM8BMPp/AUy7+5gcvaKtd9mB6J4oUyQQAfj+cNOg/eOiY2t+EFr8b+pVBDG+z
yQ+9K9g0Z9b1GTjZsHtGymdPWjwNWbhnNKKh6fvQSY5o/YBQUD0tQL/bUPEnoMW2
S0vflC10ptJ0d/NqAio2VS9bPE3zMsYNDLlt3zUw+OPugT1IOwPLOLENArbYsrSk
jcXP+f70MMPSMYx7FwNGPwE2X/izFfv+Bal7wuPqusI4SV1b8D9tO236K/bg7/g4
OjfUEXwPaWrVipFQWHm+d0uMr4O/DAQqwN09tzmhF3sqcatmLyHOXaGhO2ZS6O8i
v1XJ52ZieXOO71nIr8LsP+9VKON2kS9x+n4rX3eqGBCEdkysb/Z5IBkddYtydtDk
L6I0VuHm78CrJrkKuqq4SvFUGpXTtvKQYDZp1DZOJPCiGiGn9/o=
=Fdyp
-----END PGP SIGNATURE-----
RE: Self Signed Certificate for Load Balancer and Solr Nodes
Posted by Kelly Rusk <Ke...@rackspace.com>.
Thank you Shawn,
I can directly connect to either node without issue, it is only when the Load Balancer routes to either solr1 or solr2 that I get the security error (ex. https://solrlb.com:8983/solr). The Load Balancer is not managing HTTPS but just acting as a pure TCP proxy. Nothing more complex than sending traffic to either solr1 or solr2... however, the URL will be displayed as solrlb.com as it hides the real address of what is being routed to.
In this case, do we need a certificate for solrlb.com installed on both solr1 and solr2?
In our previous environments we used the same load balancer setup, but that worked since the Solr nodes were serving over http and not https.
Regards,
Kelly
-----Original Message-----
From: Shawn Heisey <ap...@elyograg.org>
Sent: Friday, June 1, 2018 5:25 PM
To: solr-user@lucene.apache.org
Subject: Re: Self Signed Certificate for Load Balancer and Solr Nodes
On 6/1/2018 2:01 PM, Kelly Rusk wrote:
> We have solr1.com and solr2.com self-signed certs that correspond to the two servers. We also have a load balancer with an address named solrlb.com. When we hit the load balancer it gives us an SSL error, as it is passing us back to either solr1.com or solr2.com, but since these two Solr servers only have each other's self-signed cert installed in their Keystore, it doesn't resolve when it comes in through the load balanced address of solrlb.com.
>
> We tried a san certificate that has all 3 addresses, but when we do this, we get the following error:
>
> This page can't be displayed
> Turn on TLS 1.0, TLS 1.1, and TLS 1.2 in Advanced settings and try connecting to https://b-win-solr-01.azure-dfa.com:8983 again. If this error persists, it is possible that this site uses an unsupported protocol or cipher suite such as RC4 (link for the details), which is not considered secure. Please contact your site administrator.
One really important question is whether the load balancer acts as a pure TCP proxy, or whether the load balancer is configured with a certificate and handles HTTPS itself.
If the load balancer is handling HTTPS, it's very likely that the load balancer either cannot use modern TLS protocols and/or ciphers, or that it has the modern protocols/ciphers turned off. There's probably nothing that we can do to help you in this situation. You will need to find support for your load balancer.
If the load balancer is just a TCP proxy and lets the back end server handle HTTPS, then you may need to ensure that you're running a very recent version of Java 8. You may also need to install the JCE policy files for unlimited strength encryption into your Java. I see from other messages on the list that you're running Solr 6.6.2, so it would not be a good idea for you to use Java 9 or Java 10. If you need them, the JCE policy files for Java 8 can be found here:
http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html
One thing you didn't explicitly mention is whether the connection works when talking directly to one of the Solr servers instead of the load balancer. If that works, then your Java version is probably fine, and it's even more evidence that the problem is on the load balancer.
Thanks,
Shawn
Re: [OT] Self Signed Certificate for Load Balancer and Solr Nodes
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Shawn,
On 6/1/18 5:25 PM, Shawn Heisey wrote:
> On 6/1/2018 2:01 PM, Kelly Rusk wrote:
>> We have solr1.com and solr2.com self-signed certs that correspond
>> to the two servers. We also have a load balancer with an address
>> named solrlb.com. When we hit the load balancer it gives us an
>> SSL error, as it is passing us back to either solr1.com or
>> solr2.com, but since these two Solr servers only have each
>> other's self-signed cert installed in their Keystore, it doesn't
>> resolve when it comes in through the load balanced address of
>> solrlb.com.
>>
>> We tried a san certificate that has all 3 addresses, but when we
>> do this, we get the following error:
>>
>> This page can't be displayed Turn on TLS 1.0, TLS 1.1, and TLS
>> 1.2 in Advanced settings and try connecting to
>> https://b-win-solr-01.azure-dfa.com:8983 again. If this error
>> persists, it is possible that this site uses an unsupported
>> protocol or cipher suite such as RC4 (link for the details),
>> which is not considered secure. Please contact your site
>> administrator.
>
> One really important question is whether the load balancer acts as
> a pure TCP proxy, or whether the load balancer is configured with
> a certificate and handles HTTPS itself.
>
> If the load balancer is handling HTTPS, it's very likely that the
> load balancer either cannot use modern TLS protocols and/or
> ciphers, or that it has the modern protocols/ciphers turned off.
> There's probably nothing that we can do to help you in this
> situation. You will need to find support for your load balancer.
>
> If the load balancer is just a TCP proxy and lets the back end
> server handle HTTPS, then you may need to ensure that you're
> running a very recent version of Java 8. You may also need to
> install the JCE policy files for unlimited strength encryption into
> your Java. I see from other messages on the list that you're
> running Solr 6.6.2, so it would not be a good idea for you to use
> Java 9 or Java 10. If you need them, the JCE policy files for Java
> 8 can be found here:
>
> http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-
2133166.html
Starting
>
with Oracle Java 8u151 and later, the "unlimited strength
jurisdiction policy files" are included in the default build, so you
no longer have to manually-install them.
Nice to see that Java finally got out of the 1990s mindset when it
comes to cryptography.
Unfortunately, Java 8 is close to EOL[2] so it's time to look at newer
versions of Java, which likely means newer versions of Solr if you
want to be safe and secure. I say "close to EOL" even though it's 7
months away because it can take a looong time to plan and execute an
upgrade of both Solr and Java.
- -chris
[1]
https://golb.hplar.ch/2017/10/JCE-policy-changes-in-Java-SE-8u151-and-8u
152.html
[2] http://www.oracle.com/technetwork/java/javase/eol-135779.html
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlsRwEIACgkQHPApP6U8
pFi+ng/+KhdiGaSN4PHjHvroqcNXKmmvXbMIIHcHCAARGnTS+0LuZGhopWbJA0u+
NhE/fHJTyRFtfCBaY6gL9NsumAQTXA2kCLsKpWv86WaVEWZSH55BC/0aJCNp/xOU
/QheBJ255RDBYeLZvGAngAS7mWK1wPh6BhsD0bNwtoU7xGCZQtvLt7CdQLu+F8Dm
uJczJOipp8SS/TlTJcP9t02WW3RvjqIZbn4EEr0DZj7hzy1ST8/yzu7cNpo+uQw5
AmoIDik8TmVKmT7h/gW8/frpz7brI+Zw3qm+YELpJK2SQywqhZFdhPjnnAqYKqY0
AuVJlYeC+0ivw/3oHQM/kShzqgXiMTv8bp63BbEYcWt1z9pb2Ltrx/jHsEQYr6k1
bxHAnrXXoQQTq8wm4jqYBSfEB97JyYWqCKJ04HyhxJ9Tzqv5vUwL1xXf4mY0m6dA
eDGoKQ3fjHZaMzUhc0c/zv4MwMH+KYzZ05Y5mdT1UHaYGX3sUMGhdSyNlvWZy4Np
G7ehzOdsuEO+b5+YBQQpWarei76I5soPttkz5rrvWfksn8jUHo0VoqDVs0/g6uY4
5p85OJPF/C4quLDWHN1swpVQJ2q4R3C4RdGjdb2WT+hkks6c1WkqGfkAH2ONA+DS
dxG83u9aDxm+eyoj+GvMlTIAGnqutU2nNQrErb5sGjVHkQaLLaw=
=PK53
-----END PGP SIGNATURE-----
Re: Self Signed Certificate for Load Balancer and Solr Nodes
Posted by Shawn Heisey <ap...@elyograg.org>.
On 6/1/2018 2:01 PM, Kelly Rusk wrote:
> We have solr1.com and solr2.com self-signed certs that correspond to the two servers. We also have a load balancer with an address named solrlb.com. When we hit the load balancer it gives us an SSL error, as it is passing us back to either solr1.com or solr2.com, but since these two Solr servers only have each other's self-signed cert installed in their Keystore, it doesn't resolve when it comes in through the load balanced address of solrlb.com.
>
> We tried a san certificate that has all 3 addresses, but when we do this, we get the following error:
>
> This page can't be displayed
> Turn on TLS 1.0, TLS 1.1, and TLS 1.2 in Advanced settings and try connecting to https://b-win-solr-01.azure-dfa.com:8983 again. If this error persists, it is possible that this site uses an unsupported protocol or cipher suite such as RC4 (link for the details), which is not considered secure. Please contact your site administrator.
One really important question is whether the load balancer acts as a
pure TCP proxy, or whether the load balancer is configured with a
certificate and handles HTTPS itself.
If the load balancer is handling HTTPS, it's very likely that the load
balancer either cannot use modern TLS protocols and/or ciphers, or that
it has the modern protocols/ciphers turned off. There's probably
nothing that we can do to help you in this situation. You will need to
find support for your load balancer.
If the load balancer is just a TCP proxy and lets the back end server
handle HTTPS, then you may need to ensure that you're running a very
recent version of Java 8. You may also need to install the JCE policy
files for unlimited strength encryption into your Java. I see from
other messages on the list that you're running Solr 6.6.2, so it would
not be a good idea for you to use Java 9 or Java 10. If you need them,
the JCE policy files for Java 8 can be found here:
http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html
One thing you didn't explicitly mention is whether the connection works
when talking directly to one of the Solr servers instead of the load
balancer. If that works, then your Java version is probably fine, and
it's even more evidence that the problem is on the load balancer.
Thanks,
Shawn