You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by Julius Davies <ju...@gmail.com> on 2007/01/30 04:59:11 UTC
[httpclient] [ssl] 4.0's CN verification might cause some headaches for users
As we get closer to the 4.0 release, I have one "titanic + iceberg"
feeling. Here's the head's up.
This bug fix introduces a subtle, yet potentially devastating change
to the default behaviour of "https://" when using httpclient-4.0:
HTTPCLIENT-613 "https should check CN of x509 cert"
I have this funny feeling that a lot of SOAP, REST, WS-*, and whatever
else people call server-to-server HTTP is going to go *splat* when
httpclient-4.0 starts making its way into JBoss, Websphere, Weblogic,
Axis, and all those other stacks.
Imagine even the simplest of scenarios:
1. https://soap.mydomain.com/ has a beautiful proper cert from Verisign.
2. But one of its java clients decided to save on the DNS lookup, and
calls in (using httpclient) with "https://1.2.3.4/".
3. Or people went to "https://google.com/" instead of
"https://www.google.com/".
4. The worst situation, though, is where it's a domain, but the
locked-down business-to-business communication channel has no way to
do the DNS lookup, since it doesn't have access to the target
business's DNS. It only has a measly little firewall opening for the
one IP address. So the consumer is actually forced to do
"https://1.2.3.4/" unless they want to fool around with "/etc/hosts"!
Do others have this nervous feeling? Frankly, I feel we have no
choice. We're missing a critical piece of public PKI without it. And
thank goodness we support wildcards. That will help. But nonetheless
I have a feeling best summed up by one word:
Gulp.
Just before 4.0 has its first RC, I think we should prepare for this
issue on the wiki and even on the main webpage.
=================================================
FAQ - My "https://" connections worked great with httpclient-3.x, but
ever since I upgraded to httpclient-4.x, my https connections are
broken with this error message:
javax.net.ssl.SSLException: hostname in certificate didn't match:
<foo.com> != <bar.com>
------------------------------------
Answer:
Find out what domain the Certificate is valid for. The error message
should tell you. Change your use of httpclient to use that domain
exactly.
If you see: <foo.com> != <bar.com>
Then change your httpclient calls from "https://foo.com/" to "https://bar.com/".
=================================================
People were talking about "scheme" on this thread recently. Can we
provide the following two schemes right out of the box?
https-no-host-verify://
- This one checks the server's certificate for a valid
Certificate-Chain + TrustAnchor, and requires that the server's cert
be non-expired. But this scheme doesn't verify the CN field of the
certificate. Behaves like "https://" under httpclient-3.0 and
httpclient-2.0.
(Or maybe call it https-no-cn-check://?) (other ideas?)
https-completely-insecure://
- This one doesn't check CN, doesn't check expiry, and doesn't check
for valid certificate chain + TrustAnchor. This one is completely
vulnerable to man-in-the-middle attacks, and should be avoided if at
all possible, even for development environments, since it's such a bad
habit that always seems to find a way into production. Friends don't
let friends use "https-completely-insecure://", even in the privacy of
their own development environments. We provide this scheme only as an
evil temptation for you to resist. It is not meant to ever be used.
Yes, that's right. Never use this one.
(The old convention was "https-easy://" but I don't think "easy" isn't
scaring people enough.)
--
yours,
Julius Davies
416-652-0183
http://juliusdavies.ca/
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
Re: [httpclient] [ssl] 4.0's CN verification might cause some headaches
for users
Posted by Ortwin Glück <od...@odi.ch>.
Julius Davies wrote:
> As we get closer to the 4.0 release, I have one "titanic + iceberg"
> feeling. Here's the head's up.
>
> This bug fix introduces a subtle, yet potentially devastating change
> to the default behaviour of "https://" when using httpclient-4.0:
>
> HTTPCLIENT-613 "https should check CN of x509 cert"
>
> I have this funny feeling that a lot of SOAP, REST, WS-*, and whatever
> else people call server-to-server HTTP is going to go *splat* when
> httpclient-4.0 starts making its way into JBoss, Websphere, Weblogic,
> Axis, and all those other stacks.
Integration of the new version of HttpClient is not going to be
seamless. Our users will expect changes.
> Imagine even the simplest of scenarios:
>
> 1. https://soap.mydomain.com/ has a beautiful proper cert from Verisign.
>
> 2. But one of its java clients decided to save on the DNS lookup, and
> calls in (using httpclient) with "https://1.2.3.4/".
>
> 3. Or people went to "https://google.com/" instead of
> "https://www.google.com/".
>
> 4. The worst situation, though, is where it's a domain, but the
> locked-down business-to-business communication channel has no way to
> do the DNS lookup, since it doesn't have access to the target
> business's DNS. It only has a measly little firewall opening for the
> one IP address. So the consumer is actually forced to do
> "https://1.2.3.4/" unless they want to fool around with "/etc/hosts"!
Julius, such setups are inherently broken. There is nothing to save
people who design crap like this. Somebody call me arrogant :-)
I can tell a similar story from the mobile world. We had an application
running on a Sony P900. All worked well until the customer started using
a new generation of the same phone. Its new OS was performing stricter
checks on the code signing certificate and promptly refused to install
our app because it was using only a self-signed one. Of course we fixed
our signing procedure and not the phone.
> Do others have this nervous feeling? Frankly, I feel we have no
> choice. We're missing a critical piece of public PKI without it. And
> thank goodness we support wildcards. That will help. But nonetheless
> I have a feeling best summed up by one word:
>
> Gulp.
>
> Just before 4.0 has its first RC, I think we should prepare for this
> issue on the wiki and even on the main webpage.
Of course I find it a good idea to document the new behaviour. This way
integrators can easily find a solution when they get bug reports.
> =================================================
> FAQ - My "https://" connections worked great with httpclient-3.x, but
> ever since I upgraded to httpclient-4.x, my https connections are
> broken with this error message:
>
> javax.net.ssl.SSLException: hostname in certificate didn't match:
> <foo.com> != <bar.com>
>
> ------------------------------------
>
> Answer:
>
> Find out what domain the Certificate is valid for. The error message
> should tell you. Change your use of httpclient to use that domain
> exactly.
>
> If you see: <foo.com> != <bar.com>
>
> Then change your httpclient calls from "https://foo.com/" to
> "https://bar.com/".
> =================================================
>
> People were talking about "scheme" on this thread recently. Can we
> provide the following two schemes right out of the box?
Yeah, why not. Although I am not a big fan of this abuse of schemes. You
could do the same with a simple paramter to the request.
> https-no-host-verify://
> - This one checks the server's certificate for a valid
> Certificate-Chain + TrustAnchor, and requires that the server's cert
> be non-expired. But this scheme doesn't verify the CN field of the
> certificate. Behaves like "https://" under httpclient-3.0 and
> httpclient-2.0.
>
> (Or maybe call it https-no-cn-check://?) (other ideas?)
Maybe add one that fits the common practice of self-signed certs:
https-self-signed://
Some admins actually believe that it's enough to have a self-signed
cert, because it's the first thing in all the documentation and even
generated semi-automatically by some linux distros...
>
> https-completely-insecure://
Well, it *does* encryption. So it's not completely insecure in a network
without men-in-the-middle. At least it prevents accidential eavesdropping.
> - This one doesn't check CN, doesn't check expiry, and doesn't check
> for valid certificate chain + TrustAnchor. This one is completely
> vulnerable to man-in-the-middle attacks, and should be avoided if at
> all possible, even for development environments, since it's such a bad
> habit that always seems to find a way into production. Friends don't
> let friends use "https-completely-insecure://", even in the privacy of
> their own development environments. We provide this scheme only as an
> evil temptation for you to resist. It is not meant to ever be used.
> Yes, that's right. Never use this one.
>
> (The old convention was "https-easy://" but I don't think "easy" isn't
> scaring people enough.)
>
>
--
[web] http://www.odi.ch/
[blog] http://www.odi.ch/weblog/
[pgp] key 0x81CF3416
finger print F2B1 B21F F056 D53E 5D79 A5AF 02BE 70F5 81CF 3416
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
Re: [httpclient] [ssl] 4.0's CN verification might cause some
headaches for users
Posted by Julius Davies <ju...@cucbc.com>.
Hi, Oleg, Jim, Martin, others on httpcomponents-dev,
In reply to Oleg's question:
On Mon, 2007-05-02 at 17:58 +0100, Oleg Kalnichevski wrote:
> Folks,
>
> I think there should no longer be any contrib SSL stuff in HttpClient
> 4.0, easy or otherwise. The users should simply get directed to Commons
> SSL / SSLUtils / whatever
>
> Julius,
>
> Where did the process get stuck again?
>
> Oleg
Here's a status update on not-yet-commons-ssl:
- not-yet-commons-ssl-0.3.5 released from
"http://juliusdavies.ca/commons-ssl/" on January 1st with better CN
verification (inspired by HTTPCLIENT-614 issue).
https://issues.apache.org/jira/browse/HTTPCLIENT-614)
- Mailing list started up. 4 subscribers! Averaging 2 posts per
month! :-)
http://lists.juliusdavies.ca/listinfo.cgi/not-yet-commons-ssl-juliusdavies.ca/
- I think n-y-c-ssl is averaging about 2 downloads a day.
Jim kindly offered to sign a copy of the CCLA and send it back to CUCBC.
I sent Jim a self-addressed stamped envelope (hard to find U.S. stamps
in Canada!) on December 12th, and now I'm just waiting for the CCLA to
showup at CUCBC before I send an email to the Incubation PMC.
Hopefully the S.A.S.E. arrived and was large enough to hold the CCLA!
If not, I can send another one.
I'm doing things weird by doing the CCLA first, but I think that's the
most critical issue to get out of the way so that people can safely
download the library and play around with it.
yours,
Julius
--
Julius Davies, Senior Application Developer, Product Development
T 416-652-0183 | juliusdavies@cucbc.com
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
Re: [httpclient] [ssl] 4.0's CN verification might cause some headaches
for users
Posted by Roland Weber <ht...@dubioso.net>.
Hi Oleg,
> I think there should no longer be any contrib SSL stuff in HttpClient
> 4.0, easy or otherwise. The users should simply get directed to Commons
> SSL / SSLUtils / whatever
fair enough. So it will just be clear statements in the SSL guide 4.0.
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
Re: [httpclient] [ssl] 4.0's CN verification might cause some
headaches for users
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Mon, 2007-02-05 at 17:50 +0100, Roland Weber wrote:
> Hi Julius,
>
> > Mind you, upgrading (or maybe "switching" is a better word) to
> > httpclient-4.0 should only affect the client code. This change can
> > require people to fix their client, their server, and their DNS, and
> > this issue might not showup for people until they hit their production
> > environments. I imagine it will be a little stressful!
>
> Thanks for bringing it to our attention. We will remember it by the
> time we're writing an SSL guide for 4.0. And if it hits them only
> in production, their integration and staging environments aren't
> good enough ;-)
>
> > I agree with Roland and Michael that it is best if people not use them
> > at all. But I see so many people just blindly using "easy" on the
> > "httpclient-user" mailing list, I thought maybe
> > "https-completely-insecure://" would scare them off.
>
> We'll be addressing that in the SSL 4.0 guide, too. And I'll see to
> it that the "EasySSLProtocolSocketFactory" changes it's name not only
> after the "SSL" part :-)
>
Folks,
I think there should no longer be any contrib SSL stuff in HttpClient
4.0, easy or otherwise. The users should simply get directed to Commons
SSL / SSLUtils / whatever
Julius,
Where did the process get stuck again?
Oleg
> > (I wonder if those schemes would have helped the public PKI situation
> > had they been part of the standards. Probably not.)
>
> Hardly. The point of PKI is to establish a chain of trust.
> You can't do that by removing the trust.
>
> cheers,
> Roland
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
Re: [httpclient] [ssl] 4.0's CN verification might cause some headaches
for users
Posted by Roland Weber <ht...@dubioso.net>.
Hi Julius,
> Mind you, upgrading (or maybe "switching" is a better word) to
> httpclient-4.0 should only affect the client code. This change can
> require people to fix their client, their server, and their DNS, and
> this issue might not showup for people until they hit their production
> environments. I imagine it will be a little stressful!
Thanks for bringing it to our attention. We will remember it by the
time we're writing an SSL guide for 4.0. And if it hits them only
in production, their integration and staging environments aren't
good enough ;-)
> I agree with Roland and Michael that it is best if people not use them
> at all. But I see so many people just blindly using "easy" on the
> "httpclient-user" mailing list, I thought maybe
> "https-completely-insecure://" would scare them off.
We'll be addressing that in the SSL 4.0 guide, too. And I'll see to
it that the "EasySSLProtocolSocketFactory" changes it's name not only
after the "SSL" part :-)
> (I wonder if those schemes would have helped the public PKI situation
> had they been part of the standards. Probably not.)
Hardly. The point of PKI is to establish a chain of trust.
You can't do that by removing the trust.
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
Re: [httpclient] [ssl] 4.0's CN verification might cause some headaches for users
Posted by Julius Davies <ju...@gmail.com>.
Thanks, everyone, for your comments! I didn't realize Httpclient-4.0
was going to be such a dramatic change to the consumers. Since that's
the case this isn't such a big deal.
Mind you, upgrading (or maybe "switching" is a better word) to
httpclient-4.0 should only affect the client code. This change can
require people to fix their client, their server, and their DNS, and
this issue might not showup for people until they hit their production
environments. I imagine it will be a little stressful!
Regarding:
https-no-host-verify://
https-completely-insecure://
I agree with Roland and Michael that it is best if people not use them
at all. But I see so many people just blindly using "easy" on the
"httpclient-user" mailing list, I thought maybe
"https-completely-insecure://" would scare them off.
But I also agree that I'm probably being foolish, and that including
it might just encourage more people!
(I wonder if those schemes would have helped the public PKI situation
had they been part of the standards. Probably not.)
--
yours,
Julius Davies
416-652-0183
http://juliusdavies.ca/
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
Re: [httpclient] [ssl] 4.0's CN verification might cause some headaches for users
Posted by Michael Becke <mb...@gmail.com>.
> > People were talking about "scheme" on this thread recently. Can we
> > provide the following two schemes right out of the box?
>
> We're far away from talking defaults. If any, I would suggest
> defaults that either adhere strictly to specifications or else
> map to JVM settings.
>
> > https-no-host-verify://
> > https-completely-insecure://
>
> I think it would be much better to have a good SSL guide that
> tells people how and why to set up such schemes themselves.
> A default, if we provide one, should map to the SSL socket factory
> of the JVM and perform CN checks as strictly as we can make them.
>
> > We provide this scheme only as an evil temptation for you to resist.
>
> That's a funny thing to write, and I guess I used to think so
> myself. But I have become much more pragmatic. If we don't
> want them to use it, we shouldn't put it in by default. That
> will keep more people from abusing the code as any warning.
I would just like to echo Roland's sentiments here. Having the
ability to do these special schemes is fine, but I don't think they
should be enabled out of the box. Their unintended use could lead to
inherent security issues as well as just general confusion. Out of
the box we should strive to support the "standards" with the ability
to configure for custom scenarios.
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
Re: [httpclient] [ssl] 4.0's CN verification might cause some headaches
for users
Posted by Roland Weber <ht...@dubioso.net>.
Hi Julius,
> As we get closer to the 4.0 release,
That's what I call a forward-looking statement. We're weeks away from
even getting a grip on connection management, let alone from an
HttpClient 4.0 that passes our internal reviews for an alpha1 release.
> I have one "titanic + iceberg" feeling.
Which one is us?
> HTTPCLIENT-613 "https should check CN of x509 cert"
I have no problem at all to make the default behavior strictly compliant
with the specification, as long as it is easy to change. We're shipping
HttpClient 3.0 with the RFC2109 cookie spec as default although many web
sites require the browser compatibility spec to function properly.
> I have this funny feeling that a lot of SOAP, REST, WS-*, and whatever
> else people call server-to-server HTTP is going to go *splat* when
> httpclient-4.0 starts making its way into JBoss, Websphere, Weblogic,
> Axis, and all those other stacks.
Sounds like a dream come true. The "making it's way" part of course ;-)
As Odi pointed out, HttpClient 4 will not be a drop-in replacement.
And nobody (running a serious risk) will be stupid enough to just
roll out a completely new version of a J2EE platform or web services
stack without testing their applications thoroughly. Problems will be
detected in testing, and then people will decide whether to fix their
DNS/hostname/certificate setup or whether to customize the behaviour
of HttpClient. My bet is on the latter, but at least they'll know what
they're doing.
> Just before 4.0 has its first RC, I think we should prepare for this
> issue on the wiki and even on the main webpage.
I think you're overestimating the impact of TLS/SSL certificate
verification. Have you compared the API of HttpClient 3 with what's
in HttpCore? We're not only changing all package names, we've also
removed HttpMethod which was at the center of the HttpClient 3 API.
Trust me, by the time people have adapted their code to the new API,
setting a different CN verification policy will be a trifle for them.
A code migration guide would be much more important for the current
user base, but I'm not going to put much effort into that until the
HttpClient API stabilizes. Maybe by the time we're heading up to alpha3.
I don't expect anybody to start a migration to alpha1. We can be glad
if they even look at it to assess the potential effort.
> =================================================
> FAQ - My "https://" connections worked great with httpclient-3.x, but
> ever since I upgraded to httpclient-4.x, my https connections are
> broken with this error message:
FAQ - I have an application that runs great with HttpClient 3.x, but
when I compile it against HttpClient 4.0, the compiler cancels after
100 errors, and they all read "cannot find symbol" for classes in
org.apache.commons.httpclient.
> People were talking about "scheme" on this thread recently. Can we
> provide the following two schemes right out of the box?
We're far away from talking defaults. If any, I would suggest
defaults that either adhere strictly to specifications or else
map to JVM settings.
> https-no-host-verify://
> https-completely-insecure://
I think it would be much better to have a good SSL guide that
tells people how and why to set up such schemes themselves.
A default, if we provide one, should map to the SSL socket factory
of the JVM and perform CN checks as strictly as we can make them.
> We provide this scheme only as an evil temptation for you to resist.
That's a funny thing to write, and I guess I used to think so
myself. But I have become much more pragmatic. If we don't
want them to use it, we shouldn't put it in by default. That
will keep more people from abusing the code as any warning.
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org