You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Henry Story <he...@bblfish.net> on 2011/11/01 01:05:40 UTC
resolution (partial): https client with TLS renegotiating server
I think I found the problem in Java, and a partial workaround. Essentially the java HTTP client only sends the certificate if the server requests it in NeedClientAuth, not in WantClientAuth mode. This is a subtle bug and difficult to explain, or rather it requires moving all the way from the lowest levels of TLS to the user experience in a browser. Java used to have the ambition to become a language to write a browser (hotjava) and I think this is still a valid aim - in any case every application long term will become a browser application.
So what is the UI problem? The difference between WANT and NEED mode is that in WANT TLS mode the server can ask the client for a certificate, and if the client does not have one, or does not wish to continue, the connection will be pursued but without client authentication. This means that the browser can for example return an HTTP 400 page with a human readable explanation of the problem, or better even redirect the user to a account creation page, or to another site, or whatever... If the server requests the certificate in NEED mode, then if the client does not send the certificate the TLS connection is abruptly and finally closed leaving the user with a not so beautiful and not very explanatory error message page. It's the TLS equivalent of having the door shut in one's face without an explanation. So Java clients currently, I discovered - but I may still be shown some to be wrong - only seems to send a certificate when requested in NEED mode. I hope we can fix this or find a way for java clients to send certificates in WANT mode. We should certainly open a bug report on this. But I have no idea where one reports Java bugs nowadays. bugs.sun.com seems to be very broken.
I really needed the Java clients to work at least in order to run the test suite to test the http://webid.info/ authentication in the read-write-web project. After all it would be silly not to be able to test the code that works for most desktop browsers in Java! So I had to find a workaround.
Here is the code that allows me to test the connection to a TLS protected resource that needs authentication. In line 40 I enable the TLS client certificate to user "JoeLambda", so that if the client code ever gets requested a certificate in a TLS connection it can return that users cert. We then connect to the protected resource, using the basic http libraries.
240 testKeyManager.setId("JoeLambda")
245 val scon =webidProfile.secure.to_uri.toURL.openConnection().asInstanceOf[HttpsURLConnection]
246 scon.setSSLSocketFactory(sslContext.getSocketFactory)
247 scon.setRequestProperty("Content-Type",Post.SPARQL)
248 scon.setRequestProperty("User-Agent" , "Java/1.7.0")
249 scon.setRequestMethod("POST")
250 val msg = updateFriend.format("http://bblfish.net/#hjs").getBytes("UTF-8")
251 scon.setRequestProperty("Content-Length",msg.length.toString)
252 scon.setDoOutput(true)
253 scon.setDoInput(true)
254
255 val out = scon.getOutputStream
256 out.write(msg)
257 out.flush()
258 out.close()
259 scon.connect()
260
261 val httpCode = scon.getResponseCode
262
263
264 val req =webidProfile.secure.PUT <:< Map("User-Agent" -> "Java/1.7.0","Content-Type"->Post.SPARQL)
265 val req2 = req.copy(
266 method="POST",
267 body=Some(new RefStringEntity(updateFriend.format(webID.toExternalForm),Post.SPARQL,"UTF-8"))
268 )
269
270 val httpCode = Http( req2 get_statusCode )
271 httpCode must_== 200
Because in this test case the resource is protected for POSTs the server ends up calling the authentication module, which tries to find a cert for the user. This then ends up calling the netty code below. On line 182 the server looks at the HTTP headers of the request, and if that header contains "Java" it sets the ssl engine in NeedClientAuth mode (line 182). For desktop browser the WantClientAuth mode is better, as that allows a user who does not have a certificate to log in with other technologies such as OAuth, OpenID, WebId etc... It then renegotiates the connection in line 186, and the waits for a maximum of 30 seconds for a response. By line 190 the server has the client certificate now, and can proceed with authorisation.
173 r.underlying.context.getPipeline.get(classOf[SslHandler]) match {
174 case sslh: SslHandler => try {
175 //return the client certificate in the existing session if one exists
176 Some(sslh.getEngine.getSession.getPeerCertificates)
177 } catch {
178 case e => {
179 // request a certificate from the user
180 sslh.setEnableRenegotiation(true)
181 r match {
182 case UserAgent(agent) if needAuth(agent) => sslh.getEngine.setNeedClientAuth(true)
183 case _ => sslh.getEngine.setWantClientAuth(true)
184 }
185
186 val future = sslh.handshake()
187 future.await(30000) //that's certainly way too long.
188 if (future.isDone) {
189 if (future.isSuccess) try {
190 Some(sslh.getEngine.getSession.getPeerCertificates)
191 } catch {
192 case e => None
193 } else {
194 None
195 }
196 } else {
197 None
198 }
199 }
200 }
201 case _ => None
202 }
So in summary: does anyone know if there is a way to avoid this behaviour in the TLS client?
And if not how do I put together a bug report, and to whom do I send it to?
Henry
On 28 Oct 2011, at 17:17, Oleg Kalnichevski wrote:
> On Fri, Oct 28, 2011 at 04:44:59PM +0200, Henry Story wrote:
>>
>> On 28 Oct 2011, at 16:01, Oleg Kalnichevski wrote:
>>
>>> On Fri, Oct 28, 2011 at 10:21:42AM +0200, Henry Story wrote:
>>>> I have a bit more support now in thinking that this is an issue with lack of support for TLS renegotiation.
>>>> I added the following code to my test, which calls a non TLS renegotiating server
>>>>
>>>> "testing client certs" should {
>>>> "connect to foafssl.org and ask for cert" in {
>>>> keyManager.setId("JoeLambda")
>>>> val foafssl = :/("foafssl.org",443)/"test/WebId" secure
>>>> val model = Http(foafssl as_model(baseURI(foafssl),TURTLE) )
>>>> model.write(System.out,TURTLE.jenaLang)
>>>> model.size() must_==10 //should be greater than, but anyway
>>>> }
>>>> }
>>>>
>>>> When I connect to foafssl.org - but any non TLS renegotiating server would do I believe - then the methods in my FlexiKeyManager get called in the order expected: namely first it gets asked for the aliases, then for a certificate for that alias, and finally for the private key for that alias. This does not happen when connecting to the server that does renegotiation.
>>>>
>>>> class FlexiKeyManager extends X509ExtendedKeyManager {
>>>> val keys = mutable.Map[String, Pair[Array[X509Certificate],PrivateKey]]()
>>>>
>>>> def addClientCert(alias: String,certs: Array[X509Certificate], privateKey: PrivateKey) {
>>>> keys.put(alias,Pair(certs,privateKey))
>>>> }
>>>>
>>>> var currentId: String = null
>>>>
>>>> def setId(alias: String) { currentId = if (keys.contains(alias)) alias else null }
>>>> def getClientAliases(keyType: String, issuers: Array[Principal]) =
>>>> if (currentId!=null) Array(currentId) else null
>>>> def chooseClientAlias(keyType: Array[String], issuers: Array[Principal], socket: Socket) =
>>>> currentId
>>>> def getServerAliases(keyType: String, issuers: Array[Principal]) = null
>>>> def chooseServerAlias(keyType: String, issuers: Array[Principal], socket: Socket) = ""
>>>> def getCertificateChain(alias: String) = keys.get(alias) match {
>>>> case Some(certNKey) => certNKey._1;
>>>> case None => null
>>>> }
>>>> def getPrivateKey(alias: String) = keys.get(alias).map(ck=>ck._2).getOrElse(null)
>>>>
>>>> override def chooseEngineClientAlias(keyType: Array[String], issuers: Array[Principal], engine: SSLEngine): String = currentId
>>>> }
>>>>
>>>>
>>>
>>> Hi Henry
>>>
>>> HttpClient has absolutely no control over TLS protocol aspects. It merely leverages TLS/SSL capabilities provided by Java JSSE. As far as I know the latest Java releases ship with support for the TLS renegotiation disabled. You can try enabling it using instructions below or consider using an alternative JSSE implementation.
>>>
>>> http://download.oracle.com/javase/6/docs/technotes/guides/security/jsse/JSSERefGuide.html#tlsRenegotiation
>>> http://java.sun.com/javase/javaseforbusiness/docs/TLSReadme.html
>>
>> Thanks Oleg,
>>
>> I am already running with those following properties
>>
>> -Dsun.security.ssl.allowUnsafeRenegotiation=true
>> -Dsun.security.ssl.allowLegacyHelloMessages=true
>>
>> set. I know I am doing this because I am also running the server in the same VM as the client here: the testing engine. I also just printed out all the system properties in the client code just to make sure, and they were still there.
>>
>> Btw, on the latest JVMs TLS renegotiation is back on by default
>>
>> http://download.oracle.com/javase/7/docs/technotes/guides/security/jsse/JSSERefGuide.html#descPhase2
>>
>> So could it be that there is a bug in the Java code? That would be interesting, because it would show that they had not tested their server with their own client libraries, which would be somewhat odd in fact. Perhaps I'll post a bug report then on the oracle web site.
>>
>> Henry
>>
>>
>
> I think that with SSL debug one one should be able to see whether or not the TLS renegotiation is enabled and whether or not the client attempts to renegotiate in case the support for renegotiation is active.
Thanks that was helpful.
>
> Oleg
>
>
>
>>>
>>> Oleg
>>>
>>>>
>>>>
>>>> On 28 Oct 2011, at 00:49, Henry Story wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I am working on a server that tries to ask the client for his X509 certificate only when it is sure that it will be needed. This can be done very neatly using TLS renegotiation: the server can analysing the HTTP request to see if action requested on the resource needs authentication at all. If so it requests a TLS renegotiations as show in this mini netty server written in one page of Scala [1].
>>>>>
>>>>> I am now trying to test this. Most desktop browsers accept some form of TLS renegotiation - except Opera 11 I think. But I am not sure that java http client does. I am using the dispatch scala wrapping of the httpclient, and so I am cling them this too.
>>>>>
>>>>> The code for these tests is here:
>>>>>
>>>>> https://dvcs.w3.org/hg/read-write-web/file/c0bf9b280888/src/test/scala/auth/CreateWebIDSpec.scala
>>>>>
>>>>> The test after line 234 does not return the right result. After a lot of stepping through code it occurred to me that perhaps httpclient does not do renegotiation. Perhaps I have not set it up properly to do this. But it could also be another issue. As it is late, I thought I'd ask before going to sleep.
>>>>>
>>>>> Thanks in advance,
>>>>>
>>>>> Henry
>>>>>
>>>>>
>>>>> [1] in the webid branch of the read-write-web project around line 64
>>>>> https://dvcs.w3.org/hg/read-write-web/file/9ca474c333e8/src/main/scala/netty/SslLoginTest.scala
>>>>> [2] http://dispatch.databinder.net/Dispatch.html
>>>>>
>>>>>
>>>>> Social Web Architect
>>>>> http://bblfish.net/
Re: resolution (partial): https client with TLS renegotiating server
Posted by Henry Story <he...@bblfish.net>.
I have posted a bug report on this on the openjdk mailing list
https://bugs.openjdk.java.net/show_bug.cgi?id=100213
I hope I make the problem clear there. Please vote for it if so.
Henry