You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@jclouds.apache.org by "Ka-Hing Cheung (JIRA)" <ji...@apache.org> on 2015/06/25 03:20:04 UTC
[jira] [Comment Edited] (JCLOUDS-905) Expect: 100-Continue + SSL =
timeout
[ https://issues.apache.org/jira/browse/JCLOUDS-905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600496#comment-14600496 ]
Ka-Hing Cheung edited comment on JCLOUDS-905 at 6/25/15 1:19 AM:
-----------------------------------------------------------------
I've hit a (maybe) similar problem today. I am trying to run S3BlobIntegrationLiveTest against a local S3 server, if I run the tests serially everything is fine, but if I run them in parallel, I hit an expect timeout. The backtrace when it's stuck is:
{code}
"pool-5-thread-10" #49 prio=5 os_prio=0 tid=0x00007fb9ccd99800 nid=0x176f runnable [0x00007fb94a550000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
- locked <0x00000000fb6b6e50> (a java.io.BufferedInputStream)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:818)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
at sun.net.www.protocol.http.HttpURLConnection.expect100Continue(HttpURLConnection.java:1192)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1296)
- locked <0x00000000fb6a8900> (a sun.net.www.protocol.http.HttpURLConnection)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1257)
- locked <0x00000000fb6a8900> (a sun.net.www.protocol.http.HttpURLConnection)
at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.writePayloadToConnection(JavaUrlHttpCommandExecutorService.java:294)
at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:170)
at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:64)
at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:95)
at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
at org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156)
at org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123)
at com.sun.proxy.$Proxy51.putObject(Unknown Source)
at org.jclouds.s3.blobstore.S3BlobStore.putBlob(S3BlobStore.java:283)
at org.jclouds.s3.blobstore.S3BlobStore.putBlob(S3BlobStore.java:257)
at org.jclouds.blobstore.integration.internal.BaseBlobIntegrationTest$1.call(BaseBlobIntegrationTest.java:138)
at org.jclouds.blobstore.integration.internal.BaseBlobIntegrationTest$1.call(BaseBlobIntegrationTest.java:132)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
This is with HTTP only (not HTTPS). Setting Constants.PROPERTY_STRIP_EXPECT_HEADER fixes it. From glancing at the code, it seems like JDK itself should be consuming the 100 response, so it's not possible to handle it from jclouds?
was (Author: khc):
I've hit a (maybe) similar problem today. I am trying to run S3BlobIntegrationLiveTest against a local S3 server, if I run the tests serially everything is fine, but if I run them in parallel, I hit an expect timeout. The backtrace when it's stuck is:
{code}
"pool-5-thread-10" #49 prio=5 os_prio=0 tid=0x00007fb9ccd99800 nid=0x176f runnable [0x00007fb94a550000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
- locked <0x00000000fb6b6e50> (a java.io.BufferedInputStream)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:818)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
at sun.net.www.protocol.http.HttpURLConnection.expect100Continue(HttpURLConnection.java:1192)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1296)
- locked <0x00000000fb6a8900> (a sun.net.www.protocol.http.HttpURLConnection)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1257)
- locked <0x00000000fb6a8900> (a sun.net.www.protocol.http.HttpURLConnection)
at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.writePayloadToConnection(JavaUrlHttpCommandExecutorService.java:294)
at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:170)
at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:64)
at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:95)
at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
at org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156)
at org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123)
at com.sun.proxy.$Proxy51.putObject(Unknown Source)
at org.jclouds.s3.blobstore.S3BlobStore.putBlob(S3BlobStore.java:283)
at org.jclouds.s3.blobstore.S3BlobStore.putBlob(S3BlobStore.java:257)
at org.jclouds.blobstore.integration.internal.BaseBlobIntegrationTest$1.call(BaseBlobIntegrationTest.java:138)
at org.jclouds.blobstore.integration.internal.BaseBlobIntegrationTest$1.call(BaseBlobIntegrationTest.java:132)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
This is with HTTP only (not HTTPS). From glancing at the code, it seems like JDK itself should be consuming the 100 response, so it's not possible to handle it from jclouds?
> Expect: 100-Continue + SSL = timeout
> ------------------------------------
>
> Key: JCLOUDS-905
> URL: https://issues.apache.org/jira/browse/JCLOUDS-905
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-blobstore, jclouds-core
> Affects Versions: 2.0.0, 1.9.0
> Reporter: Svetoslav Neykov
>
> A bug in the state logic of HttpsUrlConnection used by the default http driver will cause it to block and wait for headers at the wrong time when used with "Expect: 100-Continue" header. When triggered the soTimeout value is not applied on the socket which leads to indefinite block on a read call, only to be cancelled by a server timeout.
> The bug is triggered when using
> * Expect: 100-Continue
> * on Java 7 or Java 8
> * on SSL connection
> * connection reuse is enabled (by default)
> * the stream returned by the call is not closed by the caller
> * but instead is closed by the GC
> Closing the socket by GC trips the state logic of the "Expect: 100-Continue" feature, eventually leading to failure to parse the http stream coming from the server.
> The problem is not specific to objectstore or even jclouds, but is triggered here because of the use of "Expect: 100-Continue". Seems like using it in combination with SSL connections will inevitably lead to failed requests.
> A jclouds test case which reproduces the behaviour can be found at https://github.com/apache/incubator-brooklyn/blob/master/locations/jclouds/src/test/java/brooklyn/entity/rebind/persister/jclouds/JcloudsExpect100ContinueTest.java
> Note that if wire logging is enabled the problem can't be reproduced (closing streams explicitly?)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)