You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@jclouds.apache.org by "Ignasi Barrera (JIRA)" <ji...@apache.org> on 2018/10/18 09:24:00 UTC
[jira] [Commented] (JCLOUDS-1461) During multiple chunk upload on
Google Cloud Storage, errors are logged whereas upload is successful
[ https://issues.apache.org/jira/browse/JCLOUDS-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654914#comment-16654914 ]
Ignasi Barrera commented on JCLOUDS-1461:
-----------------------------------------
By default, jclouds tries to retry requests upon failure, or to follow redirects if the response is a 3xx and the requests are considered "retryable".
The problem here seems to be an incorrect use of the 308 response code by Google Cloud. Reading the docs it appears they are using it to inform that the user needs to continue uploading the file, but that does not really conform to the spec ([RFC-7538|https://tools.ietf.org/html/rfc7538]). According to it, the server [should generate a Location header|https://tools.ietf.org/html/rfc7538#section-3], just as any other 3xx responses, but given the provided error log, that header is missing.
This is clearly a 3xx (redirect) response code that is not aimed to be used as a regular redirect, so we're here with a situation where jclouds should have to deal with an incorrect implementation of the standard. Probably a way to do that is to implement a custom redirection handler for google cloud that just ignores 308 redirects, and delegates to the default one otherwise.
Looking at the code, [there is already a hanler that does that|https://github.com/jclouds/jclouds/blob/master/providers/google-cloud-storage/src/main/java/org/jclouds/googlecloudstorage/handlers/GoogleCloudStorageRedirectRetryHandler.java], but it is [not enabled|https://github.com/jclouds/jclouds/blob/master/providers/google-cloud-storage/src/main/java/org/jclouds/googlecloudstorage/config/GoogleCloudStorageHttpApiModule.java#L59]. Fixing the issue should be as trivial as configuring it as a retry handler there. [~cinlloc] would you like to send a pull request with this fix?
> During multiple chunk upload on Google Cloud Storage, errors are logged whereas upload is successful
> ----------------------------------------------------------------------------------------------------
>
> Key: JCLOUDS-1461
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1461
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-blobstore, jclouds-labs-google
> Affects Versions: 2.1.1
> Environment: CentOS Linux release 7.4.1708 (Core)
> Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
> Reporter: Clement Collin
> Priority: Minor
> Labels: google-cloud-storage
>
> *Step to reproduce*:
> Use _org.jclouds.googlecloudstorage.domain.ResumableUpload_ API to upload a file into multiple chunks onto Google Compute Engine.
> e.g.:
> {noformat}
> ResumableUpload resumableUpload = googleCloudStorageApi.getResumableUploadApi().chunkUpload(bucket, uploadId, MediaType.OCTET_STREAM.toString(), range, "bytes " + rangeFrom + "-" + rangeTo + "/" + diskFile.length(), new ByteSourcePayload(ByteSource.wrap(buf)));{noformat}
> Each request returns a _308/HTTP_PERMANENT_REDIRECT_ which is expected behavior.
> Cf. [https://cloud.google.com/storage/docs/json_api/v1/how-tos/resumable-upload:]
> {quote}If the request succeeds, the server responds with 308 Resume Incomplete, along with a Range header that identifies the total number of bytes that have been stored so far.
> {quote}
> At the end, the last chunk answer is a _200_ which confirm the upload went well.
> *Issue*:
> For every chunk upload 308 answer it receives, JClouds logs an error which seems inappropriate:
> {noformat}
> ERROR (SLF4JLogger.java:RedirectionRetryHandler:88) - Cannot retry after redirect, no host header: [method=org.jclouds.googlecloudstorage.features.ResumableUploadApi.public abstract org.jclouds.googlecloudstorage.domain.ResumableUpload org.jclouds.googlecloudstorage.features.ResumableUploadApi.chunkUpload(java.lang.String,java.lang.String,java.lang.String,java.lang.Long,java.lang.String,org.jclouds.io.Payload)[myuser, AEnB2UqFW60Vyk9CRr2yB3gHG2h3dszv1tkif51NLcrhsADPqZF3sW6gs7fIf9HwNx8SIARKqglbxaVAph6TxdEVo7aO3LNQJJS-Iw3-qr_uDd-f8qxh90o, application/octet-stream, 4194304, bytes 343932928-348127231/351108355, [content=true, contentMetadata=[cacheControl=null, contentDisposition=null, contentEncoding=null, contentLanguage=null, contentLength=4194304, contentMD5=null, contentType=application/octet-stream, expires=null], written=false, isSensitive=false]], request=PUT https://www.googleapis.com/upload/storage/v1/b/myuser/o?uploadType=resumable&upload_id=AEnB2UqFW60Vyk9CRr2yB3gHG2h3dszv1tkif51NLcrhsADPqZF3sW6gs7fIf9HwNx8SIARKqglbxaVAph6TxdEVo7aO3LNQJJS-Iw3-qr_uDd-f8qxh90o HTTP/1.1]{noformat}
> *Workaround*:
> Can configure log levels to hide this error.
> *More info:*
> * Log of two last HTTP requests
>
> {noformat}
> 2018-10-16 10:45:51,164 gce-publish-connector DEBUG (SLF4JLogger.java:InvokeHttpMethod:56) - >> invoking Object:Upload
> 2018-10-16 10:45:51,164 gce-publish-connector DEBUG (SLF4JLogger.java:JavaUrlHttpCommandExecutorService:56) - Sending request -1294939191: PUT https://www.googleapis.com/upload/storage/v1/b/myuser/o?uploadType=resumable&upload_id=AEnB2UqFW60Vyk9CRr2yB3gHG2h3dszv1tkif51NLcrhsADPqZF3sW6gs7fIf9HwNx8SIARKqglbxaVAph6TxdEVo7aO3LNQJJS-Iw3-qr_uDd-f8qxh90o HTTP/1.1
> 2018-10-16 10:45:52,234 gce-publish-connector DEBUG (SLF4JLogger.java:JavaUrlHttpCommandExecutorService:56) - Receiving response -1294939191: HTTP/1.1 308 Resume Incomplete
> 2018-10-16 10:45:52,240 gce-publish-connector ERROR (SLF4JLogger.java:RedirectionRetryHandler:88) - Cannot retry after redirect, no host header: [method=org.jclouds.googlecloudstorage.features.ResumableUploadApi.public abstract org.jclouds.googlecloudstorage.domain.ResumableUpload org.jclouds.googlecloudstorage.features.ResumableUploadApi.chunkUpload(java.lang.String,java.lang.String,java.lang.String,java.lang.Long,java.lang.String,org.jclouds.io.Payload)[myuser, AEnB2UqFW60Vyk9CRr2yB3gHG2h3dszv1tkif51NLcrhsADPqZF3sW6gs7fIf9HwNx8SIARKqglbxaVAph6TxdEVo7aO3LNQJJS-Iw3-qr_uDd-f8qxh90o, application/octet-stream, 4194304, bytes 343932928-348127231/351108355, [content=true, contentMetadata=[cacheControl=null, contentDisposition=null, contentEncoding=null, contentLanguage=null, contentLength=4194304, contentMD5=null, contentType=application/octet-stream, expires=null], written=false, isSensitive=false]], request=PUT https://www.googleapis.com/upload/storage/v1/b/myuser/o?uploadType=resumable&upload_id=AEnB2UqFW60Vyk9CRr2yB3gHG2h3dszv1tkif51NLcrhsADPqZF3sW6gs7fIf9HwNx8SIARKqglbxaVAph6TxdEVo7aO3LNQJJS-Iw3-qr_uDd-f8qxh90o HTTP/1.1]
> 2018-10-16 10:45:52,259 gce-publish-connector DEBUG (SLF4JLogger.java:InvokeHttpMethod:56) - >> invoking Object:Upload
> 2018-10-16 10:45:52,259 gce-publish-connector DEBUG (SLF4JLogger.java:JavaUrlHttpCommandExecutorService:56) - Sending request -822896246: PUT https://www.googleapis.com/upload/storage/v1/b/myuser/o?uploadType=resumable&upload_id=AEnB2UqFW60Vyk9CRr2yB3gHG2h3dszv1tkif51NLcrhsADPqZF3sW6gs7fIf9HwNx8SIARKqglbxaVAph6TxdEVo7aO3LNQJJS-Iw3-qr_uDd-f8qxh90o HTTP/1.1
> 2018-10-16 10:45:53,303 gce-publish-connector DEBUG (SLF4JLogger.java:JavaUrlHttpCommandExecutorService:56) - Receiving response -822896246: HTTP/1.1 200 OK
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)