You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@jclouds.apache.org by "Roman Novokhatsky (JIRA)" <ji...@apache.org> on 2015/07/28 16:40:04 UTC
[jira] [Updated] (JCLOUDS-977) Wrong MD5 is being set to chunk on
multipart upload
[ https://issues.apache.org/jira/browse/JCLOUDS-977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roman Novokhatsky updated JCLOUDS-977:
--------------------------------------
Description:
1. Construct a blob:
{code:java}
Blob blob = blobStore.blobBuilder("blob-name")
.payload(sourceBlob.getPayload().openStream())
.contentLength(sourceBlob.getMetadata().getContentMetadata().getContentLength())
.contentMD5(sourceBlob.getMetadata().getContentMetadata().getContentMD5AsHashCode())
.build();
{code}
2. Initiate upload:
{code:java}
blobStore.putBlob("container-name", blob, PutOptions.Builder.multipart())
{code}
3. Get an exception:
{code}
org.jclouds.aws.AWSResponseException: request PUT https://xxxxxxx.s3.amazonaws.com/file.wmv?partNumber=1&uploadId=XPTNg5O8nmxp.PTmX9jbNJ.RFBOqozKRzAcso8O7oEqHkuadwRgW_aiyuCgShoVB316pToTcUDo_FKXXrEdH5A-- HTTP/1.1 failed with code 400, error: AWSError{requestId='01DA43F9D952570F', requestToken='W1uccq2LxM3BMBeNhgz1aYLglTtOhqBtJm5tvcaAzhPZCZtegnFN96oK4JzBEy3LqKP/sBy4cSs=', code='BadDigest', message='The Content-MD5 you specified did not match what we received.', context='{CalculatedDigest=4MVdqDCufuGS2c4clr9SPg==, ExpectedDigest=e8s07WDHzryJUU7NYyE5GA==, HostId=W1uccq2LxM3BMBeNhgz1aYLglTtOhqBtJm5tvcaAzhPZCZtegnFN96oK4JzBEy3LqKP/sBy4cSs=}'}
at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:75)
at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:65)
at org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:132)
at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:101)
at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
at org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156)
at org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123)
at com.sun.proxy.$Proxy48.uploadPart(Unknown Source)
at org.jclouds.s3.blobstore.strategy.internal.SequentialMultipartUploadStrategy.prepareUploadPart(SequentialMultipartUploadStrategy.java:111)
at org.jclouds.s3.blobstore.strategy.internal.SequentialMultipartUploadStrategy.execute(SequentialMultipartUploadStrategy.java:93)
at org.jclouds.aws.s3.blobstore.AWSS3BlobStore.putBlob(AWSS3BlobStore.java:87)
{code}
Seems that the error is caused by {{org.jclouds.io.ContentMetadataBuilder.contentMD5(HashCode)}} method, which ignores any null hashcode, and the following code in {{org.jclouds.io.internal.BasePayloadSlicer.InputStreamPayloadIterator.createPayload(byte[])}} can't replace the checksum with null:
{code:java}
ContentMetadata cm = metaData.toBuilder().contentLength((long)content.length).contentMD5((HashCode) null).build();
payload.setContentMetadata(BaseMutableContentMetadata.fromContentMetadata(cm));
{code}
was:
1. Construct a blob:
{code:java}
Blob blob = blobStore.blobBuilder("blob-name")
.payload(sourceBlob.getPayload().openStream())
.contentLength(sourceBlob.getMetadata().getContentMetadata().getContentLength())
.contentMD5(sourceBlob.getMetadata().getContentMetadata().getContentMD5AsHashCode())
.build();
{code}
2. Initiate upload:
{code:java}
blobStore.putBlob("container-name", blob, PutOptions.Builder.multipart())
{code}
3. Get an exception:
{code}
org.jclouds.aws.AWSResponseException: request PUT https://xxxxxxx.s3.amazonaws.com/file.wmv?partNumber=1&uploadId=XPTNg5O8nmxp.PTmX9jbNJ.RFBOqozKRzAcso8O7oEqHkuadwRgW_aiyuCgShoVB316pToTcUDo_FKXXrEdH5A-- HTTP/1.1 failed with code 400, error: AWSError{requestId='01DA43F9D952570F', requestToken='W1uccq2LxM3BMBeNhgz1aYLglTtOhqBtJm5tvcaAzhPZCZtegnFN96oK4JzBEy3LqKP/sBy4cSs=', code='BadDigest', message='The Content-MD5 you specified did not match what we received.', context='{CalculatedDigest=4MVdqDCufuGS2c4clr9SPg==, ExpectedDigest=e8s07WDHzryJUU7NYyE5GA==, HostId=W1uccq2LxM3BMBeNhgz1aYLglTtOhqBtJm5tvcaAzhPZCZtegnFN96oK4JzBEy3LqKP/sBy4cSs=}'}
at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:75)
at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:65)
at org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:132)
at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:101)
at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
at org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156)
at org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123)
at com.sun.proxy.$Proxy48.uploadPart(Unknown Source)
at org.jclouds.s3.blobstore.strategy.internal.SequentialMultipartUploadStrategy.prepareUploadPart(SequentialMultipartUploadStrategy.java:111)
at org.jclouds.s3.blobstore.strategy.internal.SequentialMultipartUploadStrategy.execute(SequentialMultipartUploadStrategy.java:93)
at org.jclouds.aws.s3.blobstore.AWSS3BlobStore.putBlob(AWSS3BlobStore.java:87)
{code}
Seems that the error is caused by {{org.jclouds.io.ContentMetadataBuilder.contentMD5(HashCode)}} method, which ignores any null hashcode, and the following code in {{org.jclouds.io.internal.BasePayloadSlicer.InputStreamPayloadIterator.createPayload(byte[])}} can't replace the checksum that is being copied from whole blob metadata:
{code:java}
ContentMetadata cm = metaData.toBuilder().contentLength((long)content.length).contentMD5((HashCode) null).build();
payload.setContentMetadata(BaseMutableContentMetadata.fromContentMetadata(cm));
{code}
> Wrong MD5 is being set to chunk on multipart upload
> ---------------------------------------------------
>
> Key: JCLOUDS-977
> URL: https://issues.apache.org/jira/browse/JCLOUDS-977
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-blobstore
> Affects Versions: 1.9.0
> Reporter: Roman Novokhatsky
>
> 1. Construct a blob:
> {code:java}
> Blob blob = blobStore.blobBuilder("blob-name")
> .payload(sourceBlob.getPayload().openStream())
> .contentLength(sourceBlob.getMetadata().getContentMetadata().getContentLength())
> .contentMD5(sourceBlob.getMetadata().getContentMetadata().getContentMD5AsHashCode())
> .build();
> {code}
> 2. Initiate upload:
> {code:java}
> blobStore.putBlob("container-name", blob, PutOptions.Builder.multipart())
> {code}
> 3. Get an exception:
> {code}
> org.jclouds.aws.AWSResponseException: request PUT https://xxxxxxx.s3.amazonaws.com/file.wmv?partNumber=1&uploadId=XPTNg5O8nmxp.PTmX9jbNJ.RFBOqozKRzAcso8O7oEqHkuadwRgW_aiyuCgShoVB316pToTcUDo_FKXXrEdH5A-- HTTP/1.1 failed with code 400, error: AWSError{requestId='01DA43F9D952570F', requestToken='W1uccq2LxM3BMBeNhgz1aYLglTtOhqBtJm5tvcaAzhPZCZtegnFN96oK4JzBEy3LqKP/sBy4cSs=', code='BadDigest', message='The Content-MD5 you specified did not match what we received.', context='{CalculatedDigest=4MVdqDCufuGS2c4clr9SPg==, ExpectedDigest=e8s07WDHzryJUU7NYyE5GA==, HostId=W1uccq2LxM3BMBeNhgz1aYLglTtOhqBtJm5tvcaAzhPZCZtegnFN96oK4JzBEy3LqKP/sBy4cSs=}'}
> at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:75)
> at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:65)
> at org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:132)
> at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:101)
> at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
> at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
> at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
> at org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156)
> at org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123)
> at com.sun.proxy.$Proxy48.uploadPart(Unknown Source)
> at org.jclouds.s3.blobstore.strategy.internal.SequentialMultipartUploadStrategy.prepareUploadPart(SequentialMultipartUploadStrategy.java:111)
> at org.jclouds.s3.blobstore.strategy.internal.SequentialMultipartUploadStrategy.execute(SequentialMultipartUploadStrategy.java:93)
> at org.jclouds.aws.s3.blobstore.AWSS3BlobStore.putBlob(AWSS3BlobStore.java:87)
> {code}
> Seems that the error is caused by {{org.jclouds.io.ContentMetadataBuilder.contentMD5(HashCode)}} method, which ignores any null hashcode, and the following code in {{org.jclouds.io.internal.BasePayloadSlicer.InputStreamPayloadIterator.createPayload(byte[])}} can't replace the checksum with null:
> {code:java}
> ContentMetadata cm = metaData.toBuilder().contentLength((long)content.length).contentMD5((HashCode) null).build();
> payload.setContentMetadata(BaseMutableContentMetadata.fromContentMetadata(cm));
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)