You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@jclouds.apache.org by "Andrew Gaul (JIRA)" <ji...@apache.org> on 2018/01/09 21:00:04 UTC
[jira] [Resolved] (JCLOUDS-1368) GCS cannot upload multipart file
larger than 1 GB
[ https://issues.apache.org/jira/browse/JCLOUDS-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Gaul resolved JCLOUDS-1368.
----------------------------------
Resolution: Fixed
Fix Version/s: 2.0.4
2.1.0
> GCS cannot upload multipart file larger than 1 GB
> -------------------------------------------------
>
> Key: JCLOUDS-1368
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1368
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-blobstore
> Affects Versions: 2.0.3
> Reporter: Andrew Gaul
> Assignee: Andrew Gaul
> Labels: google-cloud-storage
> Fix For: 2.1.0, 2.0.4
>
>
> When uploading a file:
> {code:java}
> long size = 33L * (32L * 1024L * 1024L);
> ByteSource byteSource = TestUtils.randomByteSource().slice(0, size);
> blobStore.createContainerInLocation(null, containerName);
> Blob blob = blobStore.blobBuilder(blobName)
> .payload(byteSource)
> .contentLength(size)
> .build();
> blobStore.putBlob(containerName, blob, PutOptions.Builder.multipart());
> {code}
> I see an error:
> {noformat}
> org.jclouds.http.HttpResponseException: command: POST https://www.googleapis.com/storage/v1/b/gaul-blobstore0/o/large-blob/compose HTTP/1.1 failed with response: HTTP/1.1 400 Bad Request; content: [{
> "error": {
> "errors": [
> {
> "domain": "global",
> "reason": "invalid",
> "message": "The number of source components provided (33) exceeds the maximum (32)"
> }
> ],
> "code": 400,
> "message": "The number of source components provided (33) exceeds the maximum (32)"
> }
> }
> ]
> {noformat}
> This is due to the wonky {{MultipartUploadSlicingAlgorithm}} slicing into (31) 32 MB parts then creating a remainder 64 MB part that {{BaseBlobStore.putMultipartBlob}} does not account for. This interface seems stupid and better to create (32) 33 MB parts. Originally reported at:
> https://stackoverflow.com/questions/47048264/the-number-of-source-components-provided-78-exceeds-the-maximum-32
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)