[jira] [Commented] (JCLOUDS-1606) Cannot upload more than 32 parts to GCS

["Lari Hotari (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/JCLOUDS-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17583973#comment-17583973 ] 

Lari Hotari commented on JCLOUDS-1606:
--------------------------------------

It looks like the slicing algorithm creates 33 parts at the moment. This would have to be changed so that GCS has the limit of 10000 parts. Composition should be used to compose up to 32 parts at a time.

> Cannot upload more than 32 parts to GCS
> ---------------------------------------
>
>                 Key: JCLOUDS-1606
>                 URL: https://issues.apache.org/jira/browse/JCLOUDS-1606
>             Project: jclouds
>          Issue Type: Bug
>          Components: jclouds-blobstore
>    Affects Versions: 2.5.0
>            Reporter: Lari Hotari
>            Priority: Major
>
> There's currently a limitation in JClouds that it cannot upload more than 32 parts to GCS.
> {code:java}
> org.jclouds.http.HttpResponseException: command: POST https://www.googleapis.com/storage/v1/b/somebucket/o/ff553922-1fa3-4ceb-abcd-60106603b5c8-object-123456/compose HTTP/1.1 failed with response: HTTP/1.1 400 Bad Request; content: [{
>   "error": {
>     "code": 400,
>     "message": "The number of source components provided (35) exceeds the maximum (32)",
>     "errors": [
>       {
>         "message": "The number of source components provided (35) exceeds the maximum (32)",
>         "domain": "global",
>         "reason": "invalid"
>       }
>     ]
>   }
> } {code}
> The limitation of 32 parts is per API call to the compose endpoint.
>  
> When there are more than 32 parts, the endpoint should be called multiple times. The total limit is 10000 parts in GCS.
> [https://cloud.google.com/storage/docs/composite-objects]
>  
> {quote}When you perform a composition:
>  * The source objects are unaffected.
>  * You can use between {*}1 and 32 source objects{*}.
>  * {*}Source objects can themselves be composite objects{*}.{quote}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)