[jira] [Commented] (OAK-12165) Review Part Size Values for Azure Blob Upload (v12)

["Dikran Seropian (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-12165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18068012#comment-18068012 ] 

Dikran Seropian commented on OAK-12165:
---------------------------------------

Those values should also be externalized for more flexible operational management and tuning.

> Review Part Size Values for Azure Blob Upload (v12)
> ---------------------------------------------------
>
>                 Key: OAK-12165
>                 URL: https://issues.apache.org/jira/browse/OAK-12165
>             Project: Jackrabbit Oak
>          Issue Type: Task
>          Components: blob-cloud-azure
>    Affects Versions: 1.92.0
>            Reporter: Joerg Hoh
>            Priority: Major
>
> We should revisit the actual values for the 2 constants
> * AZURE_BLOB_MIN_MULTIPART_UPLOAD_PART_SIZE (256k)
> * AZURE_BLOB_MAX_MULTIPART_UPLOAD_PART_SIZE (4G)
> Our implementation was relying on these values and started to allocate up to 4G of a byte[] array before sending the actual buffer to the blobstore (using a simple HttpUrlConnection), which blew the heap. For that we should revisit the value of AZURE_BLOB_MAX_MULTIPART_UPLOAD_PART_SIZE.
> AZURE_BLOB_MIN_MULTIPART_UPLOAD_PART_SIZE is more important, as when you use this value to upload a large binary, the implementation will return with a large collection of URIs, which can pose a problem on its own (we calculated that we would get 500k DownloadURIs when we want to upload a 100G file.
> Where do these values come from and why were they changed so dramatically from the actual values used with V8 (min=10M, max=100M), which were working very well for us (see also OAK-12164)?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)