[jira] [Comment Edited] (OAK-8552) Minimize network calls required when creating a direct download URI

["Amit Jain (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16910176#comment-16910176 ] 

Amit Jain edited comment on OAK-8552 at 8/19/19 5:26 AM:
---------------------------------------------------------

[~mattvryan] 
1. Essentially, I guess what you are saying is that the n/w call should only be one. I don't know enough about the Azure blob APIs but maybe we can see if we can substitute the below call so that we don't need the additional download attributes call later.
{code}
CloudBlockBlob blob = getAzureContainer().getBlockBlobReference(key);
{code}

bq. But all that is really needed in this case is the reference, which can be obtained from the back end directly using the blob id - no network calls required.  
Yes it still makes a call for existence in the DataStoreBlobStore#getReference() which can be potentially removed. Not sure if there are cases when the call receives a blob id but is not available in the backend. But in such a case that will lead to situation 2 subsequently, is it not?

bq. Furthermore, the reason we are even trying to get the reference in the first place is to determine if this blob is stored inline or not.  Maybe there is a better way to determine this.
I don't think there's much we can do here. The storage inline or not is an internal implementation of the blob store and the code in oak-store-spi would have to be aware of such details to check the same.

2. Here it seems there's no alternative but to check for existence. Is that right? Also, the test that you conducted removed the existence check here only or also in the condition observed in 1.? If its just the the existence check that is newly added then that is the major cause of the slow down ~ 250 times (147 s Vs 40000). Not sure how much changing 1 will help according to your test.



was (Author: amitjain):
[~mattvryan] 
1. Essentially, I guess what you are saying is that the n/w call should only be one. I don't know enough about the Azure blob APIs but maybe we can see if we can substitute the below call so that we don't need the additional download attributes call later.
{code}
CloudBlockBlob blob = getAzureContainer().getBlockBlobReference(key);
{code}

bq. But all that is really needed in this case is the reference, which can be obtained from the back end directly using the blob id - no network calls required.  
Yes it still makes a call for existence in the DataStoreBlobStore#getReference() which can be potentially removed. Not sure if there are cases when the call receives a blob id but is not available in the backend. But in such a case that will lead to situation 2, is it not?

bq. Furthermore, the reason we are even trying to get the reference in the first place is to determine if this blob is stored inline or not.  Maybe there is a better way to determine this.
I don't think there's much we can do here. The storage inline or not is an internal implementation of the blob store and the code in oak-store-spi would have to be aware of such details to check the same.

2. Here it seems there's no alternative but to check for existence. Is that right? Also, the test that you conducted removed the existence check here only or also in the condition observed in 1.? If its just the the existence check that is newly added then that is the major cause of the slow down ~ 250 times (147 s Vs 40000). Not sure how much changing 1 will help according to your test.


> Minimize network calls required when creating a direct download URI
> -------------------------------------------------------------------
>
>                 Key: OAK-8552
>                 URL: https://issues.apache.org/jira/browse/OAK-8552
>             Project: Jackrabbit Oak
>          Issue Type: Sub-task
>          Components: blob-cloud, blob-cloud-azure
>            Reporter: Matt Ryan
>            Priority: Major
>
> We need to isolate and try to optimize network calls required to create a direct download URI.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)