You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@jclouds.apache.org by John Calcote <jo...@gmail.com> on 2020/05/18 20:00:16 UTC

Range download

I'm trying to create a sort of multi-part download mechanism using jclouds download with a GetOption of "range". I'd like to just download 5MB ranges until I run out of file to download. The doc on the use of this option is a bit sparse. Can someone help me understand the semantics of download with a range option? 

Here's what I'd like to do:

long start = 0;
long size = 5 * 1024 * 1024;
long length = size;

while (length < size) {
    long limit = start + size
    Blob blob = getBlob(container, key, range(start, limit - 1));
    ContentMetadata meta = blob.getMetadata().getContentMetadata();
    long length = meta.getContentLength();

    // stream 'length' bytes into a buffer and process the buffer here

    start = limit;
}

Assumptions:
1) content length will return the actual size downloaded if it's less than the range size specified.
2) It's OK to request a range of bytes that's outside the file extant - you'll just get back a length of zero.

Anything I'm missing?

Thanks in advance,
John

Re: Range download

Posted by Andrew Gaul <ga...@apache.org>.
On Mon, May 18, 2020 at 08:00:16PM -0000, John Calcote wrote:
> I'm trying to create a sort of multi-part download mechanism using jclouds download with a GetOption of "range". I'd like to just download 5MB ranges until I run out of file to download. The doc on the use of this option is a bit sparse. Can someone help me understand the semantics of download with a range option? 
> 
> Here's what I'd like to do:
> 
> long start = 0;
> long size = 5 * 1024 * 1024;
> long length = size;
> 
> while (length < size) {
>     long limit = start + size
>     Blob blob = getBlob(container, key, range(start, limit - 1));
>     ContentMetadata meta = blob.getMetadata().getContentMetadata();
>     long length = meta.getContentLength();
> 
>     // stream 'length' bytes into a buffer and process the buffer here
> 
>     start = limit;
> }
> 
> Assumptions:
> 1) content length will return the actual size downloaded if it's less than the range size specified.
> 2) It's OK to request a range of bytes that's outside the file extant - you'll just get back a length of zero.

BlobStore.getBlob takes an optional argument that you can populate via:

    new GetOptions().range(start, size)

Note that this range is inclusive.

-- 
Andrew Gaul
http://gaul.org/

Re: Range download

Posted by John Calcote <jo...@gmail.com>.

On 2020/05/18 20:00:16, John Calcote <jo...@gmail.com> wrote: 

> while (length < size) {
Oops! Should have been:

while (length >= size) {