Fwd: Azure Storage Service

[Kriti <ta...@gmail.com>]

Hi,

I was trying using Azure Storage Service operations but facing problems
uploading a Page Blob.

Requirement: Uploading a VHD, creating a custom image & hence a VM out of
it (using REST API/JClouds support).

Blocking Issue: Using Storage Account name & primary access key as
credentials, I successfully created a storage container but while creating
a PAGE BLOB, I was getting AzureStorageResponseException.

Doing something like this:

        AzureBlobClient blobClient =
AzureBlobClient.class.cast(context.unwrap(AzureBlobApiMetadata.CONTEXT_TOKEN).getApi());
        AzureBlob newBlob = blobClient.newBlob();
        File f = new File("C:\\sample.txt");
        newBlob.getProperties().setName("<blobname>");
        newBlob.setPayload(f);
        newBlob.getProperties().setContainer("<containername>");
        MutableBlobPropertiesImpl properties = (MutableBlobPropertiesImpl)
newBlob.getProperties();
        properties.setType(BlobType.PAGE_BLOB);
        blobClient.putBlob("<containername>", newBlob);

Is there a way to upload a file >64MB using PutBlob(Blob Type: PAGE BLOB)
operation ?

Is PUT PAGE(Windows Azure REST API) support provided by JClouds ? Can I get
a sample implementation/examples for the same.

Any help in this regard would be highly appreciable.

Thanks,
Kriti

Re: Fwd: Azure Storage Service

[Andrew Phillips <an...@apache.org>]

> Unfortunately JClouds only supports block blobs larger than 64MB; not page
> blobs [1]. This is mostly because support for large blob types was
> initially written for managing large files like zip files; not the
> random-seek VHD files which have to be aligned to 512 byte boundaries[2].

Thanks for the explanation, John! Azure blocks vs. pages - learned  
something new again there...

ap

Re: Fwd: Azure Storage Service

[Kriti <ta...@gmail.com>]

Thanks Andrew for the quick reply, I had a look at the commits for the
multi-part upload support for Azure blobs, will get a hint from it but that
is not exactly what we were looking forward. I will get back to you with
more crisp requirement if needed.

Thanks John for a detailed description & help.

Regards,
Kriti




On Mon, Sep 2, 2013 at 10:12 AM, John Kew <jo...@socrata.com> wrote:

> Kriti;
>
> Unfortunately JClouds only supports block blobs larger than 64MB; not page
> blobs [1]. This is mostly because support for large blob types was
> initially written for managing large files like zip files; not the
> random-seek VHD files which have to be aligned to 512 byte boundaries[2].
>
> The work required to get page blobs in place is not technically hard; but
> there may be a bit of work in making sure the JClouds API for azure blob
> remains clean.
>
> Personally I'd be pretty interested in this support; but my work does not
> have an immediate need for it. If you want to take a stab at it I would be
> more than willing to help out.
>
> Also; if you just need to copy vhds between storage accounts you can use
> the async copy API[3]. The only reason I mention this is that my company
> has many VHDs which largely stay within Azure; but we had the need to move
> them from storage account to storage account. You can do this with shared
> access keys[4].
>
> -John
>
> [1] Block vrs Page Blobs:
> http://msdn.microsoft.com/en-us/library/windowsazure/ee691964.aspx
> [2] Blob API:
> http://msdn.microsoft.com/en-us/library/windowsazure/ee691975.aspx
> [3] Copy API:
> http://msdn.microsoft.com/en-us/library/windowsazure/dd894037.aspx
> [4] Shared Access Keys:
> http://msdn.microsoft.com/en-us/library/windowsazure/ee395415.aspx
>
>
> On Sun, Sep 1, 2013 at 6:31 AM, Andrew Phillips <an...@apache.org>wrote:
>
>> Hi Kriti
>>
>> Could you have a look at JCLOUDS-161 [1] and the related commits? Large
>> blob should be supported on master and 1.6.2-incubating - which version(s)
>> of jclouds are you using?
>>
>> Regards
>>
>> ap
>>
>> [1] https://issues.apache.org/**jira/browse/JCLOUDS-161<https://issues.apache.org/jira/browse/JCLOUDS-161>
>>
>>
>

Re: Fwd: Azure Storage Service

[John Kew <jo...@socrata.com>]

Kriti;

Unfortunately JClouds only supports block blobs larger than 64MB; not page
blobs [1]. This is mostly because support for large blob types was
initially written for managing large files like zip files; not the
random-seek VHD files which have to be aligned to 512 byte boundaries[2].

The work required to get page blobs in place is not technically hard; but
there may be a bit of work in making sure the JClouds API for azure blob
remains clean.

Personally I'd be pretty interested in this support; but my work does not
have an immediate need for it. If you want to take a stab at it I would be
more than willing to help out.

Also; if you just need to copy vhds between storage accounts you can use
the async copy API[3]. The only reason I mention this is that my company
has many VHDs which largely stay within Azure; but we had the need to move
them from storage account to storage account. You can do this with shared
access keys[4].

-John

[1] Block vrs Page Blobs:
http://msdn.microsoft.com/en-us/library/windowsazure/ee691964.aspx
[2] Blob API:
http://msdn.microsoft.com/en-us/library/windowsazure/ee691975.aspx
[3] Copy API:
http://msdn.microsoft.com/en-us/library/windowsazure/dd894037.aspx
[4] Shared Access Keys:
http://msdn.microsoft.com/en-us/library/windowsazure/ee395415.aspx


On Sun, Sep 1, 2013 at 6:31 AM, Andrew Phillips <an...@apache.org> wrote:

> Hi Kriti
>
> Could you have a look at JCLOUDS-161 [1] and the related commits? Large
> blob should be supported on master and 1.6.2-incubating - which version(s)
> of jclouds are you using?
>
> Regards
>
> ap
>
> [1] https://issues.apache.org/**jira/browse/JCLOUDS-161<https://issues.apache.org/jira/browse/JCLOUDS-161>
>
>

Re: Fwd: Azure Storage Service

[Andrew Phillips <an...@apache.org>]

Hi Kriti

Could you have a look at JCLOUDS-161 [1] and the related commits?  
Large blob should be supported on master and 1.6.2-incubating - which  
version(s) of jclouds are you using?

Regards

ap

[1] https://issues.apache.org/jira/browse/JCLOUDS-161