Real Multi-Region BlobStore

[Adrian Cole <ad...@gmail.com>]

TL;DR;  At the planning meeting, gaul mentioned that lack of real
multi-region support in BlobStore is becoming a limiting factor for their
customers.  I've sketched a design that builds on what's already been done,
clarifying some points.  I plan to work on this over the next few days, so
please pitch in, if you have any opinions.

https://issues.apache.org/jira/browse/JCLOUDS-299

-A

Description pasted for the lazy:

For effective multi-region apps, we need to both expose which regions are
available, and also continue isolating a BlobStore to a specific one. Using
this model, users can have predictable performance (ex. one BlobStore
command won't cross regions), and isolation (one down region won't affect
the BlobStore in use).

The style of exposing an api scoped to region is also something we've
practiced for well over a year, at the provider api level.

ex. SwiftApi.getObjectApiForRegion("foo") style is commonplace now, even if
not yet in the "View" interfaces such as BlobStore.

During the last planning meeting, Maginatics (via andrew gaul) raised
supporting multiple regions in BlobStore is becoming a must have. Users
need to interact with multiple regions within Rackspace and OpenStack, for
example, and these users may not know the magic region ids, nor desire
maintaining a separate context for each.

This issue introduces a design that takes directly from the 'provider api"
practice of get*ApiForRegion("foo"), applied it to BlobStore, specifically
the word "region" as that has a fairly common understanding across
BlobStore providers.

Ex.

BlobStoreContext ctx = ...

// new command
// note we aren't propagating the rarely useful Location object, and
instead dealing w/ Strings
// also note this is *before* you get a blobstore instance, hinting that
this is a discovery command
Set<String> regionIds = ctx.configuredRegions();

// maintain current behaviour, which defers to defaults.
BlobStore defaultBS = ctx.getBlobStore();

// new command
// isolated to a specific region and will not make calls across multiple
endpoints
BlobStore defaultBS = ctx.blobStoreInRegion("foo");

Note the style above is opinionated a bit. For example, we aren't following
the javabeans practice of putting "get" in front of everything. See "how to
write a method"http://vimeo.com/74316116

Re: Real Multi-Region BlobStore

[Adrian Cole <ad...@gmail.com>]

openstack-swift in labs now includes the above design.

Please try it out and leave feedback here:
https://issues.apache.org/jira/browse/JCLOUDS-299

RegionScopedBlobStoreContext ctx = RegionScopedBlobStoreContext.class.cast(
view);

regions = ctx.configuredRegions();

blobstore = ctx.blobStoreInRegion(regionId);

deprecatedAsyncBlobstore = ctx.asyncBlobStoreInRegion(regionId);

signer = ctx.signerInRegion(regionId);

-A


On Thu, Sep 26, 2013 at 7:43 PM, Adrian Cole <ad...@gmail.com>wrote:

> TL;DR;  At the planning meeting, gaul mentioned that lack of real
> multi-region support in BlobStore is becoming a limiting factor for their
> customers.  I've sketched a design that builds on what's already been done,
> clarifying some points.  I plan to work on this over the next few days, so
> please pitch in, if you have any opinions.
>
> https://issues.apache.org/jira/browse/JCLOUDS-299
>
> -A
>
> Description pasted for the lazy:
>
> For effective multi-region apps, we need to both expose which regions are
> available, and also continue isolating a BlobStore to a specific one. Using
> this model, users can have predictable performance (ex. one BlobStore
> command won't cross regions), and isolation (one down region won't affect
> the BlobStore in use).
>
> The style of exposing an api scoped to region is also something we've
> practiced for well over a year, at the provider api level.
>
> ex. SwiftApi.getObjectApiForRegion("foo") style is commonplace now, even
> if not yet in the "View" interfaces such as BlobStore.
>
> During the last planning meeting, Maginatics (via andrew gaul) raised
> supporting multiple regions in BlobStore is becoming a must have. Users
> need to interact with multiple regions within Rackspace and OpenStack, for
> example, and these users may not know the magic region ids, nor desire
> maintaining a separate context for each.
>
> This issue introduces a design that takes directly from the 'provider api"
> practice of get*ApiForRegion("foo"), applied it to BlobStore, specifically
> the word "region" as that has a fairly common understanding across
> BlobStore providers.
>
> Ex.
>
> BlobStoreContext ctx = ...
>
> // new command
> // note we aren't propagating the rarely useful Location object, and
> instead dealing w/ Strings
> // also note this is *before* you get a blobstore instance, hinting that
> this is a discovery command
> Set<String> regionIds = ctx.configuredRegions();
>
> // maintain current behaviour, which defers to defaults.
> BlobStore defaultBS = ctx.getBlobStore();
>
> // new command
> // isolated to a specific region and will not make calls across multiple
> endpoints
> BlobStore defaultBS = ctx.blobStoreInRegion("foo");
>
> Note the style above is opinionated a bit. For example, we aren't
> following the javabeans practice of putting "get" in front of everything.
> See "how to write a method"http://vimeo.com/74316116
>
>