You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jclouds.apache.org by Janne Koskinen <jt...@iki.fi> on 2016/05/19 14:04:34 UTC
Blob dependency from Compute
Hi Folks!
I am Azure Compute ARM developer and currently working on ImageExtensionAPI. Creating custom images is straight forward with Azure but when Image is done, the listImages method of compute adapter is not listing these custom images. Only way to list custom images is to use Azure Blob Rest API which has own authentication and own endpoint and it is not related to Compute REST API.
Best available option is to use jClouds Blob API to list the images but that will make blob API dependency to Azure Compute API. Is that problem? If answer is yes, what do you recommend?
Br,
Janne
Re: Blob dependency from Compute
Posted by Ignasi Barrera <na...@apache.org>.
Ok, here is a first proposal to allow linking contexts together.
Please, review and give feedback!
https://github.com/jclouds/jclouds/pull/960
On 20 May 2016 at 16:52, Ignasi Barrera <na...@apache.org> wrote:
> If the endpoint is always the same and can be hardcoded/inferred given
> the data provided to create the "first" context, then I'd say it could
> be a good way to fix it. There is a way of performing raw http calls
> in jclouds, but you really want to use the provider API, so with this
> approach we should be creating a "child" context for the blob provider
> when getting the image extension.
>
> This said, options 2 is not so much work. I haven't explored it yet,
> but I think it should not take much to have something usable in place,
> and we get the added benefit that we have a clean way to "link"
> contexts and make them injectable where needed. Take into account that
> with option 1 we still have the problem of propagating all the
> configuration (properties that may collision, modules, etc) from one
> context to the other.
>
>
> Regarding the move image thing, I'm not very familiar with the blob
> store APIs, but I'm pretty sure Andrew Gaul or other blob store
> masters (:D) will be able to help there.
>
>
> I.
>
> On 20 May 2016 at 16:41, Janne Koskinen <jt...@iki.fi> wrote:
>> Thanks for the response!
>>
>> 1. In this case this is not too complex when endpoint is always same and we
>> can get credentials directly from StorageAccountAPI.getKeys. For user of
>> the API that will be transparent. Is there any acceptable way to make REST
>> call to another endpoint inside provider? That could be option 3.
>>
>> 2. This looks quite heavy solution for this problem when considering my
>> comment related to option 1. However if you still you think that option 1
>> (or 3) is still not good enough, we can consider this solution.
>>
>> We are still having one problem with usage of custom images and in any case
>> we need to have workaround for that before starting to do PR for core. The
>> current problem is that ImageExtensionAPI is creating the image to
>> blob/container/image and for some reason, we can't build VM from that
>> location but we need to move the image to blob/image and we don't know if
>> we can move the image with the blob API. We will study this early next week
>> to make sure that we can continue this path.
>>
>>
>> On Fri, May 20, 2016 at 5:31 PM, Ignasi Barrera <na...@apache.org> wrote:
>>
>>> On 20 May 2016 at 11:46, Ignasi Barrera <na...@apache.org> wrote:
>>> > Hi Janne,
>>> >
>>> > Currently there is no way for a provider code to access another one,
>>> > as we haven't had such a use case yet. The main issue here is one
>>> > context having to contact a different api (different endpoint,
>>> > credentials, configuration properties, etc). There are two approaches
>>> > that come to my mind to make this possible:
>>> >
>>> > 1. Let the provider use the context builder to create a connection to
>>> > the other api.
>>> > - This means that when creating the context we should have to
>>> > supply all the info (endpoint, credentials, properties) required for
>>> > the additional API.
>>> > - We cannot reuse standard properties (identity, credential, and
>>> > others such as timeouts) as they could require different values for
>>> > each context.
>>> > - We'd also have to pass the list of modules that the second
>>> > context would need.
>>> >
>>> > I see this option as very problematic, as there is many information
>>> > and possible conflicts that may appear between the two contexts.
>>> >
>>> > 2. Make one context "aware" of the existence of others, and let those
>>> > others be injected in the classes of the first provider. Something
>>> > like:
>>> >
>>> > View one = ContextBuilder.newBuilder("..."). ... .build();
>>> > View two = ContextBuilder.newBuilder("...").linkedTo(one). ...
>>> .build();
>>> >
>>> > And then make that context (or list of contexts) passed to the context
>>> > builder injectable in the code. In your case in the Azure
>>> > ImageExtension. Users should be responsible of properly creating and
>>> > closing each context, and they would be isolated.
>>> >
>>> > The ComputeService now returns an Optional<ImageExtension> which value
>>> > is computed by testing the presence of an implementation of the
>>> > interface. In the case of Azure that will have to be changed to check
>>> > for the implementation *and* for the existence of the required "linked
>>> > context".
>>> >
>>> >
>>> > If this proposal looks OK, I'll work on it and try to open a PR with
>>> > the changes required in core asap.
>>> >
>>> > (We really want this to be as generic as possible to let other
>>> > providers take benefit from this. For example the OpenStack Nova
>>> > ComputeService implemenation could look for a linked neutron context
>>> > and, if present, use neutron to perform all networking operations, and
>>> > fallback to nova networks otherwise.)
>>> >
>>> >
>>> >
>>> > I.
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > On 19 May 2016 at 16:04, Janne Koskinen <jt...@iki.fi> wrote:
>>> >> Hi Folks!
>>> >>
>>> >> I am Azure Compute ARM developer and currently working on
>>> ImageExtensionAPI. Creating custom images is straight forward with Azure
>>> but when Image is done, the listImages method of compute adapter is not
>>> listing these custom images. Only way to list custom images is to use Azure
>>> Blob Rest API which has own authentication and own endpoint and it is not
>>> related to Compute REST API.
>>> >>
>>> >> Best available option is to use jClouds Blob API to list the images but
>>> that will make blob API dependency to Azure Compute API. Is that problem?
>>> If answer is yes, what do you recommend?
>>> >>
>>> >> Br,
>>> >> Janne
>>>
>>
>>
>>
>> --
>> Br,
>> Janne Koskinen
>> 050-4863434
Re: Blob dependency from Compute
Posted by Ignasi Barrera <na...@apache.org>.
If the endpoint is always the same and can be hardcoded/inferred given
the data provided to create the "first" context, then I'd say it could
be a good way to fix it. There is a way of performing raw http calls
in jclouds, but you really want to use the provider API, so with this
approach we should be creating a "child" context for the blob provider
when getting the image extension.
This said, options 2 is not so much work. I haven't explored it yet,
but I think it should not take much to have something usable in place,
and we get the added benefit that we have a clean way to "link"
contexts and make them injectable where needed. Take into account that
with option 1 we still have the problem of propagating all the
configuration (properties that may collision, modules, etc) from one
context to the other.
Regarding the move image thing, I'm not very familiar with the blob
store APIs, but I'm pretty sure Andrew Gaul or other blob store
masters (:D) will be able to help there.
I.
On 20 May 2016 at 16:41, Janne Koskinen <jt...@iki.fi> wrote:
> Thanks for the response!
>
> 1. In this case this is not too complex when endpoint is always same and we
> can get credentials directly from StorageAccountAPI.getKeys. For user of
> the API that will be transparent. Is there any acceptable way to make REST
> call to another endpoint inside provider? That could be option 3.
>
> 2. This looks quite heavy solution for this problem when considering my
> comment related to option 1. However if you still you think that option 1
> (or 3) is still not good enough, we can consider this solution.
>
> We are still having one problem with usage of custom images and in any case
> we need to have workaround for that before starting to do PR for core. The
> current problem is that ImageExtensionAPI is creating the image to
> blob/container/image and for some reason, we can't build VM from that
> location but we need to move the image to blob/image and we don't know if
> we can move the image with the blob API. We will study this early next week
> to make sure that we can continue this path.
>
>
> On Fri, May 20, 2016 at 5:31 PM, Ignasi Barrera <na...@apache.org> wrote:
>
>> On 20 May 2016 at 11:46, Ignasi Barrera <na...@apache.org> wrote:
>> > Hi Janne,
>> >
>> > Currently there is no way for a provider code to access another one,
>> > as we haven't had such a use case yet. The main issue here is one
>> > context having to contact a different api (different endpoint,
>> > credentials, configuration properties, etc). There are two approaches
>> > that come to my mind to make this possible:
>> >
>> > 1. Let the provider use the context builder to create a connection to
>> > the other api.
>> > - This means that when creating the context we should have to
>> > supply all the info (endpoint, credentials, properties) required for
>> > the additional API.
>> > - We cannot reuse standard properties (identity, credential, and
>> > others such as timeouts) as they could require different values for
>> > each context.
>> > - We'd also have to pass the list of modules that the second
>> > context would need.
>> >
>> > I see this option as very problematic, as there is many information
>> > and possible conflicts that may appear between the two contexts.
>> >
>> > 2. Make one context "aware" of the existence of others, and let those
>> > others be injected in the classes of the first provider. Something
>> > like:
>> >
>> > View one = ContextBuilder.newBuilder("..."). ... .build();
>> > View two = ContextBuilder.newBuilder("...").linkedTo(one). ...
>> .build();
>> >
>> > And then make that context (or list of contexts) passed to the context
>> > builder injectable in the code. In your case in the Azure
>> > ImageExtension. Users should be responsible of properly creating and
>> > closing each context, and they would be isolated.
>> >
>> > The ComputeService now returns an Optional<ImageExtension> which value
>> > is computed by testing the presence of an implementation of the
>> > interface. In the case of Azure that will have to be changed to check
>> > for the implementation *and* for the existence of the required "linked
>> > context".
>> >
>> >
>> > If this proposal looks OK, I'll work on it and try to open a PR with
>> > the changes required in core asap.
>> >
>> > (We really want this to be as generic as possible to let other
>> > providers take benefit from this. For example the OpenStack Nova
>> > ComputeService implemenation could look for a linked neutron context
>> > and, if present, use neutron to perform all networking operations, and
>> > fallback to nova networks otherwise.)
>> >
>> >
>> >
>> > I.
>> >
>> >
>> >
>> >
>> >
>> >
>> > On 19 May 2016 at 16:04, Janne Koskinen <jt...@iki.fi> wrote:
>> >> Hi Folks!
>> >>
>> >> I am Azure Compute ARM developer and currently working on
>> ImageExtensionAPI. Creating custom images is straight forward with Azure
>> but when Image is done, the listImages method of compute adapter is not
>> listing these custom images. Only way to list custom images is to use Azure
>> Blob Rest API which has own authentication and own endpoint and it is not
>> related to Compute REST API.
>> >>
>> >> Best available option is to use jClouds Blob API to list the images but
>> that will make blob API dependency to Azure Compute API. Is that problem?
>> If answer is yes, what do you recommend?
>> >>
>> >> Br,
>> >> Janne
>>
>
>
>
> --
> Br,
> Janne Koskinen
> 050-4863434
Re: Blob dependency from Compute
Posted by Janne Koskinen <jt...@iki.fi>.
Thanks for the response!
1. In this case this is not too complex when endpoint is always same and we
can get credentials directly from StorageAccountAPI.getKeys. For user of
the API that will be transparent. Is there any acceptable way to make REST
call to another endpoint inside provider? That could be option 3.
2. This looks quite heavy solution for this problem when considering my
comment related to option 1. However if you still you think that option 1
(or 3) is still not good enough, we can consider this solution.
We are still having one problem with usage of custom images and in any case
we need to have workaround for that before starting to do PR for core. The
current problem is that ImageExtensionAPI is creating the image to
blob/container/image and for some reason, we can't build VM from that
location but we need to move the image to blob/image and we don't know if
we can move the image with the blob API. We will study this early next week
to make sure that we can continue this path.
On Fri, May 20, 2016 at 5:31 PM, Ignasi Barrera <na...@apache.org> wrote:
> On 20 May 2016 at 11:46, Ignasi Barrera <na...@apache.org> wrote:
> > Hi Janne,
> >
> > Currently there is no way for a provider code to access another one,
> > as we haven't had such a use case yet. The main issue here is one
> > context having to contact a different api (different endpoint,
> > credentials, configuration properties, etc). There are two approaches
> > that come to my mind to make this possible:
> >
> > 1. Let the provider use the context builder to create a connection to
> > the other api.
> > - This means that when creating the context we should have to
> > supply all the info (endpoint, credentials, properties) required for
> > the additional API.
> > - We cannot reuse standard properties (identity, credential, and
> > others such as timeouts) as they could require different values for
> > each context.
> > - We'd also have to pass the list of modules that the second
> > context would need.
> >
> > I see this option as very problematic, as there is many information
> > and possible conflicts that may appear between the two contexts.
> >
> > 2. Make one context "aware" of the existence of others, and let those
> > others be injected in the classes of the first provider. Something
> > like:
> >
> > View one = ContextBuilder.newBuilder("..."). ... .build();
> > View two = ContextBuilder.newBuilder("...").linkedTo(one). ...
> .build();
> >
> > And then make that context (or list of contexts) passed to the context
> > builder injectable in the code. In your case in the Azure
> > ImageExtension. Users should be responsible of properly creating and
> > closing each context, and they would be isolated.
> >
> > The ComputeService now returns an Optional<ImageExtension> which value
> > is computed by testing the presence of an implementation of the
> > interface. In the case of Azure that will have to be changed to check
> > for the implementation *and* for the existence of the required "linked
> > context".
> >
> >
> > If this proposal looks OK, I'll work on it and try to open a PR with
> > the changes required in core asap.
> >
> > (We really want this to be as generic as possible to let other
> > providers take benefit from this. For example the OpenStack Nova
> > ComputeService implemenation could look for a linked neutron context
> > and, if present, use neutron to perform all networking operations, and
> > fallback to nova networks otherwise.)
> >
> >
> >
> > I.
> >
> >
> >
> >
> >
> >
> > On 19 May 2016 at 16:04, Janne Koskinen <jt...@iki.fi> wrote:
> >> Hi Folks!
> >>
> >> I am Azure Compute ARM developer and currently working on
> ImageExtensionAPI. Creating custom images is straight forward with Azure
> but when Image is done, the listImages method of compute adapter is not
> listing these custom images. Only way to list custom images is to use Azure
> Blob Rest API which has own authentication and own endpoint and it is not
> related to Compute REST API.
> >>
> >> Best available option is to use jClouds Blob API to list the images but
> that will make blob API dependency to Azure Compute API. Is that problem?
> If answer is yes, what do you recommend?
> >>
> >> Br,
> >> Janne
>
--
Br,
Janne Koskinen
050-4863434
Re: Blob dependency from Compute
Posted by Ignasi Barrera <na...@apache.org>.
On 20 May 2016 at 11:46, Ignasi Barrera <na...@apache.org> wrote:
> Hi Janne,
>
> Currently there is no way for a provider code to access another one,
> as we haven't had such a use case yet. The main issue here is one
> context having to contact a different api (different endpoint,
> credentials, configuration properties, etc). There are two approaches
> that come to my mind to make this possible:
>
> 1. Let the provider use the context builder to create a connection to
> the other api.
> - This means that when creating the context we should have to
> supply all the info (endpoint, credentials, properties) required for
> the additional API.
> - We cannot reuse standard properties (identity, credential, and
> others such as timeouts) as they could require different values for
> each context.
> - We'd also have to pass the list of modules that the second
> context would need.
>
> I see this option as very problematic, as there is many information
> and possible conflicts that may appear between the two contexts.
>
> 2. Make one context "aware" of the existence of others, and let those
> others be injected in the classes of the first provider. Something
> like:
>
> View one = ContextBuilder.newBuilder("..."). ... .build();
> View two = ContextBuilder.newBuilder("...").linkedTo(one). ... .build();
>
> And then make that context (or list of contexts) passed to the context
> builder injectable in the code. In your case in the Azure
> ImageExtension. Users should be responsible of properly creating and
> closing each context, and they would be isolated.
>
> The ComputeService now returns an Optional<ImageExtension> which value
> is computed by testing the presence of an implementation of the
> interface. In the case of Azure that will have to be changed to check
> for the implementation *and* for the existence of the required "linked
> context".
>
>
> If this proposal looks OK, I'll work on it and try to open a PR with
> the changes required in core asap.
>
> (We really want this to be as generic as possible to let other
> providers take benefit from this. For example the OpenStack Nova
> ComputeService implemenation could look for a linked neutron context
> and, if present, use neutron to perform all networking operations, and
> fallback to nova networks otherwise.)
>
>
>
> I.
>
>
>
>
>
>
> On 19 May 2016 at 16:04, Janne Koskinen <jt...@iki.fi> wrote:
>> Hi Folks!
>>
>> I am Azure Compute ARM developer and currently working on ImageExtensionAPI. Creating custom images is straight forward with Azure but when Image is done, the listImages method of compute adapter is not listing these custom images. Only way to list custom images is to use Azure Blob Rest API which has own authentication and own endpoint and it is not related to Compute REST API.
>>
>> Best available option is to use jClouds Blob API to list the images but that will make blob API dependency to Azure Compute API. Is that problem? If answer is yes, what do you recommend?
>>
>> Br,
>> Janne
Re: Blob dependency from Compute
Posted by Ignasi Barrera <na...@apache.org>.
Hi Janne,
Currently there is no way for a provider code to access another one,
as we haven't had such a use case yet. The main issue here is one
context having to contact a different api (different endpoint,
credentials, configuration properties, etc). There are two approaches
that come to my mind to make this possible:
1. Let the provider use the context builder to create a connection to
the other api.
- This means that when creating the context we should have to
supply all the info (endpoint, credentials, properties) required for
the additional API.
- We cannot reuse standard properties (identity, credential, and
others such as timeouts) as they could require different values for
each context.
- We'd also have to pass the list of modules that the second
context would need.
I see this option as very problematic, as there is many information
and possible conflicts that may appear between the two contexts.
2. Make one context "aware" of the existence of others, and let those
others be injected in the classes of the first provider. Something
like:
View one = ContextBuilder.newBuilder("..."). ... .build();
View two = ContextBuilder.newBuilder("...").linkedTo(one). ... .build();
And then make that context (or list of contexts) passed to the context
builder injectable in the code. In your case in the Azure
ImageExtension. Users should be responsible of properly creating and
closing each context, and they would be isolated.
The ComputeService now returns an Optional<ImageExtension> which value
is computed by testing the presence of an implementation of the
interface. In the case of Azure that will have to be changed to check
for the implementation *and* for the existence of the required "linked
context".
If this proposal looks OK, I'll work on it and try to open a PR with
the changes required in core asap.
(We really want this to be as generic as possible to let other
providers take benefit from this. For example the OpenStack Nova
ComputeService implemenation could look for a linked neutron context
and, if present, use neutron to perform all networking operations, and
fallback to nova networks otherwise.)
I.
On 19 May 2016 at 16:04, Janne Koskinen <jt...@iki.fi> wrote:
> Hi Folks!
>
> I am Azure Compute ARM developer and currently working on ImageExtensionAPI. Creating custom images is straight forward with Azure but when Image is done, the listImages method of compute adapter is not listing these custom images. Only way to list custom images is to use Azure Blob Rest API which has own authentication and own endpoint and it is not related to Compute REST API.
>
> Best available option is to use jClouds Blob API to list the images but that will make blob API dependency to Azure Compute API. Is that problem? If answer is yes, what do you recommend?
>
> Br,
> Janne