You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Chiradeep Vittal <Ch...@citrix.com> on 2013/10/01 23:06:01 UTC
[DISCUSS] Leaky abstractions [was review requests 13238, 13896,
14320]
We have a couple of people trying to expose the advanced capabilities of the underlying physical resources to the end-user. In the case of Nicolas FOATA, he is trying to expose some of the advanced functions of XenServer/XCP, and in the case of Daan, he is trying to expose some feature of HAProxy.
Users are ideally abstracted from these details and shouldn't have to wonder which offering to choose [because they are not Xen/HAProxy experts].
After all one of the goals of CS is to hide these messy details and let users focus on their apps.
Is there a possibility of a generic way of leaking abstractions for sufficiently advanced users?
https://reviews.apache.org/r/13238/
https://reviews.apache.org/r/14320/
https://reviews.apache.org/r/13896/
BTW, I really prefer that these changes are discussed by putting up an FS on the wiki rather than submitting patch requests.
If it touches more than a few files, it is probably worth discussing with a [DISCUSS] tag line.
Also, it requires tests.
Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896,
14320]
Posted by Kelven Yang <ke...@citrix.com>.
*_details table is currently used as the generic way of passing key/value
pairs that are transparent to API layer and core CloudStack orchestration
flow. When I say transparent, the scope is only limited to API layer and
core generic orchestration flow, specific plugin elements or Gurus still
have the chance to further interpret these key/value data with current
Guru/Element pattern in the flow.
So, we do have such generic facility, but how to present in a more
domain-specific and friendly way is another problem.
Kelven
On 10/3/13 11:15 AM, "Darren Shepherd" <da...@gmail.com> wrote:
>We are always going to run into the issue where they're implementation
>specific features that can't be represented in a generic way in the
>API. First class attributes in the API need to essentially be the
>lowest common denominator. ACS already has this pattern of *_details
>tables. I honestly don't know completely how they are used, but I
>would hope that pattern could be used to address these types of
>issues. By allowing the admin to add arbitrary key/value pairs to the
>various offerings/entities we can use that to tap into provider
>specific or advanced configuration. For the http close proposal you
>could just add haproxy.httpClose=true on the network offering and the
>haproxy impl can respect that and use it. So implementations can
>document what additional key/value parameters they support to access
>advanced features.
>
>Review 13896 adds an otherOptions field to the service offering.
>Conceptually that is fine in my mind, but I just don't understand why
>the *_details tables can't be used instead. Just like XenServer has
>other_config on all types, I'd like it if ACS had a similar thing
>(which I kinda, sorta think it does?). So we should be able to
>arbitrary data onto any type. Can somebody comment on how this idea
>may relate with the existing implementation of "details" and "tags" in
>ACS?
>
>Darren
>
>On Thu, Oct 3, 2013 at 10:21 AM, Daan Hoogland <da...@gmail.com>
>wrote:
>> Murali,
>>
>> What our admins need is the ability to instantiate an abstract thing
>>called
>> a virtual redundant high available router. They need be able to tune it
>> with all sorts of features is such a way that other routers redundant or
>> not and high availible or not, may but do not have to have the same
>>tuning
>> parameters. This 'feature' of httpClose is just one of the many things
>>they
>> need to be able to tune. Others include a more fine grained control over
>> the iptables/firewall rules and monitoring of the functionality/daemons
>> running on the machines. I am not biased as to the way how to
>>do/implement
>> this control. The networkoffering seemed like the way to do it.
>>
>> Having said that I didn't think that httpClose would be considered
>>haproxy
>> specific. It seems worrying that someone could device a proxy or a
>> loadbalancer that does not support either one of the features of
>> maximizing - or pooling connections. I'm not into that market recently
>>so I
>> might be mistaking. This maybe besides the point but it discomforts me
>> somewhat.
>>
>> regards,
>> Daan
>>
>>
>> On Thu, Oct 3, 2013 at 2:22 PM, murali reddy
>><mu...@gmail.com>wrote:
>>
>>> On Thu, Oct 3, 2013 at 4:18 PM, Daan Hoogland <daan.hoogland@gmail.com
>>> >wrote:
>>>
>>> > Chiradeep,
>>> >
>>> > I have been thinking about your concern and there is something
>>>bothering
>>> me
>>> > about it. The issue CLOUDSTACK-4328 of which
>>> > https://reviews.apache.org/r/14320/ is an implementation. This issue
>>>has
>>> > been brought up by the engineers at Schuberg Philis. These are cloud
>>> > operators and domain admins. So these are the people that need to be
>>> > bothered by this tuning and they are certainly haproxy and xen
>>>experts to
>>> > an extend. For them not being able to use connection caching was a
>>>bug.
>>> And
>>> > the proposed solution still seems reasonable to me. It is exposing
>>>the
>>> > abstract feature only to those instantiating a vpc, isn't it? This
>>>last
>>> > question probably touches on the reason you are addressing my
>>>submission
>>> > together with the ones from Nicolas I see only people instantiating
>>>a
>>> vpc
>>> > having to be bothered by which netoffering to use (with respect to
>>> > httpClose functionality) If this is not the case how should I isolate
>>> this
>>> > concern from other , more mondain users?
>>> >
>>> > btw I did not create an FS page as the issue was created as a jira
>>> ticket.
>>> > I am willing to do that in hind sight but want to have a path to
>>>follow
>>> > first.
>>> >
>>> > regards,
>>> > Daan
>>> >
>>> > On Tue, Oct 1, 2013 at 11:06 PM, Chiradeep Vittal <
>>> > Chiradeep.Vittal@citrix.com> wrote:
>>> > >
>>> > > We have a couple of people trying to expose the advanced
>>>capabilities
>>> of
>>> > the underlying physical resources to the end-user. In the case of
>>>Nicolas
>>> > FOATA, he is trying to expose some of the advanced functions of
>>> > XenServer/XCP, and in the case of Daan, he is trying to expose some
>>> feature
>>> > of HAProxy.
>>> > >
>>> > > Users are ideally abstracted from these details and shouldn't have
>>>to
>>> > wonder which offering to choose [because they are not Xen/HAProxy
>>> experts].
>>> > > After all one of the goals of CS is to hide these messy details
>>>and let
>>> > users focus on their apps.
>>> > >
>>> > > Is there a possibility of a generic way of leaking abstractions for
>>> > sufficiently advanced users?
>>> > >
>>> >
>>>
>>> Generally as a principle core API's abstract and expose functionality
>>>that
>>> is commonly available across the hypervisors and network service
>>>providers
>>> etc.. I believe we should continue to enforce it.
>>>
>>> But we do have a pattern of API's configure* (configurevirtualrouter,
>>> configurenetscaler etc) where a hypervisor/network element plug-in can
>>>let
>>> admin to configure params specific to plug-in. Perhaps we can use this
>>> API's fine tune a plugin for advanced configurations. For e.g both
>>>HaProxy
>>> max connections, httpModeEnabled can be parameters that can be
>>>configured
>>> with configureVirtualRouter api.
>>>
>>> Daan,
>>>
>>> does this approach works for you?
>>>
>>> -Murali
>>>
>>>
>>> > > https://reviews.apache.org/r/13238/
>>> > > https://reviews.apache.org/r/14320/
>>> > > https://reviews.apache.org/r/13896/
>>> > >
>>> > > BTW, I really prefer that these changes are discussed by putting
>>>up an
>>> FS
>>> > on the wiki rather than submitting patch requests.
>>> > > If it touches more than a few files, it is probably worth
>>>discussing
>>> with
>>> > a [DISCUSS] tag line.
>>> > > Also, it requires tests.
>>> > >
>>> > >
>>> > >
>>> >
>>>
Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]
Posted by Daan Hoogland <da...@gmail.com>.
Having thought about the more generic issue with 'leaky abstractions'
a little more I would like to add the reverse of my scenario to the
discussion.
all implementers of a flutch have a feature Y so we implement an
option in our interface. Now the one of them (or a new kid on the
block) solves the problem that this option is for in a totally
different way. Their solution slowly becomes commonplace but only
slowly and one of the big players, producer of implementation a,
doesn't go for this solution. How do we migrate from generic option to
a key/value pair that only the provider/element/guru for an a
understands?
regards,
Daan
On Sat, Oct 5, 2013 at 11:28 AM, Daan Hoogland <da...@gmail.com> wrote:
> H Chiradeep,
>
> I am hesitating to keep on about my case of httpClose as this is about the
> more general subject you gave this thread, so please take my referals to it
> as examples.
>
> so it is in a sense keepAlive (formerly known as ! httpClose) we are talking
> about. Then there is a matter of how to implement it as I understand you
> have objections to the part where it is in the networkoffering. Let's take
> that back to the review thread, alright?
>
> Remains the question of the new feature X that will be common good in next
> generations. How art we dealing with that?
>
> And in this case maybe X is default for most but !X is default for
> implementation c. To that I'd say implement the options as and take the
> default from the largest group of implementations. In httpClose you could
> think of default is to keepAlive. I didn't do this to remain backwards
> compatible. If that is not a problem I think the engineers at Schuberg
> Philis can live with a change of hardcoded settings. I don't like that,
> hence my implementation.
>
> regards,
> Daan
>
>
> On Sat, Oct 5, 2013 at 12:56 AM, Chiradeep Vittal
> <Ch...@citrix.com> wrote:
>>
>> I've checked the Netscaler and HAProxy docs, this appears to be artifact
>> of the HAProxy implementation (inability to support keep-alive).
>> For a cloud operator that chooses Netscaler or F5 for load balancing, this
>> won't make any sense.
>>
>> On 10/3/13 12:55 PM, "Daan Hoogland" <da...@gmail.com> wrote:
>>
>> >On Thu, Oct 3, 2013 at 9:02 PM, Chip Childers
>> ><ch...@sungard.com>wrote:
>> >
>> >> a model for extensions like that makes perfect sense.
>> >
>> >
>> >
>> >This model sound fine indeed. It makes no sense for httpClose however.
>> >
>> >Here's my concern:
>> >So when an early adapter is implemented and the rest of the market comes
>> >to
>> >their senses, how do we migrate without running into migration/upgrade
>> >problems?
>> >httpClose is a flag controlling connection pooling. I probably choose the
>> >wrong name. It is something that any implementation will support or
>> > should
>> >have supported already. Am I going to implement it as a key/value now to
>> >later implemented as I have done anyway? I don't like this idea.
>> >
>> >Don't get me wrong the pattern described by you guys is fine in some
>> >situations. I don't think it is applicable to this feature.
>> >
>> >regards,
>> >Daan
>>
>
Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]
Posted by Daan Hoogland <da...@gmail.com>.
H Chiradeep,
I am hesitating to keep on about my case of httpClose as this is about the
more general subject you gave this thread, so please take my referals to it
as examples.
so it is in a sense keepAlive (formerly known as ! httpClose) we are
talking about. Then there is a matter of how to implement it as I
understand you have objections to the part where it is in the
networkoffering. Let's take that back to the review thread, alright?
Remains the question of the new feature X that will be common good in next
generations. How art we dealing with that?
And in this case maybe X is default for most but !X is default for
implementation c. To that I'd say implement the options as and take the
default from the largest group of implementations. In httpClose you could
think of default is to keepAlive. I didn't do this to remain backwards
compatible. If that is not a problem I think the engineers at Schuberg
Philis can live with a change of hardcoded settings. I don't like that,
hence my implementation.
regards,
Daan
On Sat, Oct 5, 2013 at 12:56 AM, Chiradeep Vittal <
Chiradeep.Vittal@citrix.com> wrote:
> I've checked the Netscaler and HAProxy docs, this appears to be artifact
> of the HAProxy implementation (inability to support keep-alive).
> For a cloud operator that chooses Netscaler or F5 for load balancing, this
> won't make any sense.
>
> On 10/3/13 12:55 PM, "Daan Hoogland" <da...@gmail.com> wrote:
>
> >On Thu, Oct 3, 2013 at 9:02 PM, Chip Childers
> ><ch...@sungard.com>wrote:
> >
> >> a model for extensions like that makes perfect sense.
> >
> >
> >
> >This model sound fine indeed. It makes no sense for httpClose however.
> >
> >Here's my concern:
> >So when an early adapter is implemented and the rest of the market comes
> >to
> >their senses, how do we migrate without running into migration/upgrade
> >problems?
> >httpClose is a flag controlling connection pooling. I probably choose the
> >wrong name. It is something that any implementation will support or should
> >have supported already. Am I going to implement it as a key/value now to
> >later implemented as I have done anyway? I don't like this idea.
> >
> >Don't get me wrong the pattern described by you guys is fine in some
> >situations. I don't think it is applicable to this feature.
> >
> >regards,
> >Daan
>
>
Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]
Posted by Daan Hoogland <da...@gmail.com>.
I know and I was underlining that in the part of my mail you didn't quote.
:p
On Thu, Oct 3, 2013 at 10:26 PM, Chip Childers <ch...@sungard.com>wrote:
> On Thu, Oct 03, 2013 at 09:55:41PM +0200, Daan Hoogland wrote:
> > On Thu, Oct 3, 2013 at 9:02 PM, Chip Childers <chip.childers@sungard.com
> >wrote:
> >
> > > a model for extensions like that makes perfect sense.
> >
> >
> >
> > This model sound fine indeed. It makes no sense for httpClose however.
>
> My personal comment was only about the usefulness of the approach, not
> the specific function in question. ;-)
>
> I don't have an opinion either way on this one, since I don't have
> enough information to weigh in.
>
> >
> > Here's my concern:
> > So when an early adapter is implemented and the rest of the market comes
> to
> > their senses, how do we migrate without running into migration/upgrade
> > problems?
> > httpClose is a flag controlling connection pooling. I probably choose the
> > wrong name. It is something that any implementation will support or
> should
> > have supported already. Am I going to implement it as a key/value now to
> > later implemented as I have done anyway? I don't like this idea.
> >
> > Don't get me wrong the pattern described by you guys is fine in some
> > situations. I don't think it is applicable to this feature.
> >
> > regards,
> > Daan
>
Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896,
14320]
Posted by Chip Childers <ch...@sungard.com>.
On Thu, Oct 03, 2013 at 09:55:41PM +0200, Daan Hoogland wrote:
> On Thu, Oct 3, 2013 at 9:02 PM, Chip Childers <ch...@sungard.com>wrote:
>
> > a model for extensions like that makes perfect sense.
>
>
>
> This model sound fine indeed. It makes no sense for httpClose however.
My personal comment was only about the usefulness of the approach, not
the specific function in question. ;-)
I don't have an opinion either way on this one, since I don't have
enough information to weigh in.
>
> Here's my concern:
> So when an early adapter is implemented and the rest of the market comes to
> their senses, how do we migrate without running into migration/upgrade
> problems?
> httpClose is a flag controlling connection pooling. I probably choose the
> wrong name. It is something that any implementation will support or should
> have supported already. Am I going to implement it as a key/value now to
> later implemented as I have done anyway? I don't like this idea.
>
> Don't get me wrong the pattern described by you guys is fine in some
> situations. I don't think it is applicable to this feature.
>
> regards,
> Daan
Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896,
14320]
Posted by Chiradeep Vittal <Ch...@citrix.com>.
I've checked the Netscaler and HAProxy docs, this appears to be artifact
of the HAProxy implementation (inability to support keep-alive).
For a cloud operator that chooses Netscaler or F5 for load balancing, this
won't make any sense.
On 10/3/13 12:55 PM, "Daan Hoogland" <da...@gmail.com> wrote:
>On Thu, Oct 3, 2013 at 9:02 PM, Chip Childers
><ch...@sungard.com>wrote:
>
>> a model for extensions like that makes perfect sense.
>
>
>
>This model sound fine indeed. It makes no sense for httpClose however.
>
>Here's my concern:
>So when an early adapter is implemented and the rest of the market comes
>to
>their senses, how do we migrate without running into migration/upgrade
>problems?
>httpClose is a flag controlling connection pooling. I probably choose the
>wrong name. It is something that any implementation will support or should
>have supported already. Am I going to implement it as a key/value now to
>later implemented as I have done anyway? I don't like this idea.
>
>Don't get me wrong the pattern described by you guys is fine in some
>situations. I don't think it is applicable to this feature.
>
>regards,
>Daan
Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]
Posted by Daan Hoogland <da...@gmail.com>.
On Thu, Oct 3, 2013 at 9:02 PM, Chip Childers <ch...@sungard.com>wrote:
> a model for extensions like that makes perfect sense.
This model sound fine indeed. It makes no sense for httpClose however.
Here's my concern:
So when an early adapter is implemented and the rest of the market comes to
their senses, how do we migrate without running into migration/upgrade
problems?
httpClose is a flag controlling connection pooling. I probably choose the
wrong name. It is something that any implementation will support or should
have supported already. Am I going to implement it as a key/value now to
later implemented as I have done anyway? I don't like this idea.
Don't get me wrong the pattern described by you guys is fine in some
situations. I don't think it is applicable to this feature.
regards,
Daan
Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]
Posted by Daan Hoogland <da...@gmail.com>.
Can we discuss the folowing:
Some type of device, the flutch is widely used and cloudstack supports
brands a and b. Now brand c comes to the market with feature X. It grows
it's market share slowly becouse of this feature and we, or better the
producer implement
a plugin using the new resourcedetails pattern. Now the market share
starts growing as people start realizing that a flutch without X makes no
sense. a and b implement X in their flutches. the cloudstack user does now
have to configure for his legacy a and b flutches as wel as for his brand
new flying c, X as a key value pair on each device. we want to accomodat
this by making X a standard part of the flutch-offering, don't we?
Even if a decides to implement X in their flutches but b doesn't I think we
would want to do that.
It may come as now surprise that, given the test results the Schuberg
Philis engineers produced and put in
https://issues.apache.org/jira/browse/CLOUDSTACK-4328, I feel the httpClose
feature in an X with respect to loadbalancers and proxies.
kind regards,
Daan
On Fri, Oct 4, 2013 at 7:52 AM, Murali Reddy <Mu...@citrix.com>wrote:
> On 04/10/13 12:32 AM, "Chip Childers" <ch...@sungard.com> wrote:
>
> >> On Oct 3, 2013, at 3:01 PM, Chiradeep Vittal
> >><Ch...@citrix.com> wrote:
> >>
> >> I was thinking along the same lines that a offering could have a generic
> >> key/value map (where the value could be a string/json string/whatever).
> >> CloudStack core wouldn't have any idea about the semantics of the keys
> >>or
> >> values, but the specific hypervisor/provider might decide to interpret
> >>it.
> >
> >+1 - a model for extensions like that makes perfect sense.
>
> We already have api's (add/list/removeResourceDetails) in 4.2 [1] to add
> key-value metadata to first class objects of CloudStack. At present it
> does not have semantic meaning to neither core nor plug-ins, but should
> not be hard to extend so that plug-ins can access meta-data and interpret
> it.
>
> [1] http://s.apache.org/ReK
>
> >
> >>
> >>
> >>> On 10/3/13 11:15 AM, "Darren Shepherd" <da...@gmail.com>
> >>>wrote:
> >>>
> >>> We are always going to run into the issue where they're implementation
> >>> specific features that can't be represented in a generic way in the
> >>> API. First class attributes in the API need to essentially be the
> >>> lowest common denominator. ACS already has this pattern of *_details
> >>> tables. I honestly don't know completely how they are used, but I
> >>> would hope that pattern could be used to address these types of
> >>> issues. By allowing the admin to add arbitrary key/value pairs to the
> >>> various offerings/entities we can use that to tap into provider
> >>> specific or advanced configuration. For the http close proposal you
> >>> could just add haproxy.httpClose=true on the network offering and the
> >>> haproxy impl can respect that and use it. So implementations can
> >>> document what additional key/value parameters they support to access
> >>> advanced features.
> >>>
> >>> Review 13896 adds an otherOptions field to the service offering.
> >>> Conceptually that is fine in my mind, but I just don't understand why
> >>> the *_details tables can't be used instead. Just like XenServer has
> >>> other_config on all types, I'd like it if ACS had a similar thing
> >>> (which I kinda, sorta think it does?). So we should be able to
> >>> arbitrary data onto any type. Can somebody comment on how this idea
> >>> may relate with the existing implementation of "details" and "tags" in
> >>> ACS?
> >>>
> >>> Darren
> >>>
> >>> On Thu, Oct 3, 2013 at 10:21 AM, Daan Hoogland
> >>><da...@gmail.com>
> >>> wrote:
> >>>> Murali,
> >>>>
> >>>> What our admins need is the ability to instantiate an abstract thing
> >>>> called
> >>>> a virtual redundant high available router. They need be able to tune
> >>>>it
> >>>> with all sorts of features is such a way that other routers redundant
> >>>>or
> >>>> not and high availible or not, may but do not have to have the same
> >>>> tuning
> >>>> parameters. This 'feature' of httpClose is just one of the many things
> >>>> they
> >>>> need to be able to tune. Others include a more fine grained control
> >>>>over
> >>>> the iptables/firewall rules and monitoring of the
> >>>>functionality/daemons
> >>>> running on the machines. I am not biased as to the way how to
> >>>> do/implement
> >>>> this control. The networkoffering seemed like the way to do it.
> >>>>
> >>>> Having said that I didn't think that httpClose would be considered
> >>>> haproxy
> >>>> specific. It seems worrying that someone could device a proxy or a
> >>>> loadbalancer that does not support either one of the features of
> >>>> maximizing - or pooling connections. I'm not into that market recently
> >>>> so I
> >>>> might be mistaking. This maybe besides the point but it discomforts me
> >>>> somewhat.
> >>>>
> >>>> regards,
> >>>> Daan
> >>>>
> >>>>
> >>>> On Thu, Oct 3, 2013 at 2:22 PM, murali reddy
> >>>> <mu...@gmail.com>wrote:
> >>>>
> >>>>> On Thu, Oct 3, 2013 at 4:18 PM, Daan Hoogland
> >>>>><daan.hoogland@gmail.com
> >>>>>> wrote:
> >>>>>
> >>>>>> Chiradeep,
> >>>>>>
> >>>>>> I have been thinking about your concern and there is something
> >>>>> bothering
> >>>>> me
> >>>>>> about it. The issue CLOUDSTACK-4328 of which
> >>>>>> https://reviews.apache.org/r/14320/ is an implementation. This
> issue
> >>>>> has
> >>>>>> been brought up by the engineers at Schuberg Philis. These are cloud
> >>>>>> operators and domain admins. So these are the people that need to be
> >>>>>> bothered by this tuning and they are certainly haproxy and xen
> >>>>> experts to
> >>>>>> an extend. For them not being able to use connection caching was a
> >>>>> bug.
> >>>>> And
> >>>>>> the proposed solution still seems reasonable to me. It is exposing
> >>>>> the
> >>>>>> abstract feature only to those instantiating a vpc, isn't it? This
> >>>>> last
> >>>>>> question probably touches on the reason you are addressing my
> >>>>> submission
> >>>>>> together with the ones from Nicolas I see only people instantiating
> >>>>> a
> >>>>> vpc
> >>>>>> having to be bothered by which netoffering to use (with respect to
> >>>>>> httpClose functionality) If this is not the case how should I
> >>>>>>isolate
> >>>>> this
> >>>>>> concern from other , more mondain users?
> >>>>>>
> >>>>>> btw I did not create an FS page as the issue was created as a jira
> >>>>> ticket.
> >>>>>> I am willing to do that in hind sight but want to have a path to
> >>>>> follow
> >>>>>> first.
> >>>>>>
> >>>>>> regards,
> >>>>>> Daan
> >>>>>>
> >>>>>> On Tue, Oct 1, 2013 at 11:06 PM, Chiradeep Vittal <
> >>>>>> Chiradeep.Vittal@citrix.com> wrote:
> >>>>>>>
> >>>>>>> We have a couple of people trying to expose the advanced
> >>>>> capabilities
> >>>>> of
> >>>>>> the underlying physical resources to the end-user. In the case of
> >>>>> Nicolas
> >>>>>> FOATA, he is trying to expose some of the advanced functions of
> >>>>>> XenServer/XCP, and in the case of Daan, he is trying to expose some
> >>>>> feature
> >>>>>> of HAProxy.
> >>>>>>>
> >>>>>>> Users are ideally abstracted from these details and shouldn't have
> >>>>> to
> >>>>>> wonder which offering to choose [because they are not Xen/HAProxy
> >>>>> experts].
> >>>>>>> After all one of the goals of CS is to hide these messy details
> >>>>> and let
> >>>>>> users focus on their apps.
> >>>>>>>
> >>>>>>> Is there a possibility of a generic way of leaking abstractions for
> >>>>>> sufficiently advanced users?
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>> Generally as a principle core API's abstract and expose functionality
> >>>>> that
> >>>>> is commonly available across the hypervisors and network service
> >>>>> providers
> >>>>> etc.. I believe we should continue to enforce it.
> >>>>>
> >>>>> But we do have a pattern of API's configure* (configurevirtualrouter,
> >>>>> configurenetscaler etc) where a hypervisor/network element plug-in
> >>>>>can
> >>>>> let
> >>>>> admin to configure params specific to plug-in. Perhaps we can use
> >>>>>this
> >>>>> API's fine tune a plugin for advanced configurations. For e.g both
> >>>>> HaProxy
> >>>>> max connections, httpModeEnabled can be parameters that can be
> >>>>> configured
> >>>>> with configureVirtualRouter api.
> >>>>>
> >>>>> Daan,
> >>>>>
> >>>>> does this approach works for you?
> >>>>>
> >>>>> -Murali
> >>>>>
> >>>>>
> >>>>>>> https://reviews.apache.org/r/13238/
> >>>>>>> https://reviews.apache.org/r/14320/
> >>>>>>> https://reviews.apache.org/r/13896/
> >>>>>>>
> >>>>>>> BTW, I really prefer that these changes are discussed by putting
> >>>>> up an
> >>>>> FS
> >>>>>> on the wiki rather than submitting patch requests.
> >>>>>>> If it touches more than a few files, it is probably worth
> >>>>> discussing
> >>>>> with
> >>>>>> a [DISCUSS] tag line.
> >>>>>>> Also, it requires tests.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>
> >>
> >
>
>
>
Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896,
14320]
Posted by Murali Reddy <Mu...@citrix.com>.
On 04/10/13 12:32 AM, "Chip Childers" <ch...@sungard.com> wrote:
>> On Oct 3, 2013, at 3:01 PM, Chiradeep Vittal
>><Ch...@citrix.com> wrote:
>>
>> I was thinking along the same lines that a offering could have a generic
>> key/value map (where the value could be a string/json string/whatever).
>> CloudStack core wouldn't have any idea about the semantics of the keys
>>or
>> values, but the specific hypervisor/provider might decide to interpret
>>it.
>
>+1 - a model for extensions like that makes perfect sense.
We already have api's (add/list/removeResourceDetails) in 4.2 [1] to add
key-value metadata to first class objects of CloudStack. At present it
does not have semantic meaning to neither core nor plug-ins, but should
not be hard to extend so that plug-ins can access meta-data and interpret
it.
[1] http://s.apache.org/ReK
>
>>
>>
>>> On 10/3/13 11:15 AM, "Darren Shepherd" <da...@gmail.com>
>>>wrote:
>>>
>>> We are always going to run into the issue where they're implementation
>>> specific features that can't be represented in a generic way in the
>>> API. First class attributes in the API need to essentially be the
>>> lowest common denominator. ACS already has this pattern of *_details
>>> tables. I honestly don't know completely how they are used, but I
>>> would hope that pattern could be used to address these types of
>>> issues. By allowing the admin to add arbitrary key/value pairs to the
>>> various offerings/entities we can use that to tap into provider
>>> specific or advanced configuration. For the http close proposal you
>>> could just add haproxy.httpClose=true on the network offering and the
>>> haproxy impl can respect that and use it. So implementations can
>>> document what additional key/value parameters they support to access
>>> advanced features.
>>>
>>> Review 13896 adds an otherOptions field to the service offering.
>>> Conceptually that is fine in my mind, but I just don't understand why
>>> the *_details tables can't be used instead. Just like XenServer has
>>> other_config on all types, I'd like it if ACS had a similar thing
>>> (which I kinda, sorta think it does?). So we should be able to
>>> arbitrary data onto any type. Can somebody comment on how this idea
>>> may relate with the existing implementation of "details" and "tags" in
>>> ACS?
>>>
>>> Darren
>>>
>>> On Thu, Oct 3, 2013 at 10:21 AM, Daan Hoogland
>>><da...@gmail.com>
>>> wrote:
>>>> Murali,
>>>>
>>>> What our admins need is the ability to instantiate an abstract thing
>>>> called
>>>> a virtual redundant high available router. They need be able to tune
>>>>it
>>>> with all sorts of features is such a way that other routers redundant
>>>>or
>>>> not and high availible or not, may but do not have to have the same
>>>> tuning
>>>> parameters. This 'feature' of httpClose is just one of the many things
>>>> they
>>>> need to be able to tune. Others include a more fine grained control
>>>>over
>>>> the iptables/firewall rules and monitoring of the
>>>>functionality/daemons
>>>> running on the machines. I am not biased as to the way how to
>>>> do/implement
>>>> this control. The networkoffering seemed like the way to do it.
>>>>
>>>> Having said that I didn't think that httpClose would be considered
>>>> haproxy
>>>> specific. It seems worrying that someone could device a proxy or a
>>>> loadbalancer that does not support either one of the features of
>>>> maximizing - or pooling connections. I'm not into that market recently
>>>> so I
>>>> might be mistaking. This maybe besides the point but it discomforts me
>>>> somewhat.
>>>>
>>>> regards,
>>>> Daan
>>>>
>>>>
>>>> On Thu, Oct 3, 2013 at 2:22 PM, murali reddy
>>>> <mu...@gmail.com>wrote:
>>>>
>>>>> On Thu, Oct 3, 2013 at 4:18 PM, Daan Hoogland
>>>>><daan.hoogland@gmail.com
>>>>>> wrote:
>>>>>
>>>>>> Chiradeep,
>>>>>>
>>>>>> I have been thinking about your concern and there is something
>>>>> bothering
>>>>> me
>>>>>> about it. The issue CLOUDSTACK-4328 of which
>>>>>> https://reviews.apache.org/r/14320/ is an implementation. This issue
>>>>> has
>>>>>> been brought up by the engineers at Schuberg Philis. These are cloud
>>>>>> operators and domain admins. So these are the people that need to be
>>>>>> bothered by this tuning and they are certainly haproxy and xen
>>>>> experts to
>>>>>> an extend. For them not being able to use connection caching was a
>>>>> bug.
>>>>> And
>>>>>> the proposed solution still seems reasonable to me. It is exposing
>>>>> the
>>>>>> abstract feature only to those instantiating a vpc, isn't it? This
>>>>> last
>>>>>> question probably touches on the reason you are addressing my
>>>>> submission
>>>>>> together with the ones from Nicolas I see only people instantiating
>>>>> a
>>>>> vpc
>>>>>> having to be bothered by which netoffering to use (with respect to
>>>>>> httpClose functionality) If this is not the case how should I
>>>>>>isolate
>>>>> this
>>>>>> concern from other , more mondain users?
>>>>>>
>>>>>> btw I did not create an FS page as the issue was created as a jira
>>>>> ticket.
>>>>>> I am willing to do that in hind sight but want to have a path to
>>>>> follow
>>>>>> first.
>>>>>>
>>>>>> regards,
>>>>>> Daan
>>>>>>
>>>>>> On Tue, Oct 1, 2013 at 11:06 PM, Chiradeep Vittal <
>>>>>> Chiradeep.Vittal@citrix.com> wrote:
>>>>>>>
>>>>>>> We have a couple of people trying to expose the advanced
>>>>> capabilities
>>>>> of
>>>>>> the underlying physical resources to the end-user. In the case of
>>>>> Nicolas
>>>>>> FOATA, he is trying to expose some of the advanced functions of
>>>>>> XenServer/XCP, and in the case of Daan, he is trying to expose some
>>>>> feature
>>>>>> of HAProxy.
>>>>>>>
>>>>>>> Users are ideally abstracted from these details and shouldn't have
>>>>> to
>>>>>> wonder which offering to choose [because they are not Xen/HAProxy
>>>>> experts].
>>>>>>> After all one of the goals of CS is to hide these messy details
>>>>> and let
>>>>>> users focus on their apps.
>>>>>>>
>>>>>>> Is there a possibility of a generic way of leaking abstractions for
>>>>>> sufficiently advanced users?
>>>>>>>
>>>>>>
>>>>>
>>>>> Generally as a principle core API's abstract and expose functionality
>>>>> that
>>>>> is commonly available across the hypervisors and network service
>>>>> providers
>>>>> etc.. I believe we should continue to enforce it.
>>>>>
>>>>> But we do have a pattern of API's configure* (configurevirtualrouter,
>>>>> configurenetscaler etc) where a hypervisor/network element plug-in
>>>>>can
>>>>> let
>>>>> admin to configure params specific to plug-in. Perhaps we can use
>>>>>this
>>>>> API's fine tune a plugin for advanced configurations. For e.g both
>>>>> HaProxy
>>>>> max connections, httpModeEnabled can be parameters that can be
>>>>> configured
>>>>> with configureVirtualRouter api.
>>>>>
>>>>> Daan,
>>>>>
>>>>> does this approach works for you?
>>>>>
>>>>> -Murali
>>>>>
>>>>>
>>>>>>> https://reviews.apache.org/r/13238/
>>>>>>> https://reviews.apache.org/r/14320/
>>>>>>> https://reviews.apache.org/r/13896/
>>>>>>>
>>>>>>> BTW, I really prefer that these changes are discussed by putting
>>>>> up an
>>>>> FS
>>>>>> on the wiki rather than submitting patch requests.
>>>>>>> If it touches more than a few files, it is probably worth
>>>>> discussing
>>>>> with
>>>>>> a [DISCUSS] tag line.
>>>>>>> Also, it requires tests.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>
>>
>
Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]
Posted by Chip Childers <ch...@sungard.com>.
> On Oct 3, 2013, at 3:01 PM, Chiradeep Vittal <Ch...@citrix.com> wrote:
>
> I was thinking along the same lines that a offering could have a generic
> key/value map (where the value could be a string/json string/whatever).
> CloudStack core wouldn't have any idea about the semantics of the keys or
> values, but the specific hypervisor/provider might decide to interpret it.
+1 - a model for extensions like that makes perfect sense.
>
>
>> On 10/3/13 11:15 AM, "Darren Shepherd" <da...@gmail.com> wrote:
>>
>> We are always going to run into the issue where they're implementation
>> specific features that can't be represented in a generic way in the
>> API. First class attributes in the API need to essentially be the
>> lowest common denominator. ACS already has this pattern of *_details
>> tables. I honestly don't know completely how they are used, but I
>> would hope that pattern could be used to address these types of
>> issues. By allowing the admin to add arbitrary key/value pairs to the
>> various offerings/entities we can use that to tap into provider
>> specific or advanced configuration. For the http close proposal you
>> could just add haproxy.httpClose=true on the network offering and the
>> haproxy impl can respect that and use it. So implementations can
>> document what additional key/value parameters they support to access
>> advanced features.
>>
>> Review 13896 adds an otherOptions field to the service offering.
>> Conceptually that is fine in my mind, but I just don't understand why
>> the *_details tables can't be used instead. Just like XenServer has
>> other_config on all types, I'd like it if ACS had a similar thing
>> (which I kinda, sorta think it does?). So we should be able to
>> arbitrary data onto any type. Can somebody comment on how this idea
>> may relate with the existing implementation of "details" and "tags" in
>> ACS?
>>
>> Darren
>>
>> On Thu, Oct 3, 2013 at 10:21 AM, Daan Hoogland <da...@gmail.com>
>> wrote:
>>> Murali,
>>>
>>> What our admins need is the ability to instantiate an abstract thing
>>> called
>>> a virtual redundant high available router. They need be able to tune it
>>> with all sorts of features is such a way that other routers redundant or
>>> not and high availible or not, may but do not have to have the same
>>> tuning
>>> parameters. This 'feature' of httpClose is just one of the many things
>>> they
>>> need to be able to tune. Others include a more fine grained control over
>>> the iptables/firewall rules and monitoring of the functionality/daemons
>>> running on the machines. I am not biased as to the way how to
>>> do/implement
>>> this control. The networkoffering seemed like the way to do it.
>>>
>>> Having said that I didn't think that httpClose would be considered
>>> haproxy
>>> specific. It seems worrying that someone could device a proxy or a
>>> loadbalancer that does not support either one of the features of
>>> maximizing - or pooling connections. I'm not into that market recently
>>> so I
>>> might be mistaking. This maybe besides the point but it discomforts me
>>> somewhat.
>>>
>>> regards,
>>> Daan
>>>
>>>
>>> On Thu, Oct 3, 2013 at 2:22 PM, murali reddy
>>> <mu...@gmail.com>wrote:
>>>
>>>> On Thu, Oct 3, 2013 at 4:18 PM, Daan Hoogland <daan.hoogland@gmail.com
>>>>> wrote:
>>>>
>>>>> Chiradeep,
>>>>>
>>>>> I have been thinking about your concern and there is something
>>>> bothering
>>>> me
>>>>> about it. The issue CLOUDSTACK-4328 of which
>>>>> https://reviews.apache.org/r/14320/ is an implementation. This issue
>>>> has
>>>>> been brought up by the engineers at Schuberg Philis. These are cloud
>>>>> operators and domain admins. So these are the people that need to be
>>>>> bothered by this tuning and they are certainly haproxy and xen
>>>> experts to
>>>>> an extend. For them not being able to use connection caching was a
>>>> bug.
>>>> And
>>>>> the proposed solution still seems reasonable to me. It is exposing
>>>> the
>>>>> abstract feature only to those instantiating a vpc, isn't it? This
>>>> last
>>>>> question probably touches on the reason you are addressing my
>>>> submission
>>>>> together with the ones from Nicolas I see only people instantiating
>>>> a
>>>> vpc
>>>>> having to be bothered by which netoffering to use (with respect to
>>>>> httpClose functionality) If this is not the case how should I isolate
>>>> this
>>>>> concern from other , more mondain users?
>>>>>
>>>>> btw I did not create an FS page as the issue was created as a jira
>>>> ticket.
>>>>> I am willing to do that in hind sight but want to have a path to
>>>> follow
>>>>> first.
>>>>>
>>>>> regards,
>>>>> Daan
>>>>>
>>>>> On Tue, Oct 1, 2013 at 11:06 PM, Chiradeep Vittal <
>>>>> Chiradeep.Vittal@citrix.com> wrote:
>>>>>>
>>>>>> We have a couple of people trying to expose the advanced
>>>> capabilities
>>>> of
>>>>> the underlying physical resources to the end-user. In the case of
>>>> Nicolas
>>>>> FOATA, he is trying to expose some of the advanced functions of
>>>>> XenServer/XCP, and in the case of Daan, he is trying to expose some
>>>> feature
>>>>> of HAProxy.
>>>>>>
>>>>>> Users are ideally abstracted from these details and shouldn't have
>>>> to
>>>>> wonder which offering to choose [because they are not Xen/HAProxy
>>>> experts].
>>>>>> After all one of the goals of CS is to hide these messy details
>>>> and let
>>>>> users focus on their apps.
>>>>>>
>>>>>> Is there a possibility of a generic way of leaking abstractions for
>>>>> sufficiently advanced users?
>>>>>>
>>>>>
>>>>
>>>> Generally as a principle core API's abstract and expose functionality
>>>> that
>>>> is commonly available across the hypervisors and network service
>>>> providers
>>>> etc.. I believe we should continue to enforce it.
>>>>
>>>> But we do have a pattern of API's configure* (configurevirtualrouter,
>>>> configurenetscaler etc) where a hypervisor/network element plug-in can
>>>> let
>>>> admin to configure params specific to plug-in. Perhaps we can use this
>>>> API's fine tune a plugin for advanced configurations. For e.g both
>>>> HaProxy
>>>> max connections, httpModeEnabled can be parameters that can be
>>>> configured
>>>> with configureVirtualRouter api.
>>>>
>>>> Daan,
>>>>
>>>> does this approach works for you?
>>>>
>>>> -Murali
>>>>
>>>>
>>>>>> https://reviews.apache.org/r/13238/
>>>>>> https://reviews.apache.org/r/14320/
>>>>>> https://reviews.apache.org/r/13896/
>>>>>>
>>>>>> BTW, I really prefer that these changes are discussed by putting
>>>> up an
>>>> FS
>>>>> on the wiki rather than submitting patch requests.
>>>>>> If it touches more than a few files, it is probably worth
>>>> discussing
>>>> with
>>>>> a [DISCUSS] tag line.
>>>>>> Also, it requires tests.
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>
>
Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896,
14320]
Posted by Chiradeep Vittal <Ch...@citrix.com>.
I was thinking along the same lines that a offering could have a generic
key/value map (where the value could be a string/json string/whatever).
CloudStack core wouldn't have any idea about the semantics of the keys or
values, but the specific hypervisor/provider might decide to interpret it.
On 10/3/13 11:15 AM, "Darren Shepherd" <da...@gmail.com> wrote:
>We are always going to run into the issue where they're implementation
>specific features that can't be represented in a generic way in the
>API. First class attributes in the API need to essentially be the
>lowest common denominator. ACS already has this pattern of *_details
>tables. I honestly don't know completely how they are used, but I
>would hope that pattern could be used to address these types of
>issues. By allowing the admin to add arbitrary key/value pairs to the
>various offerings/entities we can use that to tap into provider
>specific or advanced configuration. For the http close proposal you
>could just add haproxy.httpClose=true on the network offering and the
>haproxy impl can respect that and use it. So implementations can
>document what additional key/value parameters they support to access
>advanced features.
>
>Review 13896 adds an otherOptions field to the service offering.
>Conceptually that is fine in my mind, but I just don't understand why
>the *_details tables can't be used instead. Just like XenServer has
>other_config on all types, I'd like it if ACS had a similar thing
>(which I kinda, sorta think it does?). So we should be able to
>arbitrary data onto any type. Can somebody comment on how this idea
>may relate with the existing implementation of "details" and "tags" in
>ACS?
>
>Darren
>
>On Thu, Oct 3, 2013 at 10:21 AM, Daan Hoogland <da...@gmail.com>
>wrote:
>> Murali,
>>
>> What our admins need is the ability to instantiate an abstract thing
>>called
>> a virtual redundant high available router. They need be able to tune it
>> with all sorts of features is such a way that other routers redundant or
>> not and high availible or not, may but do not have to have the same
>>tuning
>> parameters. This 'feature' of httpClose is just one of the many things
>>they
>> need to be able to tune. Others include a more fine grained control over
>> the iptables/firewall rules and monitoring of the functionality/daemons
>> running on the machines. I am not biased as to the way how to
>>do/implement
>> this control. The networkoffering seemed like the way to do it.
>>
>> Having said that I didn't think that httpClose would be considered
>>haproxy
>> specific. It seems worrying that someone could device a proxy or a
>> loadbalancer that does not support either one of the features of
>> maximizing - or pooling connections. I'm not into that market recently
>>so I
>> might be mistaking. This maybe besides the point but it discomforts me
>> somewhat.
>>
>> regards,
>> Daan
>>
>>
>> On Thu, Oct 3, 2013 at 2:22 PM, murali reddy
>><mu...@gmail.com>wrote:
>>
>>> On Thu, Oct 3, 2013 at 4:18 PM, Daan Hoogland <daan.hoogland@gmail.com
>>> >wrote:
>>>
>>> > Chiradeep,
>>> >
>>> > I have been thinking about your concern and there is something
>>>bothering
>>> me
>>> > about it. The issue CLOUDSTACK-4328 of which
>>> > https://reviews.apache.org/r/14320/ is an implementation. This issue
>>>has
>>> > been brought up by the engineers at Schuberg Philis. These are cloud
>>> > operators and domain admins. So these are the people that need to be
>>> > bothered by this tuning and they are certainly haproxy and xen
>>>experts to
>>> > an extend. For them not being able to use connection caching was a
>>>bug.
>>> And
>>> > the proposed solution still seems reasonable to me. It is exposing
>>>the
>>> > abstract feature only to those instantiating a vpc, isn't it? This
>>>last
>>> > question probably touches on the reason you are addressing my
>>>submission
>>> > together with the ones from Nicolas I see only people instantiating
>>>a
>>> vpc
>>> > having to be bothered by which netoffering to use (with respect to
>>> > httpClose functionality) If this is not the case how should I isolate
>>> this
>>> > concern from other , more mondain users?
>>> >
>>> > btw I did not create an FS page as the issue was created as a jira
>>> ticket.
>>> > I am willing to do that in hind sight but want to have a path to
>>>follow
>>> > first.
>>> >
>>> > regards,
>>> > Daan
>>> >
>>> > On Tue, Oct 1, 2013 at 11:06 PM, Chiradeep Vittal <
>>> > Chiradeep.Vittal@citrix.com> wrote:
>>> > >
>>> > > We have a couple of people trying to expose the advanced
>>>capabilities
>>> of
>>> > the underlying physical resources to the end-user. In the case of
>>>Nicolas
>>> > FOATA, he is trying to expose some of the advanced functions of
>>> > XenServer/XCP, and in the case of Daan, he is trying to expose some
>>> feature
>>> > of HAProxy.
>>> > >
>>> > > Users are ideally abstracted from these details and shouldn't have
>>>to
>>> > wonder which offering to choose [because they are not Xen/HAProxy
>>> experts].
>>> > > After all one of the goals of CS is to hide these messy details
>>>and let
>>> > users focus on their apps.
>>> > >
>>> > > Is there a possibility of a generic way of leaking abstractions for
>>> > sufficiently advanced users?
>>> > >
>>> >
>>>
>>> Generally as a principle core API's abstract and expose functionality
>>>that
>>> is commonly available across the hypervisors and network service
>>>providers
>>> etc.. I believe we should continue to enforce it.
>>>
>>> But we do have a pattern of API's configure* (configurevirtualrouter,
>>> configurenetscaler etc) where a hypervisor/network element plug-in can
>>>let
>>> admin to configure params specific to plug-in. Perhaps we can use this
>>> API's fine tune a plugin for advanced configurations. For e.g both
>>>HaProxy
>>> max connections, httpModeEnabled can be parameters that can be
>>>configured
>>> with configureVirtualRouter api.
>>>
>>> Daan,
>>>
>>> does this approach works for you?
>>>
>>> -Murali
>>>
>>>
>>> > > https://reviews.apache.org/r/13238/
>>> > > https://reviews.apache.org/r/14320/
>>> > > https://reviews.apache.org/r/13896/
>>> > >
>>> > > BTW, I really prefer that these changes are discussed by putting
>>>up an
>>> FS
>>> > on the wiki rather than submitting patch requests.
>>> > > If it touches more than a few files, it is probably worth
>>>discussing
>>> with
>>> > a [DISCUSS] tag line.
>>> > > Also, it requires tests.
>>> > >
>>> > >
>>> > >
>>> >
>>>
Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]
Posted by Darren Shepherd <da...@gmail.com>.
We are always going to run into the issue where they're implementation
specific features that can't be represented in a generic way in the
API. First class attributes in the API need to essentially be the
lowest common denominator. ACS already has this pattern of *_details
tables. I honestly don't know completely how they are used, but I
would hope that pattern could be used to address these types of
issues. By allowing the admin to add arbitrary key/value pairs to the
various offerings/entities we can use that to tap into provider
specific or advanced configuration. For the http close proposal you
could just add haproxy.httpClose=true on the network offering and the
haproxy impl can respect that and use it. So implementations can
document what additional key/value parameters they support to access
advanced features.
Review 13896 adds an otherOptions field to the service offering.
Conceptually that is fine in my mind, but I just don't understand why
the *_details tables can't be used instead. Just like XenServer has
other_config on all types, I'd like it if ACS had a similar thing
(which I kinda, sorta think it does?). So we should be able to
arbitrary data onto any type. Can somebody comment on how this idea
may relate with the existing implementation of "details" and "tags" in
ACS?
Darren
On Thu, Oct 3, 2013 at 10:21 AM, Daan Hoogland <da...@gmail.com> wrote:
> Murali,
>
> What our admins need is the ability to instantiate an abstract thing called
> a virtual redundant high available router. They need be able to tune it
> with all sorts of features is such a way that other routers redundant or
> not and high availible or not, may but do not have to have the same tuning
> parameters. This 'feature' of httpClose is just one of the many things they
> need to be able to tune. Others include a more fine grained control over
> the iptables/firewall rules and monitoring of the functionality/daemons
> running on the machines. I am not biased as to the way how to do/implement
> this control. The networkoffering seemed like the way to do it.
>
> Having said that I didn't think that httpClose would be considered haproxy
> specific. It seems worrying that someone could device a proxy or a
> loadbalancer that does not support either one of the features of
> maximizing - or pooling connections. I'm not into that market recently so I
> might be mistaking. This maybe besides the point but it discomforts me
> somewhat.
>
> regards,
> Daan
>
>
> On Thu, Oct 3, 2013 at 2:22 PM, murali reddy <mu...@gmail.com>wrote:
>
>> On Thu, Oct 3, 2013 at 4:18 PM, Daan Hoogland <daan.hoogland@gmail.com
>> >wrote:
>>
>> > Chiradeep,
>> >
>> > I have been thinking about your concern and there is something bothering
>> me
>> > about it. The issue CLOUDSTACK-4328 of which
>> > https://reviews.apache.org/r/14320/ is an implementation. This issue has
>> > been brought up by the engineers at Schuberg Philis. These are cloud
>> > operators and domain admins. So these are the people that need to be
>> > bothered by this tuning and they are certainly haproxy and xen experts to
>> > an extend. For them not being able to use connection caching was a bug.
>> And
>> > the proposed solution still seems reasonable to me. It is exposing the
>> > abstract feature only to those instantiating a vpc, isn't it? This last
>> > question probably touches on the reason you are addressing my submission
>> > together with the ones from Nicolas I see only people instantiating a
>> vpc
>> > having to be bothered by which netoffering to use (with respect to
>> > httpClose functionality) If this is not the case how should I isolate
>> this
>> > concern from other , more mondain users?
>> >
>> > btw I did not create an FS page as the issue was created as a jira
>> ticket.
>> > I am willing to do that in hind sight but want to have a path to follow
>> > first.
>> >
>> > regards,
>> > Daan
>> >
>> > On Tue, Oct 1, 2013 at 11:06 PM, Chiradeep Vittal <
>> > Chiradeep.Vittal@citrix.com> wrote:
>> > >
>> > > We have a couple of people trying to expose the advanced capabilities
>> of
>> > the underlying physical resources to the end-user. In the case of Nicolas
>> > FOATA, he is trying to expose some of the advanced functions of
>> > XenServer/XCP, and in the case of Daan, he is trying to expose some
>> feature
>> > of HAProxy.
>> > >
>> > > Users are ideally abstracted from these details and shouldn't have to
>> > wonder which offering to choose [because they are not Xen/HAProxy
>> experts].
>> > > After all one of the goals of CS is to hide these messy details and let
>> > users focus on their apps.
>> > >
>> > > Is there a possibility of a generic way of leaking abstractions for
>> > sufficiently advanced users?
>> > >
>> >
>>
>> Generally as a principle core API's abstract and expose functionality that
>> is commonly available across the hypervisors and network service providers
>> etc.. I believe we should continue to enforce it.
>>
>> But we do have a pattern of API's configure* (configurevirtualrouter,
>> configurenetscaler etc) where a hypervisor/network element plug-in can let
>> admin to configure params specific to plug-in. Perhaps we can use this
>> API's fine tune a plugin for advanced configurations. For e.g both HaProxy
>> max connections, httpModeEnabled can be parameters that can be configured
>> with configureVirtualRouter api.
>>
>> Daan,
>>
>> does this approach works for you?
>>
>> -Murali
>>
>>
>> > > https://reviews.apache.org/r/13238/
>> > > https://reviews.apache.org/r/14320/
>> > > https://reviews.apache.org/r/13896/
>> > >
>> > > BTW, I really prefer that these changes are discussed by putting up an
>> FS
>> > on the wiki rather than submitting patch requests.
>> > > If it touches more than a few files, it is probably worth discussing
>> with
>> > a [DISCUSS] tag line.
>> > > Also, it requires tests.
>> > >
>> > >
>> > >
>> >
>>
Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]
Posted by Daan Hoogland <da...@gmail.com>.
Murali,
What our admins need is the ability to instantiate an abstract thing called
a virtual redundant high available router. They need be able to tune it
with all sorts of features is such a way that other routers redundant or
not and high availible or not, may but do not have to have the same tuning
parameters. This 'feature' of httpClose is just one of the many things they
need to be able to tune. Others include a more fine grained control over
the iptables/firewall rules and monitoring of the functionality/daemons
running on the machines. I am not biased as to the way how to do/implement
this control. The networkoffering seemed like the way to do it.
Having said that I didn't think that httpClose would be considered haproxy
specific. It seems worrying that someone could device a proxy or a
loadbalancer that does not support either one of the features of
maximizing - or pooling connections. I'm not into that market recently so I
might be mistaking. This maybe besides the point but it discomforts me
somewhat.
regards,
Daan
On Thu, Oct 3, 2013 at 2:22 PM, murali reddy <mu...@gmail.com>wrote:
> On Thu, Oct 3, 2013 at 4:18 PM, Daan Hoogland <daan.hoogland@gmail.com
> >wrote:
>
> > Chiradeep,
> >
> > I have been thinking about your concern and there is something bothering
> me
> > about it. The issue CLOUDSTACK-4328 of which
> > https://reviews.apache.org/r/14320/ is an implementation. This issue has
> > been brought up by the engineers at Schuberg Philis. These are cloud
> > operators and domain admins. So these are the people that need to be
> > bothered by this tuning and they are certainly haproxy and xen experts to
> > an extend. For them not being able to use connection caching was a bug.
> And
> > the proposed solution still seems reasonable to me. It is exposing the
> > abstract feature only to those instantiating a vpc, isn't it? This last
> > question probably touches on the reason you are addressing my submission
> > together with the ones from Nicolas I see only people instantiating a
> vpc
> > having to be bothered by which netoffering to use (with respect to
> > httpClose functionality) If this is not the case how should I isolate
> this
> > concern from other , more mondain users?
> >
> > btw I did not create an FS page as the issue was created as a jira
> ticket.
> > I am willing to do that in hind sight but want to have a path to follow
> > first.
> >
> > regards,
> > Daan
> >
> > On Tue, Oct 1, 2013 at 11:06 PM, Chiradeep Vittal <
> > Chiradeep.Vittal@citrix.com> wrote:
> > >
> > > We have a couple of people trying to expose the advanced capabilities
> of
> > the underlying physical resources to the end-user. In the case of Nicolas
> > FOATA, he is trying to expose some of the advanced functions of
> > XenServer/XCP, and in the case of Daan, he is trying to expose some
> feature
> > of HAProxy.
> > >
> > > Users are ideally abstracted from these details and shouldn't have to
> > wonder which offering to choose [because they are not Xen/HAProxy
> experts].
> > > After all one of the goals of CS is to hide these messy details and let
> > users focus on their apps.
> > >
> > > Is there a possibility of a generic way of leaking abstractions for
> > sufficiently advanced users?
> > >
> >
>
> Generally as a principle core API's abstract and expose functionality that
> is commonly available across the hypervisors and network service providers
> etc.. I believe we should continue to enforce it.
>
> But we do have a pattern of API's configure* (configurevirtualrouter,
> configurenetscaler etc) where a hypervisor/network element plug-in can let
> admin to configure params specific to plug-in. Perhaps we can use this
> API's fine tune a plugin for advanced configurations. For e.g both HaProxy
> max connections, httpModeEnabled can be parameters that can be configured
> with configureVirtualRouter api.
>
> Daan,
>
> does this approach works for you?
>
> -Murali
>
>
> > > https://reviews.apache.org/r/13238/
> > > https://reviews.apache.org/r/14320/
> > > https://reviews.apache.org/r/13896/
> > >
> > > BTW, I really prefer that these changes are discussed by putting up an
> FS
> > on the wiki rather than submitting patch requests.
> > > If it touches more than a few files, it is probably worth discussing
> with
> > a [DISCUSS] tag line.
> > > Also, it requires tests.
> > >
> > >
> > >
> >
>
Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]
Posted by murali reddy <mu...@gmail.com>.
On Thu, Oct 3, 2013 at 4:18 PM, Daan Hoogland <da...@gmail.com>wrote:
> Chiradeep,
>
> I have been thinking about your concern and there is something bothering me
> about it. The issue CLOUDSTACK-4328 of which
> https://reviews.apache.org/r/14320/ is an implementation. This issue has
> been brought up by the engineers at Schuberg Philis. These are cloud
> operators and domain admins. So these are the people that need to be
> bothered by this tuning and they are certainly haproxy and xen experts to
> an extend. For them not being able to use connection caching was a bug. And
> the proposed solution still seems reasonable to me. It is exposing the
> abstract feature only to those instantiating a vpc, isn't it? This last
> question probably touches on the reason you are addressing my submission
> together with the ones from Nicolas I see only people instantiating a vpc
> having to be bothered by which netoffering to use (with respect to
> httpClose functionality) If this is not the case how should I isolate this
> concern from other , more mondain users?
>
> btw I did not create an FS page as the issue was created as a jira ticket.
> I am willing to do that in hind sight but want to have a path to follow
> first.
>
> regards,
> Daan
>
> On Tue, Oct 1, 2013 at 11:06 PM, Chiradeep Vittal <
> Chiradeep.Vittal@citrix.com> wrote:
> >
> > We have a couple of people trying to expose the advanced capabilities of
> the underlying physical resources to the end-user. In the case of Nicolas
> FOATA, he is trying to expose some of the advanced functions of
> XenServer/XCP, and in the case of Daan, he is trying to expose some feature
> of HAProxy.
> >
> > Users are ideally abstracted from these details and shouldn't have to
> wonder which offering to choose [because they are not Xen/HAProxy experts].
> > After all one of the goals of CS is to hide these messy details and let
> users focus on their apps.
> >
> > Is there a possibility of a generic way of leaking abstractions for
> sufficiently advanced users?
> >
>
Generally as a principle core API's abstract and expose functionality that
is commonly available across the hypervisors and network service providers
etc.. I believe we should continue to enforce it.
But we do have a pattern of API's configure* (configurevirtualrouter,
configurenetscaler etc) where a hypervisor/network element plug-in can let
admin to configure params specific to plug-in. Perhaps we can use this
API's fine tune a plugin for advanced configurations. For e.g both HaProxy
max connections, httpModeEnabled can be parameters that can be configured
with configureVirtualRouter api.
Daan,
does this approach works for you?
-Murali
> > https://reviews.apache.org/r/13238/
> > https://reviews.apache.org/r/14320/
> > https://reviews.apache.org/r/13896/
> >
> > BTW, I really prefer that these changes are discussed by putting up an FS
> on the wiki rather than submitting patch requests.
> > If it touches more than a few files, it is probably worth discussing with
> a [DISCUSS] tag line.
> > Also, it requires tests.
> >
> >
> >
>
Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]
Posted by Daan Hoogland <da...@gmail.com>.
Chiradeep,
I have been thinking about your concern and there is something bothering me
about it. The issue CLOUDSTACK-4328 of which
https://reviews.apache.org/r/14320/ is an implementation. This issue has
been brought up by the engineers at Schuberg Philis. These are cloud
operators and domain admins. So these are the people that need to be
bothered by this tuning and they are certainly haproxy and xen experts to
an extend. For them not being able to use connection caching was a bug. And
the proposed solution still seems reasonable to me. It is exposing the
abstract feature only to those instantiating a vpc, isn't it? This last
question probably touches on the reason you are addressing my submission
together with the ones from Nicolas I see only people instantiating a vpc
having to be bothered by which netoffering to use (with respect to
httpClose functionality) If this is not the case how should I isolate this
concern from other , more mondain users?
btw I did not create an FS page as the issue was created as a jira ticket.
I am willing to do that in hind sight but want to have a path to follow
first.
regards,
Daan
On Tue, Oct 1, 2013 at 11:06 PM, Chiradeep Vittal <
Chiradeep.Vittal@citrix.com> wrote:
>
> We have a couple of people trying to expose the advanced capabilities of
the underlying physical resources to the end-user. In the case of Nicolas
FOATA, he is trying to expose some of the advanced functions of
XenServer/XCP, and in the case of Daan, he is trying to expose some feature
of HAProxy.
>
> Users are ideally abstracted from these details and shouldn't have to
wonder which offering to choose [because they are not Xen/HAProxy experts].
> After all one of the goals of CS is to hide these messy details and let
users focus on their apps.
>
> Is there a possibility of a generic way of leaking abstractions for
sufficiently advanced users?
>
> https://reviews.apache.org/r/13238/
> https://reviews.apache.org/r/14320/
> https://reviews.apache.org/r/13896/
>
> BTW, I really prefer that these changes are discussed by putting up an FS
on the wiki rather than submitting patch requests.
> If it touches more than a few files, it is probably worth discussing with
a [DISCUSS] tag line.
> Also, it requires tests.
>
>
>