You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brooklyn.apache.org by Aled Sage <al...@gmail.com> on 2017/03/01 11:54:03 UTC
[PROPOSAL/DISCUSSION] yaml DSL for invoking effectors
Hi all,
I'd like to resurrect the discussion of whether the yaml DSL should
support invoking effectors.
See https://github.com/apache/brooklyn-server/pull/155. (That was
merged, but Alex will revert it while we discuss if we want it, and if
so then how it would behave).
If folk have additional use-cases and opinions to share, that would be
very useful!
---
Below is what Andrew wrote in his email "[PROPOSAL] Enabling Effective
Effectors" on 30/05/2016, but it wasn't properly discussed then.
## Calling Effectors
The YAML blueprint specification allows entities to be defined with
sensors
and to access the value of sensors for use in configuration. However,
although an entity can include effectors defined in the Java classes, or
using scripting languages (see above) and SSH commands, it is not
possible
to execute effectors and retrieve their results anywhere in a blueprint.
The code in [brooklyn-server#155](
https://github.com/apache/brooklyn-server/pull/155) implements a new
function for the `DslComponent` class that can execute an effector on an
entity and evaluates to its return value. This new function is used as
follows:
```YAML
$brooklyn:entity("other").effector("findInformation"):
args:
arg1: "value"
arg2: 3.14159d
arg3: $brooklyn:attributeWhenReady("host.address")
```
Here we see the effector `findInformation` being evaluated with three
arguments, on the entity with id `other`. One of the arguments is an
`attributeWhenReady` call, thus causing the execution to be delayed
until
the sensor data is available.
---
Alex's philosophical objection in
https://github.com/apache/brooklyn-server/pull/155#issuecomment-283077136
is copied below for your convenience):
right, so the use case is one entity wanting to get information from
another. is there no way this can be accommodated using sensors? my
philosophical objection is that we're introducing first-class
support for a new class of dependency injection:
1. simplest - static: DEP.x is set to a constant
2. blocking - sensor: DEP.x can wait if a value isn't ready yet, ie
it is a promise, $brooklyn:entity(SRC).attributeWhenReady(...),
which once resolved is always taken as the value
3. triggering - effector: every lookup to DEP.x invokes a call
somewhere eg $brooklyn:entity(SRC).effector(...)
A widespread use of (3) scares me [Alex] and it's worth avoiding
this if at all possible. it also means lookups aren't idempotent
(which is why the SideEffecting marker is introduced here, but it
isn't going to work.
could your [use-case] be solved another way, if not with waiting on
a sensor, by the config pointing at the source entity rather than
the key value itself, and whenever it is accessed there is code
which invokes the effector to get the key on that source entity?
---
I believe the original use-case that motivated this was setting up a
docker host, with certificates dynamically generated by a CA Server
(Certificate Authority).
The blueprint has a CA Server entity and several Docker Host entities.
For setting up each Docker Host, we want a new certifiacte that is
signed by the CA. We want to put the certificate files onto the Docker
Host (so that the Docker Engine is correctly configured for TLS).
There is an effector on the CA entity, to send it a Certificate Signing
Request (CSR), and thus to get back a new certificate.
One solution in pure-yaml blueprints (simplified slightly for clarity in
this discussion) would be to have a "certificate" config key on the
Docker Host entity. This would be set to a value like:
brooklyn.config:
certificateData:
$brooklyn:entity("ca").effector("requestCertificate"):
args:
ip: $brooklyn:config("host.address")
Part of the setup script would use this config key, and create a file
using its value.
When the config key was evaluated for the first time, it would execute
the effector and thus get a new certificate. Subsequent lookups of the
config key would use the same value (rather than invoking the effector
multiple times).
Note the significant difference compared to Alex's summary, where Alex
suggested the effector would be invoked every time the config key value
was retrieved.
===
Below are some alternative ways to solve the above use-case. I don't
want us to get too distracted in this discussion of "yaml DSL for
invoking effectors". However, if we reach agreement that one of these
alternative is better then we can ignore the use-case that I've
described above.
_*Resort to Java*_
We could resort to writing some Java code (e.g. write an
"EntityInitializer" that called the requestCertificate effector, and set
the result as a sensor; this could be added to the Docker Host entity.
The rest of the Docker Host blueprint would use attributeWhenReady on
the "certificateData" sensor.
There could be a generic EntityInitializer for invoking a given effector
on a given entity, and setting a sensor with the result.
_*Defining an Effector as a Sequence of Tasks*_
Longer term, we could add YAML support for describing a sequence of
tasks. The basic Docker Host entity would be extended so that its
"start" effector first invoked a task, which would call the
"requestCertificate" effector on the CA (and probably set the result as
a sensor or config key).
Re: [PROPOSAL/DISCUSSION] yaml DSL for invoking effectors
Posted by Alex Heneveld <al...@cloudsoftcorp.com>.
Hi Geoff -
> Would it be too conservative to suggest that we don't do any more on this
> until we do have a clear and compelling use-case in mind, when it will be
> (I hope) a lot easier to see what way it should be done?
On the contrary -- it would be foolish to do something here /without/
a clear and compelling use-case to illuminate the way!
Best
Alex
On 03/03/2017 09:46, Geoff Macartney wrote:
> hi Aled,
>
> taking your points in reverse order:
>
> 4. I agree I can't see a good use case. If it's just doing a lookup with
> no side effects that would surely be more easily done by reading a sensor,
> not invoking an effector.
> 3. It seems that the ability to invoke side effects is the main reason you
> would want to be able to invoke an effector; you're making the entity _do_
> something, maybe with a result; this suggests that...
> 2. ...you would actually want to re-invoke the effector each time you read
> the config. But that sounds risky. In the CA server type example you
> probably wouldn't want to do this - you don't want to get a fresh public
> certificate each time you invoke the "sign-CSR" effector, rather it's
> something you do once when you initialise the entity that needs a
> certificate. I think the problem is that ...
> 1. ...we don't have a clear idea of the use-cases we would need to
> support; I think there are likely several possible use-cases here, but that
> they are likely to have slightly different requirements, especially around
> point 2.
>
> Would it be too conservative to suggest that we don't do any more on this
> until we do have a clear and compelling use-case in mind, when it will be
> (I hope) a lot easier to see what way it should be done? As you say the key
> question is "_should_ we support..." and until we can clearly see "ah yes,
> we need that now, in order to do XYZ" maybe we shouldn't add it in the hope
> that it might be useful.
>
> I'd be keen to hear what @grkvlt thinks on this now, - Andrew?
>
> Geoff
>
>
>
>
> On Thu, 2 Mar 2017 at 11:59 Aled Sage <al...@gmail.com> wrote:
>
>> Hi Geoff,
>>
>> I think long-term you're example would be covered by "Defining an
>> Effector as a Sequence of Tasks". That topic needs a lot more
>> (separate!) discussion, but in brief the idea is that an effector
>> definition is a first-class concept in the yaml (rather than using
>> "entity.initializers"); there would be yaml-support for defining a
>> sequence of tasks (e.g. execute some bash, do a http call, make an
>> effector call, etc). The yaml for "make an effector call" might end up
>> looking similar to this DSL discussion, but the context in which it
>> would be used is very different.
>>
>> ---
>> I think the key question for this discussion is: should we support
>> effector-invocation via the DSL *for config values*?
>>
>> If so:
>>
>> 1. What use-case(s) are we trying to support?
>> 2. Should the effector be executed every time the config value is
>> retrieved, or only once (and the result stored for subsequent lookups)?
>> 3. Should it be relied upon for side-effects (e.g. the CA certificate
>> request example etc)?
>> 4. Or should it only be used for effectors that are getters/lookups?
>>
>> For (3), I now lean towards the use of the EntityInitializer (and longer
>> term the "sequence of tasks" approach).
>>
>> For (4), I'm not sure what a good use-case is (e.g. why can't we set a
>> sensor with that value, which might require extending the "source"
>> entity to add an additional "sensor feed" that retrieves the value for us).
>>
>> For (2), I'm not sure what is most intuitive to users who are
>> reading/writing the blueprint yaml. It could be argued either way, which
>> suggests different users will expect different things! Therefore I'd
>> want to avoid relying on side-effects that you only want to happen once!
>>
>> Aled
>>
>>
>> On 02/03/2017 10:30, Geoff Macartney wrote:
>>> I don't think we should just be thinking about a use-case like the CA
>>> server, which is maybe more limited in behaviour than the more general
>> idea
>>> here of being able to call effectors.
>>>
>>> In particular, what if you wanted to embed the effector invocation on
>> some
>>> entity within the code for an effector of your own entity? i.e. you
>> have
>>> an effector on entity A that does some stuff when invoked, including
>>> invoking an effector on entity B? (And maybe using the result when
>>> calculating its own result instead of storing it in a sensor).
>>>
>>> Is that something we'd want to support? Is it a "stage 2" thing we
>> would
>>> add later on, some time after implementing the EntityInitializer
>> approach?
>>> G
>>>
>>> On Wed, 1 Mar 2017 at 12:32 Alex Heneveld <
>> alex.heneveld@cloudsoftcorp.com>
>>> wrote:
>>>
>>> An `EntityInitializer` for this purpose is a nice alternative pattern --
>>> better than the child entity I suggested at #155 -- until we have
>> sequence
>>> effector YAML. Is there anything `$brooklyn:effector` gives us that an
>>> initializer wouldn't do in a cleaner way?
>>>
>>> Best
>>> Alex
>>>
>>>
>>> On 1 March 2017 at 11:54, Aled Sage <al...@gmail.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I'd like to resurrect the discussion of whether the yaml DSL should
>>>> support invoking effectors.
>>>>
>>>> See https://github.com/apache/brooklyn-server/pull/155. (That was
>> merged,
>>>> but Alex will revert it while we discuss if we want it, and if so then
>> how
>>>> it would behave).
>>>>
>>>> If folk have additional use-cases and opinions to share, that would be
>>>> very useful!
>>>>
>>>> ---
>>>>
>>>> Below is what Andrew wrote in his email "[PROPOSAL] Enabling Effective
>>>> Effectors" on 30/05/2016, but it wasn't properly discussed then.
>>>>
>>>> ## Calling Effectors
>>>>
>>>> The YAML blueprint specification allows entities to be defined with
>>>> sensors
>>>> and to access the value of sensors for use in configuration.
>> However,
>>>> although an entity can include effectors defined in the Java
>> classes,
>>> or
>>>> using scripting languages (see above) and SSH commands, it is not
>>>> possible
>>>> to execute effectors and retrieve their results anywhere in a
>>> blueprint.
>>>> The code in [brooklyn-server#155](
>>>> https://github.com/apache/brooklyn-server/pull/155) implements a
>> new
>>>> function for the `DslComponent` class that can execute an effector
>> on
>>> an
>>>> entity and evaluates to its return value. This new function is used
>> as
>>>> follows:
>>>>
>>>> ```YAML
>>>> $brooklyn:entity("other").effector("findInformation"):
>>>> args:
>>>> arg1: "value"
>>>> arg2: 3.14159d
>>>> arg3: $brooklyn:attributeWhenReady("host.address")
>>>> ```
>>>>
>>>> Here we see the effector `findInformation` being evaluated with
>> three
>>>> arguments, on the entity with id `other`. One of the arguments is an
>>>> `attributeWhenReady` call, thus causing the execution to be delayed
>>>> until
>>>> the sensor data is available.
>>>>
>>>> ---
>>>> Alex's philosophical objection in https://github.com/apache/broo
>>>> klyn-server/pull/155#issuecomment-283077136 is copied below for your
>>>> convenience):
>>>>
>>>> right, so the use case is one entity wanting to get information from
>>>> another. is there no way this can be accommodated using sensors? my
>>>> philosophical objection is that we're introducing first-class
>>>> support for a new class of dependency injection:
>>>>
>>>> 1. simplest - static: DEP.x is set to a constant
>>>> 2. blocking - sensor: DEP.x can wait if a value isn't ready yet, ie
>>>> it is a promise, $brooklyn:entity(SRC).attributeWhenReady(...),
>>>> which once resolved is always taken as the value
>>>> 3. triggering - effector: every lookup to DEP.x invokes a call
>>>> somewhere eg $brooklyn:entity(SRC).effector(...)
>>>>
>>>> A widespread use of (3) scares me [Alex] and it's worth avoiding
>>>> this if at all possible. it also means lookups aren't idempotent
>>>> (which is why the SideEffecting marker is introduced here, but it
>>>> isn't going to work.
>>>>
>>>> could your [use-case] be solved another way, if not with waiting on
>>>> a sensor, by the config pointing at the source entity rather than
>>>> the key value itself, and whenever it is accessed there is code
>>>> which invokes the effector to get the key on that source entity?
>>>>
>>>>
>>>> ---
>>>> I believe the original use-case that motivated this was setting up a
>>>> docker host, with certificates dynamically generated by a CA Server
>>>> (Certificate Authority).
>>>>
>>>> The blueprint has a CA Server entity and several Docker Host entities.
>> For
>>>> setting up each Docker Host, we want a new certifiacte that is signed by
>>>> the CA. We want to put the certificate files onto the Docker Host (so
>> that
>>>> the Docker Engine is correctly configured for TLS).
>>>>
>>>> There is an effector on the CA entity, to send it a Certificate Signing
>>>> Request (CSR), and thus to get back a new certificate.
>>>>
>>>> One solution in pure-yaml blueprints (simplified slightly for clarity in
>>>> this discussion) would be to have a "certificate" config key on the
>> Docker
>>>> Host entity. This would be set to a value like:
>>>>
>>>> brooklyn.config:
>>>> certificateData:
>>>> $brooklyn:entity("ca").effector("requestCertificate"):
>>>> args:
>>>> ip: $brooklyn:config("host.address")
>>>>
>>>> Part of the setup script would use this config key, and create a file
>>>> using its value.
>>>>
>>>> When the config key was evaluated for the first time, it would execute
>> the
>>>> effector and thus get a new certificate. Subsequent lookups of the
>> config
>>>> key would use the same value (rather than invoking the effector multiple
>>>> times).
>>>>
>>>> Note the significant difference compared to Alex's summary, where Alex
>>>> suggested the effector would be invoked every time the config key value
>>> was
>>>> retrieved.
>>>>
>>>> ===
>>>> Below are some alternative ways to solve the above use-case. I don't
>> want
>>>> us to get too distracted in this discussion of "yaml DSL for invoking
>>>> effectors". However, if we reach agreement that one of these alternative
>>> is
>>>> better then we can ignore the use-case that I've described above.
>>>>
>>>> _*Resort to Java*_
>>>> We could resort to writing some Java code (e.g. write an
>>>> "EntityInitializer" that called the requestCertificate effector, and set
>>>> the result as a sensor; this could be added to the Docker Host entity.
>> The
>>>> rest of the Docker Host blueprint would use attributeWhenReady on the
>>>> "certificateData" sensor.
>>>>
>>>> There could be a generic EntityInitializer for invoking a given effector
>>>> on a given entity, and setting a sensor with the result.
>>>>
>>>> _*Defining an Effector as a Sequence of Tasks*_
>>>> Longer term, we could add YAML support for describing a sequence of
>> tasks.
>>>> The basic Docker Host entity would be extended so that its "start"
>>> effector
>>>> first invoked a task, which would call the "requestCertificate" effector
>>> on
>>>> the CA (and probably set the result as a sensor or config key).
>>>>
>>>>
>>
Re: [PROPOSAL/DISCUSSION] yaml DSL for invoking effectors
Posted by Geoff Macartney <ge...@cloudsoftcorp.com>.
hi Aled,
taking your points in reverse order:
4. I agree I can't see a good use case. If it's just doing a lookup with
no side effects that would surely be more easily done by reading a sensor,
not invoking an effector.
3. It seems that the ability to invoke side effects is the main reason you
would want to be able to invoke an effector; you're making the entity _do_
something, maybe with a result; this suggests that...
2. ...you would actually want to re-invoke the effector each time you read
the config. But that sounds risky. In the CA server type example you
probably wouldn't want to do this - you don't want to get a fresh public
certificate each time you invoke the "sign-CSR" effector, rather it's
something you do once when you initialise the entity that needs a
certificate. I think the problem is that ...
1. ...we don't have a clear idea of the use-cases we would need to
support; I think there are likely several possible use-cases here, but that
they are likely to have slightly different requirements, especially around
point 2.
Would it be too conservative to suggest that we don't do any more on this
until we do have a clear and compelling use-case in mind, when it will be
(I hope) a lot easier to see what way it should be done? As you say the key
question is "_should_ we support..." and until we can clearly see "ah yes,
we need that now, in order to do XYZ" maybe we shouldn't add it in the hope
that it might be useful.
I'd be keen to hear what @grkvlt thinks on this now, - Andrew?
Geoff
On Thu, 2 Mar 2017 at 11:59 Aled Sage <al...@gmail.com> wrote:
> Hi Geoff,
>
> I think long-term you're example would be covered by "Defining an
> Effector as a Sequence of Tasks". That topic needs a lot more
> (separate!) discussion, but in brief the idea is that an effector
> definition is a first-class concept in the yaml (rather than using
> "entity.initializers"); there would be yaml-support for defining a
> sequence of tasks (e.g. execute some bash, do a http call, make an
> effector call, etc). The yaml for "make an effector call" might end up
> looking similar to this DSL discussion, but the context in which it
> would be used is very different.
>
> ---
> I think the key question for this discussion is: should we support
> effector-invocation via the DSL *for config values*?
>
> If so:
>
> 1. What use-case(s) are we trying to support?
> 2. Should the effector be executed every time the config value is
> retrieved, or only once (and the result stored for subsequent lookups)?
> 3. Should it be relied upon for side-effects (e.g. the CA certificate
> request example etc)?
> 4. Or should it only be used for effectors that are getters/lookups?
>
> For (3), I now lean towards the use of the EntityInitializer (and longer
> term the "sequence of tasks" approach).
>
> For (4), I'm not sure what a good use-case is (e.g. why can't we set a
> sensor with that value, which might require extending the "source"
> entity to add an additional "sensor feed" that retrieves the value for us).
>
> For (2), I'm not sure what is most intuitive to users who are
> reading/writing the blueprint yaml. It could be argued either way, which
> suggests different users will expect different things! Therefore I'd
> want to avoid relying on side-effects that you only want to happen once!
>
> Aled
>
>
> On 02/03/2017 10:30, Geoff Macartney wrote:
> > I don't think we should just be thinking about a use-case like the CA
> > server, which is maybe more limited in behaviour than the more general
> idea
> > here of being able to call effectors.
> >
> > In particular, what if you wanted to embed the effector invocation on
> some
> > entity within the code for an effector of your own entity? i.e. you
> have
> > an effector on entity A that does some stuff when invoked, including
> > invoking an effector on entity B? (And maybe using the result when
> > calculating its own result instead of storing it in a sensor).
> >
> > Is that something we'd want to support? Is it a "stage 2" thing we
> would
> > add later on, some time after implementing the EntityInitializer
> approach?
> >
> > G
> >
> > On Wed, 1 Mar 2017 at 12:32 Alex Heneveld <
> alex.heneveld@cloudsoftcorp.com>
> > wrote:
> >
> > An `EntityInitializer` for this purpose is a nice alternative pattern --
> > better than the child entity I suggested at #155 -- until we have
> sequence
> > effector YAML. Is there anything `$brooklyn:effector` gives us that an
> > initializer wouldn't do in a cleaner way?
> >
> > Best
> > Alex
> >
> >
> > On 1 March 2017 at 11:54, Aled Sage <al...@gmail.com> wrote:
> >
> >> Hi all,
> >>
> >> I'd like to resurrect the discussion of whether the yaml DSL should
> >> support invoking effectors.
> >>
> >> See https://github.com/apache/brooklyn-server/pull/155. (That was
> merged,
> >> but Alex will revert it while we discuss if we want it, and if so then
> how
> >> it would behave).
> >>
> >> If folk have additional use-cases and opinions to share, that would be
> >> very useful!
> >>
> >> ---
> >>
> >> Below is what Andrew wrote in his email "[PROPOSAL] Enabling Effective
> >> Effectors" on 30/05/2016, but it wasn't properly discussed then.
> >>
> >> ## Calling Effectors
> >>
> >> The YAML blueprint specification allows entities to be defined with
> >> sensors
> >> and to access the value of sensors for use in configuration.
> However,
> >> although an entity can include effectors defined in the Java
> classes,
> > or
> >> using scripting languages (see above) and SSH commands, it is not
> >> possible
> >> to execute effectors and retrieve their results anywhere in a
> > blueprint.
> >> The code in [brooklyn-server#155](
> >> https://github.com/apache/brooklyn-server/pull/155) implements a
> new
> >> function for the `DslComponent` class that can execute an effector
> on
> > an
> >> entity and evaluates to its return value. This new function is used
> as
> >> follows:
> >>
> >> ```YAML
> >> $brooklyn:entity("other").effector("findInformation"):
> >> args:
> >> arg1: "value"
> >> arg2: 3.14159d
> >> arg3: $brooklyn:attributeWhenReady("host.address")
> >> ```
> >>
> >> Here we see the effector `findInformation` being evaluated with
> three
> >> arguments, on the entity with id `other`. One of the arguments is an
> >> `attributeWhenReady` call, thus causing the execution to be delayed
> >> until
> >> the sensor data is available.
> >>
> >> ---
> >> Alex's philosophical objection in https://github.com/apache/broo
> >> klyn-server/pull/155#issuecomment-283077136 is copied below for your
> >> convenience):
> >>
> >> right, so the use case is one entity wanting to get information from
> >> another. is there no way this can be accommodated using sensors? my
> >> philosophical objection is that we're introducing first-class
> >> support for a new class of dependency injection:
> >>
> >> 1. simplest - static: DEP.x is set to a constant
> >> 2. blocking - sensor: DEP.x can wait if a value isn't ready yet, ie
> >> it is a promise, $brooklyn:entity(SRC).attributeWhenReady(...),
> >> which once resolved is always taken as the value
> >> 3. triggering - effector: every lookup to DEP.x invokes a call
> >> somewhere eg $brooklyn:entity(SRC).effector(...)
> >>
> >> A widespread use of (3) scares me [Alex] and it's worth avoiding
> >> this if at all possible. it also means lookups aren't idempotent
> >> (which is why the SideEffecting marker is introduced here, but it
> >> isn't going to work.
> >>
> >> could your [use-case] be solved another way, if not with waiting on
> >> a sensor, by the config pointing at the source entity rather than
> >> the key value itself, and whenever it is accessed there is code
> >> which invokes the effector to get the key on that source entity?
> >>
> >>
> >> ---
> >> I believe the original use-case that motivated this was setting up a
> >> docker host, with certificates dynamically generated by a CA Server
> >> (Certificate Authority).
> >>
> >> The blueprint has a CA Server entity and several Docker Host entities.
> For
> >> setting up each Docker Host, we want a new certifiacte that is signed by
> >> the CA. We want to put the certificate files onto the Docker Host (so
> that
> >> the Docker Engine is correctly configured for TLS).
> >>
> >> There is an effector on the CA entity, to send it a Certificate Signing
> >> Request (CSR), and thus to get back a new certificate.
> >>
> >> One solution in pure-yaml blueprints (simplified slightly for clarity in
> >> this discussion) would be to have a "certificate" config key on the
> Docker
> >> Host entity. This would be set to a value like:
> >>
> >> brooklyn.config:
> >> certificateData:
> >> $brooklyn:entity("ca").effector("requestCertificate"):
> >> args:
> >> ip: $brooklyn:config("host.address")
> >>
> >> Part of the setup script would use this config key, and create a file
> >> using its value.
> >>
> >> When the config key was evaluated for the first time, it would execute
> the
> >> effector and thus get a new certificate. Subsequent lookups of the
> config
> >> key would use the same value (rather than invoking the effector multiple
> >> times).
> >>
> >> Note the significant difference compared to Alex's summary, where Alex
> >> suggested the effector would be invoked every time the config key value
> > was
> >> retrieved.
> >>
> >> ===
> >> Below are some alternative ways to solve the above use-case. I don't
> want
> >> us to get too distracted in this discussion of "yaml DSL for invoking
> >> effectors". However, if we reach agreement that one of these alternative
> > is
> >> better then we can ignore the use-case that I've described above.
> >>
> >> _*Resort to Java*_
> >> We could resort to writing some Java code (e.g. write an
> >> "EntityInitializer" that called the requestCertificate effector, and set
> >> the result as a sensor; this could be added to the Docker Host entity.
> The
> >> rest of the Docker Host blueprint would use attributeWhenReady on the
> >> "certificateData" sensor.
> >>
> >> There could be a generic EntityInitializer for invoking a given effector
> >> on a given entity, and setting a sensor with the result.
> >>
> >> _*Defining an Effector as a Sequence of Tasks*_
> >> Longer term, we could add YAML support for describing a sequence of
> tasks.
> >> The basic Docker Host entity would be extended so that its "start"
> > effector
> >> first invoked a task, which would call the "requestCertificate" effector
> > on
> >> the CA (and probably set the result as a sensor or config key).
> >>
> >>
>
>
Re: [PROPOSAL/DISCUSSION] yaml DSL for invoking effectors
Posted by Aled Sage <al...@gmail.com>.
Hi Geoff,
I think long-term you're example would be covered by "Defining an
Effector as a Sequence of Tasks". That topic needs a lot more
(separate!) discussion, but in brief the idea is that an effector
definition is a first-class concept in the yaml (rather than using
"entity.initializers"); there would be yaml-support for defining a
sequence of tasks (e.g. execute some bash, do a http call, make an
effector call, etc). The yaml for "make an effector call" might end up
looking similar to this DSL discussion, but the context in which it
would be used is very different.
---
I think the key question for this discussion is: should we support
effector-invocation via the DSL *for config values*?
If so:
1. What use-case(s) are we trying to support?
2. Should the effector be executed every time the config value is
retrieved, or only once (and the result stored for subsequent lookups)?
3. Should it be relied upon for side-effects (e.g. the CA certificate
request example etc)?
4. Or should it only be used for effectors that are getters/lookups?
For (3), I now lean towards the use of the EntityInitializer (and longer
term the "sequence of tasks" approach).
For (4), I'm not sure what a good use-case is (e.g. why can't we set a
sensor with that value, which might require extending the "source"
entity to add an additional "sensor feed" that retrieves the value for us).
For (2), I'm not sure what is most intuitive to users who are
reading/writing the blueprint yaml. It could be argued either way, which
suggests different users will expect different things! Therefore I'd
want to avoid relying on side-effects that you only want to happen once!
Aled
On 02/03/2017 10:30, Geoff Macartney wrote:
> I don't think we should just be thinking about a use-case like the CA
> server, which is maybe more limited in behaviour than the more general idea
> here of being able to call effectors.
>
> In particular, what if you wanted to embed the effector invocation on some
> entity within the code for an effector of your own entity? i.e. you have
> an effector on entity A that does some stuff when invoked, including
> invoking an effector on entity B? (And maybe using the result when
> calculating its own result instead of storing it in a sensor).
>
> Is that something we'd want to support? Is it a "stage 2" thing we would
> add later on, some time after implementing the EntityInitializer approach?
>
> G
>
> On Wed, 1 Mar 2017 at 12:32 Alex Heneveld <al...@cloudsoftcorp.com>
> wrote:
>
> An `EntityInitializer` for this purpose is a nice alternative pattern --
> better than the child entity I suggested at #155 -- until we have sequence
> effector YAML. Is there anything `$brooklyn:effector` gives us that an
> initializer wouldn't do in a cleaner way?
>
> Best
> Alex
>
>
> On 1 March 2017 at 11:54, Aled Sage <al...@gmail.com> wrote:
>
>> Hi all,
>>
>> I'd like to resurrect the discussion of whether the yaml DSL should
>> support invoking effectors.
>>
>> See https://github.com/apache/brooklyn-server/pull/155. (That was merged,
>> but Alex will revert it while we discuss if we want it, and if so then how
>> it would behave).
>>
>> If folk have additional use-cases and opinions to share, that would be
>> very useful!
>>
>> ---
>>
>> Below is what Andrew wrote in his email "[PROPOSAL] Enabling Effective
>> Effectors" on 30/05/2016, but it wasn't properly discussed then.
>>
>> ## Calling Effectors
>>
>> The YAML blueprint specification allows entities to be defined with
>> sensors
>> and to access the value of sensors for use in configuration. However,
>> although an entity can include effectors defined in the Java classes,
> or
>> using scripting languages (see above) and SSH commands, it is not
>> possible
>> to execute effectors and retrieve their results anywhere in a
> blueprint.
>> The code in [brooklyn-server#155](
>> https://github.com/apache/brooklyn-server/pull/155) implements a new
>> function for the `DslComponent` class that can execute an effector on
> an
>> entity and evaluates to its return value. This new function is used as
>> follows:
>>
>> ```YAML
>> $brooklyn:entity("other").effector("findInformation"):
>> args:
>> arg1: "value"
>> arg2: 3.14159d
>> arg3: $brooklyn:attributeWhenReady("host.address")
>> ```
>>
>> Here we see the effector `findInformation` being evaluated with three
>> arguments, on the entity with id `other`. One of the arguments is an
>> `attributeWhenReady` call, thus causing the execution to be delayed
>> until
>> the sensor data is available.
>>
>> ---
>> Alex's philosophical objection in https://github.com/apache/broo
>> klyn-server/pull/155#issuecomment-283077136 is copied below for your
>> convenience):
>>
>> right, so the use case is one entity wanting to get information from
>> another. is there no way this can be accommodated using sensors? my
>> philosophical objection is that we're introducing first-class
>> support for a new class of dependency injection:
>>
>> 1. simplest - static: DEP.x is set to a constant
>> 2. blocking - sensor: DEP.x can wait if a value isn't ready yet, ie
>> it is a promise, $brooklyn:entity(SRC).attributeWhenReady(...),
>> which once resolved is always taken as the value
>> 3. triggering - effector: every lookup to DEP.x invokes a call
>> somewhere eg $brooklyn:entity(SRC).effector(...)
>>
>> A widespread use of (3) scares me [Alex] and it's worth avoiding
>> this if at all possible. it also means lookups aren't idempotent
>> (which is why the SideEffecting marker is introduced here, but it
>> isn't going to work.
>>
>> could your [use-case] be solved another way, if not with waiting on
>> a sensor, by the config pointing at the source entity rather than
>> the key value itself, and whenever it is accessed there is code
>> which invokes the effector to get the key on that source entity?
>>
>>
>> ---
>> I believe the original use-case that motivated this was setting up a
>> docker host, with certificates dynamically generated by a CA Server
>> (Certificate Authority).
>>
>> The blueprint has a CA Server entity and several Docker Host entities. For
>> setting up each Docker Host, we want a new certifiacte that is signed by
>> the CA. We want to put the certificate files onto the Docker Host (so that
>> the Docker Engine is correctly configured for TLS).
>>
>> There is an effector on the CA entity, to send it a Certificate Signing
>> Request (CSR), and thus to get back a new certificate.
>>
>> One solution in pure-yaml blueprints (simplified slightly for clarity in
>> this discussion) would be to have a "certificate" config key on the Docker
>> Host entity. This would be set to a value like:
>>
>> brooklyn.config:
>> certificateData:
>> $brooklyn:entity("ca").effector("requestCertificate"):
>> args:
>> ip: $brooklyn:config("host.address")
>>
>> Part of the setup script would use this config key, and create a file
>> using its value.
>>
>> When the config key was evaluated for the first time, it would execute the
>> effector and thus get a new certificate. Subsequent lookups of the config
>> key would use the same value (rather than invoking the effector multiple
>> times).
>>
>> Note the significant difference compared to Alex's summary, where Alex
>> suggested the effector would be invoked every time the config key value
> was
>> retrieved.
>>
>> ===
>> Below are some alternative ways to solve the above use-case. I don't want
>> us to get too distracted in this discussion of "yaml DSL for invoking
>> effectors". However, if we reach agreement that one of these alternative
> is
>> better then we can ignore the use-case that I've described above.
>>
>> _*Resort to Java*_
>> We could resort to writing some Java code (e.g. write an
>> "EntityInitializer" that called the requestCertificate effector, and set
>> the result as a sensor; this could be added to the Docker Host entity. The
>> rest of the Docker Host blueprint would use attributeWhenReady on the
>> "certificateData" sensor.
>>
>> There could be a generic EntityInitializer for invoking a given effector
>> on a given entity, and setting a sensor with the result.
>>
>> _*Defining an Effector as a Sequence of Tasks*_
>> Longer term, we could add YAML support for describing a sequence of tasks.
>> The basic Docker Host entity would be extended so that its "start"
> effector
>> first invoked a task, which would call the "requestCertificate" effector
> on
>> the CA (and probably set the result as a sensor or config key).
>>
>>
Re: [PROPOSAL/DISCUSSION] yaml DSL for invoking effectors
Posted by Geoff Macartney <ge...@cloudsoftcorp.com>.
I don't think we should just be thinking about a use-case like the CA
server, which is maybe more limited in behaviour than the more general idea
here of being able to call effectors.
In particular, what if you wanted to embed the effector invocation on some
entity within the code for an effector of your own entity? i.e. you have
an effector on entity A that does some stuff when invoked, including
invoking an effector on entity B? (And maybe using the result when
calculating its own result instead of storing it in a sensor).
Is that something we'd want to support? Is it a "stage 2" thing we would
add later on, some time after implementing the EntityInitializer approach?
G
On Wed, 1 Mar 2017 at 12:32 Alex Heneveld <al...@cloudsoftcorp.com>
wrote:
An `EntityInitializer` for this purpose is a nice alternative pattern --
better than the child entity I suggested at #155 -- until we have sequence
effector YAML. Is there anything `$brooklyn:effector` gives us that an
initializer wouldn't do in a cleaner way?
Best
Alex
On 1 March 2017 at 11:54, Aled Sage <al...@gmail.com> wrote:
> Hi all,
>
> I'd like to resurrect the discussion of whether the yaml DSL should
> support invoking effectors.
>
> See https://github.com/apache/brooklyn-server/pull/155. (That was merged,
> but Alex will revert it while we discuss if we want it, and if so then how
> it would behave).
>
> If folk have additional use-cases and opinions to share, that would be
> very useful!
>
> ---
>
> Below is what Andrew wrote in his email "[PROPOSAL] Enabling Effective
> Effectors" on 30/05/2016, but it wasn't properly discussed then.
>
> ## Calling Effectors
>
> The YAML blueprint specification allows entities to be defined with
> sensors
> and to access the value of sensors for use in configuration. However,
> although an entity can include effectors defined in the Java classes,
or
> using scripting languages (see above) and SSH commands, it is not
> possible
> to execute effectors and retrieve their results anywhere in a
blueprint.
>
> The code in [brooklyn-server#155](
> https://github.com/apache/brooklyn-server/pull/155) implements a new
> function for the `DslComponent` class that can execute an effector on
an
> entity and evaluates to its return value. This new function is used as
> follows:
>
> ```YAML
> $brooklyn:entity("other").effector("findInformation"):
> args:
> arg1: "value"
> arg2: 3.14159d
> arg3: $brooklyn:attributeWhenReady("host.address")
> ```
>
> Here we see the effector `findInformation` being evaluated with three
> arguments, on the entity with id `other`. One of the arguments is an
> `attributeWhenReady` call, thus causing the execution to be delayed
> until
> the sensor data is available.
>
> ---
> Alex's philosophical objection in https://github.com/apache/broo
> klyn-server/pull/155#issuecomment-283077136 is copied below for your
> convenience):
>
> right, so the use case is one entity wanting to get information from
> another. is there no way this can be accommodated using sensors? my
> philosophical objection is that we're introducing first-class
> support for a new class of dependency injection:
>
> 1. simplest - static: DEP.x is set to a constant
> 2. blocking - sensor: DEP.x can wait if a value isn't ready yet, ie
> it is a promise, $brooklyn:entity(SRC).attributeWhenReady(...),
> which once resolved is always taken as the value
> 3. triggering - effector: every lookup to DEP.x invokes a call
> somewhere eg $brooklyn:entity(SRC).effector(...)
>
> A widespread use of (3) scares me [Alex] and it's worth avoiding
> this if at all possible. it also means lookups aren't idempotent
> (which is why the SideEffecting marker is introduced here, but it
> isn't going to work.
>
> could your [use-case] be solved another way, if not with waiting on
> a sensor, by the config pointing at the source entity rather than
> the key value itself, and whenever it is accessed there is code
> which invokes the effector to get the key on that source entity?
>
>
> ---
> I believe the original use-case that motivated this was setting up a
> docker host, with certificates dynamically generated by a CA Server
> (Certificate Authority).
>
> The blueprint has a CA Server entity and several Docker Host entities. For
> setting up each Docker Host, we want a new certifiacte that is signed by
> the CA. We want to put the certificate files onto the Docker Host (so that
> the Docker Engine is correctly configured for TLS).
>
> There is an effector on the CA entity, to send it a Certificate Signing
> Request (CSR), and thus to get back a new certificate.
>
> One solution in pure-yaml blueprints (simplified slightly for clarity in
> this discussion) would be to have a "certificate" config key on the Docker
> Host entity. This would be set to a value like:
>
> brooklyn.config:
> certificateData:
> $brooklyn:entity("ca").effector("requestCertificate"):
> args:
> ip: $brooklyn:config("host.address")
>
> Part of the setup script would use this config key, and create a file
> using its value.
>
> When the config key was evaluated for the first time, it would execute the
> effector and thus get a new certificate. Subsequent lookups of the config
> key would use the same value (rather than invoking the effector multiple
> times).
>
> Note the significant difference compared to Alex's summary, where Alex
> suggested the effector would be invoked every time the config key value
was
> retrieved.
>
> ===
> Below are some alternative ways to solve the above use-case. I don't want
> us to get too distracted in this discussion of "yaml DSL for invoking
> effectors". However, if we reach agreement that one of these alternative
is
> better then we can ignore the use-case that I've described above.
>
> _*Resort to Java*_
> We could resort to writing some Java code (e.g. write an
> "EntityInitializer" that called the requestCertificate effector, and set
> the result as a sensor; this could be added to the Docker Host entity. The
> rest of the Docker Host blueprint would use attributeWhenReady on the
> "certificateData" sensor.
>
> There could be a generic EntityInitializer for invoking a given effector
> on a given entity, and setting a sensor with the result.
>
> _*Defining an Effector as a Sequence of Tasks*_
> Longer term, we could add YAML support for describing a sequence of tasks.
> The basic Docker Host entity would be extended so that its "start"
effector
> first invoked a task, which would call the "requestCertificate" effector
on
> the CA (and probably set the result as a sensor or config key).
>
>
Re: [PROPOSAL/DISCUSSION] yaml DSL for invoking effectors
Posted by Alex Heneveld <al...@cloudsoftcorp.com>.
An `EntityInitializer` for this purpose is a nice alternative pattern --
better than the child entity I suggested at #155 -- until we have sequence
effector YAML. Is there anything `$brooklyn:effector` gives us that an
initializer wouldn't do in a cleaner way?
Best
Alex
On 1 March 2017 at 11:54, Aled Sage <al...@gmail.com> wrote:
> Hi all,
>
> I'd like to resurrect the discussion of whether the yaml DSL should
> support invoking effectors.
>
> See https://github.com/apache/brooklyn-server/pull/155. (That was merged,
> but Alex will revert it while we discuss if we want it, and if so then how
> it would behave).
>
> If folk have additional use-cases and opinions to share, that would be
> very useful!
>
> ---
>
> Below is what Andrew wrote in his email "[PROPOSAL] Enabling Effective
> Effectors" on 30/05/2016, but it wasn't properly discussed then.
>
> ## Calling Effectors
>
> The YAML blueprint specification allows entities to be defined with
> sensors
> and to access the value of sensors for use in configuration. However,
> although an entity can include effectors defined in the Java classes, or
> using scripting languages (see above) and SSH commands, it is not
> possible
> to execute effectors and retrieve their results anywhere in a blueprint.
>
> The code in [brooklyn-server#155](
> https://github.com/apache/brooklyn-server/pull/155) implements a new
> function for the `DslComponent` class that can execute an effector on an
> entity and evaluates to its return value. This new function is used as
> follows:
>
> ```YAML
> $brooklyn:entity("other").effector("findInformation"):
> args:
> arg1: "value"
> arg2: 3.14159d
> arg3: $brooklyn:attributeWhenReady("host.address")
> ```
>
> Here we see the effector `findInformation` being evaluated with three
> arguments, on the entity with id `other`. One of the arguments is an
> `attributeWhenReady` call, thus causing the execution to be delayed
> until
> the sensor data is available.
>
> ---
> Alex's philosophical objection in https://github.com/apache/broo
> klyn-server/pull/155#issuecomment-283077136 is copied below for your
> convenience):
>
> right, so the use case is one entity wanting to get information from
> another. is there no way this can be accommodated using sensors? my
> philosophical objection is that we're introducing first-class
> support for a new class of dependency injection:
>
> 1. simplest - static: DEP.x is set to a constant
> 2. blocking - sensor: DEP.x can wait if a value isn't ready yet, ie
> it is a promise, $brooklyn:entity(SRC).attributeWhenReady(...),
> which once resolved is always taken as the value
> 3. triggering - effector: every lookup to DEP.x invokes a call
> somewhere eg $brooklyn:entity(SRC).effector(...)
>
> A widespread use of (3) scares me [Alex] and it's worth avoiding
> this if at all possible. it also means lookups aren't idempotent
> (which is why the SideEffecting marker is introduced here, but it
> isn't going to work.
>
> could your [use-case] be solved another way, if not with waiting on
> a sensor, by the config pointing at the source entity rather than
> the key value itself, and whenever it is accessed there is code
> which invokes the effector to get the key on that source entity?
>
>
> ---
> I believe the original use-case that motivated this was setting up a
> docker host, with certificates dynamically generated by a CA Server
> (Certificate Authority).
>
> The blueprint has a CA Server entity and several Docker Host entities. For
> setting up each Docker Host, we want a new certifiacte that is signed by
> the CA. We want to put the certificate files onto the Docker Host (so that
> the Docker Engine is correctly configured for TLS).
>
> There is an effector on the CA entity, to send it a Certificate Signing
> Request (CSR), and thus to get back a new certificate.
>
> One solution in pure-yaml blueprints (simplified slightly for clarity in
> this discussion) would be to have a "certificate" config key on the Docker
> Host entity. This would be set to a value like:
>
> brooklyn.config:
> certificateData:
> $brooklyn:entity("ca").effector("requestCertificate"):
> args:
> ip: $brooklyn:config("host.address")
>
> Part of the setup script would use this config key, and create a file
> using its value.
>
> When the config key was evaluated for the first time, it would execute the
> effector and thus get a new certificate. Subsequent lookups of the config
> key would use the same value (rather than invoking the effector multiple
> times).
>
> Note the significant difference compared to Alex's summary, where Alex
> suggested the effector would be invoked every time the config key value was
> retrieved.
>
> ===
> Below are some alternative ways to solve the above use-case. I don't want
> us to get too distracted in this discussion of "yaml DSL for invoking
> effectors". However, if we reach agreement that one of these alternative is
> better then we can ignore the use-case that I've described above.
>
> _*Resort to Java*_
> We could resort to writing some Java code (e.g. write an
> "EntityInitializer" that called the requestCertificate effector, and set
> the result as a sensor; this could be added to the Docker Host entity. The
> rest of the Docker Host blueprint would use attributeWhenReady on the
> "certificateData" sensor.
>
> There could be a generic EntityInitializer for invoking a given effector
> on a given entity, and setting a sensor with the result.
>
> _*Defining an Effector as a Sequence of Tasks*_
> Longer term, we could add YAML support for describing a sequence of tasks.
> The basic Docker Host entity would be extended so that its "start" effector
> first invoked a task, which would call the "requestCertificate" effector on
> the CA (and probably set the result as a sensor or config key).
>
>