You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brooklyn.apache.org by Graham Ashby <Gr...@ca.ibm.com> on 2017/03/23 17:39:31 UTC
YAML Transformer propagation
This should be easy, but...
I have a successful sensor now, thanks to Aled's help.
But, it is defined on the cluster, and I want to use it on the cluster
entities, as well as other entities in the application.
I tried the following on the entity, but it just hangs:
brooklyn.config:
cm_cluster_first_host:
$Brooklyn:entity("cm_cluster").attributeWhenReady("cluster_first_host")
So, what is the "correct" way?
Thanks
Graham
From: Aled Sage <al...@gmail.com>
To: dev@brooklyn.apache.org
Date: 03/23/2017 09:15 AM
Subject: Re: Trouble with YAML Transformer
Hi Graham,
I think that is non-deterministic: it failed approx 50% of the time in
my simple unit test.
As described in BROOKLYN-458, the problem is that it will evaluate the
expression when the sensor "cluster.first.entity" is set. It will
immediately look up the "host.name" of that entity. If that sensor is
aleady set, then great. If it is not yet set, it will return immediately
without setting "cluster_first_host". Nothing will subsequently trigger
the transformer again (because "clustere.first.entity" does not change),
until Brooklyn is restarted or the first.entity is replaced.
There is a workaround suggested in BROOKLYN-458 that uses
"triggerSensors" (and an extra aggregated sensor!) to ensure that the
value is re-evaluated when the entity gets its host.name.
Aled
p.s. if you deploy this to a location where it takes a while for
"host.name" to become available and thus populated (e.g. where it
requires provisioning VMs), then it's much more likely to fail. If that
is working for you consistently then let me know as I'd be interested to
find out why!
On 23/03/2017 13:07, Graham Ashby wrote:
> Thanks all for all your help.
>
> For the record, what finally worked is:
>
> - type: org.apache.brooklyn.enricher.stock.Transformer
> brooklyn.config:
> enricher.sourceSensor:
$brooklyn:sensor("cluster.first.entity")
> enricher.targetSensor: $brooklyn:sensor("cluster_first_host")
> enricher.targetValue:
>
$brooklyn:entity($brooklyn:attributeWhenReady("cluster.first.entity")).attributeWhenReady("host.name")
>
> Graham
>
>
>
> From: Aled Sage <al...@gmail.com>
> To: dev@brooklyn.apache.org
> Date: 03/23/2017 07:47 AM
> Subject: Re: Trouble with YAML Transformer
>
>
>
> Hi Graham, Mike, all,
>
> Graham: for why your transformer isn't populating the sensor value, see
> the explanation I've written up at
> https://issues.apache.org/jira/browse/BROOKLYN-458.
>
> For a workaround, see my first comment on that jira issue:
>
>
>
https://issues.apache.org/jira/browse/BROOKLYN-458?focusedCommentId=15938141&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15938141
>
>
>
> ---
> For the single-arg `$brooklyn:sensor(...)`...
>
> `$brooklyn:sensor("cluster.first.entity")` returns a new
> `AttributeSensor<Object>` instance, with name "cluster.first.entity". It
> doesn't look up the actual value of the sensor on the entity.
>
> What it means by "the given sensor on the current entity if found" is
> that: if the entity declares a strongly typed sensor with exactly that
> name (e.g. that also has a description and/or a type that is not
> Object), then it will be the entity's own sensor instance returned
> rather than constructing a new instance.
>
> ---
> Mike: as discussed with you offline, the error for:
>
> $brooklyn:sensor("cluster.first.entity").attributeWhenReady("host.name")
>
> is reported at https://issues.apache.org/jira/browse/BROOKLYN-454, along
> with a workaround.
>
> The workaround is to use:
>
>
>
$brooklyn:entity($brooklyn:sensor("cluster.first.entity")).attributeWhenReady("host.name")
>
> ---
> For the record, Mike also hit
> https://issues.apache.org/jira/browse/BROOKLYN-457 while exploring
> possible solutions for Graham's use-case.
>
> ---
> Mike, could you please see what Brooklyn docs we can improve to make
> this clearer, and easier to use/understand? Also what should we improve
> to make investigating/debugging such problems easier?
>
> Aled
>
>
> On 22/03/2017 20:52, Graham Ashby wrote:
>> And if I try:
>>
>> enricher.targetValue:
>>
$brooklyn:sensor("cluster.first.entity").attributeWhenReady("host.name")
>>
>> I get the error:
>>
>> Error evaluating node (rethrowing)
>>
>
'$brooklyn:sensor("cluster.first.entity").attributeWhenReady("host.name")':
>> java.lang.IllegalArgumentException: No such function
> 'attributeWhenReady'
>> on sensor("cluster.first.entity")
>>
>>
>>
>> From: Mike Zaccardo <mi...@cloudsoftcorp.com>
>> To: dev@brooklyn.apache.org
>> Date: 03/22/2017 04:31 PM
>> Subject: Re: Trouble with YAML Transformer
>>
>>
>>
>> From the DSL spec [
>>
>
https://docs.cloudsoft.io/blueprints/syntax-yaml-reference.html#dsl-commands
>
>> ]:
>>
>> * $brooklyn:sensor("sensor.name") returns the given sensor on the
> current
>> entity if found, or an untyped (Object) sensor;
>>
>> * $brooklyn:sensor("com.acme.brooklyn.ContainingEntityClass",
>> "sensor.name")
>> returns the strongly typed sensor defined in the given class
>>
>> Based on that, it seemed to me that the first, single-argument value
> would
>> actually return the value of the sensor (the "object"), not the sensor
>> construct itself (the "class"). However, I tried updating your code
and
>> ended up with an empty-valued sensor.
>>
>> brooklyn.enrichers:
>> - type: org.apache.brooklyn.enricher.stock.Transformer
>> brooklyn.config:
>> enricher.sourceSensor:
>> $brooklyn:sensor("cluster.first.entity")
>> enricher.targetSensor:
> $brooklyn:sensor("cluster_first_host")
>> enricher.targetValue:
>> $brooklyn:formatString:
>> - "%s"
>> -
>>
$brooklyn:sensor("cluster.first.entity").attributeWhenReady("host.name")
>>
>> I'd like to kick the question to the rest of the community -- why does
>> that
>> ^ end up with an empty value? It should default to searching the
current
>> entity, which is a `DynamicCluster`, which has a
`cluster.first.entity`.
>>
>> -Mike
>>
>> On Wed, Mar 22, 2017 at 12:52 PM Graham Ashby <Gr...@ca.ibm.com>
>> wrote:
>>
>> OK, Stupid mistake.
>> So, when I changed it to colon, I got the error:
>> 2017-03-22 15:41:43,612 WARN o.a.b.c.b.s.d.BrooklynDslInterpreter
>> [brooklyn-jetty-server-8081-qtp434786215-42]: Error evaluating node
>> (rethrowing) '$brooklyn:sensor("org.apache.brooklyn.api.entity.Entity",
>> "cluster.first.entity").attributeWhenReady("host.name")':
>> org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: Sensor
>> cluster.first.entity not found on class
>> org.apache.brooklyn.api.entity.Entity
>>
>> So I changed the Cast:
>>
>> enricher.targetValue:
>> $brooklyn:sensor("org.apache.brooklyn.entity.group.AbstractGroup",
>> "cluster.first.entity").attributeWhenReady("host.name")
>>
>> Now I get:
>>
>> 2017-03-22 15:43:50,909 WARN o.a.b.c.b.s.d.BrooklynDslInterpreter
>> [brooklyn-jetty-server-8081-qtp434786215-2217]: Error evaluating node
>> (rethrowing)
>> '$brooklyn:sensor("org.apache.brooklyn.entity.group.AbstractGroup",
>> "cluster.first.entity").attributeWhenReady("host.name")':
>> java.lang.IllegalArgumentException: Not permitted to invoke function on
>> 'class org.apache.brooklyn.core.sensor.BasicAttributeSensor' (outside
>> allowed package scope)
>>
>> Thanks
>> G
>>
>>
>>
>> From: Mike Zaccardo <mi...@cloudsoftcorp.com>
>> To: Brooklyn dev <de...@brooklyn.apache.org>
>> Date: 03/22/2017 03:30 PM
>> Subject: Re: Trouble with YAML Transformer
>>
>>
>>
>> Hi Graham,
>>
>> In your `enricher.targetValue` -- `$brooklyn.sensor` should be
>> `$brooklyn:sensor`
>> (the dot replaced by a colon).
>>
>> If that does not solve it, try providing the `enricher.targetValue`
with
>> string, like so:
>>
>> brooklyn.enrichers:
>> - type: org.apache.brooklyn.enricher.stock.Transformer
>> brooklyn.config:
>> enricher.sourceSensor:
> $brooklyn:sensor("cluster.first.entity")
>> enricher.targetSensor:
$brooklyn:sensor("cluster_first_host")
>> enricher.targetValue:
>> $brooklyn:formatString:
>> - "%s"
>> -
>> $brooklyn:sensor("org.apache.brooklyn.api.entity.Entity",
>> "cluster.first.entity").attributeWhenReady("
>> host.name")
>>
>> I hope this helps.
>>
>> Cheers,
>> Mike
>>
>> On Wed, Mar 22, 2017 at 12:10 PM Graham Ashby <Gr...@ca.ibm.com>
>> wrote:
>>
>> What I want is an attribute that I can include in a template.
>> I'm trying to get the host.name of the first entity in the cluster, so
>> what I've got is:
>>
>> brooklyn.enrichers:
>> - type: org.apache.brooklyn.enricher.stock.Transformer
>> brooklyn.config:
>> enricher.sourceSensor:
> $brooklyn:sensor("cluster.first.entity")
>> enricher.targetSensor:
$brooklyn:sensor("cluster_first_host")
>> enricher.targetValue:
>> $brooklyn.sensor("org.apache.brooklyn.api.entity.Entity",
>> "cluster.first.entity").attributeWhenReady("host.name")
>>
>> But I get the literal value
>> "$brooklyn.sensor("org.apache.brooklyn.api.entity.Entity",
>> "cluster.first.entity").attributeWhenReady("host.name")"
>>
>> So, what am I doing wrong?
>>
>> Thanks
>> Graham
>>
>> --
>>
>> Mike Zaccardo
>>
>> Senior Software Engineer
>>
>> Skype: mike.cloudsoft
>>
>> Twitter: @ItsMeMikeZ <https://twitter.com/ItsMeMikeZ> LinkedIn:
>> linkedin.com/in/mike-zaccardo
>> <https://www.linkedin.com/in/mike-zaccardo-aa3ab812>
>>
>> [image: Cloudsoft Logo.jpg] <https://cloudsoft.io/>
>>
>> Stay up to date with everything Cloudsoft:
>>
>> [image: Twitter_Logo_White_On_Blue.png] <https://twitter.com/cloudsoft>
>> [image:
>> YouTube-social-icon_red_48px.png]
>> <https://www.youtube.com/channel/UCpbLhvXrYWz8B_osUX6rn0Q>
>>
>>
>>
>>
>
>
>
>
>
Re: YAML Transformer propagation
Posted by Aled Sage <al...@gmail.com>.
Hi Graham,
That yaml snippet looks fine to me (except it should be lower-case
$brooklyn).
Can you share more of your blueprint please?
It might hang depending when the entity tries to use the config
`cm_cluster_first_host` - if it tries to use this value before the
"host.name" sensor has been set, then it will block until that's
available (which could conceivably block the necessary step(s) to
actually get the "host.name").
The blueprint below for me works (i.e. I checked that the "Node" entity
has the config key "cm_cluster_first_host", and that the app was in a
started state - I deployed it to localhost):
brooklyn.catalog:
version: 0.2-SNAPSHOT
items:
- id: test-cluster
name: test-cluster
item:
type: org.apache.brooklyn.entity.group.DynamicCluster
id: cm_cluster
brooklyn.config:
memberSpec:
$brooklyn:entitySpec:
type:
org.apache.brooklyn.entity.software.base.EmptySoftwareProcess
name: "Node"
brooklyn.config:
cm_cluster_first_host:
$brooklyn:entity("cm_cluster").attributeWhenReady("cluster_first_host")
brooklyn.enrichers:
- type: org.apache.brooklyn.enricher.stock.Aggregator
brooklyn.config:
enricher.aggregating.fromMembers: true
enricher.transformation.untyped: list
enricher.sourceSensor: $brooklyn:sensor("host.name")
enricher.targetSensor: $brooklyn:sensor("members.hostnames")
- type: org.apache.brooklyn.enricher.stock.Transformer
brooklyn.config:
enricher.triggerSensors:
- $brooklyn:sensor("cluster.first.entity")
- $brooklyn:sensor("members.hostnames")
enricher.targetSensor: $brooklyn:sensor("cluster_first_host")
enricher.targetValue:
$brooklyn:formatString:
- "%s"
-
$brooklyn:entity($brooklyn:attributeWhenReady("cluster.first.entity")).attributeWhenReady("host.name")
On 23/03/2017 17:39, Graham Ashby wrote:
> This should be easy, but...
>
> I have a successful sensor now, thanks to Aled's help.
> But, it is defined on the cluster, and I want to use it on the cluster
> entities, as well as other entities in the application.
>
> I tried the following on the entity, but it just hangs:
>
> brooklyn.config:
> cm_cluster_first_host:
> $Brooklyn:entity("cm_cluster").attributeWhenReady("cluster_first_host")
>
> So, what is the "correct" way?
>
> Thanks
> Graham
>
>
>
> From: Aled Sage <al...@gmail.com>
> To: dev@brooklyn.apache.org
> Date: 03/23/2017 09:15 AM
> Subject: Re: Trouble with YAML Transformer
>
>
>
> Hi Graham,
>
> I think that is non-deterministic: it failed approx 50% of the time in
> my simple unit test.
>
> As described in BROOKLYN-458, the problem is that it will evaluate the
> expression when the sensor "cluster.first.entity" is set. It will
> immediately look up the "host.name" of that entity. If that sensor is
> aleady set, then great. If it is not yet set, it will return immediately
> without setting "cluster_first_host". Nothing will subsequently trigger
> the transformer again (because "clustere.first.entity" does not change),
> until Brooklyn is restarted or the first.entity is replaced.
>
> There is a workaround suggested in BROOKLYN-458 that uses
> "triggerSensors" (and an extra aggregated sensor!) to ensure that the
> value is re-evaluated when the entity gets its host.name.
>
> Aled
>
> p.s. if you deploy this to a location where it takes a while for
> "host.name" to become available and thus populated (e.g. where it
> requires provisioning VMs), then it's much more likely to fail. If that
> is working for you consistently then let me know as I'd be interested to
> find out why!
>
>
> On 23/03/2017 13:07, Graham Ashby wrote:
>> Thanks all for all your help.
>>
>> For the record, what finally worked is:
>>
>> - type: org.apache.brooklyn.enricher.stock.Transformer
>> brooklyn.config:
>> enricher.sourceSensor:
> $brooklyn:sensor("cluster.first.entity")
>> enricher.targetSensor: $brooklyn:sensor("cluster_first_host")
>> enricher.targetValue:
>>
> $brooklyn:entity($brooklyn:attributeWhenReady("cluster.first.entity")).attributeWhenReady("host.name")
>> Graham
>>
>>
>>
>> From: Aled Sage <al...@gmail.com>
>> To: dev@brooklyn.apache.org
>> Date: 03/23/2017 07:47 AM
>> Subject: Re: Trouble with YAML Transformer
>>
>>
>>
>> Hi Graham, Mike, all,
>>
>> Graham: for why your transformer isn't populating the sensor value, see
>> the explanation I've written up at
>> https://issues.apache.org/jira/browse/BROOKLYN-458.
>>
>> For a workaround, see my first comment on that jira issue:
>>
>>
>>
> https://issues.apache.org/jira/browse/BROOKLYN-458?focusedCommentId=15938141&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15938141
>
>>
>>
>> ---
>> For the single-arg `$brooklyn:sensor(...)`...
>>
>> `$brooklyn:sensor("cluster.first.entity")` returns a new
>> `AttributeSensor<Object>` instance, with name "cluster.first.entity". It
>> doesn't look up the actual value of the sensor on the entity.
>>
>> What it means by "the given sensor on the current entity if found" is
>> that: if the entity declares a strongly typed sensor with exactly that
>> name (e.g. that also has a description and/or a type that is not
>> Object), then it will be the entity's own sensor instance returned
>> rather than constructing a new instance.
>>
>> ---
>> Mike: as discussed with you offline, the error for:
>>
>> $brooklyn:sensor("cluster.first.entity").attributeWhenReady("host.name")
>>
>> is reported at https://issues.apache.org/jira/browse/BROOKLYN-454, along
>> with a workaround.
>>
>> The workaround is to use:
>>
>>
>>
> $brooklyn:entity($brooklyn:sensor("cluster.first.entity")).attributeWhenReady("host.name")
>> ---
>> For the record, Mike also hit
>> https://issues.apache.org/jira/browse/BROOKLYN-457 while exploring
>> possible solutions for Graham's use-case.
>>
>> ---
>> Mike, could you please see what Brooklyn docs we can improve to make
>> this clearer, and easier to use/understand? Also what should we improve
>> to make investigating/debugging such problems easier?
>>
>> Aled
>>
>>
>> On 22/03/2017 20:52, Graham Ashby wrote:
>>> And if I try:
>>>
>>> enricher.targetValue:
>>>
> $brooklyn:sensor("cluster.first.entity").attributeWhenReady("host.name")
>>> I get the error:
>>>
>>> Error evaluating node (rethrowing)
>>>
> '$brooklyn:sensor("cluster.first.entity").attributeWhenReady("host.name")':
>>> java.lang.IllegalArgumentException: No such function
>> 'attributeWhenReady'
>>> on sensor("cluster.first.entity")
>>>
>>>
>>>
>>> From: Mike Zaccardo <mi...@cloudsoftcorp.com>
>>> To: dev@brooklyn.apache.org
>>> Date: 03/22/2017 04:31 PM
>>> Subject: Re: Trouble with YAML Transformer
>>>
>>>
>>>
>>> From the DSL spec [
>>>
> https://docs.cloudsoft.io/blueprints/syntax-yaml-reference.html#dsl-commands
>
>>> ]:
>>>
>>> * $brooklyn:sensor("sensor.name") returns the given sensor on the
>> current
>>> entity if found, or an untyped (Object) sensor;
>>>
>>> * $brooklyn:sensor("com.acme.brooklyn.ContainingEntityClass",
>>> "sensor.name")
>>> returns the strongly typed sensor defined in the given class
>>>
>>> Based on that, it seemed to me that the first, single-argument value
>> would
>>> actually return the value of the sensor (the "object"), not the sensor
>>> construct itself (the "class"). However, I tried updating your code
> and
>>> ended up with an empty-valued sensor.
>>>
>>> brooklyn.enrichers:
>>> - type: org.apache.brooklyn.enricher.stock.Transformer
>>> brooklyn.config:
>>> enricher.sourceSensor:
>>> $brooklyn:sensor("cluster.first.entity")
>>> enricher.targetSensor:
>> $brooklyn:sensor("cluster_first_host")
>>> enricher.targetValue:
>>> $brooklyn:formatString:
>>> - "%s"
>>> -
>>>
> $brooklyn:sensor("cluster.first.entity").attributeWhenReady("host.name")
>>> I'd like to kick the question to the rest of the community -- why does
>>> that
>>> ^ end up with an empty value? It should default to searching the
> current
>>> entity, which is a `DynamicCluster`, which has a
> `cluster.first.entity`.
>>> -Mike
>>>
>>> On Wed, Mar 22, 2017 at 12:52 PM Graham Ashby <Gr...@ca.ibm.com>
>>> wrote:
>>>
>>> OK, Stupid mistake.
>>> So, when I changed it to colon, I got the error:
>>> 2017-03-22 15:41:43,612 WARN o.a.b.c.b.s.d.BrooklynDslInterpreter
>>> [brooklyn-jetty-server-8081-qtp434786215-42]: Error evaluating node
>>> (rethrowing) '$brooklyn:sensor("org.apache.brooklyn.api.entity.Entity",
>>> "cluster.first.entity").attributeWhenReady("host.name")':
>>> org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: Sensor
>>> cluster.first.entity not found on class
>>> org.apache.brooklyn.api.entity.Entity
>>>
>>> So I changed the Cast:
>>>
>>> enricher.targetValue:
>>> $brooklyn:sensor("org.apache.brooklyn.entity.group.AbstractGroup",
>>> "cluster.first.entity").attributeWhenReady("host.name")
>>>
>>> Now I get:
>>>
>>> 2017-03-22 15:43:50,909 WARN o.a.b.c.b.s.d.BrooklynDslInterpreter
>>> [brooklyn-jetty-server-8081-qtp434786215-2217]: Error evaluating node
>>> (rethrowing)
>>> '$brooklyn:sensor("org.apache.brooklyn.entity.group.AbstractGroup",
>>> "cluster.first.entity").attributeWhenReady("host.name")':
>>> java.lang.IllegalArgumentException: Not permitted to invoke function on
>>> 'class org.apache.brooklyn.core.sensor.BasicAttributeSensor' (outside
>>> allowed package scope)
>>>
>>> Thanks
>>> G
>>>
>>>
>>>
>>> From: Mike Zaccardo <mi...@cloudsoftcorp.com>
>>> To: Brooklyn dev <de...@brooklyn.apache.org>
>>> Date: 03/22/2017 03:30 PM
>>> Subject: Re: Trouble with YAML Transformer
>>>
>>>
>>>
>>> Hi Graham,
>>>
>>> In your `enricher.targetValue` -- `$brooklyn.sensor` should be
>>> `$brooklyn:sensor`
>>> (the dot replaced by a colon).
>>>
>>> If that does not solve it, try providing the `enricher.targetValue`
> with
>>> string, like so:
>>>
>>> brooklyn.enrichers:
>>> - type: org.apache.brooklyn.enricher.stock.Transformer
>>> brooklyn.config:
>>> enricher.sourceSensor:
>> $brooklyn:sensor("cluster.first.entity")
>>> enricher.targetSensor:
> $brooklyn:sensor("cluster_first_host")
>>> enricher.targetValue:
>>> $brooklyn:formatString:
>>> - "%s"
>>> -
>>> $brooklyn:sensor("org.apache.brooklyn.api.entity.Entity",
>>> "cluster.first.entity").attributeWhenReady("
>>> host.name")
>>>
>>> I hope this helps.
>>>
>>> Cheers,
>>> Mike
>>>
>>> On Wed, Mar 22, 2017 at 12:10 PM Graham Ashby <Gr...@ca.ibm.com>
>>> wrote:
>>>
>>> What I want is an attribute that I can include in a template.
>>> I'm trying to get the host.name of the first entity in the cluster, so
>>> what I've got is:
>>>
>>> brooklyn.enrichers:
>>> - type: org.apache.brooklyn.enricher.stock.Transformer
>>> brooklyn.config:
>>> enricher.sourceSensor:
>> $brooklyn:sensor("cluster.first.entity")
>>> enricher.targetSensor:
> $brooklyn:sensor("cluster_first_host")
>>> enricher.targetValue:
>>> $brooklyn.sensor("org.apache.brooklyn.api.entity.Entity",
>>> "cluster.first.entity").attributeWhenReady("host.name")
>>>
>>> But I get the literal value
>>> "$brooklyn.sensor("org.apache.brooklyn.api.entity.Entity",
>>> "cluster.first.entity").attributeWhenReady("host.name")"
>>>
>>> So, what am I doing wrong?
>>>
>>> Thanks
>>> Graham
>>>
>>> --
>>>
>>> Mike Zaccardo
>>>
>>> Senior Software Engineer
>>>
>>> Skype: mike.cloudsoft
>>>
>>> Twitter: @ItsMeMikeZ <https://twitter.com/ItsMeMikeZ> LinkedIn:
>>> linkedin.com/in/mike-zaccardo
>>> <https://www.linkedin.com/in/mike-zaccardo-aa3ab812>
>>>
>>> [image: Cloudsoft Logo.jpg] <https://cloudsoft.io/>
>>>
>>> Stay up to date with everything Cloudsoft:
>>>
>>> [image: Twitter_Logo_White_On_Blue.png] <https://twitter.com/cloudsoft>
>>> [image:
>>> YouTube-social-icon_red_48px.png]
>>> <https://www.youtube.com/channel/UCpbLhvXrYWz8B_osUX6rn0Q>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>
>
>
>
>