You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brooklyn.apache.org by ahgittin <gi...@git.apache.org> on 2015/06/22 00:02:56 UTC
[GitHub] incubator-brooklyn pull request: Catalog default niceness
GitHub user ahgittin opened a pull request:
https://github.com/apache/incubator-brooklyn/pull/705
Catalog default niceness
set up a default catalog which includes:
* four YAML application templates to get started with, teaching most of what people should know
* scanned items plus some convenient short-names for common entities
makes the Brooklyn code so that the four sample templates can give a pretty nice user experience with sensors and enrichers, and they all run without warnings in the log or ugliness in the activities view (in prep for 0.7.0 release):
* resolution of values is more thorough, respecting timeouts, and run as transient
* new enrichers for time-weighted-delta and rolling averages from YAML
also the default catalog comes from the CLI project, and is copied to the dist, so that we don't have to maintain two variants but can test from the IDE of the CLI project
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ahgittin/incubator-brooklyn catalog-default-niceness
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-brooklyn/pull/705.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #705
----
commit 4280eef873415c63a88571f23492ef30bb6c5bba
Author: Alex Heneveld <al...@cloudsoftcorp.com>
Date: 2015-06-10T18:00:21Z
exclude bom files from rat check, where these are intended for the user to modify
commit b35d671750e057380dfa76c82d5df0f86b79c6b3
Author: Alex Heneveld <al...@cloudsoftcorp.com>
Date: 2015-06-15T10:48:59Z
merge the two assembly xml files in the dist (DRY)
commit d59356d08d56b4e169cce035a983169d19afe362
Author: Alex Heneveld <al...@cloudsoftcorp.com>
Date: 2015-06-15T10:54:27Z
fix warnings in dist build
commit ef69cc3ed425af29ceb36c2a28e7971adf80fb1a
Author: Alex Heneveld <al...@cloudsoftcorp.com>
Date: 2015-06-15T16:34:16Z
copy the brooklyn/default.catalog.bom from the usage/cli project into the dist
this allows the same default bom to be used from the IDE as included in the dist,
and changes only need to be made in a single place (in the cli project)
commit ed0f1210d81e90d4d8d384ec2d3ccab946c2423a
Author: Alex Heneveld <al...@cloudsoftcorp.com>
Date: 2015-06-18T15:55:16Z
adjust synch model for controller
should not synch on class as that is used for things like getExecutionContext (and so can deadlock)
commit 02a954cbe46ec708dc4a207d1a410fc5a6554b00
Author: Alex Heneveld <al...@cloudsoftcorp.com>
Date: 2015-06-18T15:56:17Z
improve ValueResolver task+timeout, and related work on tasks
ValueResolver will now look if it is already in a task when computing the execution context,
so more things which need a context and/or have timeouts can be handled correctly --
esp when $brooklyn:config is passed from yaml to various places
More tasks are marked transient so don't clutter the GUI.
Cancellations are treated better, with fewer errors.
Also shell env vars where values are null are now treated as "" rather than "null",
and a few other string handling goodies
commit a9bf2c61d6decd00a96e5a8bcfb44c14df48c07c
Author: Alex Heneveld <al...@cloudsoftcorp.com>
Date: 2015-06-19T11:56:39Z
yaml-friendly enrichers for delta and rolling avg
commit be41a3db6bb39a5cefda6e11b5744e46b61e6f2e
Author: Alex Heneveld <al...@cloudsoftcorp.com>
Date: 2015-06-20T18:13:49Z
skip validation for "template" catalog items as that isn't always supported
prevents errors on startup and in log for catalog rest api calls
commit 0aa8f68d91079ca0f8fbe6fc948b4c8fed8781cb
Author: Alex Heneveld <al...@cloudsoftcorp.com>
Date: 2015-06-21T17:47:12Z
more enrichers ease-of-use
aggregator doesn't require "fromMembers", instead it defaults sensibly,
plus better errors and messages around enrichers and policies
commit 8be8ea7016fbe3bfb9db9cda7a005341ff6820a5
Author: Alex Heneveld <al...@cloudsoftcorp.com>
Date: 2015-06-20T21:30:33Z
in VanillaSoftware, look at shell.env for custom install and customize scripts
commit 07c0dd8a176bbef480847c6343f283abaabbd66c
Author: Alex Heneveld <al...@cloudsoftcorp.com>
Date: 2015-06-20T21:30:55Z
change how freemarker logging is set to remove warning about "logging libraray"
commit 513aa4b9b04542f3bf57675622d103b1eca3ee4f
Author: Alex Heneveld <al...@cloudsoftcorp.com>
Date: 2015-06-20T23:33:41Z
treat tasks submitted by transient tasks as transient by default
commit 5f3976aaf827664975d533bba2b3ca6161ea58b4
Author: Alex Heneveld <al...@cloudsoftcorp.com>
Date: 2015-06-21T17:49:30Z
remove unnecessary (jboss) parameters in nginx health-check
commit d5cbe49e148734a6e1e3a058127cf5dc663de934
Author: Alex Heneveld <al...@cloudsoftcorp.com>
Date: 2015-06-19T12:04:12Z
expand the default catalog to have several nice example templates
commit 4908eb10d2c2e69b42495d58c7a76c6955efe0fb
Author: Alex Heneveld <al...@cloudsoftcorp.com>
Date: 2015-06-21T18:58:50Z
expand string-to-sensor coercion to make sensors easier to write in yaml
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33119062
--- Diff: usage/dist/pom.xml ---
@@ -63,6 +63,35 @@
<build>
<plugins>
<plugin>
+ <artifactId>maven-dependency-plugin</artifactId>
+ <!-- copy the config file from the CLI project -->
+ <executions>
+ <execution>
+ <id>copy</id>
--- End diff --
I wondered if there was such a thing. That's cool. I'd have used it if I'd known.
However I do like having an artifact for the `bom` as it is easier to access directly. Maybe it makes sense for that to be a standard pattern for bundles. We'll see. For now I'll leave it but add a comment about your suggestion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114335673
According to http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-vpc.html#vpc-only-instance-types these instance types -- C4, M4, T2 -- are only supported in VPC, not in "classic". The jclouds PR includes code to prefer instance types available in both, and so would fix the problem, but the Brooklyn cut only takes the image half of that PR. So one option would be to copy more of the PR. But I'm still investigating...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-115269512
I've found the problem @mikezaccardo referred to. The logging re "location" is spurious, I've downgraded that message in 6648495077669378a6540d939e9d8f44a68a1ff6 . Currently when running `Main` in an IDE you don't have the load-balancer project and so that type does not load, and that fails the catalog load.
It's an IDE-only issue, but at least now we get sensible warnings. The dist will be fine.
The commit above also fixes the two `YamlTime...` comments @neykov made above.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33043741
--- Diff: usage/dist/pom.xml ---
@@ -63,6 +63,35 @@
<build>
<plugins>
<plugin>
+ <artifactId>maven-dependency-plugin</artifactId>
+ <!-- copy the config file from the CLI project -->
+ <executions>
+ <execution>
+ <id>copy</id>
--- End diff --
Better use `unpack` goal with `includes` filter. This will make the bom artifact type on cli unnecessary.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33042854
--- Diff: usage/dist/pom.xml ---
@@ -98,9 +132,14 @@
<configuration>
<appendAssemblyId>true</appendAssemblyId>
<descriptors>
- <descriptor>src/main/config/build-distribution-archive.xml</descriptor>
+ <descriptor>src/main/config/build-distribution.xml</descriptor>
</descriptors>
<finalName>brooklyn-${project.version}</finalName>
+ <formats>
+ <format>tar.gz</format>
+ <format>zip</format>
+ </formats>
+ <attach>false</attach>
--- End diff --
I think we should stick to the default behaviour of attaching the artifacts here and remove the explicit `attach-artifact` goal instead, see comments in https://github.com/apache/incubator-brooklyn/pull/709.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-115010672
Thanks @neykov . Great comments. Addressed in b5cc2779dc9de9d44093a8041c8cbd5591c23f47 pushed directly to master as it is just PR tweaks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114757595
@neykov I've addressed your excellent observations. Do the last few commits look alright?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114374673
@duncangrant - can you let me know if this fixes it?
all - anyone else had the chance to work through this and want to give input?
@aledsage - you've done the most on the auto-scaler; any objections to the change just made to set the default min to 1 ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33194180
--- Diff: core/src/main/java/brooklyn/util/flags/TypeCoercions.java ---
@@ -723,7 +727,31 @@ public Class apply(final String input) {
registerAdapter(String.class, AttributeSensor.class, new Function<String,AttributeSensor>() {
@Override
public AttributeSensor apply(final String input) {
- return new BasicAttributeSensor<Object>(Object.class, input);
+ Entity entity = BrooklynTaskTags.getContextEntity(Tasks.current());
+ if (entity!=null) {
+ Sensor<?> result = null;
+ if (entity!=null) {
+ result = entity.getEntityType().getSensor(input);
+ if (result instanceof AttributeSensor)
+ return (AttributeSensor) result;
+ }
+ }
+ return Sensors.newSensor(Object.class, input);
+ }
+ });
+ registerAdapter(String.class, Sensor.class, new Function<String,Sensor>() {
+ @Override
+ public AttributeSensor apply(final String input) {
+ Entity entity = BrooklynTaskTags.getContextEntity(Tasks.current());
+ if (entity!=null) {
+ Sensor<?> result = null;
+ if (entity!=null) {
--- End diff --
who wrote this mess? :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33141300
--- Diff: core/src/main/java/brooklyn/enricher/basic/YamlTimeWeightedDeltaEnricher.java ---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package brooklyn.enricher.basic;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import brooklyn.config.ConfigKey;
+import brooklyn.enricher.basic.AbstractTransformer;
+import brooklyn.entity.basic.ConfigKeys;
+import brooklyn.event.SensorEvent;
+import brooklyn.util.flags.TypeCoercions;
+import brooklyn.util.time.Duration;
+
+import com.google.common.base.Function;
+
+/**
+ * Converts an absolute count sensor into a delta sensor (i.e. the diff between the current and previous value),
+ * presented as a units/timeUnit based on the event timing.
+ * <p>
+ * For example, given a requests.count sensor, this can make a requests.per_sec sensor with {@link #DELTA_PERIOD} set to "1s" (the default).
+ * <p>
+ * Suitable for configuration from YAML.
+ */
+public class YamlTimeWeightedDeltaEnricher<T extends Number> extends AbstractTransformer<T,Double> {
+ private static final Logger LOG = LoggerFactory.getLogger(YamlTimeWeightedDeltaEnricher.class);
+
+ Number lastValue;
+ long lastTime = -1;
+
+ public static ConfigKey<Duration> DELTA_PERIOD = ConfigKeys.newConfigKey(Duration.class, "enricher.delta.period",
+ "Duration that this delta should compute for, default per second", Duration.ONE_SECOND);
+
+ @Override
+ protected Function<SensorEvent<T>, Double> getTransformation() {
+ return new Function<SensorEvent<T>, Double>() {
+ @Override
+ public Double apply(SensorEvent<T> event) {
+ Number current = TypeCoercions.coerce(event.getValue(), Double.class);
+
+ if (current == null) return null;
+
+ long eventTime = event.getTimestamp();
+ long unitMillis = getConfig(DELTA_PERIOD).toMilliseconds();
+ Double result = null;
+
+ if (eventTime > 0 && eventTime > lastTime) {
+ if (lastValue == null || lastTime < 0) {
+ // cannot calculate time-based delta with a single value
+ if (LOG.isTraceEnabled()) LOG.trace("{} received event but no last value so will not emit, null -> {} at {}", new Object[] {this, current, eventTime});
+ } else {
+ double duration = eventTime - lastTime;
+ result = (current.doubleValue() - lastValue.doubleValue()) / (duration / unitMillis);
+ }
+ }
+
+ lastValue = current;
+ lastTime = eventTime;
--- End diff --
Synchronize reads/writes in pairs?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114339240
Right, it looks like non-VPC use ("EC2-Classic") is pretty much deprecated, only available to people with older accounts and for them in regions used before or during 2013.
So I think the pragmatic thing to do is give a nice error in this case, directing people to use `eu-central-1` which launched in 2014 (and so only has VPC) or to specify a specific `networkName` e.g. `subnet-041c8373` for a subnet which auto-assigns public IP's or where Brooklyn has access or is configured to use a jump host.
I will add this nice error message and change the default location to `eu-central-1`.
For more info:
* http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-vpc.html#vpc-only-instance-types
* http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/default-vpc.html
* http://aws.amazon.com/vpc/faqs/#Default_VPCs
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33144859
--- Diff: usage/cli/src/main/resources/brooklyn/default.catalog.bom ---
@@ -1,23 +1,359 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-#
-
-# this catalog bom simply scans.
-# in a dist a default.catalog.bom from the conf/ dir takes precedence.
+
+# this catalog bom is an illustration supplying a few useful sample items
+# and templates to get started using Brooklyn
brooklyn.catalog:
- scanJavaAnnotations: true
+ version: 0.7.0-SNAPSHOT # BROOKLYN_VERSION
+ items:
+
+ # load everything in the classpath with a @Catalog annotation
+ - scanJavaAnnotations: true
+
+ - id: server
+ description: |
+ Provision a server, with customizable provisioning.properties and credentials installed,
+ but no other special software process or scripts executed.
+ item:
+ type: brooklyn.entity.basic.EmptySoftwareProcess
+ name: Server
+
+ - id: vanilla-bash-server
+ description: |
+ Provision a server, with customizable provisioning.properties and credentials installed,
+ but no other special software process or scripts executed.
+ The script should be supplied in "launch.command" as per docs on
+ brooklyn.entity.basic.VanillaSoftwareProcess.
+ item:
+ type: brooklyn.entity.basic.VanillaSoftwareProcess
+ name: Server with Launch Script (bash)
+
+ - id: load-balancer
+ description: |
+ Create a load balancer which will point at members in the group entity
+ referred to by the config key "serverPool".
+ The sensor advertising the port can be configured with the "member.sensor.portNumber" config key,
+ defaulting to `http.port`; all member entities which have published "service.up" will then be picked up.
+ item:
+ type: brooklyn.entity.proxy.nginx.NginxController
+ name: Load Balancer (nginx)
+
+ - id: cluster
+ description: |
+ Create a cluster of entities, resizable, with starting size "initialSize",
+ and using a spec supplied in the "memberSpec" key.
+ item:
+ type: brooklyn.entity.group.DynamicCluster
+
+ - id: 1-server-template
+ itemType: template
+ name: "Template 1: Server"
+ description: |
+ Sample YAML to provision a server in a cloud with illustrative VM properties
+ item:
+ name: Server (Brooklyn Example)
+
+ # this basic example shows how Brooklyn can provision a single raw VM
+ # in the cloud or location of your choice
+
+ services:
+ - type: server
--- End diff --
personal preference: The indentation looks confusing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-brooklyn/pull/705
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33144308
--- Diff: core/src/main/java/brooklyn/util/flags/TypeCoercions.java ---
@@ -723,7 +727,31 @@ public Class apply(final String input) {
registerAdapter(String.class, AttributeSensor.class, new Function<String,AttributeSensor>() {
@Override
public AttributeSensor apply(final String input) {
- return new BasicAttributeSensor<Object>(Object.class, input);
+ Entity entity = BrooklynTaskTags.getContextEntity(Tasks.current());
+ if (entity!=null) {
+ Sensor<?> result = null;
+ if (entity!=null) {
+ result = entity.getEntityType().getSensor(input);
+ if (result instanceof AttributeSensor)
+ return (AttributeSensor) result;
+ }
+ }
+ return Sensors.newSensor(Object.class, input);
+ }
+ });
+ registerAdapter(String.class, Sensor.class, new Function<String,Sensor>() {
+ @Override
+ public AttributeSensor apply(final String input) {
+ Entity entity = BrooklynTaskTags.getContextEntity(Tasks.current());
+ if (entity!=null) {
+ Sensor<?> result = null;
+ if (entity!=null) {
--- End diff --
Already checked != null above.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33182640
--- Diff: policy/src/main/java/brooklyn/enricher/TimeWeightedDeltaEnricher.java ---
@@ -41,6 +42,8 @@
* presented as a units/timeUnit based on the event timing.
* <p>
* NB for time (e.g. "total milliseconds consumed") use {@link TimeFractionDeltaEnricher}
+ * <p>
+ * See also {@link YamlTimeWeightedDeltaEnricher} designed for use from YAML.
--- End diff --
I've marked the older classes `@Beta` and added a reference. I think they should go in time, but not sure yet...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114775622
Rebased to fix conflicts with #709, and merged to prevent the build error I was getting from it. The last few commits could do with a quick scan-after-commit, though it's really just addressing comments here so not crucial.
However @rdowner if the `apache-` prefix in `usage/dist/pom.xml` was needed for something we're going to have to find a different way to achieve it I think.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33193676
--- Diff: core/src/main/java/brooklyn/util/task/BasicExecutionContext.java ---
@@ -169,6 +169,11 @@ public T call() {
taskTags.addAll(tags);
+ if (Tasks.current()!=null && BrooklynTaskTags.isTransient(Tasks.current()) && !taskTags.contains(BrooklynTaskTags.NON_TRANSIENT_TASK_TAG)) {
--- End diff --
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33120641
--- Diff: software/base/src/main/java/brooklyn/entity/software/StaticSensor.java ---
@@ -36,9 +43,17 @@ public StaticSensor(ConfigBag params) {
value = params.get(STATIC_VALUE);
}
+ @SuppressWarnings("unchecked")
@Override
public void apply(EntityLocal entity) {
super.apply(entity);
- entity.setAttribute(sensor, (T) TypeCoercions.coerce(value, sensor.getType()));
+
+ Maybe<T> v = Tasks.resolving(value).as((Class<T>)sensor.getType()).timeout(Duration.millis(200)).getMaybe();
--- End diff --
100% agree re arbitrary numbers in code. i've replace them with defined constants whose name and javadoc give a bit of reasoning behind the arbitrariness. lemme know if you think it's an improvement @neykov .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
Re: [GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by Alex Heneveld <al...@cloudsoftcorp.com>.
I see the problem. The class isn't on the classpath if you run from the CLI
project in the IDE unless you add extra projects. Unfortunate and the
logging is misleading but this won't affect a dist build.
I'll look for a fix. I also notice a couple of Svet's comments I missed but
I'll fix those.
However this shouldn't block a release.
--
Cloudsoft Corporation Limited, Registered in Scotland No: SC349230.
Registered Office: 13 Dryden Place, Edinburgh, EH9 1RP
This e-mail message is confidential and for use by the addressee only. If
the message is received by anyone other than the addressee, please return
the message to the sender by replying to it and then delete the message
from your computer. Internet e-mails are not necessarily secure. Cloudsoft
Corporation Limited does not accept responsibility for changes made to this
message after it was sent.
Whilst all reasonable care has been taken to avoid the transmission of
viruses, it is the responsibility of the recipient to ensure that the
onward transmission, opening or use of this message and any attachments
will not adversely affect its systems or data. No responsibility is
accepted by Cloudsoft Corporation Limited in this regard and the recipient
should carry out such virus and other checks as it considers appropriate.
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-115024562
This is bizarre. It's trying to load it as a location. And I can't
reproduce. Is this with a distinctive build and no other catalog? Can you
paste the full stack trace from the debug log?
On 24 Jun 2015 13:53, "Mike Zaccardo" <no...@github.com> wrote:
> I get this output when running master now:
>
> `2015-06-24 13:52:20,214 WARN Location resolution failed for
> 'brooklyn.entity.proxy.nginx.NginxController' (will fail shortly): known
> resolvers are: [id, named, brooklyn.catalog, localhost, byon, single, host,
> multi, portForwardManager, pool, jclouds, jcloudsByon]
> 2015-06-24 13:52:20,219 WARN Error importing catalog from
> classpath:/brooklyn/default.catalog.bom:
> brooklyn.util.exceptions.CompoundRuntimeException: Could not resolve item
> load-balancer:
> type: brooklyn.entity.proxy.nginx.NginxController
> name: Load Balancer (nginx)
>
> brooklyn.util.exceptions.CompoundRuntimeException: Could not resolve item
> load-balancer:
> type: brooklyn.entity.proxy.nginx.NginxController
> name: Load Balancer (nginx)
>
> at brooklyn.util.exceptions.Exceptions.create(Exceptions.java:278) ~[classes/:na]`
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-115009336>
> .
>
--
Cloudsoft Corporation Limited, Registered in Scotland No: SC349230.
Registered Office: 13 Dryden Place, Edinburgh, EH9 1RP
This e-mail message is confidential and for use by the addressee only. If
the message is received by anyone other than the addressee, please return
the message to the sender by replying to it and then delete the message
from your computer. Internet e-mails are not necessarily secure. Cloudsoft
Corporation Limited does not accept responsibility for changes made to this
message after it was sent.
Whilst all reasonable care has been taken to avoid the transmission of
viruses, it is the responsibility of the recipient to ensure that the
onward transmission, opening or use of this message and any attachments
will not adversely affect its systems or data. No responsibility is
accepted by Cloudsoft Corporation Limited in this regard and the recipient
should carry out such virus and other checks as it considers appropriate.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114334161
confirmed for me too, specifying `minRam: 2gb` in AWS results in:
```
2015-06-22 19:33:29,968 ERROR Failed to start VM for aws-ec2:eu-west-1@NginxControllerImpl{id=m8aiWnk1}: request POST https://ec2.eu-west-1.amazonaws.com/ HTTP/1.1 failed with code 400, error: AWSError{requestId='4e534c1d-1884-4776-a3e6-5efd470c9c79', requestToken='null', code='VPCResourceNotSpecified', message='The specified instance type can only be used in a VPC. A subnet ID or network interface ID is required to carry out the request.', context='{Response=, Errors=}'}
```
We hit issues like this when writing https://github.com/jclouds/jclouds/pull/732 and temporary local Brooklyn patch/copy https://github.com/apache/incubator-brooklyn/pull/610 -- these focussed on fixing the issue with selecting incompatible images, so it's possible we need more fixes from the former PR added to Brooklyn. Or possibly AWS have made more restrictive which instance types can run outwith a VPC. More anon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33194695
--- Diff: usage/cli/src/main/resources/brooklyn/default.catalog.bom ---
@@ -1,23 +1,359 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-#
-
-# this catalog bom simply scans.
-# in a dist a default.catalog.bom from the conf/ dir takes precedence.
+
+# this catalog bom is an illustration supplying a few useful sample items
+# and templates to get started using Brooklyn
brooklyn.catalog:
- scanJavaAnnotations: true
+ version: 0.7.0-SNAPSHOT # BROOKLYN_VERSION
+ items:
+
+ # load everything in the classpath with a @Catalog annotation
+ - scanJavaAnnotations: true
+
+ - id: server
+ description: |
+ Provision a server, with customizable provisioning.properties and credentials installed,
+ but no other special software process or scripts executed.
+ item:
+ type: brooklyn.entity.basic.EmptySoftwareProcess
+ name: Server
+
+ - id: vanilla-bash-server
+ description: |
+ Provision a server, with customizable provisioning.properties and credentials installed,
+ but no other special software process or scripts executed.
+ The script should be supplied in "launch.command" as per docs on
+ brooklyn.entity.basic.VanillaSoftwareProcess.
+ item:
+ type: brooklyn.entity.basic.VanillaSoftwareProcess
+ name: Server with Launch Script (bash)
+
+ - id: load-balancer
+ description: |
+ Create a load balancer which will point at members in the group entity
+ referred to by the config key "serverPool".
+ The sensor advertising the port can be configured with the "member.sensor.portNumber" config key,
+ defaulting to `http.port`; all member entities which have published "service.up" will then be picked up.
+ item:
+ type: brooklyn.entity.proxy.nginx.NginxController
+ name: Load Balancer (nginx)
+
+ - id: cluster
+ description: |
+ Create a cluster of entities, resizable, with starting size "initialSize",
+ and using a spec supplied in the "memberSpec" key.
+ item:
+ type: brooklyn.entity.group.DynamicCluster
+
+ - id: 1-server-template
+ itemType: template
+ name: "Template 1: Server"
+ description: |
+ Sample YAML to provision a server in a cloud with illustrative VM properties
+ item:
+ name: Server (Brooklyn Example)
+
+ # this basic example shows how Brooklyn can provision a single raw VM
+ # in the cloud or location of your choice
+
+ services:
+ - type: server
--- End diff --
the *values* are aligned -- it's a format @alasdairhodge showed me, i quite like it ... but we can revert it at any point if people don't!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33143934
--- Diff: core/src/main/java/brooklyn/enricher/basic/Transformer.java ---
@@ -24,25 +24,20 @@
import org.slf4j.LoggerFactory;
import brooklyn.config.ConfigKey;
-import brooklyn.entity.Entity;
import brooklyn.entity.basic.ConfigKeys;
-import brooklyn.entity.basic.EntityLocal;
-import brooklyn.event.AttributeSensor;
-import brooklyn.event.Sensor;
import brooklyn.event.SensorEvent;
-import brooklyn.event.SensorEventListener;
-import brooklyn.event.basic.BasicSensorEvent;
import brooklyn.util.collections.MutableSet;
import brooklyn.util.task.Tasks;
-import brooklyn.util.time.Duration;
+import brooklyn.util.task.ValueResolver;
import com.google.common.base.Function;
import com.google.common.reflect.TypeToken;
//@Catalog(name="Transformer", description="Transforms attributes of an entity; see Enrichers.builder().transforming(...)")
@SuppressWarnings("serial")
-public class Transformer<T,U> extends AbstractEnricher implements SensorEventListener<T> {
+public class Transformer<T,U> extends AbstractTransformer<T,U> {
+ @SuppressWarnings("unused")
--- End diff --
What's the point of keeping the logger, better delete it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33183153
--- Diff: utils/common/src/main/java/brooklyn/util/yaml/Yamls.java ---
@@ -350,7 +351,11 @@ public String getMatchedYamlTextOrWarn() {
if (e instanceof KnownClassVersionException) {
log.debug("Known class version exception; no yaml text being matched for "+this+": "+e);
} else {
- log.warn("Unable to match yaml text in "+this+": "+e, e);
+ if (e instanceof UserFacingException) {
+ log.warn("Unable to match yaml text in "+this+": "+e.getMessage());
--- End diff --
not sure it would actually -- possibly trace but an error here should be easy to reproduce
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by duncangrant <gi...@git.apache.org>.
Github user duncangrant commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114416055
@ahgittin that change certainly seems to fix both issues I was having
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by duncangrant <gi...@git.apache.org>.
Github user duncangrant commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114076473
The Resilient Load-Balanced Web Cluster fails for me with a VPCResourceNotSpecified error.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33122425
--- Diff: usage/dist/pom.xml ---
@@ -98,9 +132,14 @@
<configuration>
<appendAssemblyId>true</appendAssemblyId>
<descriptors>
- <descriptor>src/main/config/build-distribution-archive.xml</descriptor>
+ <descriptor>src/main/config/build-distribution.xml</descriptor>
</descriptors>
<finalName>brooklyn-${project.version}</finalName>
+ <formats>
+ <format>tar.gz</format>
+ <format>zip</format>
+ </formats>
+ <attach>false</attach>
--- End diff --
Yes, better to `<attach>false</attach>` only for the `dir`. I've removed `attach-artifact` and it works well -- though it was still archiving twice. Changing to `<phase>package</phase>` seems to fix that, and runs it after tests.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114191666
@duncangrant That sounds like it isn't due to this PR, but this class of error from AWS should be a showstopper for a GA release. Is it one component or all of them? Did the other items work? Can you give any more details? (If you're busy fixing this then ignore me for now but if not I want to make sure someone can pick this up.)
@aledsage @rdowner do you agree this should block a GA release?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33143409
--- Diff: core/src/main/java/brooklyn/entity/basic/BrooklynTaskTags.java ---
@@ -267,7 +267,12 @@ public static WrappedStream stream(Task<?> task, String streamType) {
public static void setInessential(Task<?> task) { addTagDynamically(task, INESSENTIAL_TASK); }
public static void setTransient(Task<?> task) { addTagDynamically(task, TRANSIENT_TASK_TAG); }
- public static boolean isTransient(Task<?> task) { return hasTag(task, TRANSIENT_TASK_TAG); }
+ public static boolean isTransient(Task<?> task) {
+ if (hasTag(task, TRANSIENT_TASK_TAG)) return true;
+ if (hasTag(task, NON_TRANSIENT_TASK_TAG)) return true;
--- End diff --
should be `return false`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33142638
--- Diff: usage/rest-server/src/main/java/brooklyn/rest/transform/CatalogTransformer.java ---
@@ -59,21 +59,28 @@
private static final org.slf4j.Logger log = LoggerFactory.getLogger(CatalogTransformer.class);
public static CatalogEntitySummary catalogEntitySummary(BrooklynRestResourceUtils b, CatalogItem<? extends Entity,EntitySpec<?>> item) {
- EntitySpec<?> spec = b.getCatalog().createSpec(item);
- EntityDynamicType typeMap = BrooklynTypes.getDefinedEntityType(spec.getType());
- EntityType type = typeMap.getSnapshot();
-
Set<EntityConfigSummary> config = Sets.newTreeSet(SummaryComparators.nameComparator());
Set<SensorSummary> sensors = Sets.newTreeSet(SummaryComparators.nameComparator());
Set<EffectorSummary> effectors = Sets.newTreeSet(SummaryComparators.nameComparator());
- for (ConfigKey<?> x: type.getConfigKeys())
- config.add(EntityTransformer.entityConfigSummary(x, typeMap.getConfigKeyField(x.getName())));
- for (Sensor<?> x: type.getSensors())
- sensors.add(SensorTransformer.sensorSummaryForCatalog(x));
- for (Effector<?> x: type.getEffectors())
- effectors.add(EffectorTransformer.effectorSummaryForCatalog(x));
+ try {
+ EntitySpec<?> spec = b.getCatalog().createSpec(item);
+ EntityDynamicType typeMap = BrooklynTypes.getDefinedEntityType(spec.getType());
+ EntityType type = typeMap.getSnapshot();
+ for (ConfigKey<?> x: type.getConfigKeys())
+ config.add(EntityTransformer.entityConfigSummary(x, typeMap.getConfigKeyField(x.getName())));
+ for (Sensor<?> x: type.getSensors())
+ sensors.add(SensorTransformer.sensorSummaryForCatalog(x));
+ for (Effector<?> x: type.getEffectors())
+ effectors.add(EffectorTransformer.effectorSummaryForCatalog(x));
+
+ } catch (Exception e) {
+ // templates with multiple entities can't have spec created in the manner above; just ignore
--- End diff --
Rethrow if type==Entity (or at least log an error), also propagateIfFatal.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r32995565
--- Diff: core/src/main/java/brooklyn/enricher/basic/AbstractAggregator.java ---
@@ -187,6 +187,24 @@ protected void setEntityAfterSubscribingProducerChildrenEvents() {
}
}
+ /** true if this should aggregate members */
+ protected boolean isAggregatingMembers() {
+ if (Boolean.TRUE.equals(fromMembers)) return true;
--- End diff --
remind me: is setEntity(EntityLocal) called on rebind? OR could `fromMembers` potentially be null after rebeind?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33193332
--- Diff: core/src/main/java/brooklyn/entity/basic/BrooklynTaskTags.java ---
@@ -267,7 +267,12 @@ public static WrappedStream stream(Task<?> task, String streamType) {
public static void setInessential(Task<?> task) { addTagDynamically(task, INESSENTIAL_TASK); }
public static void setTransient(Task<?> task) { addTagDynamically(task, TRANSIENT_TASK_TAG); }
- public static boolean isTransient(Task<?> task) { return hasTag(task, TRANSIENT_TASK_TAG); }
+ public static boolean isTransient(Task<?> task) {
+ if (hasTag(task, TRANSIENT_TASK_TAG)) return true;
+ if (hasTag(task, NON_TRANSIENT_TASK_TAG)) return true;
--- End diff --
great spot!!!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114372985
Got to the bottom of it:
* `networkName` sets `TemplateOptions.networks(...)` which it looks like is only supported for CloudStack and OpenStack. @andreaturli can you confirm? i've mapped this to `subnetId` for AWS, and log warning in SoftLayer. we might want other warnings in future.
* the `not available after creating` message appears to be a very occasional race behaviour; on a retry it was fine. I've upped the timeout in hopes of reducing it.
* however for *non-default* VPC's (i.e. if you set `networkId: subnet-ID`) jclouds does not set up security groups, so that suggestion now says to configure the default security group for that VPC to allow all the access you want. not ideal, but advanced-networking is wanted when working with this so we can live ith it.
the message for "classic" users who get the VPC error now says:
LOG.error("Detected that your EC2 account is a legacy 'classic' account, but the recommended instance type requires VPC. "
+ "You can specify the 'eu-central-1' region to avoid this problem, or you can specify a classic-compatible instance type, "
+ "or you can specify a subnet to use with 'networkName' "
+ "(taking care that the subnet auto-assigns public IP's and allows ingress on all ports, "
+ "as Brooklyn does not currently configure security groups for non-default VPC's; "
+ "or setting up Brooklyn to be in the subnet or have a jump host or other subnet access configuration). "
+ "For more information on VPC vs classic see http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-vpc.html.");
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by andreaturli <gi...@git.apache.org>.
Github user andreaturli commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114405734
@ahgittin I think your analysis is very good and complete!
Personally I think jclouds needs some clean up in this area: the generic `TemplateOptions` contains `networks` so I think for this needs to be used in most of the cases, instead of adding a custom option.
There are already TODOs or @deprecated in the code so it is ongoing.
a couple of comments:
- Cloudstack uses custom `securityGroupIds` and `networkIds` not the generic `networks`
- Openstack Nova uses `securityGroupNames` which is marked as @deprecated (suggests to use `groups` which is maybe even more confusing)
- `Azure` supports the custom `networkSecurityGroupName`
so maybe we should warn the Openstack, Cloudstack and Azure users as you did for SoftLayer.
- `GCE` seems the only one using `templateOptions.networks()` correctly
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114734977
Thanks @tbouron @andreaturli . I've updated the warnings and code comments to reflect this sad state of affairs with `TemplateOptions.networks(...)`. @tbouron your confirmation was helpful; you can now use `networkName` directly, it is mapped to `subnetId` for AWS.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33119484
--- Diff: software/base/src/main/java/brooklyn/entity/software/StaticSensor.java ---
@@ -36,9 +43,17 @@ public StaticSensor(ConfigBag params) {
value = params.get(STATIC_VALUE);
}
+ @SuppressWarnings("unchecked")
@Override
public void apply(EntityLocal entity) {
super.apply(entity);
- entity.setAttribute(sensor, (T) TypeCoercions.coerce(value, sensor.getType()));
+
+ Maybe<T> v = Tasks.resolving(value).as((Class<T>)sensor.getType()).timeout(Duration.millis(200)).getMaybe();
--- End diff --
This is an important distinction which I don't think we make explicit.
Config supports tasks/suppliers which are resolved whenever read. Sensors are values published at a point in time so don't normally support resolution (except for this one).
I've added javadoc:
```
/**
* Provides an initializer/feed which simply sets a given value.
* <p>
* {@link Task}/{@link Supplier} values are resolved when written,
* unlike config values which are resolved on each read.
* <p>
* This supports a {@link StaticSensor#SENSOR_PERIOD}
* which can be useful if the supplied value is such a function.
* However when the source is another sensor,
* consider using {@link Propagator} which listens for changes instead. */
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33142977
--- Diff: utils/common/src/main/java/brooklyn/util/yaml/Yamls.java ---
@@ -350,7 +351,11 @@ public String getMatchedYamlTextOrWarn() {
if (e instanceof KnownClassVersionException) {
log.debug("Known class version exception; no yaml text being matched for "+this+": "+e);
} else {
- log.warn("Unable to match yaml text in "+this+": "+e, e);
+ if (e instanceof UserFacingException) {
+ log.warn("Unable to match yaml text in "+this+": "+e.getMessage());
--- End diff --
nitpick: The stack trace would still be useful in the debug log.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33005245
--- Diff: core/src/main/java/brooklyn/enricher/basic/AbstractAggregator.java ---
@@ -187,6 +187,24 @@ protected void setEntityAfterSubscribingProducerChildrenEvents() {
}
}
+ /** true if this should aggregate members */
+ protected boolean isAggregatingMembers() {
+ if (Boolean.TRUE.equals(fromMembers)) return true;
--- End diff --
yes, rebind calls `Entity.addEnricher` which calls `Enricher.setEntity`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by tbouron <gi...@git.apache.org>.
Github user tbouron commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114384441
@ahgittin Regarding VPC and security groups configuration, here is what works for me:
* As you said, subnet ID is configured through `subnetId` for AWS
* To define my own security groups, I add a `templateOptions.securityGroupIds: [[abc, xyz]]`. While this works, the specified security groups have to created manually first. They also have to be assigned to the same VPC as the `subnetId` you choose.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33052674
--- Diff: software/base/src/main/java/brooklyn/entity/software/StaticSensor.java ---
@@ -36,9 +43,17 @@ public StaticSensor(ConfigBag params) {
value = params.get(STATIC_VALUE);
}
+ @SuppressWarnings("unchecked")
@Override
public void apply(EntityLocal entity) {
super.apply(entity);
- entity.setAttribute(sensor, (T) TypeCoercions.coerce(value, sensor.getType()));
+
+ Maybe<T> v = Tasks.resolving(value).as((Class<T>)sensor.getType()).timeout(Duration.millis(200)).getMaybe();
--- End diff --
Why not set the raw value and resolve & coerce at read time?
The resolve timeouts seem to be arbitrarily chosen - I've seen 50ms at other places. Better use a global constant instead?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114869539
Finished review (a bit late :) ). At least a couple of the comments worth addressing in a new PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by duncangrant <gi...@git.apache.org>.
Github user duncangrant commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114259415
@ahgittin All the components fail.
I can easily replicate with the following using master so you're right :
name: Empty large System
location: jclouds:aws-ec2:eu-west-1
services:
- type: brooklyn.entity.basic.EmptySoftwareProcess
provisioning.properties:
minRam: 2gb
A simple fix for the above is to ask for 8gb. Doing that does seem to fix the issue for the nginx node but the node in the cluster fails. I'm not entirely sure why it fails but the node seems to stop and then it seems to retry waiting for main.uri a few times. And then the node is torn down.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by mikezaccardo <gi...@git.apache.org>.
Github user mikezaccardo commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-115009336
I get this output when running master now:
`2015-06-24 13:52:20,214 WARN Location resolution failed for 'brooklyn.entity.proxy.nginx.NginxController' (will fail shortly): known resolvers are: [id, named, brooklyn.catalog, localhost, byon, single, host, multi, portForwardManager, pool, jclouds, jcloudsByon]
2015-06-24 13:52:20,219 WARN Error importing catalog from classpath:/brooklyn/default.catalog.bom: brooklyn.util.exceptions.CompoundRuntimeException: Could not resolve item load-balancer:
type: brooklyn.entity.proxy.nginx.NginxController
name: Load Balancer (nginx)
brooklyn.util.exceptions.CompoundRuntimeException: Could not resolve item load-balancer:
type: brooklyn.entity.proxy.nginx.NginxController
name: Load Balancer (nginx)
at brooklyn.util.exceptions.Exceptions.create(Exceptions.java:278) ~[classes/:na]`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by rdowner <gi...@git.apache.org>.
Github user rdowner commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114263313
Ok that's pretty clear, the release will have to be paused until we understand this problem and have a solution. As it is unrelated to this PR let's continue the discussion on the dev@brooklyn list instead of here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33140824
--- Diff: core/src/main/java/brooklyn/enricher/basic/YamlTimeWeightedDeltaEnricher.java ---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package brooklyn.enricher.basic;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import brooklyn.config.ConfigKey;
+import brooklyn.enricher.basic.AbstractTransformer;
+import brooklyn.entity.basic.ConfigKeys;
+import brooklyn.event.SensorEvent;
+import brooklyn.util.flags.TypeCoercions;
+import brooklyn.util.time.Duration;
+
+import com.google.common.base.Function;
+
+/**
+ * Converts an absolute count sensor into a delta sensor (i.e. the diff between the current and previous value),
+ * presented as a units/timeUnit based on the event timing.
+ * <p>
+ * For example, given a requests.count sensor, this can make a requests.per_sec sensor with {@link #DELTA_PERIOD} set to "1s" (the default).
+ * <p>
+ * Suitable for configuration from YAML.
+ */
+public class YamlTimeWeightedDeltaEnricher<T extends Number> extends AbstractTransformer<T,Double> {
+ private static final Logger LOG = LoggerFactory.getLogger(YamlTimeWeightedDeltaEnricher.class);
+
+ Number lastValue;
+ long lastTime = -1;
+
+ public static ConfigKey<Duration> DELTA_PERIOD = ConfigKeys.newConfigKey(Duration.class, "enricher.delta.period",
+ "Duration that this delta should compute for, default per second", Duration.ONE_SECOND);
+
+ @Override
+ protected Function<SensorEvent<T>, Double> getTransformation() {
+ return new Function<SensorEvent<T>, Double>() {
+ @Override
+ public Double apply(SensorEvent<T> event) {
+ Number current = TypeCoercions.coerce(event.getValue(), Double.class);
--- End diff --
Is this necessary, when you are already using `.doubleValue()` further down?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33146037
--- Diff: usage/cli/src/main/resources/brooklyn/default.catalog.bom ---
@@ -1,23 +1,359 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-#
-
-# this catalog bom simply scans.
-# in a dist a default.catalog.bom from the conf/ dir takes precedence.
+
+# this catalog bom is an illustration supplying a few useful sample items
+# and templates to get started using Brooklyn
brooklyn.catalog:
- scanJavaAnnotations: true
+ version: 0.7.0-SNAPSHOT # BROOKLYN_VERSION
+ items:
+
+ # load everything in the classpath with a @Catalog annotation
+ - scanJavaAnnotations: true
+
+ - id: server
+ description: |
+ Provision a server, with customizable provisioning.properties and credentials installed,
+ but no other special software process or scripts executed.
+ item:
+ type: brooklyn.entity.basic.EmptySoftwareProcess
+ name: Server
+
+ - id: vanilla-bash-server
+ description: |
+ Provision a server, with customizable provisioning.properties and credentials installed,
+ but no other special software process or scripts executed.
+ The script should be supplied in "launch.command" as per docs on
+ brooklyn.entity.basic.VanillaSoftwareProcess.
+ item:
+ type: brooklyn.entity.basic.VanillaSoftwareProcess
+ name: Server with Launch Script (bash)
+
+ - id: load-balancer
+ description: |
+ Create a load balancer which will point at members in the group entity
+ referred to by the config key "serverPool".
+ The sensor advertising the port can be configured with the "member.sensor.portNumber" config key,
+ defaulting to `http.port`; all member entities which have published "service.up" will then be picked up.
+ item:
+ type: brooklyn.entity.proxy.nginx.NginxController
+ name: Load Balancer (nginx)
+
+ - id: cluster
+ description: |
+ Create a cluster of entities, resizable, with starting size "initialSize",
+ and using a spec supplied in the "memberSpec" key.
+ item:
+ type: brooklyn.entity.group.DynamicCluster
+
+ - id: 1-server-template
+ itemType: template
+ name: "Template 1: Server"
+ description: |
+ Sample YAML to provision a server in a cloud with illustrative VM properties
+ item:
+ name: Server (Brooklyn Example)
+
+ # this basic example shows how Brooklyn can provision a single raw VM
+ # in the cloud or location of your choice
+
+ services:
+ - type: server
+ name: My VM
+
+ # location can be e.g. `softlayer` or `jclouds:openstack-nova:https://9.9.9.9:9999/v2.0/`,
+ # or `localhost` or `byon: { nodes: [ 10.0.0.1, 10.0.0.2, 10.0.1.{1,2} ] }`
+ location:
+ jclouds:aws-ec2:
+ # edit these to use your credential (or delete if credentials specified in brooklyn.properties)
+ identity: <REPLACE>
+ credential: <REPLACE>
+
+ region: eu-central-1
+
+ # we want Ubuntu, with a lot of RAM
+ osFamily: ubuntu
+ minRam: 8gb
+
+ # set up this user and password (default is to authorize a public key)
+ user: sample
+ password: s4mpl3
+
+ - id: 2-bash-web-server-template
+ itemType: template
+ name: "Template 2: Bash Web Server"
+ description: |
+ Sample YAML building on Template 1,
+ adding bash commands to launch a Python-based web server
+ on port 8020
+ item:
+ name: Python Web Server (Brooklyn Example)
+
+ # this example builds on the previous one,
+ # adding some scripts to initialize the VM
+
+ services:
+ - type: vanilla-bash-server
+ name: My Bash Web Server VM
+ brooklyn.config:
+ install.command: |
+ # install python if not present
+ which python || \
+ { apt-get update && apt-get install python ; } || \
+ { yum update && yum install python ; } || \
+ { echo WARNING: cannot install python && exit 1 ; }
+
+ customize.command: |
+ # create the web page to serve
+ cat > index.html << EOF
+
+ Hello world.
+ <p>
+ I am ${ENTITY_INFO}, ${MESSAGE:-a Brooklyn sample}.
+ <p>
+ Created at: `date`
+ <p>
+ I am running at ${HOSTNAME}, with on-box IP configuration:
+ <pre>
+ `ifconfig | grep inet`
+ </pre>
+
+ EOF
+
+ launch.command: |
+ # launch in background (ensuring no streams open), and record PID to file
+ nohup python -m SimpleHTTPServer ${PORT:-8020} < /dev/null > output.txt 2>&1 &
+ echo $! > ${PID_FILE:-pid.txt}
+ sleep 5
+ ps -p `cat ${PID_FILE:-pid.txt}`
+ if [ $? -ne 0 ] ; then
+ cat output.txt
+ echo WARNING: python web server not running
+ exit 1
+ fi
+
+ shell.env:
+ HOSTNAME: $brooklyn:attributeWhenReady("host.name")
+ PORT: $brooklyn:config("my.app.port")
+ ENTITY_INFO: $brooklyn:component("this", "")
+ MESSAGE: $brooklyn:config("my.message")
+
+ # custom
+ my.app.port: 8020
+ my.message: "good to meet you"
+
+ brooklyn.enrichers:
+ # publish the URL as a sensor; the GUI will pick this up (main.uri)
+ - type: brooklyn.enricher.basic.Transformer
+ brooklyn.config:
+ uniqueTag: url-generator
+ enricher.sourceSensor: host.subnet.hostname
+ # use the definition from Attributes class, as it has a RendererHint so GUI makes it a link
+ enricher.targetSensor: $brooklyn:sensor("brooklyn.entity.basic.Attributes", "main.uri")
+ enricher.targetValue:
+ $brooklyn:formatString:
+ - "http://%s:%s/"
+ - $brooklyn:attributeWhenReady("host.subnet.hostname")
+ - $brooklyn:config("my.app.port")
+
+ location:
+ jclouds:aws-ec2:
+ region: eu-central-1
+ # edit these (or delete if credentials specified in brooklyn.properties)
+ identity: <REPLACE>
+ credential: <REPLACE>
+
+ - id: 3-bash-web-and-riak-template
+ itemType: template
+ name: "Template 3: Bash Web Server and Scaling Riak Cluster"
+ description: |
+ Sample YAML building on Template 2,
+ composing that blueprint with a Riak cluster and injecting the URL
+ item:
+ name: Bash Web Server and Riak Cluster (Brooklyn Example)
+
+ # this example *references* the previous one,
+ # combining it with a stock blueprint for a Riak cluster,
+ # and shows how a sensor from the latter can be injected
+
+ services:
+
+ # reference template 2, overriding message to point at riak
+ - type: 2-bash-web-server-template
+ brooklyn.config:
+ my.message: $brooklyn:formatString("connected to Riak at %s",
+ $brooklyn:entity("riak-cluster").attributeWhenReady("main.uri"))
+
+ # use the off-the-shelf Riak cluster
+ - type: brooklyn.entity.nosql.riak.RiakCluster
+ id: riak-cluster
+ initialSize: 3
+ # and add a policy to scale based on ops per minute
+ brooklyn.policies:
+ - type: brooklyn.policy.autoscaling.AutoScalerPolicy
+ brooklyn.config:
+ metric: riak.node.ops.1m.perNode
+ # more than 100 ops per second (6k/min) scales out, less than 50 scales back
+ # up to a max of 8 riak nodes here (can be changed in GUI / REST API afterwards)
+ metricLowerBound: 3000
+ metricUpperBound: 6000
+ minPoolSize: 3
+ maxPoolSize: 8
+ resizeUpStabilizationDelay: 30s
+ resizeDownStabilizationDelay: 5m
+
+ location:
+ jclouds:aws-ec2:
+ region: eu-central-1
+ # edit these (or delete if credentials specified in brooklyn.properties)
+ identity: <REPLACE>
+ credential: <REPLACE>
+
+ - id: 4-resilient-bash-web-cluster-template
+ itemType: template
+ name: "Template 4: Resilient Load-Balanced Bash Web Cluster with Sensors"
+ description: |
+ Sample YAML to provision a cluster of the bash/python web server nodes,
+ with sensors configured, and a load balancer pointing at them,
+ and resilience policies for node replacement and scaling
+ item:
+ name: Resilient Load-Balanced Bash Web Cluster (Brooklyn Example)
+
+ # this final example shows some of the advanced functionality:
+ # defining custom sensors, and a cluster with a "spec",
+ # policies for resilience and scaling based on that sensor,
+ # and wiring a load balancer in front of the cluster
+
+ # combining this with the riak cluster in the previous example
+ # is left as a suggested exercise for the user
+
+ services:
+
+ # define a cluster of the web nodes
+ - type: cluster
+ name: Cluster of Bash Web Nodes
+ id: my-web-cluster
+ brooklyn.config:
+ initialSize: 1
+ memberSpec:
+ $brooklyn:entitySpec:
+ # template 2 is used as the spec for items in this cluster
+ # with a new message overwriting the previous,
+ # and a lot of sensors defined
+ type: 2-bash-web-server-template
+ name: My Bash Web Server VM with Sensors
+
+ brooklyn.config:
+ my.message: "part of the cluster"
+
+ brooklyn.initializers:
+ # make a simple request-count sensor, by counting the number of 200 responses in output.txt
+ - type: brooklyn.entity.software.ssh.SshCommandSensor
+ brooklyn.config:
+ name: reqs.count
+ targetType: int
+ period: 5s
+ command: "cat output.txt | grep HTTP | grep 200 | wc | awk '{print $1}'"
+ # and publish the port as a sensor so the load-balancer can pick it up
+ - type: brooklyn.entity.software.StaticSensor
+ brooklyn.config:
+ name: app.port
+ targetType: int
+ static.value: $brooklyn:config("my.app.port")
+
+ brooklyn.enrichers:
+ # derive reqs.per_sec from reqs.count
+ - type: brooklyn.enricher.basic.YamlTimeWeightedDeltaEnricher
+ brooklyn.config:
+ enricher.sourceSensor: reqs.count
+ enricher.targetSensor: reqs.per_sec
+ enricher.delta.period: 1s
+ # and take an average over 30s for reqs.per_sec into reqs.per_sec.windowed_30s
+ - type: brooklyn.enricher.basic.YamlRollingTimeWindowMeanEnricher
+ brooklyn.config:
+ enricher.sourceSensor: reqs.per_sec
+ enricher.targetSensor: reqs.per_sec.windowed_30s
+ enricher.window.duration: 30s
+
+ # emit failure sensor if a failure connecting to the service is sustained for 30s
+ - type: brooklyn.policy.ha.ServiceFailureDetector
+ brooklyn.config:
+ entityFailed.stabilizationDelay: 30s
+
+ brooklyn.policies:
+ # restart if a failure is detected (with a max of one restart in 2m, sensor will propagate otherwise)
+ - type: brooklyn.policy.ha.ServiceRestarter
+ brooklyn.config:
+ failOnRecurringFailuresInThisDuration: 2m
+
+ # back at the cluster, create a total per-sec and some per-node average
+ brooklyn.enrichers:
+ - type: brooklyn.enricher.basic.Aggregator
+ brooklyn.config:
+ enricher.sourceSensor: reqs.per_sec
+ enricher.targetSensor: reqs.per_sec
+ transformation: sum
+ - type: brooklyn.enricher.basic.Aggregator
+ brooklyn.config:
+ enricher.sourceSensor: reqs.per_sec
+ enricher.targetSensor: reqs.per_sec.per_node
+ transformation: average
+ - type: brooklyn.enricher.basic.Aggregator
+ brooklyn.config:
+ enricher.sourceSensor: reqs.per_sec.windowed_30s
+ enricher.targetSensor: reqs.per_sec.windowed_30s.per_node
+ transformation: average
+
+ brooklyn.policies:
+ # resilience: if a per-node restart policy fails,
+ # just throw that node away and create a new one
+ - type: brooklyn.policy.ha.ServiceReplacer
+
+ # and scale based on reqs/sec
+ - type: brooklyn.policy.autoscaling.AutoScalerPolicy
+ brooklyn.config:
+ # scale based on reqs/sec (though in a real-world situation,
+ # reqs.per_sec.windowed_30s.per_node might be a better choice)
+ metric: reqs.per_sec.per_node
+
+ # really low numbers, so you can trigger a scale-out just by hitting reload a lot
+ metricUpperBound: 3
+ metricLowerBound: 1
+
+ # sustain 3 reqs/sec for 2s and it will scale out
+ resizeUpStabilizationDelay: 2s
+ # only scale down when sustained for 1m
+ resizeDownStabilizationDelay: 1m
+
+ maxPoolSize: 10
+
+ # and add a load-balancer pointing at the cluster
+ - type: load-balancer
+ id: load-bal
+ brooklyn.config:
+ # point this load balancer at the cluster, specifying port to forward to
+ loadbalancer.serverpool: $brooklyn:entity("my-web-cluster")
+ member.sensor.portNumber: app.port
+
+ brooklyn.enrichers:
+ # publish a few useful info sensors and KPI's to the root of the app
+ - type: brooklyn.enricher.basic.Propagator
+ brooklyn.config:
+ uniqueTag: propagate-load-balancer-url
+ producer: $brooklyn:entity("load-bal")
+ propagating:
+ - main.uri
+ - type: brooklyn.enricher.basic.Propagator
+ brooklyn.config:
+ uniqueTag: propagate-reqs-per-sec
+ producer: $brooklyn:entity("my-web-cluster")
+ propagating:
+ - reqs.per_sec
+ - reqs.per_sec.windowed_30s.per_node
+
+ location:
+ jclouds:aws-ec2:
+ # edit these (or delete if credentials specified in brooklyn.properties)
+ identity: <REPLACE>
+ credential: <REPLACE>
+
+ region: eu-central-1
+ minRam: 2gb
--- End diff --
Excellent examples! A comprehensive showcase of the power of Brooklyn.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33183162
--- Diff: usage/rest-server/src/main/java/brooklyn/rest/transform/CatalogTransformer.java ---
@@ -59,21 +59,28 @@
private static final org.slf4j.Logger log = LoggerFactory.getLogger(CatalogTransformer.class);
public static CatalogEntitySummary catalogEntitySummary(BrooklynRestResourceUtils b, CatalogItem<? extends Entity,EntitySpec<?>> item) {
- EntitySpec<?> spec = b.getCatalog().createSpec(item);
- EntityDynamicType typeMap = BrooklynTypes.getDefinedEntityType(spec.getType());
- EntityType type = typeMap.getSnapshot();
-
Set<EntityConfigSummary> config = Sets.newTreeSet(SummaryComparators.nameComparator());
Set<SensorSummary> sensors = Sets.newTreeSet(SummaryComparators.nameComparator());
Set<EffectorSummary> effectors = Sets.newTreeSet(SummaryComparators.nameComparator());
- for (ConfigKey<?> x: type.getConfigKeys())
- config.add(EntityTransformer.entityConfigSummary(x, typeMap.getConfigKeyField(x.getName())));
- for (Sensor<?> x: type.getSensors())
- sensors.add(SensorTransformer.sensorSummaryForCatalog(x));
- for (Effector<?> x: type.getEffectors())
- effectors.add(EffectorTransformer.effectorSummaryForCatalog(x));
+ try {
+ EntitySpec<?> spec = b.getCatalog().createSpec(item);
+ EntityDynamicType typeMap = BrooklynTypes.getDefinedEntityType(spec.getType());
+ EntityType type = typeMap.getSnapshot();
+ for (ConfigKey<?> x: type.getConfigKeys())
+ config.add(EntityTransformer.entityConfigSummary(x, typeMap.getConfigKeyField(x.getName())));
+ for (Sensor<?> x: type.getSensors())
+ sensors.add(SensorTransformer.sensorSummaryForCatalog(x));
+ for (Effector<?> x: type.getEffectors())
+ effectors.add(EffectorTransformer.effectorSummaryForCatalog(x));
+
+ } catch (Exception e) {
+ // templates with multiple entities can't have spec created in the manner above; just ignore
--- End diff --
+1 and +1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33141815
--- Diff: policy/src/main/java/brooklyn/enricher/TimeWeightedDeltaEnricher.java ---
@@ -41,6 +42,8 @@
* presented as a units/timeUnit based on the event timing.
* <p>
* NB for time (e.g. "total milliseconds consumed") use {@link TimeFractionDeltaEnricher}
+ * <p>
+ * See also {@link YamlTimeWeightedDeltaEnricher} designed for use from YAML.
--- End diff --
If both this and the YAML alternative are going to be used long-term, why not re-use the code? Alternatively deprecate this one?
Analogous comment for `RollingTimeWindowMeanEnricher` would be nice.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33143702
--- Diff: core/src/main/java/brooklyn/util/task/BasicExecutionContext.java ---
@@ -169,6 +169,11 @@ public T call() {
taskTags.addAll(tags);
+ if (Tasks.current()!=null && BrooklynTaskTags.isTransient(Tasks.current()) && !taskTags.contains(BrooklynTaskTags.NON_TRANSIENT_TASK_TAG)) {
--- End diff --
Check for `!taskTags.contains(TRANSIENT_TASK_TAG)` to prevent multiple entries (in case already set by the user).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by rdowner <gi...@git.apache.org>.
Github user rdowner commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114258571
Definitely worth holding back the release until we know if this is an issue affecting `master` or not. I wouldn't like to make a GA release with a known problem affecting the world's favourite compute cloud...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Catalog default niceness
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/705#issuecomment-114345523
Okay, I've added a more helpful error if this situation is encountered, and that seems to be working:
| 2015-06-22 20:30:33,643 ERROR Detected that your EC2 account is a legacy 'classic' account, but the recommended instance type requires VPC. You can specify the 'eu-central-1' region to avoid this problem, or you can specify a classic-compatible instance type, or you can specify a subnet to use (taking care that the subnet auto-assigns public IP's, or Brooklyn can see the subnet or is configured to use a jump host for this location). For more information see http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-vpc.html.
However the "fixes" aren't working:
* `networkName: subnet-ID` gives the same `VPCResourceNotSpecified` error; maybe the syntax is off?
* `eu-central-1` (VPC) gives `com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: security group eu-central-1/jclouds#brooklyn-bxza-alex-eu-central-shoul-u2jy-nginx-ielm is not available after creating` -- I saw one of those pages suggested max security groups was 5, so maybe that's it
I'm investigating.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---