You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brooklyn.apache.org by johnmccabe <gi...@git.apache.org> on 2015/11/13 17:48:04 UTC
[GitHub] incubator-brooklyn pull request: yaml testing docs
GitHub user johnmccabe opened a pull request:
https://github.com/apache/incubator-brooklyn/pull/1031
yaml testing docs
Added details of the Test Entities to the YAML Blueprints guide.
Have explicitly *not* added a reference to the Java Blueprints section at this point, shout if you'd like that added.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/johnmccabe/incubator-brooklyn yaml-test-entity-docs
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-brooklyn/pull/1031.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1031
----
commit cc25df3fa92a7b4d50cae72e44afb61e50ff0cd1
Author: John McCabe <jo...@johnmccabe.net>
Date: 2015-11-13T16:45:30Z
yaml testing docs
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by johnmccabe <gi...@git.apache.org>.
Github user johnmccabe commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r44918698
--- Diff: docs/guide/yaml/test/test-entities.md ---
@@ -0,0 +1,72 @@
+---
+title: Blueprint Test Entities
+title_in_menu: Test Entities
+layout: website-normal
+---
+
+{% include fields.md %}
+
+## Structural Test Entities
+
+### TestCase
+The `TestCase` entity acts as a container for a list child entities which are started *sequentially*.
+{% highlight yaml %}
+{% readj example_yaml/testcase-entity.yaml %}
+{% endhighlight %}
+The ability to sequentially run entities is used to allow applications to fully deploy before attempting to start any test entities.
+
+If your application blueprint consists of multiple services then you will also require the `ParallelTestCase` entity.
--- End diff --
Both approaches are feasible, I had documented the approach that seemed simplest to me. We could take an agnostic approach and describe both test approaches, I'd prefer to have one canonical way of approaching a test though - both sequential and parallel have their pros and cons.
For example with the parallel approach this is what an effector test would look like, is this more like you were thinking?
```
name: Parallel Effector Test Example
services:
- type: org.apache.brooklyn.entity.webapp.ControlledDynamicWebAppCluster
name: My Web
id: webappcluster
brooklyn.config:
wars.root: http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war
java.sysprops:
brooklyn.example.db.url: >
$brooklyn:formatString("jdbc:%s%s?user=%s&password=%s",
component("db").attributeWhenReady("datastore.url"),
"visitors", "brooklyn", "br00k11n")
- type: org.apache.brooklyn.entity.database.mariadb.MariaDbNode
id: db
name: My DB
brooklyn.config:
creationScriptUrl: https://bit.ly/brooklyn-visitors-creation-script
- type: org.apache.brooklyn.test.framework.TestCase
name: Test Deploy Effector
brooklyn.children:
- type: org.apache.brooklyn.test.framework.TestSensor
name: webappcluster service.isUp
targetId: webappcluster
sensor: service.isUp
timeout: 15m
assert:
equals: true
- type: org.apache.brooklyn.test.framework.TestSensor
name: db service.isUp
targetId: db
sensor: service.isUp
timeout: 5m
assert:
equals: true
- type: org.apache.brooklyn.test.framework.TestEffector
name: Deploy WAR in /newcontext
targetId: webappcluster
effector: deploy
params:
url: http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war
targetName: newcontext
- type: org.apache.brooklyn.test.framework.TestHttpCall
name: /newcontext Status Code 200
timeout: 5m
url: $brooklyn:formatString("%s/newcontext/", component("webappcluster").attributeWhenReady("webapp.url"))
assert:
status: 404
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r44860930
--- Diff: docs/guide/yaml/test/example_yaml/simple-tomcat-app-sensor-test.yaml ---
@@ -0,0 +1,24 @@
+brooklyn.catalog:
+ id: simple-tomcat-app-sensor-test
+ version: 1.0
+ itemType: template
+ iconUrl: http://tomcat.apache.org/images/tomcat.png
+ name: Simple Tomcat Application Sensor Test
+ license: Apache-2.0
+ item:
+ brooklyn.config:
+ simple.confg: someValue
--- End diff --
Can we delete this (from here and from all the other examples)? I'm a fan of keeping the examples as simple as possible (so people don't get a false impression of how long it has to be). If we want to demonstrate other thing such as config, we can give a separate example for that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by johnmccabe <gi...@git.apache.org>.
Github user johnmccabe commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r44861020
--- Diff: docs/guide/yaml/test/example_yaml/simple-tomcat-app-sensor-test.yaml ---
@@ -0,0 +1,24 @@
+brooklyn.catalog:
+ id: simple-tomcat-app-sensor-test
+ version: 1.0
+ itemType: template
+ iconUrl: http://tomcat.apache.org/images/tomcat.png
+ name: Simple Tomcat Application Sensor Test
+ license: Apache-2.0
+ item:
+ brooklyn.config:
+ simple.confg: someValue
--- End diff --
Agreed, specifics of packaging up tests can be dealt with in the context of a test runner.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#issuecomment-158100345
Thanks @johnmccabe looks good. Only a couple of additional minor comments. Ping me when you've taken a look at those, and I'll merge it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by johnmccabe <gi...@git.apache.org>.
Github user johnmccabe commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r45282713
--- Diff: docs/guide/yaml/test/index.md ---
@@ -0,0 +1,24 @@
+---
+title: Testing YAML Blueprints
+layout: website-normal
+children:
+- test-entities.md
+- usage-examples.md
+---
+
+Brooklyn provides a selection of basic test entities which can be used to validate Blueprints via YAML. These are divided into two groups, structural, which effect the order in which child entities are started, and validation, which are used to confirm the application is deployed as intended:
--- End diff --
Done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by johnmccabe <gi...@git.apache.org>.
Github user johnmccabe commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r45217301
--- Diff: docs/guide/yaml/test/usage-examples.md ---
@@ -0,0 +1,57 @@
+---
+title: Example Blueprint Tests
+title_in_menu: Example Tests
+layout: website-normal
+---
+
+{% include fields.md %}
+
+## Introduction
+This section provides example tests carried out on the following `simple-tomcat-app` web application catalog item:
+
+{% highlight yaml %}
+{% readj example_yaml/simple-tomcat-app.yaml %}
--- End diff --
Dropping the catalogs from these examples
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r44860606
--- Diff: docs/guide/yaml/test/example_yaml/paralleltestcase-example.yaml ---
@@ -0,0 +1,46 @@
+brooklyn.catalog:
+ id: parallel-test
+ version: 1.0
+ itemType: template
+ iconUrl: http://tomcat.apache.org/images/tomcat.png
+ name: Parallel Tomcat and DB Example Test
+ license: Apache-2.0
+ item:
+ brooklyn.config:
+ simple.confg: someValue
+ services:
+ - type: org.apache.brooklyn.test.framework.TestCase
+ name: Parallel Testcase
+ brooklyn.children:
+ - type: org.apache.brooklyn.test.framework.TestCase
+ name: Parallel Testcase
+ brooklyn.children:
+ - type: org.apache.brooklyn.entity.webapp.ControlledDynamicWebAppCluster
--- End diff --
I think I missed some earlier discussion perhaps - why do we put the entity-under-test as a child of the `TestCase` entity?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r45358954
--- Diff: docs/guide/yaml/test/index.md ---
@@ -0,0 +1,24 @@
+---
+title: Testing YAML Blueprints
+layout: website-normal
+children:
+- test-entities.md
+- usage-examples.md
+---
+
+Brooklyn provides a selection of basic test entities which can be used to validate Blueprints via YAML. These are divided into two groups; structural, which effect the order in which child entities are started; and validation, which are used to confirm the application is deployed as intended.
--- End diff --
Instead of "order in which child entities are started", perhaps relate it to the order in which child test-things are run. But obviously not with that wording! I'm fine with leaving it as-is if we can't come up with a better term for "child test-things".
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by johnmccabe <gi...@git.apache.org>.
Github user johnmccabe commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r45367667
--- Diff: docs/guide/yaml/test/example_yaml/testcases/getting-started-test-example.yaml ---
@@ -0,0 +1,68 @@
+name: Getting Started Blueprint Test
+
+location:
+ jclouds:aws-ec2:
+ identity: ABCDEFGHIJKLMNOPQRST
+ credential: s3cr3tsq1rr3ls3cr3tsq1rr3ls3cr3tsq1rr3l
+
+services:
+- type: org.apache.brooklyn.entity.webapp.ControlledDynamicWebAppCluster
+ name: My Web
+ id: webappcluster
+ brooklyn.config:
+ wars.root: http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war
+ java.sysprops:
+ brooklyn.example.db.url: >
+ $brooklyn:formatString("jdbc:%s%s?user=%s&password=%s",
+ component("db").attributeWhenReady("datastore.url"),
+ "visitors", "brooklyn", "br00k11n")
+- type: org.apache.brooklyn.entity.database.mysql.MySqlNode
+ id: db
+ name: My DB
+ brooklyn.config:
+ creationScriptUrl: https://bit.ly/brooklyn-visitors-creation-script
+- type: org.apache.brooklyn.test.framework.TestHttpCall
+ name: Check HTTP Response Status Code
+ url: >
+ $brooklyn:formatString("http://%s:%s",
--- End diff --
We do, but its current being assigned a non-public address for the NginxController (on SL), I would have expected the webapp.url to be a public address (per the change in #959), had only spotted this last night and have not gotten round to raising it - will do so now - wanted to make sure the examples would be functional in the immediate term so pushed in this workaround.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by johnmccabe <gi...@git.apache.org>.
Github user johnmccabe commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r45217230
--- Diff: docs/guide/yaml/test/test-entities.md ---
@@ -0,0 +1,72 @@
+---
+title: Blueprint Test Entities
+title_in_menu: Test Entities
+layout: website-normal
+---
+
+{% include fields.md %}
+
+## Structural Test Entities
+
+### TestCase
+The `TestCase` entity acts as a container for a list child entities which are started *sequentially*.
+{% highlight yaml %}
+{% readj example_yaml/testcase-entity.yaml %}
+{% endhighlight %}
+The ability to sequentially run entities is used to allow applications to fully deploy before attempting to start any test entities.
+
+If your application blueprint consists of multiple services then you will also require the `ParallelTestCase` entity.
+
+### ParallelTestCase
+The `ParallelTestCase` entity acts as a container for a list of child entities which are started in *parallel*.
+{% highlight yaml %}
+{% readj example_yaml/paralleltestcase-entity.yaml %}
+{% endhighlight %}
+This entity is used when your application blueprint consists of multiple services that run in parallel.
+
+
+## Validation Test Entities
+
+### TestSensor
+The `TestSensor` entity performs an assertion on a specified sensor.
+{% highlight yaml %}
+{% readj example_yaml/testsensor-entity.yaml %}
+{% endhighlight %}
+
+#### Parameters
+- `target` - entity whose sensor will be tested, specified via DSL. For example, `$brooklyn:component("tomcat")`. See also the `targetId` parameter.
+- `targetId` - alternative to the `target` parameter which wraps the DSL component lookup. For example, `tomcat`.
+- `sensor` - sensor to evaluate. For example `service.isUp`.
+- `timeout` - time to wait for assertion to return a result.
+- `assert` - assertion to perform on the specified sensor, options include
+ - `equals` - tests that the sensor equals the supplied value. For example `true`.
+ - `regex` - tests that the sensor value matches the supplied [regex pattern](http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html?is-external=true), for example `".*hello.*"`.
+ - `isNull` - tests that the sensor value is `null`.
+
+### TestEffector
+The `TestEffector` entity invokes the specified effector on a target entity.
+{% highlight yaml %}
+{% readj example_yaml/testeffector-entity.yaml %}
+{% endhighlight %}
+
+#### Parameters
+- `target` - entity whose effector will be invoked, specified via DSL. For example, `$brooklyn:component("tomcat")`. See also the `targetId` parameter.
+- `targetId` - alternative to the `target` parameter which wraps the DSL component lookup. For example, `tomcat`.
+- `timeout` - time to wait for the effector task to complete.
+- `effector` - effector to invoke, for example `deploy`.
+- `params` - parameters to pass to the effector, these will depend on the entity and effector being tested. The example above shows the `url` and `targetName` parameters being passed to Tomcats `deploy` effector.
+
+### TestHttpCall
+The `TestHttpCall` entity performs a HTTP GET on the specified URL and performs an assertion on the response.
+{% highlight yaml %}
+{% readj example_yaml/testhttpcall-entity.yaml %}
+{% endhighlight %}
+
+#### Parameters
+- `url` - URL to perform GET request on, for example `$brooklyn:component("tomcat").attributeWhenReady("webapp.url")``.
+- `timeout` - time to wait for a HTTP response. For example `10s`, `10m`, etc
+- `assertions` - perform one of the following assertions.
+ - `status` - response must match the specified status code. The example above shows an assertions on a `404` response.
--- End diff --
I'll give you access to this PR branch so you can push in your assertion related updates @m4rkmckenna
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by m4rkmckenna <gi...@git.apache.org>.
Github user m4rkmckenna commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r44863181
--- Diff: docs/guide/yaml/test/test-entities.md ---
@@ -0,0 +1,72 @@
+---
+title: Blueprint Test Entities
+title_in_menu: Test Entities
+layout: website-normal
+---
+
+{% include fields.md %}
+
+## Structural Test Entities
+
+### TestCase
+The `TestCase` entity acts as a container for a list child entities which are started *sequentially*.
+{% highlight yaml %}
+{% readj example_yaml/testcase-entity.yaml %}
+{% endhighlight %}
+The ability to sequentially run entities is used to allow applications to fully deploy before attempting to start any test entities.
+
+If your application blueprint consists of multiple services then you will also require the `ParallelTestCase` entity.
+
+### ParallelTestCase
+The `ParallelTestCase` entity acts as a container for a list of child entities which are started in *parallel*.
+{% highlight yaml %}
+{% readj example_yaml/paralleltestcase-entity.yaml %}
+{% endhighlight %}
+This entity is used when your application blueprint consists of multiple services that run in parallel.
+
+
+## Validation Test Entities
+
+### TestSensor
+The `TestSensor` entity performs an assertion on a specified sensor.
+{% highlight yaml %}
+{% readj example_yaml/testsensor-entity.yaml %}
+{% endhighlight %}
+
+#### Parameters
+- `target` - entity whose sensor will be tested, specified via DSL. For example, `$brooklyn:component("tomcat")`. See also the `targetId` parameter.
+- `targetId` - alternative to the `target` parameter which wraps the DSL component lookup. For example, `tomcat`.
+- `sensor` - sensor to evaluate. For example `service.isUp`.
+- `timeout` - time to wait for assertion to return a result.
+- `assert` - assertion to perform on the specified sensor, options include
+ - `equals` - tests that the sensor equals the supplied value. For example `true`.
+ - `regex` - tests that the sensor value matches the supplied [regex pattern](http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html?is-external=true), for example `".*hello.*"`.
+ - `isNull` - tests that the sensor value is `null`.
+
+### TestEffector
+The `TestEffector` entity invokes the specified effector on a target entity.
+{% highlight yaml %}
+{% readj example_yaml/testeffector-entity.yaml %}
+{% endhighlight %}
+
+#### Parameters
+- `target` - entity whose effector will be invoked, specified via DSL. For example, `$brooklyn:component("tomcat")`. See also the `targetId` parameter.
+- `targetId` - alternative to the `target` parameter which wraps the DSL component lookup. For example, `tomcat`.
+- `timeout` - time to wait for the effector task to complete.
+- `effector` - effector to invoke, for example `deploy`.
+- `params` - parameters to pass to the effector, these will depend on the entity and effector being tested. The example above shows the `url` and `targetName` parameters being passed to Tomcats `deploy` effector.
+
+### TestHttpCall
+The `TestHttpCall` entity performs a HTTP GET on the specified URL and performs an assertion on the response.
+{% highlight yaml %}
+{% readj example_yaml/testhttpcall-entity.yaml %}
+{% endhighlight %}
+
+#### Parameters
+- `url` - URL to perform GET request on, for example `$brooklyn:component("tomcat").attributeWhenReady("webapp.url")``.
+- `timeout` - time to wait for a HTTP response. For example `10s`, `10m`, etc
+- `assertions` - perform one of the following assertions.
+ - `status` - response must match the specified status code. The example above shows an assertions on a `404` response.
--- End diff --
@aledsage I have a change locally based from comments on #999 by @ahgittin that will address this. Basically there will be a common set of assertions so the above would be covered by a regular expression
```
assert:
matches: 2[0-9][0-9]
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r45358446
--- Diff: docs/guide/yaml/test/example_yaml/testcases/getting-started-test-example.yaml ---
@@ -0,0 +1,68 @@
+name: Getting Started Blueprint Test
+
+location:
+ jclouds:aws-ec2:
+ identity: ABCDEFGHIJKLMNOPQRST
+ credential: s3cr3tsq1rr3ls3cr3tsq1rr3ls3cr3tsq1rr3l
+
+services:
+- type: org.apache.brooklyn.entity.webapp.ControlledDynamicWebAppCluster
+ name: My Web
+ id: webappcluster
+ brooklyn.config:
+ wars.root: http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war
+ java.sysprops:
+ brooklyn.example.db.url: >
+ $brooklyn:formatString("jdbc:%s%s?user=%s&password=%s",
+ component("db").attributeWhenReady("datastore.url"),
+ "visitors", "brooklyn", "br00k11n")
+- type: org.apache.brooklyn.entity.database.mysql.MySqlNode
+ id: db
+ name: My DB
+ brooklyn.config:
+ creationScriptUrl: https://bit.ly/brooklyn-visitors-creation-script
+- type: org.apache.brooklyn.test.framework.TestHttpCall
+ name: Check HTTP Response Status Code
+ url: >
+ $brooklyn:formatString("http://%s:%s",
--- End diff --
Do we not advertise this combined root URI as a sensor? The code looks like it should propagate "main.uri" to the ControlledDynamicWebAppCluster instance.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by johnmccabe <gi...@git.apache.org>.
Github user johnmccabe commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r44860964
--- Diff: docs/guide/yaml/test/example_yaml/paralleltestcase-example.yaml ---
@@ -0,0 +1,46 @@
+brooklyn.catalog:
+ id: parallel-test
+ version: 1.0
+ itemType: template
+ iconUrl: http://tomcat.apache.org/images/tomcat.png
+ name: Parallel Tomcat and DB Example Test
+ license: Apache-2.0
+ item:
+ brooklyn.config:
+ simple.confg: someValue
+ services:
+ - type: org.apache.brooklyn.test.framework.TestCase
+ name: Parallel Testcase
+ brooklyn.children:
+ - type: org.apache.brooklyn.test.framework.TestCase
+ name: Parallel Testcase
--- End diff --
Both having the name 'Parallel Testcase' is a mistake, the second should be something along the lines of 'Deploy Application'.
The first TestCase is used to allow sequential execution of the children, ie deploy app, then run tests. The ParallelTestCase starts the `ControlledDynamicWebAppCluster` and `MySqlNode` in parallel to reproduce the behaviour without tests (ie just nested under `-services:`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r44862665
--- Diff: docs/guide/yaml/test/usage-examples.md ---
@@ -0,0 +1,57 @@
+---
+title: Example Blueprint Tests
+title_in_menu: Example Tests
+layout: website-normal
+---
+
+{% include fields.md %}
+
+## Introduction
+This section provides example tests carried out on the following `simple-tomcat-app` web application catalog item:
+
+{% highlight yaml %}
+{% readj example_yaml/simple-tomcat-app.yaml %}
--- End diff --
Do we want to show all these examples as catalog items, or just as YAML for deploying an app directly? The latter would be shorter examples.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r44862655
--- Diff: docs/guide/yaml/test/test-entities.md ---
@@ -0,0 +1,72 @@
+---
+title: Blueprint Test Entities
+title_in_menu: Test Entities
+layout: website-normal
+---
+
+{% include fields.md %}
+
+## Structural Test Entities
+
+### TestCase
+The `TestCase` entity acts as a container for a list child entities which are started *sequentially*.
+{% highlight yaml %}
+{% readj example_yaml/testcase-entity.yaml %}
+{% endhighlight %}
+The ability to sequentially run entities is used to allow applications to fully deploy before attempting to start any test entities.
+
+If your application blueprint consists of multiple services then you will also require the `ParallelTestCase` entity.
+
+### ParallelTestCase
+The `ParallelTestCase` entity acts as a container for a list of child entities which are started in *parallel*.
+{% highlight yaml %}
+{% readj example_yaml/paralleltestcase-entity.yaml %}
+{% endhighlight %}
+This entity is used when your application blueprint consists of multiple services that run in parallel.
+
+
+## Validation Test Entities
+
+### TestSensor
+The `TestSensor` entity performs an assertion on a specified sensor.
+{% highlight yaml %}
+{% readj example_yaml/testsensor-entity.yaml %}
+{% endhighlight %}
+
+#### Parameters
+- `target` - entity whose sensor will be tested, specified via DSL. For example, `$brooklyn:component("tomcat")`. See also the `targetId` parameter.
+- `targetId` - alternative to the `target` parameter which wraps the DSL component lookup. For example, `tomcat`.
+- `sensor` - sensor to evaluate. For example `service.isUp`.
+- `timeout` - time to wait for assertion to return a result.
+- `assert` - assertion to perform on the specified sensor, options include
+ - `equals` - tests that the sensor equals the supplied value. For example `true`.
+ - `regex` - tests that the sensor value matches the supplied [regex pattern](http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html?is-external=true), for example `".*hello.*"`.
+ - `isNull` - tests that the sensor value is `null`.
+
+### TestEffector
+The `TestEffector` entity invokes the specified effector on a target entity.
+{% highlight yaml %}
+{% readj example_yaml/testeffector-entity.yaml %}
+{% endhighlight %}
+
+#### Parameters
+- `target` - entity whose effector will be invoked, specified via DSL. For example, `$brooklyn:component("tomcat")`. See also the `targetId` parameter.
+- `targetId` - alternative to the `target` parameter which wraps the DSL component lookup. For example, `tomcat`.
+- `timeout` - time to wait for the effector task to complete.
+- `effector` - effector to invoke, for example `deploy`.
+- `params` - parameters to pass to the effector, these will depend on the entity and effector being tested. The example above shows the `url` and `targetName` parameters being passed to Tomcats `deploy` effector.
+
+### TestHttpCall
+The `TestHttpCall` entity performs a HTTP GET on the specified URL and performs an assertion on the response.
+{% highlight yaml %}
+{% readj example_yaml/testhttpcall-entity.yaml %}
+{% endhighlight %}
+
+#### Parameters
+- `url` - URL to perform GET request on, for example `$brooklyn:component("tomcat").attributeWhenReady("webapp.url")``.
+- `timeout` - time to wait for a HTTP response. For example `10s`, `10m`, etc
+- `assertions` - perform one of the following assertions.
+ - `status` - response must match the specified status code. The example above shows an assertions on a `404` response.
--- End diff --
Longer term, would be nice to also allow a range (e.g. "200-299") (similar to `PortRanges.fromString()`)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by johnmccabe <gi...@git.apache.org>.
Github user johnmccabe commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r44860969
--- Diff: docs/guide/yaml/test/example_yaml/paralleltestcase-example.yaml ---
@@ -0,0 +1,46 @@
+brooklyn.catalog:
+ id: parallel-test
+ version: 1.0
+ itemType: template
+ iconUrl: http://tomcat.apache.org/images/tomcat.png
+ name: Parallel Tomcat and DB Example Test
+ license: Apache-2.0
+ item:
+ brooklyn.config:
+ simple.confg: someValue
+ services:
+ - type: org.apache.brooklyn.test.framework.TestCase
+ name: Parallel Testcase
+ brooklyn.children:
+ - type: org.apache.brooklyn.test.framework.TestCase
+ name: Parallel Testcase
+ brooklyn.children:
+ - type: org.apache.brooklyn.entity.webapp.ControlledDynamicWebAppCluster
--- End diff --
see response above
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by johnmccabe <gi...@git.apache.org>.
Github user johnmccabe commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r45217023
--- Diff: docs/guide/yaml/test/example_yaml/paralleltestcase-example.yaml ---
@@ -0,0 +1,46 @@
+brooklyn.catalog:
+ id: parallel-test
+ version: 1.0
+ itemType: template
+ iconUrl: http://tomcat.apache.org/images/tomcat.png
+ name: Parallel Tomcat and DB Example Test
+ license: Apache-2.0
+ item:
+ brooklyn.config:
+ simple.confg: someValue
+ services:
+ - type: org.apache.brooklyn.test.framework.TestCase
+ name: Parallel Testcase
+ brooklyn.children:
+ - type: org.apache.brooklyn.test.framework.TestCase
+ name: Parallel Testcase
--- End diff --
Switching to the parallel example mentioned [here](https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r44918698) and rewording the structural test entity descriptions to be less prescriptive on their usage.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-brooklyn/pull/1031
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by johnmccabe <gi...@git.apache.org>.
Github user johnmccabe commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r45368784
--- Diff: docs/guide/yaml/test/index.md ---
@@ -0,0 +1,24 @@
+---
+title: Testing YAML Blueprints
+layout: website-normal
+children:
+- test-entities.md
+- usage-examples.md
+---
+
+Brooklyn provides a selection of basic test entities which can be used to validate Blueprints via YAML. These are divided into two groups; structural, which effect the order in which child entities are started; and validation, which are used to confirm the application is deployed as intended.
--- End diff --
Change "_structural, which effect the order in which child entities are started_" to "_structural, which control the sequencing of service entity deployment and test entity execution_"
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r44862630
--- Diff: docs/guide/yaml/test/test-entities.md ---
@@ -0,0 +1,72 @@
+---
+title: Blueprint Test Entities
+title_in_menu: Test Entities
+layout: website-normal
+---
+
+{% include fields.md %}
+
+## Structural Test Entities
+
+### TestCase
+The `TestCase` entity acts as a container for a list child entities which are started *sequentially*.
+{% highlight yaml %}
+{% readj example_yaml/testcase-entity.yaml %}
+{% endhighlight %}
+The ability to sequentially run entities is used to allow applications to fully deploy before attempting to start any test entities.
+
+If your application blueprint consists of multiple services then you will also require the `ParallelTestCase` entity.
--- End diff --
I was looking at it as the entity-under-test and the TestCase could execute concurrently. The first assertion could then be an assertion that within 10 minutes the serviceUp was true.
The sequential TestCase also allows a series of assertions, such as invoking an effector and then asserting that a sensor has the expected value (or that it is reachable using the http test, or whatever).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r45357768
--- Diff: docs/guide/yaml/test/test-entities.md ---
@@ -0,0 +1,72 @@
+---
+title: Blueprint Test Entities
+title_in_menu: Test Entities
+layout: website-normal
+---
+
+{% include fields.md %}
+
+## Structural Test Entities
+
+### TestCase
+The `TestCase` entity acts as a container for a list child entities which are started *sequentially*.
+{% highlight yaml %}
+{% readj example_yaml/testcase-entity.yaml %}
+{% endhighlight %}
+The ability to sequentially run entities is used to allow applications to fully deploy before attempting to start any test entities.
+
+If your application blueprint consists of multiple services then you will also require the `ParallelTestCase` entity.
--- End diff --
Yes, that's what I was thinking of. That feels more like you are adding the test assertions, rather than having to "restructure" their app by putting it inside a `ParallelTest entity`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r44860947
--- Diff: docs/guide/yaml/test/index.md ---
@@ -0,0 +1,24 @@
+---
+title: Testing YAML Blueprints
+layout: website-normal
+children:
+- test-entities.md
+- usage-examples.md
+---
+
+Brooklyn provides a selection of basic test entities which can be used to validate Blueprints via YAML. These are divided into two groups, structural, which effect the order in which child entities are started, and validation, which are used to confirm the application is deployed as intended:
--- End diff --
I'd fo for a colon after the "two groups", a semi-colon after the "are started", and a full stop at the end of the sentence.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: yaml testing docs
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/1031#discussion_r44860617
--- Diff: docs/guide/yaml/test/example_yaml/paralleltestcase-example.yaml ---
@@ -0,0 +1,46 @@
+brooklyn.catalog:
+ id: parallel-test
+ version: 1.0
+ itemType: template
+ iconUrl: http://tomcat.apache.org/images/tomcat.png
+ name: Parallel Tomcat and DB Example Test
+ license: Apache-2.0
+ item:
+ brooklyn.config:
+ simple.confg: someValue
+ services:
+ - type: org.apache.brooklyn.test.framework.TestCase
+ name: Parallel Testcase
+ brooklyn.children:
+ - type: org.apache.brooklyn.test.framework.TestCase
+ name: Parallel Testcase
--- End diff --
Why a TestCase inside a TestCase, both with name "Parallel Testcase", and then a child of that being the entity-under-test?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---