You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@brooklyn.apache.org by ri...@apache.org on 2014/07/02 11:38:35 UTC
[1/3] git commit: Adds docs for Brooklyn persistence
Repository: incubator-brooklyn
Updated Branches:
refs/heads/master f4656743d -> 9bb861bf9
Adds docs for Brooklyn persistence
Project: http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/commit/a32f7364
Tree: http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/tree/a32f7364
Diff: http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/diff/a32f7364
Branch: refs/heads/master
Commit: a32f7364939e6ecb7fa3d9de995d8394337759ba
Parents: f465674
Author: Aled Sage <al...@gmail.com>
Authored: Wed Jul 2 09:59:07 2014 +0100
Committer: Aled Sage <al...@gmail.com>
Committed: Wed Jul 2 09:59:07 2014 +0100
----------------------------------------------------------------------
docs/use/guide/persistence/index.md | 219 +++++++++++++++++++++++++++++++
docs/use/guide/persistence/toc.json | 14 ++
docs/use/guide/toc.json | 3 +
3 files changed, 236 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a32f7364/docs/use/guide/persistence/index.md
----------------------------------------------------------------------
diff --git a/docs/use/guide/persistence/index.md b/docs/use/guide/persistence/index.md
new file mode 100644
index 0000000..57ce2a4
--- /dev/null
+++ b/docs/use/guide/persistence/index.md
@@ -0,0 +1,219 @@
+---
+title: Persistence
+layout: page
+toc: ../guide_toc.json
+categories: [use, guide]
+
+---
+<a name="introduction"></a>
+Brooklyn can be configured to persist its state so that the Brooklyn server can be restarted,
+or so that a high availability standby server can take over.
+
+Brooklyn can persist its state to one of two places: the file system, or to an Object Store
+of your choice.
+
+<a name="command-line-options"></a>
+Command Line Options
+--------------------
+
+To configure brooklyn, the relevant command line options are:
+
+* `--persist` <persistence mode>
+ The persistence mode.
+* `--persistenceDir` <persistence dir>
+ The directory to read/write persisted state (or container name if using an object store).
+* `--persistenceLocation` <persistence location>
+ The location spec for an object store to read/write persisted state.
+
+For the persistence mode, the possible values are:
+
+* `disabled` means that no state will be persisted or read; when Brooklyn stops all state is lost.
+* `rebind` means that it will read existing state, and recreate entities, locations and policies
+ from that. If there is no existing state, startup will fail.
+* `clean` means that any existing state will be deleted, and Brooklyn will be started afresh.
+* `auto` means Brooklyn will rebind if there is any existing state, or will start afresh if
+ there is no state.
+
+
+<a name="file-based-persistence"></a>
+File-based Persistence
+----------------------
+
+To persist to the file system, start brooklyn with:
+
+{% highlight bash %}
+brooklyn launch --persist auto --persistenceDir /path/to/myPersistenceDir
+{% endhighlight %}
+
+an
+If there is already data at `/path/to/myPersistenceDir`, then a backup of the directory will
+be made. This will have a name like `/path/to/myPersistenceDir.20140701-142101345.bak`.
+
+The state is written to the given path. The file structure under that path is:
+
+* `./entities/`
+* `./locations/`
+* `./policies/`
+* `./enrichers/`
+
+In each of those directories, an XML file will be created per item - for example a file per
+entity in ./entities/. This file will capture all of the state - for example, an
+entity's: id; display name; type; config; attributes; tags; relationships to locations, child
+entities, group membership, policies and enrichers; and dynamically added effectors and sensors.
+
+
+<a name="object-store-persistence"></a>
+Object Store Persistence
+------------------------
+
+Brooklyn can persist its state to any Object Store API that jclouds supports including
+S3, Swift and Azure. This gives access to any compatible Object Store product or cloud provider
+including AWS-S3, SoftLayer, Rackspace, HP and Microsoft Azure.
+
+To configure the Object Store, add the credentials to `~/.brooklyn/brooklyn.properties` such as:
+
+{% highlight properties %}
+brooklyn.location.named.aws-s3-eu-west-1:aws-s3:eu-west-1
+brooklyn.location.named.aws-s3-eu-west-1.identity=ABCDEFGHIJKLMNOPQRSTU
+brooklyn.location.named.aws-s3-eu-west-1.credential=abcdefghijklmnopqrstuvwxyz1234567890ab/c
+{% endhighlight %}
+
+or:
+
+{% highlight properties %}
+brooklyn.location.named.softlayer-swift-ams01=jclouds:swift:https://ams01.objectstorage.softlayer.net/auth/v1.0
+brooklyn.location.named.softlayer-swift-ams01.identity=ABCDEFGHIJKLM:myname
+brooklyn.location.named.softlayer-swift-ams01.credential=abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz12
+{% endhighlight %}
+
+Start Brooklyn pointing at this target object store, e.g.:
+
+{% highlight bash %}
+brooklyn launch --persist auto --persistenceDir myContainerName --persistenceLocation named:softlayer-swift-ams01
+{% endhighlight %}
+
+
+<a name="rebind"></a>
+Rebind
+------
+
+When Brooklyn starts up pointing at existing state, it will recreate the entities, locations
+and policies based on that persisted state.
+
+Once all have been created, Brooklyn will "manage" the entities. This will bind to the
+underlying entities under management to update the each entity's sensors (e.g. to poll over
+HTTP or JMX). This new state will be reported in the web-console and can also trigger
+any registered policies.
+
+
+<a name="handling-rebind-failures"></a>
+Handling Rebind Failures
+------------------------
+If rebind were to fail for any reason, details of the underlying failures will be reported
+in the brooklyn.debug.log. There are several approaches to resolving problems.
+
+<a name="rebind-failures-determine-underlying-cause"></a>
+### Determine Underlying Cause
+
+The problems reported in brooklyn.debug.log will indicate where the problem lies - which
+entities, locations or policies, and in what way it failed.
+
+<a name="rebind-failures-ignore-errors"></a>
+### Ignore Errors
+
+The `~/.brooklyn/brooklyn.properties` has several configuration options:
+
+{% highlight properties %}
+rebind.failureMode.danglingRef=continue
+rebind.failureMode.loadPolicy=continue
+rebind.failureMode.addPolicy=continue
+rebind.failureMode.rebind=fail_at_end
+{% endhighlight %}
+
+For each of these configuration options, the possible values are:
+
+* `fail_fast`: stop rebind immediately upon errors; do not try to rebind other entities
+* `fail_at_end`: continue rebinding all entities, but then fail so that all errors
+ encountered are reported
+* `continue`: log a warning, but ignore the error to continue rebinding. Depending on the
+ type of error, this can cause serious problems later (e.g. if the state of an entity
+ was entirely missing, then all its children would be orphaned).
+
+The meaning of the configuration options is:
+
+* `rebind.failureMode.dangingRef`: if there is a reference to an entity, location or policy
+ that is missing... whether to continue (discarding the reference) or fail.
+* `rebind.failureMode.loadPolicy`: if there is an error instantiate or reconstituting the
+ state of a policy or enricher... whether to continue (discarding the policy or enricher)
+ or fail.
+* `rebind.failureMode.addPolicy`: if there is an error re-adding the policy or enricher to
+ its associated entity... whether to continue (discarding the policy or enricher)
+ or fail.
+* `rebind.failureMode.rebind`: any errors on rebind not covered by the more specific error cases described above.
+
+
+<a name="rebind-failures-seek-help"></a>
+### Seek Help
+
+Help can be found at `dev@brooklyn.incubator.apache.org`, where folk will be able to investigate
+issues and suggest work-arounds.
+
+By sharing the persisted state (with credentials removed), Brooklyn developers will be able to
+reproduce and debug the problem.
+
+
+<a name="rebind-failures-determine-fix-up-the-state"></a>
+### Fix-up the State
+
+The state of each entity, location, policy and enricher is persisted in XML.
+It is thus human readable and editable.
+
+After first taking a backup of the state, it is possible to modify the state. For example,
+an offending entity could be removed, or references to that entity removed, or its XML
+could be fixed to remove the problem.
+
+
+### Fixing with Groovy Scripts
+
+The final (powerful and dangerous!) tool is to execute Groovy code on the running Brooklyn
+instance. If authorized, the REST api allows arbitrary Groovy scripts to be passed in and
+executed. This allows the state of entities to be modified (and thus fixed) at runtime.
+
+If used, it is strongly recommended that Groovy scripts are run against a disconnected Brooklyn
+instance. After fixing the entities, locations and/or policies, the Brooklyn instance's
+new persisted state can be copied and used to fix the production instance.
+
+
+<a name="writing-persistable-code"></a>
+Writing Persistable Code
+------------------------
+The most common problem on rebind is that custom entity code has not been written in a way
+that can be persisted and/or rebound.
+
+The rule of thumb when implementing new entities, locations and policies is that all state
+must be persistable. All state must be stored as config or as attributes, and must be
+serializable (e.g. avoid the use of anonymous inner classes).
+
+Below is a guide for when implementing an entity in Java (or any other JVM language):
+
+* Don't store state in artibrary fields - the field will not be persisted (this is a design
+ decision, because Brooklyn cannot intercept the field being written to, so cannot know
+ when to persist).
+* Store runtime state as attributes.
+* Ensure values can be serialized. This (currently) uses xstream, which means it does not
+ need to implement `Serializable`. However, if declaring your own classes that are to be
+ persisted as state of the entity then declare them as static (or top-level) classes
+ rather than anonymous inner classes or non-static inner classes.
+* By extending `SoftwareProcess`, entities get a lot of the rebind logic for free. For
+ example, the default `rebind()` method will call `connectSensors()`.
+* If necessary, implement rebind. The `entity.rebind()` is called automatically by the
+ Brooklyn framework on rebind, after configuring the entity's config/attributes but before
+ the entity is managed.
+ Note that `init()` will not be called on rebind.
+* For special cases, it is possible to call `entity.requestPerist()` which will trigger
+ asynchronous persistence of the entity.
+
+For locations, policies and enrichers they (currently) do not have attributes. However,
+config is persisted automatically. Normally the state of a policy or enricher is transient -
+on rebind it starts afresh, for example with monitoring the performance or health metrics
+rather than relying on the persisted values.
http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a32f7364/docs/use/guide/persistence/toc.json
----------------------------------------------------------------------
diff --git a/docs/use/guide/persistence/toc.json b/docs/use/guide/persistence/toc.json
new file mode 100644
index 0000000..718d8d3
--- /dev/null
+++ b/docs/use/guide/persistence/toc.json
@@ -0,0 +1,14 @@
+[{ "title": "Introduction",
+ "file": "{{ site.url }}/use/guide/persistence/index.html#introduction" },
+{ "title": "Command Line Options",
+ "file": "{{ site.url }}/use/guide/persistence/index.html#command-line-options" },
+{ "title": "File-based Persistence",
+ "file": "{{ site.url }}/use/guide/persistence/index.html#file-based-persistence" },
+{ "title": "Object Store Persistence",
+ "file": "{{ site.url }}/use/guide/persistence/index.html#object-store-persistence" },
+{ "title": "Rebind",
+ "file": "{{ site.url }}/use/guide/persistence/index.html#rebind" },
+{ "title": "Handling Rebind Failures",
+ "file": "{{ site.url }}/use/guide/persistence/index.html#handling-rebind-failures" },
+{ "title": "Writing Persistable Code",
+ "file": "{{ site.url }}/use/guide/persistence/index.html#riting-persistable-code" }]
http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a32f7364/docs/use/guide/toc.json
----------------------------------------------------------------------
diff --git a/docs/use/guide/toc.json b/docs/use/guide/toc.json
index 7c4aa13..9a59b2a 100644
--- a/docs/use/guide/toc.json
+++ b/docs/use/guide/toc.json
@@ -8,6 +8,9 @@
{ "title": "Management",
"file": "{{ site.url }}/use/guide/management/index.html" ,
"children": {% readj ./management/toc.json %} },
+{ "title": "Persistence",
+ "file": "{{ site.url }}/use/guide/persistence/index.html" ,
+ "children": {% readj ./persistence/toc.json %} },
{ "title": "Locations",
"file": "{{ site.url }}/use/guide/locations/index.html" },
{ "title": "Policies",
[3/3] git commit: Merge branch PR #37 with fix
Posted by ri...@apache.org.
Merge branch PR #37 with fix
This closes PR #37.
This also includes an extra commit which fixes a broken link.
Project: http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/commit/9bb861bf
Tree: http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/tree/9bb861bf
Diff: http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/diff/9bb861bf
Branch: refs/heads/master
Commit: 9bb861bf951a1d0318a84c56c2acd04c23744706
Parents: f465674 66d2ad4
Author: Richard Downer <ri...@apache.org>
Authored: Wed Jul 2 10:37:39 2014 +0100
Committer: Richard Downer <ri...@apache.org>
Committed: Wed Jul 2 10:37:39 2014 +0100
----------------------------------------------------------------------
docs/use/guide/persistence/index.md | 219 +++++++++++++++++++++++++++++++
docs/use/guide/persistence/toc.json | 14 ++
docs/use/guide/toc.json | 3 +
3 files changed, 236 insertions(+)
----------------------------------------------------------------------
[2/3] git commit: Fix typo in hyperlink anchor reference
Posted by ri...@apache.org.
Fix typo in hyperlink anchor reference
Project: http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/commit/66d2ad4d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/tree/66d2ad4d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/diff/66d2ad4d
Branch: refs/heads/master
Commit: 66d2ad4da0e0a320acbb1cfdcc2498775c0e6666
Parents: a32f736
Author: Richard Downer <ri...@apache.org>
Authored: Wed Jul 2 10:36:32 2014 +0100
Committer: Richard Downer <ri...@apache.org>
Committed: Wed Jul 2 10:36:32 2014 +0100
----------------------------------------------------------------------
docs/use/guide/persistence/toc.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/66d2ad4d/docs/use/guide/persistence/toc.json
----------------------------------------------------------------------
diff --git a/docs/use/guide/persistence/toc.json b/docs/use/guide/persistence/toc.json
index 718d8d3..d6a13d7 100644
--- a/docs/use/guide/persistence/toc.json
+++ b/docs/use/guide/persistence/toc.json
@@ -11,4 +11,4 @@
{ "title": "Handling Rebind Failures",
"file": "{{ site.url }}/use/guide/persistence/index.html#handling-rebind-failures" },
{ "title": "Writing Persistable Code",
- "file": "{{ site.url }}/use/guide/persistence/index.html#riting-persistable-code" }]
+ "file": "{{ site.url }}/use/guide/persistence/index.html#writing-persistable-code" }]