Brooklyn Task Finalizer

[Andrew Kennedy <an...@cloudsoftcorp.com>]

All,

Have you ever seen the Java 'Finalizer' thread get into a state like this:

```
"Finalizer" daemon prio=10 tid=0x00007fd13c096800 nid=0xc5ac runnable
[0x00007fc9248dc000]
   java.lang.Thread.State: RUNNABLE
at
com.google.common.util.concurrent.ForwardingFuture.isCancelled(ForwardingFuture.java:53)
at
org.apache.brooklyn.util.core.task.BasicTask.isCancelled(BasicTask.java:323)
at
org.apache.brooklyn.util.core.task.Tasks.isAncestorCancelled(Tasks.java:247)
at
org.apache.brooklyn.util.core.task.Tasks.isAncestorCancelled(Tasks.java:248)
at
org.apache.brooklyn.util.core.task.Tasks.isAncestorCancelled(Tasks.java:248)
at
org.apache.brooklyn.util.core.task.Tasks.isAncestorCancelled(Tasks.java:248)
at
org.apache.brooklyn.util.core.task.Tasks.isAncestorCancelled(Tasks.java:248)
        ...
        at
org.apache.brooklyn.util.core.task.Tasks.isAncestorCancelled(Tasks.java:248)
```

Essentially there were over *1000* recursive calls to the
`#isAncestorCancelled(Task)` method; which seems excessive! The calls
appear to originate from the `#finalize()` method on the `BasicTask` class,
which will normally (this is the default implementation of `TaskFinalizer`)
execute this code:

```Java
    public static final TaskFinalizer WARN_IF_NOT_RUN = new TaskFinalizer()
{
        @Override
        public void onTaskFinalization(Task<?> t) {
            if (!Tasks.isAncestorCancelled(t) && !t.isSubmitted()) {
                // ...
```

As you can see, this starts off by calling the
`#isAncestorCancelled(Task)`, which does indeed make a recursive call to
itself if it finds an ancestor. What I _couldn't_ work out is why the call
sequence never terminates? The `submittedByTask` is what is checked in the
recursive call, and there _is_ a null check for the returned `Task` object
(from `#getSubmittedByTask()`) that is used as the ancestor. I did wonder
if there might be a problem with the following line, that sets the
submitting task in `BasicTask#setSubmittedByTask(Task)`:

```Java
    submittedByTask = (Maybe)Maybe.softThen((Task)task,
(Maybe)Maybe.of(BasicTask.newGoneTaskFor(task)));
```

So, this will create a new empty `Task` object if `task` has been garbage
collected, I believe. But I still can't see how this might result in the
continued recursive calls.

Anyway, if anyone has insight into this please let me know, as I am worried
that it is causing stability issues when running Brooklyn.

Andrew.
-- 

Andrew Kennedy ; Founder clocker.io project ; @grkvlt ; Cloudsoft