You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@brooklyn.apache.org by he...@apache.org on 2021/09/14 21:56:19 UTC
[brooklyn-server] branch master updated (c1991d1 -> 7df3bba)
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git.
from c1991d1 lots more exclusions for read-only mode, and ability to interrogate them
new 96e7a3a prevent race/error in session lookup
new ae27393 further fix for warning on http session retrieval race condition
new 0925e5d more logging and error edge cases catches
new 85d9651 tidy task before/after model, ensure scheduled tasks clear their task context
new 88fc1a5 GC periodic-persistence tasks immediately
new c060b0c skip extra MDC task logging
new cc0ec89 fix typos, and tidy-up MDC logging in case of leaks (not sure it happens but be safe)
new 78b3129 better task logging of entitlement context
new 37c331e tidy rebind, skip duplicate hot-standby activation
new bcf0fc6 improve logging for tasks
new 783b243 only delete tasks which are done including completion if cancelled
new a6cdd85 logging about bundles being persisted and loaded
new 2d20fff when rebinding, install persisted bundles before default.catalog.bom; but start after
new ce9a796 allow access to catalog item path on RO entities
new 49ab36d better logging for HA promotion and for scheduled tasks
new 751b7ff better cleanup on switch from RO/hot
new 109876b stop tasks when ending RO mode also, more cleanly restart on rebind/promotion
new 8c98f10 fix tests which previously only passed due to scheduled tasks not being cancelled!
new 6057b9c tweak to cancelling and deleting tasks
new f6c5d85 add option to clear orig mgmt state, and test that it is doing what is expected
new b760aea fix race where persister might still be reading from hot standby
new 8477a27 tidy up a few more places where we get warns on unmanaging
new b1a9a3c downgrade pre-pre-managed location warning, as it seems to occur a lot
new 698b2d3 make sure task GC doesn't kill new entity tasks
new 1d99b34 fix a few more readonly/rebind logs and exclusions
new 750a63a suppress more warnings on planned rebind interruption
new 7df3bba This closes #1257
The 27 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../brooklyn/api/mgmt/rebind/RebindManager.java | 3 +
.../rebind/mementos/BrooklynMementoPersister.java | 2 +
.../catalog/internal/CatalogInitialization.java | 101 ++-
.../core/catalog/internal/CatalogUtils.java | 8 +-
.../brooklyn/core/entity/AbstractEntity.java | 34 +-
.../org/apache/brooklyn/core/entity/Entities.java | 9 +-
.../internal/EntityTransientCopyInternal.java | 4 +-
.../core/entity/lifecycle/ServiceStateLogic.java | 11 +-
.../brooklyn/core/feed/AttributePollHandler.java | 27 +-
.../core/mgmt/ha/HighAvailabilityManagerImpl.java | 56 +-
.../mgmt/internal/BrooklynGarbageCollector.java | 49 +-
.../mgmt/internal/EntityManagementSupport.java | 10 +-
.../core/mgmt/internal/LocalLocationManager.java | 7 +-
.../mgmt/internal/LocalSubscriptionManager.java | 13 +-
.../internal/NonDeploymentManagementContext.java | 5 +
.../BrooklynMementoPersisterToObjectStore.java | 75 +-
.../mgmt/rebind/RebindExceptionHandlerImpl.java | 35 +
.../brooklyn/core/mgmt/rebind/RebindIteration.java | 592 +++++++-------
.../core/mgmt/rebind/RebindManagerImpl.java | 88 ++-
.../mgmt/rebind/dto/BasicManagedBundleMemento.java | 7 +
.../brooklyn/core/typereg/BasicManagedBundle.java | 1 +
.../util/core/task/BasicExecutionContext.java | 55 +-
.../util/core/task/BasicExecutionManager.java | 873 +++++++++++++--------
.../apache/brooklyn/util/core/task/BasicTask.java | 3 +-
.../brooklyn/util/core/task/ScheduledTask.java | 7 +
.../mgmt/internal/EntityExecutionManagerTest.java | 23 +-
.../core/mgmt/rebind/ManagementPlaneIdTest.java | 6 +-
.../brooklyn/core/mgmt/rebind/RebindOptions.java | 6 +
.../core/mgmt/rebind/RebindTestFixture.java | 6 +-
.../brooklyn/core/mgmt/rebind/RebindTestUtils.java | 11 +-
.../util/core/task/ScheduledExecutionTest.java | 248 ++++--
.../BrooklynLauncherRebindCatalogOsgiTest.java | 4 +-
.../policy/failover/ElectPrimaryPolicy.java | 2 +-
.../policy/failover/PrimaryRunningEnricher.java | 5 +
.../rest/util/MultiSessionAttributeAdapter.java | 27 +-
.../feed/windows/WinRmCommandSensorTest.java | 21 +-
.../WindowsPerformanceCounterSensorsTest.java | 6 +-
.../brooklyn/util/exceptions/Exceptions.java | 2 +-
.../exceptions/RuntimeInterruptedException.java | 12 +-
39 files changed, 1586 insertions(+), 868 deletions(-)
[brooklyn-server] 07/27: fix typos,
and tidy-up MDC logging in case of leaks (not sure it happens but
be safe)
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit cc0ec89496aded36f8f059528bc621284dea303f
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Mon Sep 13 22:51:07 2021 +0100
fix typos, and tidy-up MDC logging in case of leaks (not sure it happens but be safe)
---
.../java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java | 5 +++--
.../org/apache/brooklyn/util/core/task/BasicExecutionManager.java | 5 ++++-
2 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
index 8f6d6af..57395c7 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
@@ -114,7 +114,8 @@ public class RebindManagerImpl implements RebindManager {
public static final Logger LOG = LoggerFactory.getLogger(RebindManagerImpl.class);
private final ManagementContextInternal managementContext;
-
+
+ // TODO separate times for persist and read-only rebind, allow configurable, default at least for RO rebind larger
private volatile Duration periodicPersistPeriod = Duration.ONE_SECOND;
private volatile boolean persistenceRunning = false;
@@ -340,7 +341,7 @@ public class RebindManagerImpl implements RebindManager {
Callable<Task<?>> taskFactory = new Callable<Task<?>>() {
@Override public Task<Void> call() {
- return Tasks.<Void>builder().dynamic(false).displayName("rebind (periodic run").body(new Callable<Void>() {
+ return Tasks.<Void>builder().dynamic(false).displayName("rebind (periodic run)").body(new Callable<Void>() {
@Override
public Void call() {
try {
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
index 706edeb..487b155 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
@@ -163,9 +163,12 @@ public class BasicExecutionManager implements ExecutionManager {
}
taskMdc = MDC.putCloseable(LOGGING_MDC_KEY_TASK_ID, task.getId());
}
+ prevEntityMdc = MDC.get(LOGGING_MDC_KEY_ENTITY_IDS);
if (entity != null) {
- prevEntityMdc = MDC.get(LOGGING_MDC_KEY_ENTITY_IDS);
entityMdc = MDC.putCloseable(LOGGING_MDC_KEY_ENTITY_IDS, "[" + entity.getApplicationId() + "," + entity.getId() + "]");
+ } else if (prevTaskMdc != null) {
+ // just in case some MDC logger leaked, make it explicit there is no entity here
+ entityMdc = MDC.putCloseable(LOGGING_MDC_KEY_ENTITY_IDS, "");
}
if (BrooklynLoggingCategories.TASK_LIFECYCLE_LOG.isDebugEnabled() || BrooklynLoggingCategories.TASK_LIFECYCLE_LOG.isTraceEnabled()) {
[brooklyn-server] 01/27: prevent race/error in session lookup
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 96e7a3a91e14e7f55e9d7da2209d50e933a1f02e
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Mon Sep 13 10:48:02 2021 +0100
prevent race/error in session lookup
---
.../rest/util/MultiSessionAttributeAdapter.java | 17 ++++++++++++++++-
1 file changed, 16 insertions(+), 1 deletion(-)
diff --git a/rest/rest-resources/src/main/java/org/apache/brooklyn/rest/util/MultiSessionAttributeAdapter.java b/rest/rest-resources/src/main/java/org/apache/brooklyn/rest/util/MultiSessionAttributeAdapter.java
index f950786..0b1878e 100644
--- a/rest/rest-resources/src/main/java/org/apache/brooklyn/rest/util/MultiSessionAttributeAdapter.java
+++ b/rest/rest-resources/src/main/java/org/apache/brooklyn/rest/util/MultiSessionAttributeAdapter.java
@@ -152,7 +152,22 @@ public class MultiSessionAttributeAdapter {
if (localSession==null) {
preferredSession = FACTORY.findValidPreferredSession(null, r);
if(preferredSession!=null) {
- localSession = r.getSession();
+ // need to create a local session so the ID/session is registered with this ui module
+ if (r instanceof Request) {
+ // but synch on the session handler to avoid race conditions in the underlying code
+// 2021-09-13T08:12:33,186Z - WARN 254 o.e.j.s.session [p1568796312-1154]
+// java.lang.IllegalStateException: Session node0171nuqxrc6qsf1tbrmxztok6xc4 already in cache
+// at org.eclipse.jetty.server.session.AbstractSessionCache.add(AbstractSessionCache.java:467) ~[!/:9.4.39.v20210325]
+// at org.eclipse.jetty.server.session.SessionHandler.newHttpSession(SessionHandler.java:770) ~[!/:9.4.39.v20210325]
+// at org.eclipse.jetty.server.Request.getSession(Request.java:1628) ~[!/:9.4.39.v20210325]
+// at org.eclipse.jetty.server.Request.getSession(Request.java:1602) ~[!/:9.4.39.v20210325]
+// at org.apache.brooklyn.rest.util.MultiSessionAttributeAdapter.of(MultiSessionAttributeAdapter.java:155) ~[!/:1.1.0-SNAPSHOT]
+ synchronized (((Request)r).getSessionHandler()) {
+ localSession = r.getSession();
+ }
+ } else {
+ localSession = r.getSession();
+ }
}
} else {
preferredSession = FACTORY.findPreferredSession(r);
[brooklyn-server] 06/27: skip extra MDC task logging
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit c060b0c60385833f73fef3335793bfa552482a72
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Mon Sep 13 22:30:37 2021 +0100
skip extra MDC task logging
---
.../brooklyn/util/core/task/BasicExecutionManager.java | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
index d7af2c0..706edeb 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
@@ -28,6 +28,7 @@ import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
+import java.util.Objects;
import java.util.Set;
import java.util.concurrent.Callable;
import java.util.concurrent.CancellationException;
@@ -135,6 +136,7 @@ public class BasicExecutionManager implements ExecutionManager {
return new BrooklynTaskLoggingMdc().withTask(task);
}
+ boolean isRedundant = false;
Task task;
MDC.MDCCloseable taskMdc=null, entityMdc=null;
String prevTaskMdc, prevEntityMdc;
@@ -155,6 +157,10 @@ public class BasicExecutionManager implements ExecutionManager {
// can misleadingly point point to the task which triggered the executor
if (task!=null) {
prevTaskMdc = MDC.get(LOGGING_MDC_KEY_TASK_ID);
+ if (Objects.equals(task.getId(), prevTaskMdc)) {
+ isRedundant = true;
+ return this;
+ }
taskMdc = MDC.putCloseable(LOGGING_MDC_KEY_TASK_ID, task.getId());
}
if (entity != null) {
@@ -162,7 +168,7 @@ public class BasicExecutionManager implements ExecutionManager {
entityMdc = MDC.putCloseable(LOGGING_MDC_KEY_ENTITY_IDS, "[" + entity.getApplicationId() + "," + entity.getId() + "]");
}
- if (BrooklynLoggingCategories.TASK_LIFECYCLE_LOG.isDebugEnabled() || BrooklynLoggingCategories.TASK_LIFECYCLE_LOG.isTraceEnabled()){
+ if (BrooklynLoggingCategories.TASK_LIFECYCLE_LOG.isDebugEnabled() || BrooklynLoggingCategories.TASK_LIFECYCLE_LOG.isTraceEnabled()) {
String taskName = task.getDisplayName();
String message = "Starting task " + task.getId() +
(Strings.isNonBlank(taskName) ? " ("+taskName+")" : "") +
@@ -179,6 +185,9 @@ public class BasicExecutionManager implements ExecutionManager {
}
public void finish() {
+ if (isRedundant) {
+ return;
+ }
if (BrooklynLoggingCategories.TASK_LIFECYCLE_LOG.isDebugEnabled() || BrooklynLoggingCategories.TASK_LIFECYCLE_LOG.isTraceEnabled()){
String taskName = task.getDisplayName();
BrooklynLogging.log(BrooklynLoggingCategories.TASK_LIFECYCLE_LOG,
@@ -1088,7 +1097,7 @@ public class BasicExecutionManager implements ExecutionManager {
taskWasSubmittedAndNotYetEnded = incompleteTaskIds.remove(task.getId());
// this method might be called more than once, eg if cancelled, so use the above as a guard where single invocation is needed (eg counts)
- if (!skipDecrementCounter && taskWasSubmittedAndNotYetEnded) {
+ if (taskWasSubmittedAndNotYetEnded) {
activeTaskCount.decrementAndGet();
}
[brooklyn-server] 27/27: This closes #1257
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 7df3bba5c91c82cce41bf6b3b5b71b1c175b402a
Merge: c1991d1 750a63a
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 22:56:11 2021 +0100
This closes #1257
.../brooklyn/api/mgmt/rebind/RebindManager.java | 3 +
.../rebind/mementos/BrooklynMementoPersister.java | 2 +
.../catalog/internal/CatalogInitialization.java | 101 ++-
.../core/catalog/internal/CatalogUtils.java | 8 +-
.../brooklyn/core/entity/AbstractEntity.java | 34 +-
.../org/apache/brooklyn/core/entity/Entities.java | 9 +-
.../internal/EntityTransientCopyInternal.java | 4 +-
.../core/entity/lifecycle/ServiceStateLogic.java | 11 +-
.../brooklyn/core/feed/AttributePollHandler.java | 27 +-
.../core/mgmt/ha/HighAvailabilityManagerImpl.java | 56 +-
.../mgmt/internal/BrooklynGarbageCollector.java | 49 +-
.../mgmt/internal/EntityManagementSupport.java | 10 +-
.../core/mgmt/internal/LocalLocationManager.java | 7 +-
.../mgmt/internal/LocalSubscriptionManager.java | 13 +-
.../internal/NonDeploymentManagementContext.java | 5 +
.../BrooklynMementoPersisterToObjectStore.java | 75 +-
.../mgmt/rebind/RebindExceptionHandlerImpl.java | 35 +
.../brooklyn/core/mgmt/rebind/RebindIteration.java | 592 +++++++-------
.../core/mgmt/rebind/RebindManagerImpl.java | 88 ++-
.../mgmt/rebind/dto/BasicManagedBundleMemento.java | 7 +
.../brooklyn/core/typereg/BasicManagedBundle.java | 1 +
.../util/core/task/BasicExecutionContext.java | 55 +-
.../util/core/task/BasicExecutionManager.java | 873 +++++++++++++--------
.../apache/brooklyn/util/core/task/BasicTask.java | 3 +-
.../brooklyn/util/core/task/ScheduledTask.java | 7 +
.../mgmt/internal/EntityExecutionManagerTest.java | 23 +-
.../core/mgmt/rebind/ManagementPlaneIdTest.java | 6 +-
.../brooklyn/core/mgmt/rebind/RebindOptions.java | 6 +
.../core/mgmt/rebind/RebindTestFixture.java | 6 +-
.../brooklyn/core/mgmt/rebind/RebindTestUtils.java | 11 +-
.../util/core/task/ScheduledExecutionTest.java | 248 ++++--
.../BrooklynLauncherRebindCatalogOsgiTest.java | 4 +-
.../policy/failover/ElectPrimaryPolicy.java | 2 +-
.../policy/failover/PrimaryRunningEnricher.java | 5 +
.../rest/util/MultiSessionAttributeAdapter.java | 27 +-
.../feed/windows/WinRmCommandSensorTest.java | 21 +-
.../WindowsPerformanceCounterSensorsTest.java | 6 +-
.../brooklyn/util/exceptions/Exceptions.java | 2 +-
.../exceptions/RuntimeInterruptedException.java | 12 +-
39 files changed, 1586 insertions(+), 868 deletions(-)
[brooklyn-server] 19/27: tweak to cancelling and deleting tasks
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 6057b9c323b7d5e6a5638497a1925a75888598a4
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 14:57:23 2021 +0100
tweak to cancelling and deleting tasks
---
.../apache/brooklyn/api/mgmt/rebind/RebindManager.java | 3 +++
.../core/mgmt/ha/HighAvailabilityManagerImpl.java | 9 +++++----
.../mgmt/internal/NonDeploymentManagementContext.java | 5 +++++
.../brooklyn/core/mgmt/rebind/RebindIteration.java | 9 +--------
.../brooklyn/core/mgmt/rebind/RebindManagerImpl.java | 17 ++++++++---------
5 files changed, 22 insertions(+), 21 deletions(-)
diff --git a/api/src/main/java/org/apache/brooklyn/api/mgmt/rebind/RebindManager.java b/api/src/main/java/org/apache/brooklyn/api/mgmt/rebind/RebindManager.java
index d9d4422..79fc49f 100644
--- a/api/src/main/java/org/apache/brooklyn/api/mgmt/rebind/RebindManager.java
+++ b/api/src/main/java/org/apache/brooklyn/api/mgmt/rebind/RebindManager.java
@@ -97,6 +97,9 @@ public interface RebindManager {
* Interrupts any current activity and waits for it to cease. */
public void stopReadOnly();
+ @Beta
+ public void stopEntityTasksAndCleanUp(String reason, Duration delayBeforeCancelling, Duration delayBeforeAbandoning);
+
public boolean isReadOnly();
/**
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
index d3ad012..9bbd00e 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
@@ -990,9 +990,6 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
* (e.g. during the periodic rebind as hot_stanby we will not repeatedly clear the brooklyn-managed-bundles).
*/
protected void clearManagedItems(ManagementTransitionMode mode) {
- // note, tasks are cancelled prior to this, when coming from RO mode, via
- // RebindManagerImpl.stopEntityAndDoneTasksBeforeRebinding
-
// log this because it may be surprising, it is just HA transitions,
// not symmetric with usual single-node start
LOG.info("Clearing all managed items on transition to "+mode);
@@ -1022,7 +1019,11 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
((BasicBrooklynTypeRegistry)managementContext.getTypeRegistry()).clear();
managementContext.getCatalogInitialization().clearBrooklynManagedBundles();
- ((LocalManagementContext)managementContext).getGarbageCollector().gcTasks();
+ // note, tasks are also cancelled prior to this when coming from RO mode, via
+ // RebindManagerImpl.stopReadOnly call to same method
+ managementContext.getRebindManager().stopEntityTasksAndCleanUp("when clearing mgmt on HA change",
+ Duration.millis(500),
+ Duration.seconds(2));
}
/** Starts hot standby or hot backup, in foreground
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/NonDeploymentManagementContext.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/NonDeploymentManagementContext.java
index 5613e91..e88e08e 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/NonDeploymentManagementContext.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/NonDeploymentManagementContext.java
@@ -603,6 +603,11 @@ public class NonDeploymentManagementContext implements ManagementContextInternal
}
@Override
+ public void stopEntityTasksAndCleanUp(String reason, Duration delayBeforeCancelling, Duration delayBeforeAbandoning) {
+ // no-op
+ }
+
+ @Override
public void reset() {
throw new IllegalStateException("Non-deployment context "+NonDeploymentManagementContext.this+" is not valid for this operation.");
}
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
index bf28d65..cf96700 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
@@ -20,8 +20,6 @@ package org.apache.brooklyn.core.mgmt.rebind;
import static com.google.common.base.Preconditions.checkNotNull;
import java.util.function.Supplier;
-import java.util.stream.Collectors;
-import org.apache.brooklyn.api.mgmt.Task;
import static org.apache.brooklyn.core.BrooklynFeatureEnablement.FEATURE_AUTO_FIX_CATALOG_REF_ON_REBIND;
import static org.apache.brooklyn.core.BrooklynFeatureEnablement.FEATURE_BACKWARDS_COMPATIBILITY_INFER_CATALOG_ITEM_ON_REBIND;
import static org.apache.brooklyn.core.catalog.internal.CatalogUtils.newClassLoadingContextForCatalogItems;
@@ -44,7 +42,6 @@ import org.apache.brooklyn.api.catalog.CatalogItem;
import org.apache.brooklyn.api.entity.Application;
import org.apache.brooklyn.api.entity.Entity;
import org.apache.brooklyn.api.location.Location;
-import org.apache.brooklyn.api.mgmt.ExecutionContext;
import org.apache.brooklyn.api.mgmt.classloading.BrooklynClassLoadingContext;
import org.apache.brooklyn.api.mgmt.ha.ManagementNodeState;
import org.apache.brooklyn.api.mgmt.rebind.RebindContext;
@@ -85,7 +82,6 @@ import org.apache.brooklyn.core.entity.EntityInternal;
import org.apache.brooklyn.core.feed.AbstractFeed;
import org.apache.brooklyn.core.location.AbstractLocation;
import org.apache.brooklyn.core.location.internal.LocationInternal;
-import org.apache.brooklyn.core.mgmt.BrooklynTaskTags;
import org.apache.brooklyn.core.mgmt.classloading.BrooklynClassLoadingContextSequential;
import org.apache.brooklyn.core.mgmt.classloading.JavaBrooklynClassLoadingContext;
import org.apache.brooklyn.core.mgmt.ha.OsgiManager;
@@ -113,8 +109,6 @@ import org.apache.brooklyn.util.collections.MutableList;
import org.apache.brooklyn.util.collections.MutableMap;
import org.apache.brooklyn.util.core.ClassLoaderUtils;
import org.apache.brooklyn.util.core.flags.FlagUtils;
-import org.apache.brooklyn.util.core.task.BasicExecutionContext;
-import org.apache.brooklyn.util.core.task.BasicExecutionManager;
import org.apache.brooklyn.util.core.task.Tasks;
import org.apache.brooklyn.util.exceptions.Exceptions;
import org.apache.brooklyn.util.guava.Maybe;
@@ -122,7 +116,6 @@ import org.apache.brooklyn.util.javalang.Reflections;
import org.apache.brooklyn.util.osgi.VersionedName;
import org.apache.brooklyn.util.stream.InputStreamSource;
import org.apache.brooklyn.util.text.Strings;
-import org.apache.brooklyn.util.time.CountdownTimer;
import org.apache.brooklyn.util.time.Duration;
import org.apache.brooklyn.util.time.Time;
import org.slf4j.Logger;
@@ -265,7 +258,7 @@ public abstract class RebindIteration {
protected void doRun() throws Exception {
if (readOnlyRebindCount.get() > 1) {
// prevent leaking
- rebindManager.stopEntityAndDoneTasksBeforeRebinding("before next read-only rebind", Duration.seconds(10), Duration.seconds(20));
+ rebindManager.stopEntityTasksAndCleanUp("before next read-only rebind", Duration.seconds(10), Duration.seconds(20));
}
loadManifestFiles();
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
index 781a8c1..899e10b 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
@@ -47,7 +47,6 @@ import org.apache.brooklyn.api.mgmt.rebind.mementos.TreeNode;
import org.apache.brooklyn.api.objs.BrooklynObject;
import org.apache.brooklyn.config.ConfigKey;
import org.apache.brooklyn.core.BrooklynFeatureEnablement;
-import org.apache.brooklyn.core.BrooklynVersion;
import org.apache.brooklyn.core.config.ConfigKeys;
import org.apache.brooklyn.core.enricher.AbstractEnricher;
import org.apache.brooklyn.core.entity.Entities;
@@ -398,18 +397,18 @@ public class RebindManagerImpl implements RebindManager {
}
readOnlyTask = null;
LOG.debug("Stopped read-only rebinding ("+this+"), mgmt "+managementContext.getManagementNodeId());
+
+ // short waits when promoting
+ stopEntityTasksAndCleanUp("when stopping hot proxy read-only mode",
+ Duration.seconds(2),
+ Duration.seconds(5));
+ // note, items are subsequently unmanaged via:
+ // HighAvailabilityManagerImpl.clearManagedItems
}
- // short waits when promoting
- stopEntityAndDoneTasksBeforeRebinding("when stopping hot proxy read-only mode",
- Duration.seconds(2),
- Duration.seconds(5));
- // note, items are subsequently unmanaged via:
- // HighAvailabilityManagerImpl.clearManagedItems
}
- public void stopEntityAndDoneTasksBeforeRebinding(String reason, Duration delayBeforeCancelling, Duration delayBeforeAbandoning) {
+ public void stopEntityTasksAndCleanUp(String reason, Duration delayBeforeCancelling, Duration delayBeforeAbandoning) {
// TODO inputs should be configurable
-
if (!managementContext.isRunning() || managementContext.getExecutionManager().isShutdown()) {
return;
}
[brooklyn-server] 09/27: tidy rebind,
skip duplicate hot-standby activation
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 37c331ea43f85e7958d9f03e8de3e3fc5e1b1743
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Mon Sep 13 23:17:08 2021 +0100
tidy rebind, skip duplicate hot-standby activation
---
.../brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java | 12 ++++++++++--
.../brooklyn/util/core/task/BasicExecutionManager.java | 2 +-
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
index 2bd080d..fab4ffc 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
@@ -314,7 +314,8 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
// catch error in some tests where mgmt context has a different HA manager
if (managementContext.getHighAvailabilityManager()!=this)
throw new IllegalStateException("Cannot start an HA manager on a management context with a different HA manager!");
-
+
+ boolean newModeApplied = false;
if (weAreMasterLocally) {
// demotion may be required; do this before triggering an election
switch (startMode) {
@@ -326,13 +327,20 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
case HOT_STANDBY:
case HOT_BACKUP:
case STANDBY:
- demoteTo(ManagementNodeState.of(startMode).get()); break;
+ demoteTo(ManagementNodeState.of(startMode).get());
+ newModeApplied = true;
+ break;
default:
throw new IllegalStateException("Unexpected high availability mode "+startMode+" requested for "+this);
}
}
ManagementNodeState oldState = getInternalNodeState();
+ if (newModeApplied && Objects.equal(oldState, startMode)) {
+ // successfully applied new mode, as part of demoteTo call above;
+ // skip the duplicate logic below which re-applies it
+ return;
+ }
// now do election
switch (startMode) {
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
index 8b44490..793c5ca 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
@@ -390,7 +390,7 @@ public class BasicExecutionManager implements ExecutionManager {
if (context!=null && !Entities.isManaged(context)) {
log.debug("Forgetting about active task on unmanagement of "+context+": "+removed);
} else {
- log.warn("Deleting submitted task before completion: "+removed+"; this task will continue to run in the background outwith "+this+", but perhaps it should have been cancelled?");
+ log.warn("Deleting submitted task before completion: "+removed+" (tags "+removed.getTags()+"); this task will continue to run in the background outwith "+this+", but perhaps it should have been cancelled?");
}
}
task.getTags().forEach(t -> {
[brooklyn-server] 24/27: make sure task GC doesn't kill new entity
tasks
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 698b2d38397134fca4d68ae2e19942e621b39c80
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 19:48:08 2021 +0100
make sure task GC doesn't kill new entity tasks
previously when replacing an entity (eg on promotion) the new live entity's tasks could get deleted accidentally
---
.../mgmt/internal/BrooklynGarbageCollector.java | 8 +++---
.../util/core/task/BasicExecutionManager.java | 30 ++++++++++++++++++++++
.../mgmt/internal/EntityExecutionManagerTest.java | 23 +++++++++--------
3 files changed, 46 insertions(+), 15 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/BrooklynGarbageCollector.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/BrooklynGarbageCollector.java
index 74f89a4..f780b5c 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/BrooklynGarbageCollector.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/BrooklynGarbageCollector.java
@@ -280,10 +280,10 @@ public class BrooklynGarbageCollector {
public void deleteTasksForEntity(Entity entity) {
// remove all references to this entity from tasks
// (note that cancellation for most tasks will have been done by LocalEntityManager.stopTasks)
- executionManager.deleteTag(entity);
- executionManager.deleteTag(BrooklynTaskTags.tagForContextEntity(entity));
- executionManager.deleteTag(BrooklynTaskTags.tagForCallerEntity(entity));
- executionManager.deleteTag(BrooklynTaskTags.tagForTargetEntity(entity));
+ executionManager.deleteDoneInTag(entity);
+ executionManager.deleteDoneInTag(BrooklynTaskTags.tagForContextEntity(entity));
+ executionManager.deleteDoneInTag(BrooklynTaskTags.tagForCallerEntity(entity));
+ executionManager.deleteDoneInTag(BrooklynTaskTags.tagForTargetEntity(entity));
}
public void onUnmanaged(Location loc) {
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
index 0498c4f..5f94d40 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
@@ -69,6 +69,7 @@ import org.apache.brooklyn.core.mgmt.entitlement.Entitlements;
import org.apache.brooklyn.util.collections.MutableList;
import org.apache.brooklyn.util.collections.MutableMap;
import org.apache.brooklyn.util.collections.MutableSet;
+import org.apache.brooklyn.util.core.task.BasicExecutionManager.BrooklynTaskLoggingMdc;
import org.apache.brooklyn.util.core.task.TaskInternal.TaskCancellationMode;
import org.apache.brooklyn.util.exceptions.Exceptions;
import org.apache.brooklyn.util.exceptions.RuntimeInterruptedException;
@@ -373,7 +374,9 @@ public class BasicExecutionManager implements ExecutionManager {
* <p>
* Useful, for example, if an entity is being expunged so that we don't keep holding
* a reference to it as a tag.
+ * @deprecated since 1.1 use deleteDoneInTag
*/
+ @Deprecated
public void deleteTag(Object tag) {
Set<Task<?>> tasks;
synchronized (tasksByTag) {
@@ -386,6 +389,33 @@ public class BasicExecutionManager implements ExecutionManager {
}
}
+ /**
+ * Deletes all truly done tasks in the given tag, and the tag also if empty when completed.
+ * @return if all tasks were done and the tag has been deleted
+ */
+ public boolean deleteDoneInTag(Object tag) {
+ Set<Task<?>> tasks;
+ boolean tagEmpty = true;
+ synchronized (tasksByTag) {
+ tasks = MutableSet.copyOf(tasksByTag.get(tag));
+ }
+ if (tasks != null) {
+ for (Task<?> task : tasks) {
+ if (task.isDone(true)) {
+ deleteTask(task);
+ } else {
+ tagEmpty = false;
+ }
+ }
+ }
+ if (tagEmpty) {
+ if (!tasksByTag.containsKey(tag)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
public void deleteTask(Task<?> task) {
boolean removed = deleteTaskNonRecursive(task);
if (!removed) return;
diff --git a/core/src/test/java/org/apache/brooklyn/core/mgmt/internal/EntityExecutionManagerTest.java b/core/src/test/java/org/apache/brooklyn/core/mgmt/internal/EntityExecutionManagerTest.java
index 4f581f5..6168942 100644
--- a/core/src/test/java/org/apache/brooklyn/core/mgmt/internal/EntityExecutionManagerTest.java
+++ b/core/src/test/java/org/apache/brooklyn/core/mgmt/internal/EntityExecutionManagerTest.java
@@ -314,18 +314,19 @@ public class EntityExecutionManagerTest extends BrooklynAppUnitTestSupport {
Entities.destroy(e);
forceGc();
-
- Set<Object> tags2 = app.getManagementContext().getExecutionManager().getTaskTags();
- for (Object tag : tags2) {
- if (tag instanceof Entity && ((Entity)tag).getId().equals(eId)) {
- fail("tags contains unmanaged entity "+tag);
- }
- if ((tag instanceof WrappedEntity) && ((WrappedEntity)tag).unwrap().getId().equals(eId)
- && ((WrappedItem<?>)tag).getWrappingType().equals(BrooklynTaskTags.CONTEXT_ENTITY)) {
- fail("tags contains unmanaged entity (wrapped) "+tag);
+
+ Asserts.succeedsEventually(() -> {
+ Set<Object> tags2 = app.getManagementContext().getExecutionManager().getTaskTags();
+ for (Object tag : tags2) {
+ if (tag instanceof Entity && ((Entity) tag).getId().equals(eId)) {
+ fail("tags contains unmanaged entity " + tag + "; tasks: " + app.getManagementContext().getExecutionManager().getTasksWithTag(tag));
+ }
+ if ((tag instanceof WrappedEntity) && ((WrappedEntity) tag).unwrap().getId().equals(eId)
+ && ((WrappedItem<?>) tag).getWrappingType().equals(BrooklynTaskTags.CONTEXT_ENTITY)) {
+ fail("tags contains unmanaged entity (wrapped) " + tag + "; tasks: " + app.getManagementContext().getExecutionManager().getTasksWithTag(tag));
+ }
}
- }
- return;
+ });
}
@Test(groups="Integration")
[brooklyn-server] 25/27: fix a few more readonly/rebind logs and
exclusions
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 1d99b34816dca8d1e803cfcff6ac09040b460b38
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 22:12:27 2021 +0100
fix a few more readonly/rebind logs and exclusions
---
.../brooklyn/core/entity/AbstractEntity.java | 32 +++++++++++-----------
.../brooklyn/core/feed/AttributePollHandler.java | 27 ++++++++++--------
2 files changed, 32 insertions(+), 27 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/core/entity/AbstractEntity.java b/core/src/main/java/org/apache/brooklyn/core/entity/AbstractEntity.java
index b796f61..fe2a8d9 100644
--- a/core/src/main/java/org/apache/brooklyn/core/entity/AbstractEntity.java
+++ b/core/src/main/java/org/apache/brooklyn/core/entity/AbstractEntity.java
@@ -924,7 +924,7 @@ public abstract class AbstractEntity extends AbstractBrooklynObject implements E
return sensors().get(attribute);
}
- static Set<String> WARNED_READ_ONLY_ATTRIBUTES = Collections.synchronizedSet(MutableSet.<String>of());
+ private static Set<String> LOGGED_PROMINENT_READ_ONLY_ATTRIBUTES = Collections.synchronizedSet(MutableSet.<String>of());
// -------- CONFIGURATION --------------
@@ -971,33 +971,33 @@ public abstract class AbstractEntity extends AbstractBrooklynObject implements E
if (Equals.approximately(val, oldVal)) {
// ignore, probably an enricher resetting values or something on init
} else {
- String message = AbstractEntity.this+" setting "+attribute+" = "+val+" (was "+oldVal+") in read only mode; will have very little effect";
+ String message = AbstractEntity.this+" setting "+attribute+" = "+val+" (was "+oldVal+") in read only mode; will not be persisted or published";
if (!getManagementSupport().isDeployed()) {
if (getManagementSupport().wasDeployed()) message += " (no longer deployed)";
else message += " (not yet deployed)";
}
- if (WARNED_READ_ONLY_ATTRIBUTES.add(attribute.getName())) {
- LOG.warn(message + " (future messages for this sensor logged at trace)");
+ if (LOGGED_PROMINENT_READ_ONLY_ATTRIBUTES.add(attribute.getName())) {
+ LOG.info(message + " (future messages for this sensor logged at trace)");
} else if (LOG.isTraceEnabled()) {
LOG.trace(message);
}
}
- }
- T result = attributesInternal.update(attribute, val);
- if (!Entities.isReadOnly(AbstractEntity.this)) {
// suppress notifications if read only
+ return attributesInternal.updateInternalWithoutLockOrPublish(attribute, val);
+ }
- if (result == null) {
- // could be this is a new sensor
- entityType.addSensorIfAbsent(attribute);
- }
+ T result = attributesInternal.update(attribute, val);
- if (!Objects.equal(result, val)) {
- getManagementSupport().getEntityChangeListener().onAttributeChanged(attribute);
- }
+ if (result == null) {
+ // could be this is a new sensor
+ entityType.addSensorIfAbsent(attribute);
}
-
+
+ if (!Objects.equal(result, val)) {
+ getManagementSupport().getEntityChangeListener().onAttributeChanged(attribute);
+ }
+
return result;
}
@@ -1032,7 +1032,7 @@ public abstract class AbstractEntity extends AbstractBrooklynObject implements E
if (getManagementSupport().wasDeployed()) message += " (no longer deployed)";
else message += " (not yet deployed)";
}
- if (WARNED_READ_ONLY_ATTRIBUTES.add(attribute.getName())) {
+ if (LOGGED_PROMINENT_READ_ONLY_ATTRIBUTES.add(attribute.getName())) {
LOG.warn(message + " (future messages for this sensor logged at trace)");
} else if (LOG.isTraceEnabled()) {
LOG.trace(message);
diff --git a/core/src/main/java/org/apache/brooklyn/core/feed/AttributePollHandler.java b/core/src/main/java/org/apache/brooklyn/core/feed/AttributePollHandler.java
index 0377eed..c599169 100644
--- a/core/src/main/java/org/apache/brooklyn/core/feed/AttributePollHandler.java
+++ b/core/src/main/java/org/apache/brooklyn/core/feed/AttributePollHandler.java
@@ -158,13 +158,13 @@ public class AttributePollHandler<V> implements PollHandler<V> {
if (!lastWasProblem) {
if (expiryTime <= nowTime) {
currentProblemLoggedAsWarning = true;
- if (entity==null || !Entities.isNoLongerManaged(entity)) {
+ if (entity==null || Entities.isManagedActive(entity)) {
log.warn("Read of " + getBriefDescription() + " gave " + type + ": " + val);
+ if (log.isDebugEnabled() && val instanceof Throwable)
+ log.debug("Trace for "+type+" reading "+getBriefDescription()+": "+val, (Throwable)val);
} else {
- log.debug("Read of " + getBriefDescription() + " gave " + type + ": " + val);
+ log.debug("Read (unmanaged) of " + getBriefDescription() + " gave " + type + ": " + val);
}
- if (log.isDebugEnabled() && val instanceof Throwable)
- log.debug("Trace for "+type+" reading "+getBriefDescription()+": "+val, (Throwable)val);
} else {
if (log.isDebugEnabled())
log.debug("Read of " + getBriefDescription() + " gave " + type + " (in grace period): " + val);
@@ -174,13 +174,18 @@ public class AttributePollHandler<V> implements PollHandler<V> {
} else {
if (expiryTime <= nowTime) {
currentProblemLoggedAsWarning = true;
- log.warn("Read of " + getBriefDescription() + " gave " + type +
- " (grace period expired, occurring for "+Duration.millis(nowTime - currentProblemStartTimeCache)+
- (config.hasExceptionHandler() ? "" : ", no exception handler set for sensor")+
- ")"+
- ": " + val);
- if (log.isDebugEnabled() && val instanceof Throwable)
- log.debug("Trace for "+type+" reading "+getBriefDescription()+": "+val, (Throwable)val);
+ if (entity==null || Entities.isManagedActive(entity)) {
+ log.warn("Read of " + getBriefDescription() + " gave " + type +
+ " (grace period expired, occurring for " + Duration.millis(nowTime - currentProblemStartTimeCache) +
+ (config.hasExceptionHandler() ? "" : ", no exception handler set for sensor") +
+ ")" +
+ ": " + val);
+ if (log.isDebugEnabled() && val instanceof Throwable)
+ log.debug("Trace for " + type + " reading " + getBriefDescription() + ": " + val, (Throwable) val);
+ } else {
+ if (log.isDebugEnabled())
+ log.debug("Read (unmanaged) of " + getBriefDescription() + " gave " + type + " (grace period expired): " + val);
+ }
} else {
if (log.isDebugEnabled())
log.debug("Recurring {} reading {} in {} (still in grace period): {}", new Object[] {type, this, getBriefDescription(), val});
[brooklyn-server] 23/27: downgrade pre-pre-managed location warning,
as it seems to occur a lot
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit b1a9a3c4d2c05525fa13714f2fc390427a1c3516
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 19:47:40 2021 +0100
downgrade pre-pre-managed location warning, as it seems to occur a lot
esp on rebind, and not have any ill effects
---
.../apache/brooklyn/core/mgmt/internal/LocalLocationManager.java | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/LocalLocationManager.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/LocalLocationManager.java
index 2555209..2b4d54c 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/LocalLocationManager.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/LocalLocationManager.java
@@ -153,8 +153,11 @@ public class LocalLocationManager implements LocationManagerInternal {
synchronized void prePreManage(Location loc) {
if (isPreRegistered(loc)) {
- log.warn(""+this+" redundant call to pre-pre-manage location "+loc+"; skipping",
- new Exception("source of duplicate pre-pre-manage of "+loc));
+ if (log.isTraceEnabled()) {
+ // seems to occur regularly on rebind, and not be a problem; ignore
+ log.trace("" + this + " redundant call to pre-pre-manage location " + loc + "; skipping",
+ new Exception("source of duplicate pre-pre-manage of " + loc));
+ }
return;
}
preRegisteredLocationsById.put(loc.getId(), loc);
[brooklyn-server] 20/27: add option to clear orig mgmt state,
and test that it is doing what is expected
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit f6c5d85e0e00ed5949d0854ba9339551007e9883
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 15:08:54 2021 +0100
add option to clear orig mgmt state, and test that it is doing what is expected
---
.../core/mgmt/ha/HighAvailabilityManagerImpl.java | 3 ++-
.../apache/brooklyn/core/mgmt/rebind/RebindOptions.java | 6 ++++++
.../brooklyn/core/mgmt/rebind/RebindTestFixture.java | 6 +++++-
.../apache/brooklyn/core/mgmt/rebind/RebindTestUtils.java | 11 ++++++++++-
.../brooklyn/feed/windows/WinRmCommandSensorTest.java | 15 +++++++++++++--
5 files changed, 36 insertions(+), 5 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
index 9bbd00e..8c63775 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
@@ -989,7 +989,8 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
* which is comparatively more expensive, so we only do it when we stop being a hotProxy or when we are demoted
* (e.g. during the periodic rebind as hot_stanby we will not repeatedly clear the brooklyn-managed-bundles).
*/
- protected void clearManagedItems(ManagementTransitionMode mode) {
+ @VisibleForTesting
+ public void clearManagedItems(ManagementTransitionMode mode) {
// log this because it may be surprising, it is just HA transitions,
// not symmetric with usual single-node start
LOG.info("Clearing all managed items on transition to "+mode);
diff --git a/core/src/test/java/org/apache/brooklyn/core/mgmt/rebind/RebindOptions.java b/core/src/test/java/org/apache/brooklyn/core/mgmt/rebind/RebindOptions.java
index 72e956f..14e40cb 100644
--- a/core/src/test/java/org/apache/brooklyn/core/mgmt/rebind/RebindOptions.java
+++ b/core/src/test/java/org/apache/brooklyn/core/mgmt/rebind/RebindOptions.java
@@ -39,6 +39,7 @@ import com.google.common.collect.Iterables;
public class RebindOptions {
public boolean checkSerializable;
+ public boolean clearOrigManagementContext;
public boolean terminateOrigManagementContext;
public RebindExceptionHandler exceptionHandler = RebindExceptionHandlerImpl.builder().strict().build();
public ManagementContext origManagementContext;
@@ -58,6 +59,7 @@ public class RebindOptions {
public static RebindOptions create(RebindOptions options) {
RebindOptions result = create();
result.checkSerializable(options.checkSerializable);
+ result.clearOrigManagementContext(options.clearOrigManagementContext);
result.terminateOrigManagementContext(options.terminateOrigManagementContext);
result.exceptionHandler(options.exceptionHandler);
result.origManagementContext(options.origManagementContext);
@@ -80,6 +82,10 @@ public class RebindOptions {
this.terminateOrigManagementContext = val;
return this;
}
+ public RebindOptions clearOrigManagementContext(boolean val) {
+ this.clearOrigManagementContext = val;
+ return this;
+ }
public RebindOptions defaultExceptionHandler() {
this.exceptionHandler = null;
return this;
diff --git a/core/src/test/java/org/apache/brooklyn/core/mgmt/rebind/RebindTestFixture.java b/core/src/test/java/org/apache/brooklyn/core/mgmt/rebind/RebindTestFixture.java
index b09aba6..b2296e8 100644
--- a/core/src/test/java/org/apache/brooklyn/core/mgmt/rebind/RebindTestFixture.java
+++ b/core/src/test/java/org/apache/brooklyn/core/mgmt/rebind/RebindTestFixture.java
@@ -18,6 +18,7 @@
*/
package org.apache.brooklyn.core.mgmt.rebind;
+import static com.google.common.base.Preconditions.checkNotNull;
import java.io.File;
import java.io.IOException;
import java.util.List;
@@ -38,7 +39,10 @@ import org.apache.brooklyn.core.entity.EntityPredicates;
import org.apache.brooklyn.core.entity.StartableApplication;
import org.apache.brooklyn.core.entity.trait.Startable;
import org.apache.brooklyn.core.internal.BrooklynProperties;
+import org.apache.brooklyn.core.mgmt.ha.HighAvailabilityManagerImpl;
+import org.apache.brooklyn.core.mgmt.internal.BrooklynObjectManagementMode;
import org.apache.brooklyn.core.mgmt.internal.LocalManagementContext;
+import org.apache.brooklyn.core.mgmt.internal.ManagementTransitionMode;
import org.apache.brooklyn.core.mgmt.persist.BrooklynMementoPersisterToObjectStore;
import org.apache.brooklyn.core.mgmt.persist.FileBasedObjectStore;
import org.apache.brooklyn.core.mgmt.persist.PersistMode;
@@ -123,7 +127,7 @@ public abstract class RebindTestFixture<T extends StartableApplication> {
return true;
}
- /** As {@link #createNewManagementContext(File)} using the default memento dir */
+ /** As {@link #createNewManagementContext(File, Map)} using the default memento dir */
protected LocalManagementContext createNewManagementContext() {
return createNewManagementContext(mementoDir, null);
}
diff --git a/core/src/test/java/org/apache/brooklyn/core/mgmt/rebind/RebindTestUtils.java b/core/src/test/java/org/apache/brooklyn/core/mgmt/rebind/RebindTestUtils.java
index 4107735..128d50c 100644
--- a/core/src/test/java/org/apache/brooklyn/core/mgmt/rebind/RebindTestUtils.java
+++ b/core/src/test/java/org/apache/brooklyn/core/mgmt/rebind/RebindTestUtils.java
@@ -45,9 +45,12 @@ import org.apache.brooklyn.api.objs.Identifiable;
import org.apache.brooklyn.core.entity.EntityInternal;
import org.apache.brooklyn.core.internal.BrooklynProperties;
import org.apache.brooklyn.core.location.internal.LocationInternal;
+import org.apache.brooklyn.core.mgmt.ha.HighAvailabilityManagerImpl;
import org.apache.brooklyn.core.mgmt.ha.ManagementPlaneSyncRecordPersisterToObjectStore;
+import org.apache.brooklyn.core.mgmt.internal.BrooklynObjectManagementMode;
import org.apache.brooklyn.core.mgmt.internal.LocalManagementContext;
import org.apache.brooklyn.core.mgmt.internal.ManagementContextInternal;
+import org.apache.brooklyn.core.mgmt.internal.ManagementTransitionMode;
import org.apache.brooklyn.core.mgmt.persist.BrooklynMementoPersisterToObjectStore;
import org.apache.brooklyn.core.mgmt.persist.FileBasedObjectStore;
import org.apache.brooklyn.core.mgmt.persist.PersistMode;
@@ -354,6 +357,7 @@ public class RebindTestUtils {
boolean hasHaPersister = newManagementContext != null && newManagementContext.getHighAvailabilityManager().getPersister() != null;
boolean checkSerializable = options.checkSerializable;
boolean terminateOrigManagementContext = options.terminateOrigManagementContext;
+ boolean clearOrigManagementContext = options.clearOrigManagementContext;
Function<BrooklynMementoPersister, Void> stateTransformer = options.stateTransformer;
LOG.info("Rebinding app, using mementoDir " + mementoDir + "; object store " + objectStore);
@@ -385,11 +389,16 @@ public class RebindTestUtils {
RebindTestUtils.checkCurrentMementoSerializable(origManagementContext);
}
+ if (clearOrigManagementContext) {
+ checkNotNull(origManagementContext, "must supply origManagementContext with terminateOrigManagementContext");
+ ((HighAvailabilityManagerImpl)origManagementContext.getHighAvailabilityManager()).clearManagedItems(
+ ManagementTransitionMode.transitioning(BrooklynObjectManagementMode.MANAGED_PRIMARY, BrooklynObjectManagementMode.UNMANAGED_PERSISTED) );
+ }
if (terminateOrigManagementContext) {
checkNotNull(origManagementContext, "must supply origManagementContext with terminateOrigManagementContext");
origManagementContext.terminate();
}
-
+
if (mementoDirBackup != null) {
FileUtil.copyDir(mementoDir, mementoDirBackup);
FileUtil.setFilePermissionsTo700(mementoDirBackup);
diff --git a/software/winrm/src/test/java/org/apache/brooklyn/feed/windows/WinRmCommandSensorTest.java b/software/winrm/src/test/java/org/apache/brooklyn/feed/windows/WinRmCommandSensorTest.java
index 75e10c4..543570f 100644
--- a/software/winrm/src/test/java/org/apache/brooklyn/feed/windows/WinRmCommandSensorTest.java
+++ b/software/winrm/src/test/java/org/apache/brooklyn/feed/windows/WinRmCommandSensorTest.java
@@ -22,9 +22,11 @@ import org.apache.brooklyn.api.entity.Entity;
import org.apache.brooklyn.api.entity.EntitySpec;
import org.apache.brooklyn.api.location.Location;
import org.apache.brooklyn.api.location.LocationSpec;
+import org.apache.brooklyn.api.mgmt.ha.HighAvailabilityMode;
import org.apache.brooklyn.api.sensor.AttributeSensor;
import org.apache.brooklyn.core.entity.EntityAsserts;
import org.apache.brooklyn.core.entity.RecordingSensorEventListener;
+import org.apache.brooklyn.core.mgmt.rebind.RebindOptions;
import org.apache.brooklyn.core.mgmt.rebind.RebindTestFixtureWithApp;
import org.apache.brooklyn.core.sensor.Sensors;
import org.apache.brooklyn.core.sensor.http.HttpRequestSensor;
@@ -65,6 +67,12 @@ public class WinRmCommandSensorTest extends RebindTestFixtureWithApp {
RecordingWinRmTool.clear();
}
+ @Override
+ protected TestApplication rebind() throws Exception {
+ // trigger demotion to stop threads
+ return rebind(RebindOptions.create().clearOrigManagementContext(true));
+ }
+
@Test
public void testRebind() throws Exception {
RecordingWinRmTool.setCustomResponse(".*mycommand.*", new RecordingWinRmTool.CustomResponse(0, "myval", ""));
@@ -81,9 +89,12 @@ public class WinRmCommandSensorTest extends RebindTestFixtureWithApp {
TestApplication app2 = rebind();
- entity = app2.getChildren().iterator().next();
+ Entity entity2 = app2.getChildren().iterator().next();
RecordingWinRmTool.setCustomResponse(".*mycommand.*", new RecordingWinRmTool.CustomResponse(0, "myval2", ""));
- EntityAsserts.assertAttributeEqualsEventually(entity, Sensors.newStringSensor("mysensor"), "myval2");
+ EntityAsserts.assertAttributeEqualsEventually(entity2, Sensors.newStringSensor("mysensor"), "myval2");
+
+ // entity is not updated
+ EntityAsserts.assertAttributeEquals(entity, Sensors.newStringSensor("mysensor"), "myval");
}
// "Integration" because takes a second
[brooklyn-server] 26/27: suppress more warnings on planned rebind
interruption
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 750a63a9bbcc70b332254fa8b434311449ff3bf4
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 22:30:35 2021 +0100
suppress more warnings on planned rebind interruption
---
.../BrooklynMementoPersisterToObjectStore.java | 17 +++++++----
.../mgmt/rebind/RebindExceptionHandlerImpl.java | 35 ++++++++++++++++++++++
.../brooklyn/util/exceptions/Exceptions.java | 2 +-
.../exceptions/RuntimeInterruptedException.java | 9 ++----
4 files changed, 50 insertions(+), 13 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/persist/BrooklynMementoPersisterToObjectStore.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/persist/BrooklynMementoPersisterToObjectStore.java
index e3b3540..31e2019 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/persist/BrooklynMementoPersisterToObjectStore.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/persist/BrooklynMementoPersisterToObjectStore.java
@@ -568,10 +568,17 @@ public class BrooklynMementoPersisterToObjectStore implements BrooklynMementoPer
@Override
public void run() {
try {
- visitor.visit(type, objectIdAndData.getKey(), objectIdAndData.getValue());
- } catch (Exception e) {
- Exceptions.propagateIfFatal(e);
- exceptionHandler.onLoadMementoFailed(type, "memento "+objectIdAndData.getKey()+" "+phase+" error", e);
+ try {
+ visitor.visit(type, objectIdAndData.getKey(), objectIdAndData.getValue());
+ } catch (Exception e) {
+ Exceptions.propagateIfFatal(e);
+ if (Thread.currentThread().isInterrupted()) {
+ throw new RuntimeInterruptedException("Interruption discovered", e);
+ }
+ exceptionHandler.onLoadMementoFailed(type, "memento " + objectIdAndData.getKey() + " " + phase + " error", e);
+ }
+ } catch (RuntimeInterruptedException e) {
+ LOG.debug("Ending persistence on interruption, probably cancelled when server about to transition: "+e);
}
}
}
@@ -594,7 +601,7 @@ public class BrooklynMementoPersisterToObjectStore implements BrooklynMementoPer
if (future.isDone()) {
try {
future.get();
- } catch (InterruptedException e2) {
+ } catch (InterruptedException|RuntimeInterruptedException e2) {
throw Exceptions.propagate(e2);
} catch (ExecutionException e2) {
LOG.warn("Problem loading memento ("+phase+"): "+e2, e2);
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindExceptionHandlerImpl.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindExceptionHandlerImpl.java
index a705748..30b2984 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindExceptionHandlerImpl.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindExceptionHandlerImpl.java
@@ -44,6 +44,7 @@ import org.apache.brooklyn.util.collections.MutableList;
import org.apache.brooklyn.util.collections.QuorumCheck;
import org.apache.brooklyn.util.collections.QuorumCheck.QuorumChecks;
import org.apache.brooklyn.util.exceptions.Exceptions;
+import org.apache.brooklyn.util.exceptions.RuntimeInterruptedException;
import org.apache.brooklyn.util.text.Strings;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -166,6 +167,10 @@ public class RebindExceptionHandlerImpl implements RebindExceptionHandler {
@Override
public void onLoadMementoFailed(BrooklynObjectType type, String msg, Exception e) {
Exceptions.propagateIfFatal(e);
+ if (Thread.currentThread().isInterrupted()) {
+ throw new RuntimeInterruptedException("Interruption discovered when loading memento, "+msg);
+ }
+
String errmsg = "problem loading memento: "+msg;
switch (type) {
@@ -193,6 +198,9 @@ public class RebindExceptionHandlerImpl implements RebindExceptionHandler {
@Override
public Entity onDanglingEntityRef(String id) {
+ if (Thread.currentThread().isInterrupted()) {
+ throw new RuntimeInterruptedException("Interruption discovered when recording dangling entity "+id);
+ }
missingEntities.add(id);
if (danglingRefFailureMode == RebindManager.RebindFailureMode.FAIL_FAST) {
throw new IllegalStateException("No entity found with id "+id);
@@ -204,6 +212,9 @@ public class RebindExceptionHandlerImpl implements RebindExceptionHandler {
@Override
public Location onDanglingLocationRef(String id) {
+ if (Thread.currentThread().isInterrupted()) {
+ throw new RuntimeInterruptedException("Interruption discovered when recording dangling location "+id);
+ }
missingLocations.add(id);
if (danglingRefFailureMode == RebindManager.RebindFailureMode.FAIL_FAST) {
throw new IllegalStateException("No location found with id "+id);
@@ -215,6 +226,10 @@ public class RebindExceptionHandlerImpl implements RebindExceptionHandler {
@Override
public Policy onDanglingPolicyRef(String id) {
+ if (Thread.currentThread().isInterrupted()) {
+ throw new RuntimeInterruptedException("Interruption discovered when recording dangling policy "+id);
+ }
+
missingPolicies.add(id);
if (danglingRefFailureMode == RebindManager.RebindFailureMode.FAIL_FAST) {
throw new IllegalStateException("No policy found with id "+id);
@@ -226,6 +241,10 @@ public class RebindExceptionHandlerImpl implements RebindExceptionHandler {
@Override
public Enricher onDanglingEnricherRef(String id) {
+ if (Thread.currentThread().isInterrupted()) {
+ throw new RuntimeInterruptedException("Interruption discovered when recording dangling enricher "+id);
+ }
+
missingEnrichers.add(id);
if (danglingRefFailureMode == RebindManager.RebindFailureMode.FAIL_FAST) {
throw new IllegalStateException("No enricher found with id "+id);
@@ -237,6 +256,10 @@ public class RebindExceptionHandlerImpl implements RebindExceptionHandler {
@Override
public Feed onDanglingFeedRef(String id) {
+ if (Thread.currentThread().isInterrupted()) {
+ throw new RuntimeInterruptedException("Interruption discovered when recording dangling feed "+id);
+ }
+
missingFeeds.add(id);
if (danglingRefFailureMode == RebindManager.RebindFailureMode.FAIL_FAST) {
throw new IllegalStateException("No feed found with id "+id);
@@ -248,6 +271,10 @@ public class RebindExceptionHandlerImpl implements RebindExceptionHandler {
@Override
public CatalogItem<?, ?> onDanglingCatalogItemRef(String id) {
+ if (Thread.currentThread().isInterrupted()) {
+ throw new RuntimeInterruptedException("Interruption discovered when recording dangling catalog item "+id);
+ }
+
missingCatalogItems.add(id);
if (danglingRefFailureMode == RebindManager.RebindFailureMode.FAIL_FAST) {
throw new IllegalStateException("No catalog item found with id "+id);
@@ -264,6 +291,10 @@ public class RebindExceptionHandlerImpl implements RebindExceptionHandler {
@Override
public BrooklynObject onDanglingUntypedItemRef(String id) {
+ if (Thread.currentThread().isInterrupted()) {
+ throw new RuntimeInterruptedException("Interruption discovered when recording dangling untyped item "+id);
+ }
+
missingUntypedItems.add(id);
if (danglingRefFailureMode == RebindManager.RebindFailureMode.FAIL_FAST) {
throw new IllegalStateException("No item found with id "+id);
@@ -413,6 +444,10 @@ public class RebindExceptionHandlerImpl implements RebindExceptionHandler {
}
protected void onErrorImpl(String errmsg, Exception e) {
+ if (Thread.currentThread().isInterrupted()) {
+ throw new RuntimeInterruptedException("Interruption discovered when recording error: "+errmsg);
+ }
+
if (rebindFailureMode == RebindManager.RebindFailureMode.FAIL_FAST) {
throw new IllegalStateException("Rebind: aborting due to "+errmsg, e);
} else {
diff --git a/utils/common/src/main/java/org/apache/brooklyn/util/exceptions/Exceptions.java b/utils/common/src/main/java/org/apache/brooklyn/util/exceptions/Exceptions.java
index 4d3946c..b91279b 100644
--- a/utils/common/src/main/java/org/apache/brooklyn/util/exceptions/Exceptions.java
+++ b/utils/common/src/main/java/org/apache/brooklyn/util/exceptions/Exceptions.java
@@ -186,7 +186,7 @@ public class Exceptions {
*/
public static void propagateIfInterrupt(Throwable throwable) {
if (throwable instanceof InterruptedException) {
- throw new RuntimeInterruptedException((InterruptedException) throwable);
+ throw new RuntimeInterruptedException(throwable);
} else if (throwable instanceof RuntimeInterruptedException) {
Thread.currentThread().interrupt();
throw (RuntimeInterruptedException) throwable;
diff --git a/utils/common/src/main/java/org/apache/brooklyn/util/exceptions/RuntimeInterruptedException.java b/utils/common/src/main/java/org/apache/brooklyn/util/exceptions/RuntimeInterruptedException.java
index f207001..600d830 100644
--- a/utils/common/src/main/java/org/apache/brooklyn/util/exceptions/RuntimeInterruptedException.java
+++ b/utils/common/src/main/java/org/apache/brooklyn/util/exceptions/RuntimeInterruptedException.java
@@ -38,17 +38,12 @@ public class RuntimeInterruptedException extends RuntimeException {
Thread.currentThread().interrupt();
}
- public RuntimeInterruptedException(InterruptedException cause) {
+ public RuntimeInterruptedException(Throwable cause) {
super(cause);
Thread.currentThread().interrupt();
}
- public RuntimeInterruptedException(String msg, InterruptedException cause) {
- super(msg, cause);
- Thread.currentThread().interrupt();
- }
-
- public RuntimeInterruptedException(String msg, RuntimeInterruptedException cause) {
+ public RuntimeInterruptedException(String msg, Throwable cause) {
super(msg, cause);
Thread.currentThread().interrupt();
}
[brooklyn-server] 04/27: tidy task before/after model,
ensure scheduled tasks clear their task context
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 85d965153a491789f1367d3265d690f8e479422c
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Mon Sep 13 17:08:04 2021 +0100
tidy task before/after model, ensure scheduled tasks clear their task context
---
.../util/core/task/BasicExecutionContext.java | 50 +++--
.../util/core/task/BasicExecutionManager.java | 196 ++++++++++------
.../apache/brooklyn/util/core/task/BasicTask.java | 3 +-
.../brooklyn/util/core/task/ScheduledTask.java | 2 +
.../util/core/task/ScheduledExecutionTest.java | 248 +++++++++++++++------
5 files changed, 338 insertions(+), 161 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionContext.java b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionContext.java
index ca9e158..70322ce 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionContext.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionContext.java
@@ -361,33 +361,41 @@ public class BasicExecutionContext extends AbstractExecutionContext {
taskTags.addAll(tags);
- if (Tasks.current()!=null && BrooklynTaskTags.isTransient(Tasks.current())
+ if (Tasks.current()!=null && BrooklynTaskTags.isTransient(Tasks.current())
&& !taskTags.contains(BrooklynTaskTags.NON_TRANSIENT_TASK_TAG) && !taskTags.contains(BrooklynTaskTags.TRANSIENT_TASK_TAG)) {
// tag as transient if submitter is transient, unless explicitly tagged as non-transient
taskTags.add(BrooklynTaskTags.TRANSIENT_TASK_TAG);
}
- final Object startCallback = properties.get("newTaskStartCallback");
- properties.put("newTaskStartCallback", new Function<Task<?>,Void>() {
- @Override
- public Void apply(Task<?> it) {
- registerPerThreadExecutionContext();
- if (startCallback!=null) BasicExecutionManager.invokeCallback(startCallback, it);
- return null;
- }});
-
- final Object endCallback = properties.get("newTaskEndCallback");
- properties.put("newTaskEndCallback", new Function<Task<?>,Void>() {
- @Override
- public Void apply(Task<?> it) {
- try {
- if (endCallback!=null) BasicExecutionManager.invokeCallback(endCallback, it);
- } finally {
- clearPerThreadExecutionContext();
+ if (task instanceof ScheduledTask) {
+ // not run for scheduler
+ ((ScheduledTask)task).executionContext = this;
+
+ } else {
+ final Object startCallback = properties.get("newTaskStartCallback");
+ properties.put("newTaskStartCallback", new Function<Task<?>, Void>() {
+ @Override
+ public Void apply(Task<?> it) {
+ registerPerThreadExecutionContext();
+ if (startCallback != null) BasicExecutionManager.invokeCallback(startCallback, it);
+ return null;
}
- return null;
- }});
-
+ });
+
+ final Object endCallback = properties.get("newTaskEndCallback");
+ properties.put("newTaskEndCallback", new Function<Task<?>, Void>() {
+ @Override
+ public Void apply(Task<?> it) {
+ try {
+ if (endCallback != null) BasicExecutionManager.invokeCallback(endCallback, it);
+ } finally {
+ clearPerThreadExecutionContext();
+ }
+ return null;
+ }
+ });
+ }
+
if (task instanceof Task) {
return executionManager.submit(properties, (Task)task);
} else if (task instanceof Callable) {
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
index 29984e2..f1496fc 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
@@ -50,6 +50,7 @@ import java.util.concurrent.atomic.AtomicLong;
import java.util.stream.Collectors;
import org.apache.brooklyn.api.entity.Entity;
import org.apache.brooklyn.api.internal.BrooklynLoggingCategories;
+import org.apache.brooklyn.api.mgmt.ExecutionContext;
import org.apache.brooklyn.api.mgmt.ExecutionManager;
import org.apache.brooklyn.api.mgmt.HasTaskChildren;
import org.apache.brooklyn.api.mgmt.Task;
@@ -69,6 +70,7 @@ import org.apache.brooklyn.util.core.task.TaskInternal.TaskCancellationMode;
import org.apache.brooklyn.util.exceptions.Exceptions;
import org.apache.brooklyn.util.exceptions.RuntimeInterruptedException;
import org.apache.brooklyn.util.guava.Maybe;
+import org.apache.brooklyn.util.javalang.JavaClassNames;
import org.apache.brooklyn.util.text.Identifiers;
import org.apache.brooklyn.util.text.Strings;
import org.apache.brooklyn.util.time.CountdownTimer;
@@ -106,6 +108,8 @@ public class BasicExecutionManager implements ExecutionManager {
public static final String LOGGING_MDC_KEY_ENTITY_IDS = "entity.ids";
public static final String LOGGING_MDC_KEY_TASK_ID = "task.id";
+ private static final boolean SCHEDULED_TASKS_COUNT_AS_ACTIVE = false;
+
private boolean jitterThreads = BrooklynFeatureEnablement.isEnabled(BrooklynFeatureEnablement.FEATURE_JITTER_THREADS);
private int jitterThreadsMaxDelay = Integer.getInteger(JITTER_THREADS_MAX_DELAY_PROPERTY, 200);
@@ -525,10 +529,13 @@ public class BasicExecutionManager implements ExecutionManager {
}
protected Task<?> submitNewScheduledTask(final Map<?,?> flags, final ScheduledTask task) {
- beforeSubmitScheduledTaskAllIterations(flags, task);
-
- if (!submitSubsequentScheduledTask(flags, task)) {
- afterEndScheduledTaskAllIterations(flags, task, null);
+ boolean result = false;
+ try {
+ result = submitSubsequentScheduledTask(flags, task);
+ } finally {
+ if (!result) {
+ afterEndScheduledTaskAllIterations(flags, task, null);
+ }
}
return task;
}
@@ -555,27 +562,39 @@ public class BasicExecutionManager implements ExecutionManager {
@Override
@SuppressWarnings({ "rawtypes", "unchecked" })
public Object call() {
- if (task.startTimeUtc==-1) {
- // this is overwritten on each run; not sure if that's best or not
- task.startTimeUtc = System.currentTimeMillis();
- }
- TaskInternal<?> taskScheduled = null;
+ TaskInternal<?> taskIteration = null;
Throwable error = null;
try {
- taskScheduled = (TaskInternal<?>) task.newTask();
- taskScheduled.setSubmittedByTask(task);
- beforeStartScheduledTaskSubmissionIteration(flags, task, taskScheduled);
- final Callable<?> oldJob = taskScheduled.getJob();
- final TaskInternal<?> taskScheduledF = taskScheduled;
- taskScheduled.setJob(new Callable() { @Override public Object call() {
+ if (task.startTimeUtc==-1) {
+ beforeSubmitScheduledTaskAllIterations(flags, task);
+ beforeStartScheduledTaskAllIterations(flags, task);
+
+ task.startTimeUtc = System.currentTimeMillis();
+ }
+
+ taskIteration = (TaskInternal<?>) task.newTask();
+ taskIteration.setSubmittedByTask(task);
+
+ beforeSubmitScheduledTaskSubmissionIteration(flags, task);
+
+ final Callable<?> oldJob = taskIteration.getJob();
+ final TaskInternal<?> taskIterationF = taskIteration;
+ taskIteration.setJob(new Callable() { @Override public Object call() {
if (task.isCancelled()) {
- afterEndScheduledTaskAllIterations(flags, task, new CancellationException("cancel detected"));
- throw new CancellationException("cancel detected"); // above throws, but for good measure
+ CancellationException cancelDetected = new CancellationException("cancel detected");
+ try {
+ afterEndScheduledTaskSubmissionIteration(flags, task, taskIterationF, cancelDetected);
+ } finally {
+ // do in finally block so runs even if above throws cancelDetected
+ afterEndScheduledTaskAllIterations(flags, task, cancelDetected);
+ }
+ throw cancelDetected;
}
Throwable lastError = null;
boolean shouldResubmit = true;
- task.recentRun = taskScheduledF;
- try (BrooklynTaskLoggingMdc mdc = BrooklynTaskLoggingMdc.create(task).start()) {
+ task.recentRun = taskIterationF;
+ try (BrooklynTaskLoggingMdc mdc = BrooklynTaskLoggingMdc.create(taskIterationF).start()) {
+ beforeStartScheduledTaskSubmissionIteration(flags, task, taskIterationF);
synchronized (task) {
task.notifyAll();
}
@@ -590,26 +609,33 @@ public class BasicExecutionManager implements ExecutionManager {
}
return result;
} finally {
- // do in finally block in case we were interrupted
- if (shouldResubmit && resubmit()) {
- // resubmitted fine, no-op
- } else {
- // not resubmitted, note ending
- afterEndScheduledTaskAllIterations(flags, task, lastError);
+ if (!task.isCancelled() || task.getEndTimeUtc()<=0) {
+ // don't re-run on cancellation
+
+ afterEndScheduledTaskSubmissionIteration(flags, task, taskIterationF, lastError);
+ // do in finally block in case we were interrupted
+ if (shouldResubmit && resubmit()) {
+ // resubmitted fine, no-op
+ } else {
+ // not resubmitted, note ending
+ afterEndScheduledTaskAllIterations(flags, task, lastError);
+ }
}
}
}});
- task.nextRun = taskScheduled;
- BasicExecutionContext ec = BasicExecutionContext.getCurrentExecutionContext();
- if (ec!=null) return ec.submit(taskScheduled);
- else return submit(taskScheduled);
+ task.nextRun = taskIteration;
+ ExecutionContext ec =
+ // no longer associated the execution context on each execution;
+// BasicExecutionContext.getCurrentExecutionContext();
+ // instead it is set on the task
+ task.executionContext;
+ if (ec!=null) return ec.submit(taskIteration);
+ else return submit(taskIteration);
} catch (Exception e) {
error = e;
+ afterEndScheduledTaskSubmissionIteration(flags, task, taskIteration, error);
throw Exceptions.propagate(e);
-
- } finally {
- afterEndScheduledTaskSubmissionIteration(flags, task, taskScheduled, error);
}
}
@@ -804,8 +830,9 @@ public class BasicExecutionManager implements ExecutionManager {
}
}
- if (task instanceof ScheduledTask)
- return (Task<T>) submitNewScheduledTask(flags, (ScheduledTask)task);
+ if (task instanceof ScheduledTask) {
+ return (Task<T>) submitNewScheduledTask(flags, (ScheduledTask) task);
+ }
beforeSubmitAtomicTask(flags, task);
@@ -850,6 +877,11 @@ public class BasicExecutionManager implements ExecutionManager {
}
protected void beforeSubmitScheduledTaskAllIterations(Map<?,?> flags, Task<?> task) {
+ // for these, beforeSubmitAtomicTask is not called,
+ // but beforeStartAtomic and afterSubmitAtomic _are_ called
+ internalBeforeSubmit(flags, task);
+ }
+ protected void beforeSubmitScheduledTaskSubmissionIteration(Map<?,?> flags, Task<?> task) {
internalBeforeSubmit(flags, task);
}
protected void beforeSubmitAtomicTask(Map<?,?> flags, Task<?> task) {
@@ -916,20 +948,25 @@ public class BasicExecutionManager implements ExecutionManager {
}
}
- protected void beforeStartScheduledTaskSubmissionIteration(Map<?,?> flags, Task<?> taskRepeatedlyScheduling, Task<?> taskIteration) {
- internalBeforeStart(flags, taskRepeatedlyScheduling, true);
+ /** normally (if not interrupted) called once for each call to {@link #beforeSubmitScheduledTaskAllIterations(Map, Task)} */
+ protected void beforeStartScheduledTaskAllIterations(Map<?,?> flags, Task<?> taskDoingTheInitialSchedule) {
+ internalBeforeStart(flags, taskDoingTheInitialSchedule, !SCHEDULED_TASKS_COUNT_AS_ACTIVE, true, true);
+ }
+ protected void beforeStartScheduledTaskSubmissionIteration(Map<?,?> flags, Task<?> taskDoingTheScheduling, Task<?> taskIteration) {
+ // no-op, because handled as an atomic task
+ // internalBeforeStart(flags, taskIteration, true, false);
}
protected void beforeStartAtomicTask(Map<?,?> flags, Task<?> task) {
- internalBeforeStart(flags, task, true);
+ internalBeforeStart(flags, task, false, true, false);
}
protected void beforeStartInSameThreadTask(Map<?,?> flags, Task<?> task) {
- internalBeforeStart(flags, task, false);
+ internalBeforeStart(flags, task, false, false, false);
}
/** invoked in a task's thread when a task is starting to run (may be some time after submitted),
* but before doing any of the task's work, so that we can update bookkeeping and notify callbacks */
- protected void internalBeforeStart(Map<?,?> flags, Task<?> task, boolean allowJitter) {
- int count = activeTaskCount.incrementAndGet();
+ protected void internalBeforeStart(Map<?,?> flags, Task<?> task, boolean skipIncrementCounter, boolean allowJitter, boolean startingThisThreadMightEndElsewhere) {
+ int count = skipIncrementCounter ? activeTaskCount.get() : activeTaskCount.incrementAndGet();
if (count % 1000==0) {
log.warn("High number of active tasks: task #"+count+" is "+task);
}
@@ -939,19 +976,21 @@ public class BasicExecutionManager implements ExecutionManager {
if (!task.isCancelled()) {
Thread thread = Thread.currentThread();
((TaskInternal<?>)task).setThread(thread);
- if (RENAME_THREADS) {
- threadOriginalName.set(thread.getName());
- String newThreadName = "brooklyn-" + CaseFormat.LOWER_HYPHEN.to(CaseFormat.LOWER_CAMEL, task.getDisplayName().replace(" ", "")) + "-" + task.getId().substring(0, 8);
- thread.setName(newThreadName);
+ if (!startingThisThreadMightEndElsewhere) {
+ if (RENAME_THREADS) {
+ threadOriginalName.set(thread.getName());
+ String newThreadName = "brooklyn-" + CaseFormat.LOWER_HYPHEN.to(CaseFormat.LOWER_CAMEL, task.getDisplayName().replace(" ", "")) + "-" + task.getId().substring(0, 8);
+ thread.setName(newThreadName);
+ }
+ PerThreadCurrentTaskHolder.perThreadCurrentTask.set(task);
}
- PerThreadCurrentTaskHolder.perThreadCurrentTask.set(task);
((TaskInternal<?>)task).setStartTimeUtc(System.currentTimeMillis());
}
if (allowJitter) {
jitterThreadStart(task);
}
- if (flags!=null) {
+ if (flags!=null && !startingThisThreadMightEndElsewhere) {
invokeCallback(flags.get("newTaskStartCallback"), task);
}
}
@@ -992,23 +1031,33 @@ public class BasicExecutionManager implements ExecutionManager {
}
private static boolean loggedClosureDeprecatedInInvokeCallback;
- /** normally (if not interrupted) called once for each call to {@link #beforeSubmitScheduledTaskAllIterations(Map, Task)} */
- protected void afterEndScheduledTaskAllIterations(Map<?,?> flags, Task<?> taskRepeatedlyScheduling, Throwable error) {
- internalAfterEnd(flags, taskRepeatedlyScheduling, false, true, error);
+ /** normally (if not interrupted) called once for each call to {@link #beforeStartScheduledTaskAllIterations(Map, Task)} */
+ protected void afterEndScheduledTaskAllIterations(Map<?,?> flags, Task<?> taskDoingTheInitialSchedule, Throwable error) {
+ boolean taskWasSubmittedAndNotYetEnded = true;
+ try {
+ taskWasSubmittedAndNotYetEnded = internalAfterEnd(flags, taskDoingTheInitialSchedule, !SCHEDULED_TASKS_COUNT_AS_ACTIVE, false, error);
+ } finally {
+ synchronized (taskDoingTheInitialSchedule) { taskDoingTheInitialSchedule.notifyAll(); }
+ if (taskWasSubmittedAndNotYetEnded) {
+ // prevent from running twice on cancellation after start
+ ((TaskInternal<?>) taskDoingTheInitialSchedule).runListeners();
+ }
+ }
}
/** called once for each call to {@link #beforeStartScheduledTaskSubmissionIteration(Map, Task, Task)},
* with a per-iteration task generated by the surrounding scheduled task */
- protected void afterEndScheduledTaskSubmissionIteration(Map<?,?> flags, Task<?> taskRepeatedlyScheduling, Task<?> taskIteration, Throwable error) {
- internalAfterEnd(flags, taskRepeatedlyScheduling, true, false, error);
+ protected void afterEndScheduledTaskSubmissionIteration(Map<?,?> flags, Task<?> taskDoingTheInitialSchedule, Task<?> taskIteration, Throwable error) {
+ // no-op because handled as an atomic task
+ // internalAfterEnd(flags, taskIteration, false, true, error);
}
/** called once for each task on which {@link #beforeStartAtomicTask(Map, Task)} is invoked,
* and normally (if not interrupted prior to start)
* called once for each task on which {@link #beforeSubmitAtomicTask(Map, Task)} */
protected void afterEndAtomicTask(Map<?,?> flags, Task<?> task, Throwable error) {
- internalAfterEnd(flags, task, true, true, error);
+ internalAfterEnd(flags, task, false, true, error);
}
protected void afterEndInSameThreadTask(Map<?,?> flags, Task<?> task, Throwable error) {
- internalAfterEnd(flags, task, true, true, error);
+ internalAfterEnd(flags, task, false, true, error);
}
protected void afterEndForCancelBeforeStart(Map<?,?> flags, Task<?> task, boolean calledFromCanceller) {
if (calledFromCanceller) {
@@ -1025,28 +1074,37 @@ public class BasicExecutionManager implements ExecutionManager {
// to ensure listeners and callback only invoked once
}
}
- internalAfterEnd(flags, task, !calledFromCanceller, true, null);
+ internalAfterEnd(flags, task, true, !calledFromCanceller, null);
}
/** normally (if not interrupted) called once for each call to {@link #internalBeforeSubmit(Map, Task)},
* and, for atomic tasks and scheduled-task submission iterations where
- * always called once if {@link #internalBeforeStart(Map, Task, boolean)} is invoked and in the same thread as that method */
- protected void internalAfterEnd(Map<?,?> flags, Task<?> task, boolean startedInThisThread, boolean isEndingAllIterations, Throwable error) {
+ * always called once if {@link #internalBeforeStart(Map, Task, boolean, boolean, boolean)} is invoked and if possible
+ * (but not possible for isEndingAllIterations) in the same thread as that method */
+ protected boolean internalAfterEnd(Map<?,?> flags, Task<?> task, boolean skipDecrementCounter, boolean startedGuaranteedToEndInSameThreadAndEndingSameThread, Throwable error) {
boolean taskWasSubmittedAndNotYetEnded = true;
try {
if (log.isTraceEnabled()) log.trace(this+" afterEnd, task: "+task);
- if (startedInThisThread) {
+ taskWasSubmittedAndNotYetEnded = incompleteTaskIds.remove(task.getId());
+ // this method might be called more than once, eg if cancelled, so use the above as a guard where single invocation is needed (eg counts)
+
+ if (!skipDecrementCounter && taskWasSubmittedAndNotYetEnded) {
activeTaskCount.decrementAndGet();
}
- if (isEndingAllIterations) {
- taskWasSubmittedAndNotYetEnded = incompleteTaskIds.remove(task.getId());
- if (flags!=null && taskWasSubmittedAndNotYetEnded) {
- invokeCallback(flags.get("newTaskEndCallback"), task);
+
+ if (flags!=null && taskWasSubmittedAndNotYetEnded && startedGuaranteedToEndInSameThreadAndEndingSameThread) {
+ invokeCallback(flags.get("newTaskEndCallback"), task);
+ }
+ if (task.getEndTimeUtc()>0) {
+ if (taskWasSubmittedAndNotYetEnded) {
+ // shouldn't happen
+ log.debug("Task "+task+" has end time "+task.getEndTimeUtc()+" but was marked as incomplete");
}
- ((TaskInternal<?>)task).setEndTimeUtc(System.currentTimeMillis());
+ } else {
+ ((TaskInternal<?>) task).setEndTimeUtc(System.currentTimeMillis());
}
-
- if (startedInThisThread) {
+
+ if (startedGuaranteedToEndInSameThreadAndEndingSameThread) {
PerThreadCurrentTaskHolder.perThreadCurrentTask.remove();
//clear thread _after_ endTime set, so we won't get a null thread when there is no end-time
if (RENAME_THREADS) {
@@ -1058,15 +1116,18 @@ public class BasicExecutionManager implements ExecutionManager {
threadOriginalName.remove();
}
}
- ((TaskInternal<?>)task).setThread(null);
}
+ ((TaskInternal<?>)task).setThread(null);
+
} finally {
try {
if (error!=null) {
/* we throw, after logging debug.
* the throw means the error is available for task submitters to monitor.
* however it is possible no one is monitoring it, in which case we will have debug logging only for errors.
- * (the alternative, of warn-level logging in lots of places where we don't want it, seems worse!)
+ * (the alternative, of warn-level logging in lots of places where we don't want it, seems worse!)
+ *
+ * Note in particular that scheduled tasks will typically swallow this and simply re-submit
*/
if (log.isDebugEnabled()) {
// debug only here, because most submitters will handle failures
@@ -1088,12 +1149,13 @@ public class BasicExecutionManager implements ExecutionManager {
}
} finally {
synchronized (task) { task.notifyAll(); }
- if (isEndingAllIterations && taskWasSubmittedAndNotYetEnded) {
+ if (taskWasSubmittedAndNotYetEnded) {
// prevent from running twice on cancellation after start
((TaskInternal<?>)task).runListeners();
}
}
}
+ return taskWasSubmittedAndNotYetEnded;
}
public TaskScheduler getTaskSchedulerForTag(Object tag) {
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicTask.java b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicTask.java
index b5304e1..a6fe8d0 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicTask.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicTask.java
@@ -553,8 +553,7 @@ public class BasicTask<T> implements TaskInternal<T> {
else if (!isCancelled() && startTimeUtc <= 0) {
rv = "Submitted for execution";
if (verbosity>0) {
- long elapsed = System.currentTimeMillis() - submitTimeUtc;
- rv += " "+Time.makeTimeStringRoundedSince(elapsed)+" ago";
+ rv += " "+Time.makeTimeStringRoundedSince(submitTimeUtc)+" ago";
}
if (verbosity >= 2 && getExtraStatusText()!=null) {
rv += "\n\n"+getExtraStatusText();
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/ScheduledTask.java b/core/src/main/java/org/apache/brooklyn/util/core/task/ScheduledTask.java
index beabe30..8851971 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/ScheduledTask.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/ScheduledTask.java
@@ -18,6 +18,7 @@
*/
package org.apache.brooklyn.util.core.task;
+import org.apache.brooklyn.api.mgmt.ExecutionContext;
import static org.apache.brooklyn.util.JavaGroovyEquivalents.elvis;
import static org.apache.brooklyn.util.JavaGroovyEquivalents.groovyTruth;
@@ -73,6 +74,7 @@ public class ScheduledTask extends BasicTask<Object> {
*/
protected boolean cancelOnException = true;
+ protected ExecutionContext executionContext;
protected int runCount=0;
protected Task<?> recentRun, nextRun;
Class<? extends Exception> lastThrownType;
diff --git a/core/src/test/java/org/apache/brooklyn/util/core/task/ScheduledExecutionTest.java b/core/src/test/java/org/apache/brooklyn/util/core/task/ScheduledExecutionTest.java
index 166da59..fea7234 100644
--- a/core/src/test/java/org/apache/brooklyn/util/core/task/ScheduledExecutionTest.java
+++ b/core/src/test/java/org/apache/brooklyn/util/core/task/ScheduledExecutionTest.java
@@ -18,6 +18,10 @@
*/
package org.apache.brooklyn.util.core.task;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.core.mgmt.BrooklynTaskTags;
+import org.apache.brooklyn.core.test.entity.TestEntityImpl;
+import org.apache.brooklyn.util.collections.MutableList;
import static org.testng.Assert.assertEquals;
import static org.testng.Assert.assertTrue;
import static org.testng.Assert.fail;
@@ -49,36 +53,37 @@ import com.google.common.collect.Lists;
public class ScheduledExecutionTest {
public static final Logger log = LoggerFactory.getLogger(ScheduledExecutionTest.class);
-
+
@Test
public void testScheduledTask() throws Exception {
Duration PERIOD = Duration.millis(20);
BasicExecutionManager m = new BasicExecutionManager("mycontextid");
final AtomicInteger i = new AtomicInteger(0);
ScheduledTask t = ScheduledTask.builder(() -> new BasicTask<Integer>(() -> {
- log.info("task running: "+Tasks.current()+" "+Tasks.current().getStatusDetail(false));
- return i.incrementAndGet();
- }))
+ log.info("task running: " + Tasks.current() + " " + Tasks.current().getStatusDetail(false));
+ return i.incrementAndGet();
+ }))
.displayName("test-1")
.delay(PERIOD.multiply(2))
.period(PERIOD)
.maxIterations(5)
.build();
-
+
log.info("submitting {} {}", t, t.getStatusDetail(false));
m.submit(t);
log.info("submitted {} {}", t, t.getStatusDetail(false));
Integer interimResult = (Integer) t.get();
- log.info("done one ({}) {} {}", new Object[] {interimResult, t, t.getStatusDetail(false)});
- assertTrue(i.get() > 0, "i="+i);
+ log.info("done one ({}) {} {}", new Object[]{interimResult, t, t.getStatusDetail(false)});
+ assertTrue(i.get() > 0, "i=" + i);
t.blockUntilEnded();
Integer finalResult = (Integer) t.get();
- log.info("ended ({}) {} {}", new Object[] {finalResult, t, t.getStatusDetail(false)});
- assertEquals(finalResult, (Integer)5);
+ log.info("ended ({}) {} {}", new Object[]{finalResult, t, t.getStatusDetail(false)});
+ assertEquals(finalResult, (Integer) 5);
assertEquals(i.get(), 5);
+ Asserts.eventually(m::getNumActiveTasks, l -> l==0);
}
- @Test
+ @Test(groups="Integration")
public void testScheduledTaskCancelledIfExceptionThrown() throws Exception {
BasicExecutionManager m = new BasicExecutionManager("mycontextid");
final AtomicInteger calls = new AtomicInteger(0);
@@ -90,12 +95,15 @@ public class ScheduledExecutionTest {
public Integer call() {
calls.incrementAndGet();
throw new RuntimeException("boo");
- }});
- }});
+ }
+ });
+ }
+ });
m.submit(t);
Runnable callsIsOne = new Runnable() {
- @Override public void run() {
+ @Override
+ public void run() {
if (calls.get() != 1) {
throw new RuntimeException("not yet");
}
@@ -104,6 +112,7 @@ public class ScheduledExecutionTest {
};
Asserts.succeedsEventually(callsIsOne);
Asserts.succeedsContinually(callsIsOne);
+ Asserts.eventually(m::getNumActiveTasks, l -> l==0);
}
@Test
@@ -118,44 +127,52 @@ public class ScheduledExecutionTest {
public Integer call() {
calls.incrementAndGet();
throw new RuntimeException("boo");
- }});
- }});
+ }
+ });
+ }
+ });
m.submit(t);
t.blockUntilEnded();
assertEquals(calls.get(), 5, "Expected task to be resubmitted despite throwing an exception");
+ Asserts.eventually(m::getNumActiveTasks, l -> l==0);
}
- /** like testScheduledTask but the loop is terminated by the task itself adjusting the period */
+ /**
+ * like testScheduledTask but the loop is terminated by the task itself adjusting the period
+ */
@Test
public void testScheduledTaskSelfEnding() throws Exception {
int PERIOD = 20;
BasicExecutionManager m = new BasicExecutionManager("mycontextid");
final AtomicInteger i = new AtomicInteger(0);
- ScheduledTask t = new ScheduledTask(MutableMap.of("delay", 2*PERIOD, "period", PERIOD), new Callable<Task<?>>() {
+ ScheduledTask t = new ScheduledTask(MutableMap.of("delay", 2 * PERIOD, "period", PERIOD), new Callable<Task<?>>() {
@Override
public Task<?> call() throws Exception {
return new BasicTask<Integer>(new Callable<Integer>() {
@Override
public Integer call() {
- ScheduledTask submitter = (ScheduledTask) ((BasicTask)Tasks.current()).getSubmittedByTask();
+ ScheduledTask submitter = (ScheduledTask) ((BasicTask) Tasks.current()).getSubmittedByTask();
if (i.get() >= 4) submitter.period = null;
- log.info("task running ("+i+"): "+Tasks.current()+" "+Tasks.current().getStatusDetail(false));
+ log.info("task running (" + i + "): " + Tasks.current() + " " + Tasks.current().getStatusDetail(false));
return i.incrementAndGet();
- }});
- }});
-
+ }
+ });
+ }
+ });
+
log.info("submitting {} {}", t, t.getStatusDetail(false));
m.submit(t);
log.info("submitted {} {}", t, t.getStatusDetail(false));
Integer interimResult = (Integer) t.get();
- log.info("done one ({}) {} {}", new Object[] {interimResult, t, t.getStatusDetail(false)});
+ log.info("done one ({}) {} {}", new Object[]{interimResult, t, t.getStatusDetail(false)});
assertTrue(i.get() > 0);
t.blockUntilEnded();
Integer finalResult = (Integer) t.get();
- log.info("ended ({}) {} {}", new Object[] {finalResult, t, t.getStatusDetail(false)});
- assertEquals(finalResult, (Integer)5);
+ log.info("ended ({}) {} {}", new Object[]{finalResult, t, t.getStatusDetail(false)});
+ assertEquals(finalResult, (Integer) 5);
assertEquals(i.get(), 5);
+ Asserts.eventually(m::getNumActiveTasks, l -> l==0);
}
@Test
@@ -169,35 +186,38 @@ public class ScheduledExecutionTest {
return new BasicTask<Integer>(new Callable<Integer>() {
@Override
public Integer call() {
- log.info("task running ("+i+"): "+Tasks.current()+" "+Tasks.current().getStatusDetail(false));
- ScheduledTask submitter = (ScheduledTask) ((BasicTask)Tasks.current()).getSubmittedByTask();
+ log.info("task running (" + i + "): " + Tasks.current() + " " + Tasks.current().getStatusDetail(false));
+ ScheduledTask submitter = (ScheduledTask) ((BasicTask) Tasks.current()).getSubmittedByTask();
i.incrementAndGet();
if (i.get() >= 5) submitter.cancel();
return i.get();
- }});
- }});
-
- log.info(JavaClassNames.niceClassAndMethod()+" - submitting {} {}", t, t.getStatusDetail(false));
+ }
+ });
+ }
+ });
+
+ log.info(JavaClassNames.niceClassAndMethod() + " - submitting {} {}", t, t.getStatusDetail(false));
m.submit(t);
log.info("submitted {} {}", t, t.getStatusDetail(false));
Integer interimResult = (Integer) t.get();
- log.info("done one ({}) {} {}", new Object[] {interimResult, t, t.getStatusDetail(false)});
+ log.info("done one ({}) {} {}", new Object[]{interimResult, t, t.getStatusDetail(false)});
assertTrue(i.get() > 0);
t.blockUntilEnded();
// int finalResult = t.get()
- log.info("ended ({}) {} {}", new Object[] {i, t, t.getStatusDetail(false)});
+ log.info("ended ({}) {} {}", new Object[]{i, t, t.getStatusDetail(false)});
// assertEquals(finalResult, 5)
assertEquals(i.get(), 5);
+ Asserts.eventually(m::getNumActiveTasks, l -> l==0);
}
- @Test(groups="Integration")
+ @Test(groups = "Integration")
public void testScheduledTaskCancelOuter() throws Exception {
final Duration PERIOD = Duration.millis(20);
final Duration CYCLE_DELAY = Duration.ONE_SECOND;
// this should be enough to start the next cycle, but not so much that the cycle ends;
// and enough that when a task is interrupted it terminates within this period
final Duration SMALL_FRACTION_OF_CYCLE_DELAY = PERIOD.add(CYCLE_DELAY.multiply(0.1));
-
+
BasicExecutionManager m = new BasicExecutionManager("mycontextid");
final AtomicInteger i = new AtomicInteger();
ScheduledTask t = new ScheduledTask(MutableMap.of("delay", PERIOD.times(2), "period", PERIOD), new Callable<Task<?>>() {
@@ -206,48 +226,51 @@ public class ScheduledExecutionTest {
return new BasicTask<Integer>(new Callable<Integer>() {
@Override
public Integer call() {
- log.info("task running ("+i+"): "+Tasks.current()+" "+Tasks.current().getStatusDetail(false));
+ log.info("task running (" + i + "): " + Tasks.current() + " " + Tasks.current().getStatusDetail(false));
Time.sleep(CYCLE_DELAY);
i.incrementAndGet();
return i.get();
- }});
- }});
-
- log.info(JavaClassNames.niceClassAndMethod()+" - submitting {} {}", t, t.getStatusDetail(false));
+ }
+ });
+ }
+ });
+
+ log.info(JavaClassNames.niceClassAndMethod() + " - submitting {} {}", t, t.getStatusDetail(false));
m.submit(t);
log.info("submitted {} {}", t, t.getStatusDetail(false));
Integer interimResult = (Integer) t.get();
- log.info("done one ({}) {} {}", new Object[] {interimResult, t, t.getStatusDetail(false)});
+ log.info("done one ({}) {} {}", new Object[]{interimResult, t, t.getStatusDetail(false)});
assertEquals(i.get(), 1);
-
+
Time.sleep(SMALL_FRACTION_OF_CYCLE_DELAY);
assertEquals(t.get(), 2);
-
+
Time.sleep(SMALL_FRACTION_OF_CYCLE_DELAY);
Stopwatch timer = Stopwatch.createUnstarted();
t.cancel(true);
t.blockUntilEnded();
// int finalResult = t.get()
- log.info("blocked until ended ({}) {} {}, in {}", new Object[] {i, t, t.getStatusDetail(false), Duration.of(timer)});
+ log.info("blocked until ended ({}) {} {}, in {}", new Object[]{i, t, t.getStatusDetail(false), Duration.of(timer)});
try {
t.get();
- Assert.fail("Should have failed getting result of cancelled "+t);
+ Assert.fail("Should have failed getting result of cancelled " + t);
} catch (Exception e) {
/* expected */
}
assertEquals(i.get(), 2);
- log.info("ended ({}) {} {}, in {}", new Object[] {i, t, t.getStatusDetail(false), Duration.of(timer)});
+ log.info("ended ({}) {} {}, in {}", new Object[]{i, t, t.getStatusDetail(false), Duration.of(timer)});
Assert.assertTrue(Duration.of(timer).isShorterThan(SMALL_FRACTION_OF_CYCLE_DELAY));
+ Asserts.eventually(m::getNumActiveTasks, l -> l==0);
}
- @Test(groups="Integration")
+ @Test(groups = "Integration")
public void testScheduledTaskCancelInterrupts() throws Exception {
final Duration PERIOD = Duration.millis(20);
final Duration CYCLE_DELAY = Duration.ONE_SECOND;
// this should be enough to start the next cycle, but not so much that the cycle ends;
// and enough that when a task is interrupted it terminates within this period
final Duration SMALL_FRACTION_OF_CYCLE_DELAY = PERIOD.add(CYCLE_DELAY.multiply(0.1));
-
+
BasicExecutionManager m = new BasicExecutionManager("mycontextid");
final Semaphore interruptedSemaphore = new Semaphore(0);
final AtomicInteger i = new AtomicInteger();
@@ -258,7 +281,7 @@ public class ScheduledExecutionTest {
@Override
public Integer call() {
try {
- log.info("task running ("+i+"): "+Tasks.current()+" "+Tasks.current().getStatusDetail(false));
+ log.info("task running (" + i + "): " + Tasks.current() + " " + Tasks.current().getStatusDetail(false));
Time.sleep(CYCLE_DELAY);
i.incrementAndGet();
return i.get();
@@ -266,45 +289,48 @@ public class ScheduledExecutionTest {
interruptedSemaphore.release();
throw Exceptions.propagate(e);
}
- }});
- }});
-
- log.info(JavaClassNames.niceClassAndMethod()+" - submitting {} {}", t, t.getStatusDetail(false));
+ }
+ });
+ }
+ });
+
+ log.info(JavaClassNames.niceClassAndMethod() + " - submitting {} {}", t, t.getStatusDetail(false));
m.submit(t);
log.info("submitted {} {}", t, t.getStatusDetail(false));
Integer interimResult = (Integer) t.get();
- log.info("done one ({}) {} {}", new Object[] {interimResult, t, t.getStatusDetail(false)});
+ log.info("done one ({}) {} {}", new Object[]{interimResult, t, t.getStatusDetail(false)});
assertEquals(i.get(), 1);
-
+
Time.sleep(SMALL_FRACTION_OF_CYCLE_DELAY);
assertEquals(t.get(), 2);
-
+
Time.sleep(SMALL_FRACTION_OF_CYCLE_DELAY);
Stopwatch timer = Stopwatch.createUnstarted();
t.cancel(true);
t.blockUntilEnded();
// int finalResult = t.get()
- log.info("blocked until ended ({}) {} {}, in {}", new Object[] {i, t, t.getStatusDetail(false), Duration.of(timer)});
+ log.info("blocked until ended ({}) {} {}, in {}", new Object[]{i, t, t.getStatusDetail(false), Duration.of(timer)});
try {
t.get();
- Assert.fail("Should have failed getting result of cancelled "+t);
+ Assert.fail("Should have failed getting result of cancelled " + t);
} catch (Exception e) {
/* expected */
}
assertEquals(i.get(), 2);
Assert.assertTrue(interruptedSemaphore.tryAcquire(1, SMALL_FRACTION_OF_CYCLE_DELAY.toMilliseconds(), TimeUnit.MILLISECONDS), "child thread was not interrupted");
- log.info("ended ({}) {} {}, in {}", new Object[] {i, t, t.getStatusDetail(false), Duration.of(timer)});
+ log.info("ended ({}) {} {}, in {}", new Object[]{i, t, t.getStatusDetail(false), Duration.of(timer)});
Assert.assertTrue(Duration.of(timer).isShorterThan(SMALL_FRACTION_OF_CYCLE_DELAY));
+ Asserts.eventually(m::getNumActiveTasks, l -> l==0);
}
- @Test(groups="Integration")
+ @Test(groups = "Integration")
public void testScheduledTaskTakesLongerThanPeriod() throws Exception {
final int PERIOD = 1;
final int SLEEP_TIME = 100;
final int EARLY_RETURN_GRACE = 10;
BasicExecutionManager m = new BasicExecutionManager("mycontextid");
final List<Long> execTimes = new CopyOnWriteArrayList<Long>();
-
+
ScheduledTask t = new ScheduledTask(MutableMap.of("delay", PERIOD, "period", PERIOD), new Callable<Task<?>>() {
@Override
public Task<?> call() throws Exception {
@@ -317,17 +343,20 @@ public class ScheduledExecutionTest {
} catch (InterruptedException e) {
throw Exceptions.propagate(e);
}
- }});
- }});
-
+ }
+ });
+ }
+ });
+
m.submit(t);
-
+
Asserts.succeedsEventually(new Runnable() {
@Override
public void run() {
- assertTrue(execTimes.size() > 3, "size="+execTimes.size());
- }});
-
+ assertTrue(execTimes.size() > 3, "size=" + execTimes.size());
+ }
+ });
+
List<Long> timeDiffs = Lists.newArrayList();
long prevExecTime = -1;
for (Long execTime : execTimes) {
@@ -338,9 +367,86 @@ public class ScheduledExecutionTest {
prevExecTime = execTime;
}
}
-
+
for (Long timeDiff : timeDiffs) {
- if (timeDiff < (SLEEP_TIME - EARLY_RETURN_GRACE)) fail("timeDiffs="+timeDiffs+"; execTimes="+execTimes);
+ if (timeDiff < (SLEEP_TIME - EARLY_RETURN_GRACE))
+ fail("timeDiffs=" + timeDiffs + "; execTimes=" + execTimes);
}
+
+ t.cancel();
+ Asserts.eventually(m::getNumActiveTasks, l -> l==0);
}
-}
+
+ @Test(groups="Integration") // because slow
+ public void testScheduledTaskInContextClearing() throws Exception {
+ Duration PERIOD = Duration.millis(50);
+ int COUNT = 10;
+ BasicExecutionManager m = new BasicExecutionManager("mycontextid");
+
+ final List<String> errors = MutableList.of();
+
+ final AtomicInteger i2 = new AtomicInteger();
+ // now start other task, in entity context
+ ScheduledTask t2 = new ScheduledTask(MutableMap.of("displayName", "t2", "period", PERIOD), new Callable<Task<?>>() {
+ @Override
+ public Task<?> call() throws Exception {
+ return new BasicTask<Integer>(MutableMap.of("displayName", "t2-i"), new Callable<Integer>() {
+ @Override
+ public Integer call() {
+ Entity ce = BrooklynTaskTags.getContextEntity(Tasks.current());
+ log.info("entity task t2 running (" + i2 + "): " + Thread.currentThread() + " " + Tasks.current() + " " + Tasks.current().getStatusDetail(false)+" - "+ce);
+ if (ce==null) {
+ errors.add("Scheduled task t2 missing context entity");
+ }
+
+ ScheduledTask submitter = (ScheduledTask) ((BasicTask) Tasks.current()).getSubmittedByTask();
+ i2.incrementAndGet();
+ if (i2.get() >= COUNT) submitter.cancel();
+ return i2.get();
+ }
+ });
+ }
+ });
+ Entity contextEntity = new TestEntityImpl();
+ BasicExecutionContext exec = new BasicExecutionContext(m, MutableList.of(BrooklynTaskTags.tagForContextEntity(contextEntity)));
+ exec.submit(t2);
+
+ final AtomicInteger i1 = new AtomicInteger();
+
+ ScheduledTask t1 = new ScheduledTask(MutableMap.of("displayName", "t1", "period", PERIOD), new Callable<Task<?>>() {
+ @Override
+ public Task<?> call() throws Exception {
+ return new BasicTask<Integer>(MutableMap.of("displayName", "t1-i"), new Callable<Integer>() {
+ @Override
+ public Integer call() {
+ Entity ce = BrooklynTaskTags.getContextEntity(Tasks.current());
+ log.info("non-entity task t1 running (" + i1 + "): " + Thread.currentThread() + " " + Tasks.current() + " " + Tasks.current().getStatusDetail(false)+" - "+ce);
+ if (ce!=null) {
+ errors.add("Scheduled task t1 has context entity "+ce);
+ }
+
+ ScheduledTask submitter = (ScheduledTask) ((BasicTask) Tasks.current()).getSubmittedByTask();
+ i1.incrementAndGet();
+ if (i1.get() >= COUNT) submitter.cancel();
+ return i1.get();
+ }
+ });
+ }
+ });
+
+ Time.sleep(70);
+ log.info(JavaClassNames.niceClassAndMethod() + " - submitting {} {}", t1, t1.getStatusDetail(false));
+ m.submit(t1);
+ log.info("submitted {} {}", t1, t1.getStatusDetail(false));
+ Integer interimResult = (Integer) t1.get();
+ log.info("done one ({}) {} {}", new Object[]{interimResult, t1, t1.getStatusDetail(false)});
+ assertTrue(i1.get() > 0);
+
+ t1.blockUntilEnded();
+ t2.blockUntilEnded();
+
+ Asserts.assertSize(errors, 0);
+ Asserts.eventually(m::getNumActiveTasks, l -> l==0);
+ }
+
+}
\ No newline at end of file
[brooklyn-server] 14/27: allow access to catalog item path on RO
entities
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit ce9a7962f6d6f1b0bf8d7321f97252df42ac74fb
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 13:50:58 2021 +0100
allow access to catalog item path on RO entities
---
.../brooklyn/core/entity/internal/EntityTransientCopyInternal.java | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/core/src/main/java/org/apache/brooklyn/core/entity/internal/EntityTransientCopyInternal.java b/core/src/main/java/org/apache/brooklyn/core/entity/internal/EntityTransientCopyInternal.java
index 9106ec2..753de46 100644
--- a/core/src/main/java/org/apache/brooklyn/core/entity/internal/EntityTransientCopyInternal.java
+++ b/core/src/main/java/org/apache/brooklyn/core/entity/internal/EntityTransientCopyInternal.java
@@ -20,6 +20,7 @@ package org.apache.brooklyn.core.entity.internal;
import java.util.Collection;
+import java.util.List;
import javax.annotation.Nullable;
import org.apache.brooklyn.api.effector.Effector;
@@ -84,6 +85,7 @@ public interface EntityTransientCopyInternal {
GroupSupport groups();
RelationSupport<Entity> relations();
String getCatalogItemId();
+ List<String> getCatalogItemIdSearchPath();
// from EntityInternal:
@@ -95,7 +97,7 @@ public interface EntityTransientCopyInternal {
// for REST calls on read-only entities which want to resolve values
ExecutionContext getExecutionContext();
void setCatalogItemId(String id);
-
+
/** more methods, but which are only on selected entities */
public interface SpecialEntityTransientCopyInternal {
// from Group
[brooklyn-server] 18/27: fix tests which previously only passed due
to scheduled tasks not being cancelled!
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 8c98f10fd7f70211b18cd61dc77b30336c491f60
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 14:43:30 2021 +0100
fix tests which previously only passed due to scheduled tasks not being cancelled!
---
.../org/apache/brooklyn/feed/windows/WinRmCommandSensorTest.java | 8 +++++---
.../feed/windows/WindowsPerformanceCounterSensorsTest.java | 6 ++++--
2 files changed, 9 insertions(+), 5 deletions(-)
diff --git a/software/winrm/src/test/java/org/apache/brooklyn/feed/windows/WinRmCommandSensorTest.java b/software/winrm/src/test/java/org/apache/brooklyn/feed/windows/WinRmCommandSensorTest.java
index 7bf481c..75e10c4 100644
--- a/software/winrm/src/test/java/org/apache/brooklyn/feed/windows/WinRmCommandSensorTest.java
+++ b/software/winrm/src/test/java/org/apache/brooklyn/feed/windows/WinRmCommandSensorTest.java
@@ -29,6 +29,7 @@ import org.apache.brooklyn.core.mgmt.rebind.RebindTestFixtureWithApp;
import org.apache.brooklyn.core.sensor.Sensors;
import org.apache.brooklyn.core.sensor.http.HttpRequestSensor;
import org.apache.brooklyn.core.sensor.windows.WinRmCommandSensor;
+import org.apache.brooklyn.core.test.entity.TestApplication;
import org.apache.brooklyn.core.test.entity.TestEntity;
import org.apache.brooklyn.location.winrm.WinRmMachineLocation;
import org.apache.brooklyn.test.Asserts;
@@ -77,9 +78,10 @@ public class WinRmCommandSensorTest extends RebindTestFixtureWithApp {
app().start(ImmutableList.of(loc));
EntityAsserts.assertAttributeEqualsEventually(entity, Sensors.newStringSensor("mysensor"), "myval");
-
- rebind();
-
+
+ TestApplication app2 = rebind();
+
+ entity = app2.getChildren().iterator().next();
RecordingWinRmTool.setCustomResponse(".*mycommand.*", new RecordingWinRmTool.CustomResponse(0, "myval2", ""));
EntityAsserts.assertAttributeEqualsEventually(entity, Sensors.newStringSensor("mysensor"), "myval2");
}
diff --git a/software/winrm/src/test/java/org/apache/brooklyn/feed/windows/WindowsPerformanceCounterSensorsTest.java b/software/winrm/src/test/java/org/apache/brooklyn/feed/windows/WindowsPerformanceCounterSensorsTest.java
index 5df4f5c..744c84c 100644
--- a/software/winrm/src/test/java/org/apache/brooklyn/feed/windows/WindowsPerformanceCounterSensorsTest.java
+++ b/software/winrm/src/test/java/org/apache/brooklyn/feed/windows/WindowsPerformanceCounterSensorsTest.java
@@ -81,10 +81,12 @@ public class WindowsPerformanceCounterSensorsTest extends RebindTestFixtureWithA
"counter", "\\mycounter")))))));
app().start(ImmutableList.of(loc));
-
+ entity = app().getChildren().iterator().next();
+
EntityAsserts.assertAttributeEqualsEventually(entity, Sensors.newStringSensor("mysensor"), "myval");
-
+
rebind();
+ entity = app().getChildren().iterator().next();
String response2 = generateCounterReponse("mycounter", "myval2");
RecordingWinRmTool.setCustomResponse(".*mycounter.*", new RecordingWinRmTool.CustomResponse(0, response2, ""));
[brooklyn-server] 10/27: improve logging for tasks
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit bcf0fc6aae0fd97d50b57b23e0aff8c49891a891
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 09:13:17 2021 +0100
improve logging for tasks
---
.../apache/brooklyn/core/entity/AbstractEntity.java | 2 +-
.../core/mgmt/internal/BrooklynGarbageCollector.java | 20 ++++++++++++++------
.../util/core/task/BasicExecutionManager.java | 14 +++++++++++---
.../brooklyn/util/core/task/ScheduledTask.java | 3 +++
4 files changed, 29 insertions(+), 10 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/core/entity/AbstractEntity.java b/core/src/main/java/org/apache/brooklyn/core/entity/AbstractEntity.java
index 7c9607e..b796f61 100644
--- a/core/src/main/java/org/apache/brooklyn/core/entity/AbstractEntity.java
+++ b/core/src/main/java/org/apache/brooklyn/core/entity/AbstractEntity.java
@@ -838,7 +838,7 @@ public abstract class AbstractEntity extends AbstractBrooklynObject implements E
for (Location loc : newLocations) {
NamedStringTag ownerEntityTag = BrooklynTags.findFirstNamedStringTag(BrooklynTags.OWNER_ENTITY_ID, loc.tags().getTags());
if (ownerEntityTag != null) {
- if (!getId().equals(ownerEntityTag.getContents())) {
+ if (!getId().equals(ownerEntityTag.getContents()) && !Entities.isReadOnly(this)) {
// A location is "owned" if it was created as part of the EntitySpec of an entity (by Brooklyn).
// To share a location between entities create it yourself and pass it to any entities that needs it.
LOG.info("Adding location {} to entity {}, which is already owned by another entity {}. " +
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/BrooklynGarbageCollector.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/BrooklynGarbageCollector.java
index ca1e075..b1bc870 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/BrooklynGarbageCollector.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/BrooklynGarbageCollector.java
@@ -611,17 +611,25 @@ public class BrooklynGarbageCollector {
LOG.debug("Got CME inspecting tasks, with "+tasksToConsiderDeleting.size()+" found for deletion: "+e);
}
+ if (tasksToConsiderDeleting.isEmpty()) {
+ return 0;
+ }
+
Collections.sort(tasksToConsiderDeleting, TASKS_OLDEST_FIRST_COMPARATOR);
if (LOG.isDebugEnabled()) {
- MutableList<Task<?>> tasksToConsiderDeletingNewest = MutableList.copyOf(tasksToConsiderDeleting);
- Collections.sort(tasksToConsiderDeletingNewest, TASKS_NEWEST_FIRST_COMPARATOR);
+ List<Object> tasksToLog = MutableList.copyOf(tasksToConsiderDeleting);
+ if (tasksToConsiderDeleting.size()>10) {
+ tasksToConsiderDeleting.stream().limit(5).forEach(tasksToLog::add);
+ tasksToLog.add("...");
+ tasksToConsiderDeleting.stream().skip(tasksToConsiderDeleting.size()-5).forEach(tasksToLog::add);
+ } else {
+ tasksToLog.addAll(tasksToConsiderDeleting);
+ }
LOG.debug("brooklyn-gc detected " + taskTagsInCategoryOverCapacity.size() + " " + category + " "
- + "tags over capacity, expiring old tasks; "
+ + "tag(s) over capacity, expiring old tasks; "
+ tasksToConsiderDeleting.size() + " tasks under consideration; categories are: "
- + taskTagsInCategoryOverCapacity + "; including "
- + tasksToConsiderDeleting.stream().limit(5).collect(Collectors.toList()) + " ... "
- + tasksToConsiderDeletingNewest.stream().limit(5).collect(Collectors.toList()) );
+ + taskTagsInCategoryOverCapacity + "; including " + tasksToConsiderDeleting);
}
// now try deleting tasks which are overcapacity for each (non-entity) tag
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
index 793c5ca..b7e5adf 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
@@ -206,10 +206,12 @@ public class BasicExecutionManager implements ExecutionManager {
if (entityMdc != null) {
entityMdc.close();
if (prevEntityMdc!=null) MDC.put(LOGGING_MDC_KEY_ENTITY_IDS, prevEntityMdc);
+ prevEntityMdc = null;
}
if (taskMdc != null) {
taskMdc.close();
if (prevTaskMdc!=null) MDC.put(LOGGING_MDC_KEY_TASK_ID, prevTaskMdc);
+ prevTaskMdc = null;
}
}
@@ -967,7 +969,9 @@ public class BasicExecutionManager implements ExecutionManager {
}
/** normally (if not interrupted) called once for each call to {@link #beforeSubmitScheduledTaskAllIterations(Map, Task)} */
- protected void beforeStartScheduledTaskAllIterations(Map<?,?> flags, Task<?> taskDoingTheInitialSchedule) {
+ protected void beforeStartScheduledTaskAllIterations(Map<?,?> flags, ScheduledTask taskDoingTheInitialSchedule) {
+ taskDoingTheInitialSchedule.mdc = BrooklynTaskLoggingMdc.create(taskDoingTheInitialSchedule).start();
+
internalBeforeStart(flags, taskDoingTheInitialSchedule, !SCHEDULED_TASKS_COUNT_AS_ACTIVE, true, true);
}
protected void beforeStartScheduledTaskSubmissionIteration(Map<?,?> flags, Task<?> taskDoingTheScheduling, Task<?> taskIteration) {
@@ -1049,12 +1053,16 @@ public class BasicExecutionManager implements ExecutionManager {
}
private static boolean loggedClosureDeprecatedInInvokeCallback;
- /** normally (if not interrupted) called once for each call to {@link #beforeStartScheduledTaskAllIterations(Map, Task)} */
- protected void afterEndScheduledTaskAllIterations(Map<?,?> flags, Task<?> taskDoingTheInitialSchedule, Throwable error) {
+ /** normally (if not interrupted) called once for each call to {@link #beforeStartScheduledTaskAllIterations(Map, ScheduledTask)} */
+ protected void afterEndScheduledTaskAllIterations(Map<?,?> flags, ScheduledTask taskDoingTheInitialSchedule, Throwable error) {
boolean taskWasSubmittedAndNotYetEnded = true;
try {
taskWasSubmittedAndNotYetEnded = internalAfterEnd(flags, taskDoingTheInitialSchedule, !SCHEDULED_TASKS_COUNT_AS_ACTIVE, false, error);
} finally {
+ if (taskDoingTheInitialSchedule.mdc!=null) {
+ taskDoingTheInitialSchedule.mdc.close();
+ taskDoingTheInitialSchedule.mdc = null;
+ }
synchronized (taskDoingTheInitialSchedule) { taskDoingTheInitialSchedule.notifyAll(); }
if (taskWasSubmittedAndNotYetEnded) {
// prevent from running twice on cancellation after start
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/ScheduledTask.java b/core/src/main/java/org/apache/brooklyn/util/core/task/ScheduledTask.java
index 8851971..4422209 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/ScheduledTask.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/ScheduledTask.java
@@ -34,6 +34,7 @@ import org.apache.brooklyn.api.mgmt.Task;
import org.apache.brooklyn.core.mgmt.BrooklynTaskTags;
import org.apache.brooklyn.util.collections.MutableList;
import org.apache.brooklyn.util.collections.MutableMap;
+import org.apache.brooklyn.util.core.task.BasicExecutionManager.BrooklynTaskLoggingMdc;
import org.apache.brooklyn.util.exceptions.Exceptions;
import org.apache.brooklyn.util.time.Duration;
@@ -52,6 +53,8 @@ public class ScheduledTask extends BasicTask<Object> {
final Callable<Task<?>> taskFactory;
+ protected BrooklynTaskLoggingMdc mdc;
+
/**
* Initial delay before running, set as flag in constructor; defaults to 0
*/
[brooklyn-server] 12/27: logging about bundles being persisted and
loaded
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit a6cdd854075e6cb65316daebcd6338cf6b1b20dd
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 11:16:17 2021 +0100
logging about bundles being persisted and loaded
---
.../core/mgmt/persist/BrooklynMementoPersisterToObjectStore.java | 1 +
.../main/java/org/apache/brooklyn/core/typereg/BasicManagedBundle.java | 1 +
2 files changed, 2 insertions(+)
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/persist/BrooklynMementoPersisterToObjectStore.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/persist/BrooklynMementoPersisterToObjectStore.java
index 3ae3bc9..fb579df 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/persist/BrooklynMementoPersisterToObjectStore.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/persist/BrooklynMementoPersisterToObjectStore.java
@@ -741,6 +741,7 @@ public class BrooklynMementoPersisterToObjectStore implements BrooklynMementoPer
throw new IllegalStateException("Cannot persist bundles without a management context");
}
final ManagedBundle mb = ((ManagementContextInternal)mgmt).getOsgiManager().get().getManagedBundles().get(id);
+ LOG.debug("Persisting managed bundle "+id+": "+mb);
if (mb==null) {
LOG.warn("Cannot find managed bundle for added bundle "+id+"; ignoring");
return;
diff --git a/core/src/main/java/org/apache/brooklyn/core/typereg/BasicManagedBundle.java b/core/src/main/java/org/apache/brooklyn/core/typereg/BasicManagedBundle.java
index 92571d4..c921a31 100644
--- a/core/src/main/java/org/apache/brooklyn/core/typereg/BasicManagedBundle.java
+++ b/core/src/main/java/org/apache/brooklyn/core/typereg/BasicManagedBundle.java
@@ -179,6 +179,7 @@ public class BasicManagedBundle extends AbstractBrooklynObject implements Manage
.add("version", version)
.add("format", format)
.add("url", url)
+ .add("osgiUniqueUrl", osgiUniqueUrl)
.toString();
}
[brooklyn-server] 15/27: better logging for HA promotion and for
scheduled tasks
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 49ab36d5138f2cc4637e2f50d4a05f70084c27ad
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 13:52:05 2021 +0100
better logging for HA promotion and for scheduled tasks
---
.../brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java | 10 ++++++----
.../apache/brooklyn/util/core/task/BasicExecutionManager.java | 11 ++++++++---
2 files changed, 14 insertions(+), 7 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
index fab4ffc..00f511a 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
@@ -567,7 +567,7 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
if (ManagementNodeState.isHotProxy(oldState) && !ManagementNodeState.isHotProxy(newState)) {
// could perhaps promote standby items on some transitions; but for now we stop the old read-only and re-load them
// TODO ideally there'd be an incremental rebind as well as an incremental persist
- LOG.debug("Resetting rebind on hot->cold transition, from "+oldState+" to "+newState);
+ LOG.debug("Resetting rebind on transition from hot proxy ("+oldState+") to "+newState);
managementContext.getRebindManager().stopReadOnly();
clearManagedItems(ManagementTransitionMode.transitioning(BrooklynObjectManagementMode.LOADED_READ_ONLY, BrooklynObjectManagementMode.UNMANAGED_PERSISTED));
managementContext.getRebindManager().reset();
@@ -781,7 +781,8 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
// if failed or hot backup then we can't promote ourselves, so no point in checking who is master
return;
}
-
+
+ ManagementNodeState ourPrevState = getNodeState();
updateLocalPlaneId(memento);
String currMasterNodeId = memento.getMasterNodeId();
@@ -852,7 +853,8 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
});
}
String message = "Management node "+ownNodeId+" detected ";
- String currMasterSummary = currMasterNodeId + " (" + (currMasterNodeRecord==null ? "<none>" : timestampString(currMasterNodeRecord.getRemoteTimestamp())) + ")";
+ String currMasterSummary =
+ (Strings.isNonBlank(currMasterNodeId) ? currMasterNodeId+" " : "") + "(" + (currMasterNodeRecord==null ? "<none>" : timestampString(currMasterNodeRecord.getRemoteTimestamp())) + ")";
if (weAreNewMaster && (ownNodeRecord.getStatus() == ManagementNodeState.MASTER)) {
LOG.warn(message + "we must reassert master status, as we believe we should be master and other master "+
(currMasterNodeRecord==null ? "(a node which has gone away)" : currMasterSummary)+" has failed");
@@ -863,7 +865,7 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
if (!initializing) {
if (weAreNewMaster) {
- message += "we should be master, changing from ";
+ message += "we should be master, promoting from "+ourPrevState+"; master changing from ";
}
else if (currMasterNodeRecord==null && newMasterNodeId==null) message += "master change attempted but no candidates ";
else message += "master change, from ";
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
index 995c49e..41a704e 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
@@ -172,9 +172,15 @@ public class BasicExecutionManager implements ExecutionManager {
entityMdc = MDC.putCloseable(LOGGING_MDC_KEY_ENTITY_IDS, "");
}
+ logEvent("Starting task", task, entity);
+
+ return this;
+ }
+
+ public static void logEvent(String prefix, Task task, Entity entity) {
if (BrooklynLoggingCategories.TASK_LIFECYCLE_LOG.isDebugEnabled() || BrooklynLoggingCategories.TASK_LIFECYCLE_LOG.isTraceEnabled()) {
String taskName = task.getDisplayName();
- String message = "Starting task " + task.getId() +
+ String message = prefix + " " + task.getId() +
(Strings.isNonBlank(taskName) ? " ("+taskName+")" : "") +
(entity == null ? "" : " on entity " + entity.getId()) +
(Strings.isNonBlank(task.getSubmittedByTaskId()) ? " from task " + task.getSubmittedByTaskId() : "") +
@@ -189,8 +195,6 @@ public class BasicExecutionManager implements ExecutionManager {
message);
}
-
- return this;
}
public void finish() {
@@ -557,6 +561,7 @@ public class BasicExecutionManager implements ExecutionManager {
protected Task<?> submitNewScheduledTask(final Map<?,?> flags, final ScheduledTask task) {
boolean result = false;
try {
+ BrooklynTaskLoggingMdc.logEvent("Submitting scheduled task", task, BrooklynTaskTags.getTargetOrContextEntity(Tasks.current()));
result = submitSubsequentScheduledTask(flags, task);
} finally {
if (!result) {
[brooklyn-server] 08/27: better task logging of entitlement context
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 78b3129cfaf919389c01bf07cd830949eab44a04
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Mon Sep 13 23:03:17 2021 +0100
better task logging of entitlement context
---
.../org/apache/brooklyn/util/core/task/BasicExecutionContext.java | 5 +++--
.../org/apache/brooklyn/util/core/task/BasicExecutionManager.java | 8 +++++++-
2 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionContext.java b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionContext.java
index 70322ce..9d329e5 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionContext.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionContext.java
@@ -353,8 +353,9 @@ public class BasicExecutionContext extends AbstractExecutionContext {
}
EntitlementContext entitlementContext = BrooklynTaskTags.getEntitlement(taskTags);
- if (entitlementContext==null)
- entitlementContext = Entitlements.getEntitlementContext();
+ if (entitlementContext==null) {
+ entitlementContext = Entitlements.getEntitlementContext();
+ }
if (entitlementContext!=null) {
taskTags.add(BrooklynTaskTags.tagForEntitlement(entitlementContext));
}
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
index 487b155..8b44490 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
@@ -56,6 +56,7 @@ import org.apache.brooklyn.api.mgmt.ExecutionManager;
import org.apache.brooklyn.api.mgmt.HasTaskChildren;
import org.apache.brooklyn.api.mgmt.Task;
import org.apache.brooklyn.api.mgmt.TaskAdaptable;
+import org.apache.brooklyn.api.mgmt.entitlement.EntitlementContext;
import org.apache.brooklyn.core.BrooklynFeatureEnablement;
import org.apache.brooklyn.core.BrooklynLogging;
import org.apache.brooklyn.core.config.Sanitizer;
@@ -177,7 +178,12 @@ public class BasicExecutionManager implements ExecutionManager {
(Strings.isNonBlank(taskName) ? " ("+taskName+")" : "") +
(entity == null ? "" : " on entity " + entity.getId()) +
(Strings.isNonBlank(task.getSubmittedByTaskId()) ? " from task " + task.getSubmittedByTaskId() : "") +
- Entitlements.getEntitlementContextUserMaybe().map(s -> " for user " + s).or("");
+ Entitlements.getEntitlementContextUserMaybe().
+ orMaybe(() -> Maybe.ofDisallowingNull(BrooklynTaskTags.getEntitlement(task))
+ // entitlements might not be set in thread context, so also look on task
+ .map(EntitlementContext::user)
+ .mapMaybe(s -> Strings.isNonBlank(s) ? Maybe.of(s) : Maybe.absent())
+ ).map(s -> " for user " + s).or("");
BrooklynLogging.log(BrooklynLoggingCategories.TASK_LIFECYCLE_LOG,
UNINTERESTING_TASK_NAMES.contains(taskName) ? BrooklynLogging.LoggingLevel.TRACE : BrooklynLogging.LoggingLevel.DEBUG,
message);
[brooklyn-server] 02/27: further fix for warning on http session
retrieval race condition
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit ae273935e30e0ffae0ad6ff141afc34e213e4fb1
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 11:15:41 2021 +0100
further fix for warning on http session retrieval race condition
---
.../brooklyn/rest/util/MultiSessionAttributeAdapter.java | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/rest/rest-resources/src/main/java/org/apache/brooklyn/rest/util/MultiSessionAttributeAdapter.java b/rest/rest-resources/src/main/java/org/apache/brooklyn/rest/util/MultiSessionAttributeAdapter.java
index 0b1878e..2958a6e 100644
--- a/rest/rest-resources/src/main/java/org/apache/brooklyn/rest/util/MultiSessionAttributeAdapter.java
+++ b/rest/rest-resources/src/main/java/org/apache/brooklyn/rest/util/MultiSessionAttributeAdapter.java
@@ -154,7 +154,9 @@ public class MultiSessionAttributeAdapter {
if(preferredSession!=null) {
// need to create a local session so the ID/session is registered with this ui module
if (r instanceof Request) {
- // but synch on the session handler to avoid race conditions in the underlying code
+
+ // synch and own lookup to avoid the following warning
+
// 2021-09-13T08:12:33,186Z - WARN 254 o.e.j.s.session [p1568796312-1154]
// java.lang.IllegalStateException: Session node0171nuqxrc6qsf1tbrmxztok6xc4 already in cache
// at org.eclipse.jetty.server.session.AbstractSessionCache.add(AbstractSessionCache.java:467) ~[!/:9.4.39.v20210325]
@@ -163,7 +165,15 @@ public class MultiSessionAttributeAdapter {
// at org.eclipse.jetty.server.Request.getSession(Request.java:1602) ~[!/:9.4.39.v20210325]
// at org.apache.brooklyn.rest.util.MultiSessionAttributeAdapter.of(MultiSessionAttributeAdapter.java:155) ~[!/:1.1.0-SNAPSHOT]
synchronized (((Request)r).getSessionHandler()) {
- localSession = r.getSession();
+ try {
+ String id = ((Request) r).getSessionHandler().getSessionIdManager().newSessionId(r, System.currentTimeMillis());
+ localSession = ((Request) r).getSessionHandler().getSession(id);
+ } catch (Exception e) {
+ log.debug("Unable to retrieve session via safe override, falling back to default: "+e);
+ }
+ if (localSession==null) {
+ localSession = r.getSession();
+ }
}
} else {
localSession = r.getSession();
[brooklyn-server] 13/27: when rebinding, install persisted bundles
before default.catalog.bom; but start after
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 2d20fffa2407e4da48fe6edeebc3335e140edccb
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 12:18:13 2021 +0100
when rebinding, install persisted bundles before default.catalog.bom; but start after
this means the IDs in persistence will be re-used, but start order unchanged.
previously it did default.catalog.bom before installing and starting persisted bundles,
with the result that IDs changed on each rebind for things which came from the default.catalog.bom.
behavior indeterminate wrt snapshot bundles (might switch to preferring those in default.catalog.bom).
---
.../catalog/internal/CatalogInitialization.java | 86 ++++++++++++++--------
.../BrooklynMementoPersisterToObjectStore.java | 12 +--
.../brooklyn/core/mgmt/rebind/RebindIteration.java | 11 ++-
.../mgmt/rebind/dto/BasicManagedBundleMemento.java | 7 ++
.../BrooklynLauncherRebindCatalogOsgiTest.java | 4 +-
5 files changed, 79 insertions(+), 41 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/core/catalog/internal/CatalogInitialization.java b/core/src/main/java/org/apache/brooklyn/core/catalog/internal/CatalogInitialization.java
index 4928e8f..b65e1c7 100644
--- a/core/src/main/java/org/apache/brooklyn/core/catalog/internal/CatalogInitialization.java
+++ b/core/src/main/java/org/apache/brooklyn/core/catalog/internal/CatalogInitialization.java
@@ -68,10 +68,10 @@ import org.apache.brooklyn.util.guava.Maybe;
import org.apache.brooklyn.util.javalang.JavaClassNames;
import org.apache.brooklyn.util.os.Os;
import org.apache.brooklyn.util.osgi.VersionedName;
-import org.apache.brooklyn.util.stream.InputStreamSource;
import org.apache.brooklyn.util.text.Strings;
import org.apache.brooklyn.util.time.Duration;
import org.apache.commons.lang3.tuple.Pair;
+import org.osgi.framework.Bundle;
import org.osgi.framework.BundleContext;
import org.osgi.framework.BundleException;
import org.slf4j.Logger;
@@ -253,7 +253,11 @@ public class CatalogInitialization implements ManagementContextInjectable {
} else if (hasRunInitialCatalogInitialization()) {
throw new IllegalStateException("Catalog initialization already run for initial catalog by mechanism other than populating persisted state; mode="+mode);
}
-
+
+ // Always install the bundles from persisted state; installed (but not started) prior to catalog,
+ // so that OSGi unique IDs might be picked up when initial catalog is populated
+ Map<InstallableManagedBundle, OsgiBundleInstallationResult> persistenceInstalls = installPersistedBundlesDontStart(persistedState.getBundles(), exceptionHandler, rebindLogger);
+
populateInitialCatalogImpl(true);
final Maybe<OsgiManager> maybesOsgiManager = managementContext.getOsgiManager();
@@ -267,9 +271,20 @@ public class CatalogInitialization implements ManagementContextInjectable {
catalogUpgradeScanner.scan(osgiManager, bundleContext, rebindLogger);
CatalogUpgrades.storeInManagementContext(catalogUpgrades, managementContext);
}
+
PersistedCatalogState filteredPersistedState = filterBundlesAndCatalogInPersistedState(persistedState, rebindLogger);
- addPersistedCatalogImpl(filteredPersistedState, exceptionHandler, rebindLogger);
-
+
+ // previously we effectively installed here, after populating; but now we do it before and then uninstall if needed, to preserve IDs
+// Map<InstallableManagedBundle, OsgiBundleInstallationResult> persistenceInstalls = installPersistedBundlesDontStart(filteredPersistedState.getBundles(), exceptionHandler, rebindLogger);
+
+ try {
+ startPersistedBundles(filteredPersistedState, persistenceInstalls, exceptionHandler, rebindLogger);
+ BrooklynCatalog catalog = managementContext.getCatalog();
+ catalog.addCatalogLegacyItemsOnRebind(filteredPersistedState.getLegacyCatalogItems());
+ } finally {
+ hasRunPersistenceInitialization = true;
+ }
+
if (mode == ManagementNodeState.MASTER) {
// TODO ideally this would remain false until it has *persisted* the changed catalog;
// if there is a subsequent startup failure the forced additions will not be persisted,
@@ -407,20 +422,6 @@ public class CatalogInitialization implements ManagementContextInjectable {
}
}
- private void addPersistedCatalogImpl(PersistedCatalogState persistedState, RebindExceptionHandler exceptionHandler, RebindLogger rebindLogger) {
- assert Thread.holdsLock(populatingCatalogMutex);
-
- try {
- // Always installing the bundles from persisted state
- installPersistedBundles(persistedState.getBundles(), exceptionHandler, rebindLogger);
-
- BrooklynCatalog catalog = managementContext.getCatalog();
- catalog.addCatalogLegacyItemsOnRebind(persistedState.getLegacyCatalogItems());
- } finally {
- hasRunPersistenceInitialization = true;
- }
- }
-
private void onFinalCatalog() {
assert Thread.holdsLock(populatingCatalogMutex);
@@ -504,12 +505,12 @@ public class CatalogInitialization implements ManagementContextInjectable {
return false;
}
- private void installPersistedBundles(Map<VersionedName, InstallableManagedBundle> bundles, RebindExceptionHandler exceptionHandler, RebindLogger rebindLogger) {
+ private Map<InstallableManagedBundle, OsgiBundleInstallationResult> installPersistedBundlesDontStart(Map<VersionedName, InstallableManagedBundle> bundles, RebindExceptionHandler exceptionHandler, RebindLogger rebindLogger) {
Map<InstallableManagedBundle, OsgiBundleInstallationResult> installs = MutableMap.of();
// Install the bundles
Map<VersionedName, InstallableManagedBundle> remaining = MutableMap.copyOf(bundles);
- Set<Pair<Entry<VersionedName, InstallableManagedBundle>,Exception>> errors = MutableSet.of();
+ Set<Pair<Entry<VersionedName, InstallableManagedBundle>, Exception>> errors = MutableSet.of();
while (!remaining.isEmpty()) {
int installed = 0;
for (Entry<VersionedName, InstallableManagedBundle> entry : MutableSet.copyOf(remaining.entrySet())) {
@@ -519,35 +520,58 @@ public class CatalogInitialization implements ManagementContextInjectable {
remaining.remove(entry.getKey());
installed++;
} catch (Exception e) {
- rebindLogger.debug("Unable to install bundle "+entry.getKey()+", but may re-try in case it has a dependency on another bundle ("+e+")");
+ rebindLogger.debug("Unable to install bundle " + entry.getKey() + ", but may re-try in case it has a dependency on another bundle (" + e + ")");
errors.add(Pair.of(entry, e));
}
}
- if (installed==0) {
+ if (installed == 0) {
// keep trying until either nothing is installed or nothing is left to install
break;
}
}
rebindLogger.debug("RebindManager installed bundles {}, {} errors", installs.keySet(), errors.size());
errors.forEach(err -> exceptionHandler.onCreateFailed(BrooklynObjectType.MANAGED_BUNDLE,
- err.getLeft().getKey().toString(), err.getLeft().getValue().getManagedBundle().getSymbolicName(), err.getRight()) );
+ err.getLeft().getKey().toString(), err.getLeft().getValue().getManagedBundle().getSymbolicName(), err.getRight()));
- // Start the bundles (now that we've installed them all)
+ return installs;
+ }
- Set<RegisteredType> installedTypes = MutableSet.of();
+ private void startPersistedBundles(PersistedCatalogState filteredPersistedState, Map<InstallableManagedBundle, OsgiBundleInstallationResult> installs, RebindExceptionHandler exceptionHandler, RebindLogger rebindLogger) {
+ // Start the bundles (now that we've installed them all)
// start order is: OSGi and not catalog; then OSGi and catalog; then not catalog nor OSGi; then catalog and not OSGi
// (we need OSGi and not catalog to start first; the others are less important)
- Set<OsgiBundleInstallationResult> bundlesInOrder = MutableSet.copyOf(installs.values());
- MutableSet.copyOf(bundlesInOrder).stream().filter(b -> b.getBundle()!=null && b.getBundle().getResource("/catalog.bom")!=null).forEach(b -> {
- bundlesInOrder.remove(b); bundlesInOrder.add(b); // then move catalog.bom items to the end
+ Set<OsgiBundleInstallationResult> bundlesInOrder = MutableSet.of();
+ Set<OsgiBundleInstallationResult> bundlesToRemove = MutableSet.of();
+ installs.values().stream().forEach(candidate -> {
+ if (filteredPersistedState.getBundles().containsKey(candidate.getVersionedName())) {
+ bundlesInOrder.add(candidate);
+ } else {
+ log.debug("Skipping start of persisted bundle "+candidate+" due to catalog upgrade metadata instructions");
+ bundlesToRemove.add(candidate);
+ }
});
- MutableSet.copyOf(bundlesInOrder).stream().filter(b -> b.getBundle()!=null && b.getBundle().getResource("/META-INF/MANIFEST.MF")==null).forEach(b -> {
- bundlesInOrder.remove(b); bundlesInOrder.add(b); // move non-osgi items to the end
+ bundlesToRemove.forEach(b -> {
+ ManagedBundle mb = getManagementContext().getOsgiManager().get().getManagedBundle(b.getVersionedName());
+ if (b.getBundle().getState() >= Bundle.INSTALLED && b.getBundle().getState() < Bundle.STARTING) {
+ // we installed it, catalog did not start it, so let's uninstall it
+ OsgiBundleInstallationResult result = getManagementContext().getOsgiManager().get().uninstallUploadedBundle(b.getMetadata());
+ log.debug("Result of uninstalling "+b+" due to due to catalog upgrade metadata instructions: "+result);
+ }
+ });
+
+ MutableSet.copyOf(bundlesInOrder).stream().filter(b -> b.getBundle() != null && b.getBundle().getResource("/catalog.bom") != null).forEach(b -> {
+ bundlesInOrder.remove(b);
+ bundlesInOrder.add(b); // then move catalog.bom items to the end
+ });
+ MutableSet.copyOf(bundlesInOrder).stream().filter(b -> b.getBundle() != null && b.getBundle().getResource("/META-INF/MANIFEST.MF") == null).forEach(b -> {
+ bundlesInOrder.remove(b);
+ bundlesInOrder.add(b); // move non-osgi items to the end
});
if (!bundlesInOrder.isEmpty()) {
- log.debug("Rebind bundle start order is: "+bundlesInOrder);
+ log.debug("Rebind bundle start order is: " + bundlesInOrder);
}
+ Set<RegisteredType> installedTypes = MutableSet.of();
for (OsgiBundleInstallationResult br : bundlesInOrder) {
try {
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/persist/BrooklynMementoPersisterToObjectStore.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/persist/BrooklynMementoPersisterToObjectStore.java
index fb579df..5f16faf 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/persist/BrooklynMementoPersisterToObjectStore.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/persist/BrooklynMementoPersisterToObjectStore.java
@@ -619,7 +619,7 @@ public class BrooklynMementoPersisterToObjectStore implements BrooklynMementoPer
futures.add(asyncUpdatePlaneId(newMemento.getPlaneId(), exceptionHandler));
for (BrooklynObjectType type: BrooklynPersistenceUtils.STANDARD_BROOKLYN_OBJECT_TYPE_PERSISTENCE_ORDER) {
for (Map.Entry<String, String> entry : newMemento.getObjectsOfType(type).entrySet()) {
- addPersistContentIfManagedBundle(type, entry.getKey(), futures, exceptionHandler);
+ addPersistContentIfManagedBundle(type, entry.getKey(), entry.getValue(), futures, exceptionHandler);
futures.add(asyncPersist(type.getSubPathName(), type, entry.getKey(), entry.getValue(), exceptionHandler));
}
}
@@ -705,7 +705,7 @@ public class BrooklynMementoPersisterToObjectStore implements BrooklynMementoPer
for (BrooklynObjectType type: BrooklynPersistenceUtils.STANDARD_BROOKLYN_OBJECT_TYPE_PERSISTENCE_ORDER) {
for (Memento item : delta.getObjectsOfType(type)) {
if (!deletedIds.contains(item.getId())) {
- addPersistContentIfManagedBundle(type, item.getId(), futures, exceptionHandler);
+ addPersistContentIfManagedBundle(type, item.getId(), ""+item.getCatalogItemId()+"/"+item.getDisplayName(), futures, exceptionHandler);
futures.add(asyncPersist(type.getSubPathName(), item, exceptionHandler));
}
}
@@ -735,15 +735,17 @@ public class BrooklynMementoPersisterToObjectStore implements BrooklynMementoPer
return lastErrors;
}
- private void addPersistContentIfManagedBundle(final BrooklynObjectType type, final String id, List<ListenableFuture<?>> futures, final PersistenceExceptionHandler exceptionHandler) {
+ private void addPersistContentIfManagedBundle(final BrooklynObjectType type, final String id, final String summaryOrContents, List<ListenableFuture<?>> futures, final PersistenceExceptionHandler exceptionHandler) {
if (type==BrooklynObjectType.MANAGED_BUNDLE) {
if (mgmt==null) {
throw new IllegalStateException("Cannot persist bundles without a management context");
}
final ManagedBundle mb = ((ManagementContextInternal)mgmt).getOsgiManager().get().getManagedBundles().get(id);
- LOG.debug("Persisting managed bundle "+id+": "+mb);
+ LOG.debug("Persisting managed bundle "+id+": "+mb+" - "+summaryOrContents);
if (mb==null) {
- LOG.warn("Cannot find managed bundle for added bundle "+id+"; ignoring");
+ // previously happened on rebind because new osgi unique id was made; now it should use the same so we shouldn't see this,
+ // but if we do the log will contain a summary or contents for comparison which will help
+ LOG.warn("Cannot find managed bundle for added bundle "+id+"; ignoring (probably uninstalled or reinstalled with another OSGi ID; see debug log for contents)");
return;
}
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
index ede24a9..cfbd400 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
@@ -862,8 +862,11 @@ public abstract class RebindIteration {
if (!isEmpty) {
BrooklynLogging.log(LOG, shouldLogRebinding() ? LoggingLevel.INFO : LoggingLevel.DEBUG,
- "Rebind complete " + "(" + mode + (readOnlyRebindCount.get() >= 0 ? ", iteration " + readOnlyRebindCount : "") + ")" +
- " in {}: {} app{}, {} entit{}, {} location{}, {} polic{}, {} enricher{}, {} feed{}, {} catalog item{}, {} catalog bundle{}",
+ "Rebind complete" +
+ (!exceptionHandler.getExceptions().isEmpty() ? ", with errors" :
+ !exceptionHandler.getWarnings().isEmpty() ? ", with warnings" : "") +
+ " (" + mode + (readOnlyRebindCount.get() >= 0 ? ", iteration " + readOnlyRebindCount : "") + ")" +
+ " in {}: {} app{}, {} entit{}, {} location{}, {} polic{}, {} enricher{}, {} feed{}, {} catalog item{}, {} catalog bundle{}{}{}",
Time.makeTimeStringRounded(timer), applications.size(), Strings.s(applications),
rebindContext.getEntities().size(), Strings.ies(rebindContext.getEntities()),
rebindContext.getLocations().size(), Strings.s(rebindContext.getLocations()),
@@ -871,7 +874,9 @@ public abstract class RebindIteration {
rebindContext.getEnrichers().size(), Strings.s(rebindContext.getEnrichers()),
rebindContext.getFeeds().size(), Strings.s(rebindContext.getFeeds()),
rebindContext.getCatalogItems().size(), Strings.s(rebindContext.getCatalogItems()),
- rebindContext.getBundles().size(), Strings.s(rebindContext.getBundles())
+ rebindContext.getBundles().size(), Strings.s(rebindContext.getBundles()),
+ (!exceptionHandler.getExceptions().isEmpty() ? "; errors="+exceptionHandler.getExceptions() : ""),
+ (!exceptionHandler.getWarnings().isEmpty() ? "; warnings="+exceptionHandler.getWarnings() : "")
);
}
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/dto/BasicManagedBundleMemento.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/dto/BasicManagedBundleMemento.java
index 850da94..6d74587 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/dto/BasicManagedBundleMemento.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/dto/BasicManagedBundleMemento.java
@@ -161,4 +161,11 @@ public class BasicManagedBundleMemento extends AbstractMemento implements Manage
.add("url", getUrl())
.add("checksum", getChecksum());
}
+
+ @Override
+ public String toString() {
+ // include more details on toString here, so we can see what/where it is being persisted
+ return toVerboseString();
+ }
+
}
diff --git a/launcher/src/test/java/org/apache/brooklyn/launcher/BrooklynLauncherRebindCatalogOsgiTest.java b/launcher/src/test/java/org/apache/brooklyn/launcher/BrooklynLauncherRebindCatalogOsgiTest.java
index a90812b..5cc4f66 100644
--- a/launcher/src/test/java/org/apache/brooklyn/launcher/BrooklynLauncherRebindCatalogOsgiTest.java
+++ b/launcher/src/test/java/org/apache/brooklyn/launcher/BrooklynLauncherRebindCatalogOsgiTest.java
@@ -631,8 +631,8 @@ public abstract class BrooklynLauncherRebindCatalogOsgiTest extends AbstractBroo
assertEquals(getPersistenceListing(BrooklynObjectType.MANAGED_BUNDLE), ImmutableSet.of(bundlePersistenceId2));
if (isT1KeptRunningWhenT2Starts()) {
- // would like if we could make them the same but currently code should _always_ change
- Assert.assertNotEquals(bundlePersistenceId1, bundlePersistenceId2);
+ // now we keep these the same!
+ Assert.assertEquals(bundlePersistenceId1, bundlePersistenceId2);
}
}
[brooklyn-server] 21/27: fix race where persister might still be
reading from hot standby
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit b760aead06eea6f2a0e395497a4de48a5fc722f8
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 17:03:28 2021 +0100
fix race where persister might still be reading from hot standby
doesn't affect behaviour, just prevents errors.
also improve scheduled task logging.
---
.../rebind/mementos/BrooklynMementoPersister.java | 2 +
.../core/mgmt/ha/HighAvailabilityManagerImpl.java | 5 +
.../BrooklynMementoPersisterToObjectStore.java | 47 +-
.../core/mgmt/rebind/RebindManagerImpl.java | 7 +
.../util/core/task/BasicExecutionManager.java | 682 ++++++++++++---------
.../brooklyn/util/core/task/ScheduledTask.java | 6 +-
.../exceptions/RuntimeInterruptedException.java | 5 +
7 files changed, 456 insertions(+), 298 deletions(-)
diff --git a/api/src/main/java/org/apache/brooklyn/api/mgmt/rebind/mementos/BrooklynMementoPersister.java b/api/src/main/java/org/apache/brooklyn/api/mgmt/rebind/mementos/BrooklynMementoPersister.java
index 2090c80..163a0ff 100644
--- a/api/src/main/java/org/apache/brooklyn/api/mgmt/rebind/mementos/BrooklynMementoPersister.java
+++ b/api/src/main/java/org/apache/brooklyn/api/mgmt/rebind/mementos/BrooklynMementoPersister.java
@@ -109,6 +109,8 @@ public interface BrooklynMementoPersister {
void disableWriteAccess(boolean graceful);
/** permanently shuts down all access to the remote store */
void stop(boolean graceful);
+ /** cancels/waits for all current tasks */
+ void reset();
@VisibleForTesting
void waitForWritesCompleted(Duration timeout) throws InterruptedException, TimeoutException;
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
index 8c63775..9fa2ead 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
@@ -891,6 +891,11 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
protected void promoteToMaster() {
LOG.debug("Promoting to master: "+this);
+ if (Tasks.current()!=null) {
+ // let us check if promotion is happening in the right context
+ Task task = Tasks.current();
+ LOG.debug("Task context for master promotion: "+task+" ("+task.getTags()+"); "+task.getStatusSummary());
+ }
if (!running) {
LOG.warn("Ignoring promote-to-master request, as HighAvailabilityManager is not running");
return;
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/persist/BrooklynMementoPersisterToObjectStore.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/persist/BrooklynMementoPersisterToObjectStore.java
index 5f16faf..e3b3540 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/persist/BrooklynMementoPersisterToObjectStore.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/persist/BrooklynMementoPersisterToObjectStore.java
@@ -43,6 +43,7 @@ import org.apache.brooklyn.api.mgmt.rebind.RebindExceptionHandler;
import org.apache.brooklyn.api.mgmt.rebind.mementos.BrooklynMemento;
import org.apache.brooklyn.api.mgmt.rebind.mementos.BrooklynMementoManifest;
import org.apache.brooklyn.api.mgmt.rebind.mementos.BrooklynMementoPersister;
+import org.apache.brooklyn.api.mgmt.rebind.mementos.BrooklynMementoPersister.LookupContext;
import org.apache.brooklyn.api.mgmt.rebind.mementos.BrooklynMementoRawData;
import org.apache.brooklyn.api.mgmt.rebind.mementos.CatalogItemMemento;
import org.apache.brooklyn.api.mgmt.rebind.mementos.ManagedBundleMemento;
@@ -71,6 +72,7 @@ import org.apache.brooklyn.util.collections.MutableSet;
import org.apache.brooklyn.util.core.xstream.XmlUtil;
import org.apache.brooklyn.util.exceptions.CompoundRuntimeException;
import org.apache.brooklyn.util.exceptions.Exceptions;
+import org.apache.brooklyn.util.exceptions.RuntimeInterruptedException;
import org.apache.brooklyn.util.text.Strings;
import org.apache.brooklyn.util.time.Duration;
import org.apache.brooklyn.util.time.Time;
@@ -117,7 +119,7 @@ public class BrooklynMementoPersisterToObjectStore implements BrooklynMementoPer
private final Map<String, StoreObjectAccessorWithLock> writers = new LinkedHashMap<String, PersistenceObjectStore.StoreObjectAccessorWithLock>();
- private final ListeningExecutorService executor;
+ private ListeningExecutorService executor;
private volatile boolean writesAllowed = false;
private volatile boolean writesShuttingDown = false;
@@ -154,8 +156,6 @@ public class BrooklynMementoPersisterToObjectStore implements BrooklynMementoPer
.withClassLoader(classLoader).build();
this.serializerWithStandardClassLoader = new RetryingMementoSerializer<Object>(rawSerializer, maxSerializationAttempts);
- int maxThreadPoolSize = brooklynProperties.getConfig(PERSISTER_MAX_THREAD_POOL_SIZE);
-
objectStore.createSubPath("entities");
objectStore.createSubPath("locations");
objectStore.createSubPath("policies");
@@ -166,11 +166,11 @@ public class BrooklynMementoPersisterToObjectStore implements BrooklynMementoPer
// FIXME does it belong here or to ManagementPlaneSyncRecordPersisterToObjectStore ?
objectStore.createSubPath("plane");
- executor = MoreExecutors.listeningDecorator(Executors.newFixedThreadPool(maxThreadPoolSize, new ThreadFactory() {
- @Override public Thread newThread(Runnable r) {
- // Note: Thread name referenced in logback-includes' ThreadNameDiscriminator
- return new Thread(r, "brooklyn-persister");
- }}));
+ resetExecutor();
+ }
+
+ private Integer maxThreadPoolSize() {
+ return brooklynProperties.getConfig(PERSISTER_MAX_THREAD_POOL_SIZE);
}
public MementoSerializer<Object> getMementoSerializer() {
@@ -210,8 +210,14 @@ public class BrooklynMementoPersisterToObjectStore implements BrooklynMementoPer
final ManagementContext managementContext = lookupContext.lookupManagementContext();
RegisteredType catalogItem = managementContext.getTypeRegistry().get(catalogItemId);
if (catalogItem == null) {
- // TODO do we need to only log once, rather than risk log.warn too often? I think this only happens on rebind, so ok.
- LOG.warn("Unable to load catalog item "+catalogItemId
+ // will happen on rebind if our execution should have ended
+ if (Thread.interrupted()) {
+ LOG.debug("Aborting (probably old) rebind iteration");
+ throw new RuntimeInterruptedException("Rebind iteration cancelled");
+ }
+
+ // might come here for other reasons too
+ LOG.debug("Unable to load registered type "+catalogItemId
+" for custom class loader of " + type + " " + objectId + "; will use default class loader");
return null;
} else {
@@ -245,7 +251,24 @@ public class BrooklynMementoPersisterToObjectStore implements BrooklynMementoPer
@Override
public void stop(boolean graceful) {
disableWriteAccess(graceful);
-
+ stopExecutor(graceful);
+ }
+
+ @Override
+ public void reset() {
+ resetExecutor();
+ }
+
+ public void resetExecutor() {
+ stopExecutor(false);
+ executor = MoreExecutors.listeningDecorator(Executors.newFixedThreadPool(maxThreadPoolSize(), new ThreadFactory() {
+ @Override public Thread newThread(Runnable r) {
+ // Note: Thread name referenced in logback-includes' ThreadNameDiscriminator
+ return new Thread(r, "brooklyn-persister");
+ }}));
+ }
+
+ public void stopExecutor(boolean graceful) {
if (executor != null) {
if (graceful) {
executor.shutdown();
@@ -473,6 +496,8 @@ public class BrooklynMementoPersisterToObjectStore implements BrooklynMementoPer
@Override
public BrooklynMemento loadMemento(BrooklynMementoRawData mementoData, final LookupContext lookupContext, final RebindExceptionHandler exceptionHandler) throws IOException {
+ LOG.debug("Loading mementos");
+
if (mementoData==null)
mementoData = loadMementoRawData(exceptionHandler);
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
index 899e10b..c45c96f 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
@@ -396,6 +396,9 @@ public class RebindManagerImpl implements RebindManager {
LOG.warn("Rebind (read-only) tasks took too long to die after interrupt (ignoring): "+readOnlyTask);
}
readOnlyTask = null;
+ if (persistenceStoreAccess!=null) {
+ persistenceStoreAccess.reset();
+ }
LOG.debug("Stopped read-only rebinding ("+this+"), mgmt "+managementContext.getManagementNodeId());
// short waits when promoting
@@ -404,6 +407,7 @@ public class RebindManagerImpl implements RebindManager {
Duration.seconds(5));
// note, items are subsequently unmanaged via:
// HighAvailabilityManagerImpl.clearManagedItems
+ readOnlyRebindCount.set(0);
}
}
@@ -467,6 +471,9 @@ public class RebindManagerImpl implements RebindManager {
@Override
public void reset() {
if (persistenceRealChangeListener != null && !persistenceRealChangeListener.isActive()) persistenceRealChangeListener.reset();
+ if (persistenceStoreAccess!=null) {
+ persistenceStoreAccess.reset();
+ }
}
@Override
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
index 41a704e..88812e0 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
@@ -59,6 +59,7 @@ import org.apache.brooklyn.api.mgmt.TaskAdaptable;
import org.apache.brooklyn.api.mgmt.entitlement.EntitlementContext;
import org.apache.brooklyn.core.BrooklynFeatureEnablement;
import org.apache.brooklyn.core.BrooklynLogging;
+import org.apache.brooklyn.core.BrooklynLogging.LoggingLevel;
import org.apache.brooklyn.core.config.Sanitizer;
import org.apache.brooklyn.core.entity.Entities;
import org.apache.brooklyn.core.mgmt.BrooklynTaskTags;
@@ -133,16 +134,18 @@ public class BasicExecutionManager implements ExecutionManager {
public static BrooklynTaskLoggingMdc create() {
return new BrooklynTaskLoggingMdc();
}
+
public static BrooklynTaskLoggingMdc create(Task task) {
return new BrooklynTaskLoggingMdc().withTask(task);
}
boolean isRedundant = false;
Task task;
- MDC.MDCCloseable taskMdc=null, entityMdc=null;
+ MDC.MDCCloseable taskMdc = null, entityMdc = null;
String prevTaskMdc, prevEntityMdc;
- private BrooklynTaskLoggingMdc() {}
+ private BrooklynTaskLoggingMdc() {
+ }
public BrooklynTaskLoggingMdc withTask(Task task) {
this.task = task;
@@ -156,7 +159,7 @@ public class BasicExecutionManager implements ExecutionManager {
// the log _message_ shows who submitted it, which is more reliable than
// the prevTaskMdc as we might be in a different thread and/or the prevTaskMdc
// can misleadingly point point to the task which triggered the executor
- if (task!=null) {
+ if (task != null) {
prevTaskMdc = MDC.get(LOGGING_MDC_KEY_TASK_ID);
if (Objects.equals(task.getId(), prevTaskMdc)) {
isRedundant = true;
@@ -172,16 +175,20 @@ public class BasicExecutionManager implements ExecutionManager {
entityMdc = MDC.putCloseable(LOGGING_MDC_KEY_ENTITY_IDS, "");
}
- logEvent("Starting task", task, entity);
+ logStartEvent("Starting task", task, entity);
return this;
}
- public static void logEvent(String prefix, Task task, Entity entity) {
+ public static void logStartEvent(String prefix, Task task, Entity entity) {
if (BrooklynLoggingCategories.TASK_LIFECYCLE_LOG.isDebugEnabled() || BrooklynLoggingCategories.TASK_LIFECYCLE_LOG.isTraceEnabled()) {
+ if (entity==null) {
+ entity = BrooklynTaskTags.getTargetOrContextEntity(Tasks.current());
+ }
+
String taskName = task.getDisplayName();
String message = prefix + " " + task.getId() +
- (Strings.isNonBlank(taskName) ? " ("+taskName+")" : "") +
+ (Strings.isNonBlank(taskName) ? " (" + taskName + ")" : "") +
(entity == null ? "" : " on entity " + entity.getId()) +
(Strings.isNonBlank(task.getSubmittedByTaskId()) ? " from task " + task.getSubmittedByTaskId() : "") +
Entitlements.getEntitlementContextUserMaybe().
@@ -197,24 +204,29 @@ public class BasicExecutionManager implements ExecutionManager {
}
}
- public void finish() {
- if (isRedundant) {
- return;
- }
- if (BrooklynLoggingCategories.TASK_LIFECYCLE_LOG.isDebugEnabled() || BrooklynLoggingCategories.TASK_LIFECYCLE_LOG.isTraceEnabled()){
+ public static void logEndEvent(String prefix, Task task) {
+ if (BrooklynLoggingCategories.TASK_LIFECYCLE_LOG.isDebugEnabled() || BrooklynLoggingCategories.TASK_LIFECYCLE_LOG.isTraceEnabled()) {
String taskName = task.getDisplayName();
BrooklynLogging.log(BrooklynLoggingCategories.TASK_LIFECYCLE_LOG,
UNINTERESTING_TASK_NAMES.contains(taskName) ? BrooklynLogging.LoggingLevel.TRACE : BrooklynLogging.LoggingLevel.DEBUG,
- "Ending task {}", task.getId());
+ prefix + " " + task.getId());
+ }
+ }
+
+ public void finish() {
+ if (isRedundant) {
+ return;
}
+ // not normally used for scheduled tasks
+ logEndEvent(task instanceof ScheduledTask ? "Ending scheduled task context" : "Ending task", task);
if (entityMdc != null) {
entityMdc.close();
- if (prevEntityMdc!=null) MDC.put(LOGGING_MDC_KEY_ENTITY_IDS, prevEntityMdc);
+ if (prevEntityMdc != null) MDC.put(LOGGING_MDC_KEY_ENTITY_IDS, prevEntityMdc);
prevEntityMdc = null;
}
if (taskMdc != null) {
taskMdc.close();
- if (prevTaskMdc!=null) MDC.put(LOGGING_MDC_KEY_TASK_ID, prevTaskMdc);
+ if (prevTaskMdc != null) MDC.put(LOGGING_MDC_KEY_TASK_ID, prevTaskMdc);
prevTaskMdc = null;
}
}
@@ -224,131 +236,141 @@ public class BasicExecutionManager implements ExecutionManager {
}
}
- public static void registerUninterestingTaskName(String taskName){
- registerUninterestingTaskName(taskName,false);
+ public static void registerUninterestingTaskName(String taskName) {
+ registerUninterestingTaskName(taskName, false);
}
- public static void registerUninterestingTaskName(String taskName, boolean registerScheduledPrefix){
- log.debug("Registering '{}' as UninterestingTaskName. Starting finishing trace will be log as trace",taskName);
+
+ public static void registerUninterestingTaskName(String taskName, boolean registerScheduledPrefix) {
+ log.debug("Registering '{}' as UninterestingTaskName. Starting finishing trace will be log as trace", taskName);
UNINTERESTING_TASK_NAMES.add(taskName);
- if(registerScheduledPrefix) {
+ if (registerScheduledPrefix) {
UNINTERESTING_TASK_NAMES.add(ScheduledTask.prefixScheduledName(taskName));
}
}
private final ThreadFactory threadFactory;
-
+
private final ThreadFactory daemonThreadFactory;
-
+
private final ExecutorService runner;
-
+
private final ScheduledExecutorService delayedRunner;
// inefficient having so many records, and also doing searches through ...
// many things in here could be more efficient however (different types of lookup etc),
// do that when we need to.
-
+
//access to this field AND to members in this field is synchronized,
//to allow us to preserve order while guaranteeing thread-safe
//(but more testing is needed before we are completely sure it is thread-safe!)
//synch blocks are as finely grained as possible for efficiency;
//NB CopyOnWriteArraySet is a perf bottleneck, and the simple map makes it easier to remove when a tag is empty
- private Map<Object,Set<Task<?>>> tasksByTag = new HashMap<Object,Set<Task<?>>>();
-
- private ConcurrentMap<String,Task<?>> tasksById = new ConcurrentHashMap<String,Task<?>>();
+ private Map<Object, Set<Task<?>>> tasksByTag = new HashMap<Object, Set<Task<?>>>();
+
+ private ConcurrentMap<String, Task<?>> tasksById = new ConcurrentHashMap<String, Task<?>>();
private ConcurrentMap<Object, TaskScheduler> schedulerByTag = new ConcurrentHashMap<Object, TaskScheduler>();
- /** count of all tasks submitted, including finished */
+ /**
+ * count of all tasks submitted, including finished
+ */
private final AtomicLong totalTaskCount = new AtomicLong();
-
- /** tasks submitted but not yet done (or in cases of interruption/cancelled not yet GC'd) */
+
+ /**
+ * tasks submitted but not yet done (or in cases of interruption/cancelled not yet GC'd)
+ */
private Set<String> incompleteTaskIds = Sets.newConcurrentHashSet();
-
- /** tasks started but not yet finished */
+
+ /**
+ * tasks started but not yet finished
+ */
private final AtomicInteger activeTaskCount = new AtomicInteger();
-
+
private final List<ExecutionListener> listeners = new CopyOnWriteArrayList<ExecutionListener>();
-
+
private final static ThreadLocal<String> threadOriginalName = new ThreadLocal<String>() {
@Override
protected String initialValue() {
// should not happen, as only access is in _afterEnd with a check that _beforeStart was invoked
- log.warn("No original name recorded for thread "+Thread.currentThread().getName()+"; task "+Tasks.current());
- return "brooklyn-thread-pool-"+Identifiers.makeRandomId(8);
+ log.warn("No original name recorded for thread " + Thread.currentThread().getName() + "; task " + Tasks.current());
+ return "brooklyn-thread-pool-" + Identifiers.makeRandomId(8);
}
};
-
+
public BasicExecutionManager(String contextid) {
threadFactory = newThreadFactory(contextid);
daemonThreadFactory = new ThreadFactoryBuilder()
.setThreadFactory(threadFactory)
.setDaemon(true)
.build();
-
+
// use Executors.newCachedThreadPool(daemonThreadFactory), but timeout of 1s rather than 60s for better shutdown!
- runner = new ThreadPoolExecutor(0, Integer.MAX_VALUE, 10L, TimeUnit.SECONDS, new SynchronousQueue<Runnable>(),
+ runner = new ThreadPoolExecutor(0, Integer.MAX_VALUE, 10L, TimeUnit.SECONDS, new SynchronousQueue<Runnable>(),
daemonThreadFactory);
-
+
delayedRunner = new ScheduledThreadPoolExecutor(1, daemonThreadFactory);
if (jitterThreads) {
log.info("Task startup jittering enabled with a maximum of " + jitterThreadsMaxDelay + " delay.");
}
}
-
+
private final static class UncaughtExceptionHandlerImplementation implements Thread.UncaughtExceptionHandler {
@Override
public void uncaughtException(Thread t, Throwable e) {
- log.error("Uncaught exception in thread "+t.getName(), e);
+ log.error("Uncaught exception in thread " + t.getName(), e);
}
}
-
- /**
+
+ /**
* For use by overriders to use custom thread factory.
* But be extremely careful: called by constructor, so before sub-class' constructor will
* have been invoked!
*/
protected ThreadFactory newThreadFactory(String contextid) {
return new ThreadFactoryBuilder()
- .setNameFormat("brooklyn-execmanager-"+contextid+"-%d")
+ .setNameFormat("brooklyn-execmanager-" + contextid + "-%d")
.setUncaughtExceptionHandler(new UncaughtExceptionHandlerImplementation())
.build();
}
-
+
public void shutdownNow() {
shutdownNow(null);
}
-
- /** shuts down the executor, and if a duration is supplied awaits termination for that long.
+
+ /**
+ * shuts down the executor, and if a duration is supplied awaits termination for that long.
+ *
* @return whether everything is terminated
*/
@Beta
public boolean shutdownNow(Duration howLongToWaitForTermination) {
runner.shutdownNow();
delayedRunner.shutdownNow();
- if (howLongToWaitForTermination!=null) {
+ if (howLongToWaitForTermination != null) {
CountdownTimer timer = howLongToWaitForTermination.countdownTimer();
try {
runner.awaitTermination(timer.getDurationRemaining().toMilliseconds(), TimeUnit.MILLISECONDS);
- if (timer.isLive()) delayedRunner.awaitTermination(timer.getDurationRemaining().toMilliseconds(), TimeUnit.MILLISECONDS);
+ if (timer.isLive())
+ delayedRunner.awaitTermination(timer.getDurationRemaining().toMilliseconds(), TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
throw Exceptions.propagate(e);
}
}
return runner.isTerminated() && delayedRunner.isTerminated();
}
-
+
public void addListener(ExecutionListener listener) {
listeners.add(listener);
}
-
+
public void removeListener(ExecutionListener listener) {
listeners.remove(listener);
}
-
+
/**
* Deletes the given tag, including all tasks using this tag.
- *
+ * <p>
* Useful, for example, if an entity is being expunged so that we don't keep holding
* a reference to it as a tag.
*/
@@ -367,9 +389,9 @@ public class BasicExecutionManager implements ExecutionManager {
public void deleteTask(Task<?> task) {
boolean removed = deleteTaskNonRecursive(task);
if (!removed) return;
-
+
if (task instanceof HasTaskChildren) {
- List<Task<?>> children = ImmutableList.copyOf(((HasTaskChildren)task).getChildren());
+ List<Task<?>> children = ImmutableList.copyOf(((HasTaskChildren) task).getChildren());
for (Task<?> child : children) {
deleteTask(child);
}
@@ -391,10 +413,10 @@ public class BasicExecutionManager implements ExecutionManager {
}
Task<?> removed = tasksById.remove(task.getId());
incompleteTaskIds.remove(task.getId());
- if (removed!=null && removed.isSubmitted() && !removed.isDone(true)) {
+ if (removed != null && removed.isSubmitted() && !removed.isDone(true)) {
Entity context = BrooklynTaskTags.getContextEntity(removed);
- if (context!=null && !Entities.isManaged(context)) {
- log.debug("Deleting active task on unmanagement of "+context+": "+removed);
+ if (context != null && !Entities.isManaged(context)) {
+ log.debug("Deleting active task on unmanagement of " + context + ": " + removed);
} else {
if (removed.isDone()) {
log.debug("Deleting cancelled task before completion: " + removed + "; this task will continue to run in the background outwith " + this);
@@ -419,23 +441,31 @@ public class BasicExecutionManager implements ExecutionManager {
public boolean isShutdown() {
return runner.isShutdown();
}
-
- /** count of all tasks submitted */
+
+ /**
+ * count of all tasks submitted
+ */
public long getTotalTasksSubmitted() {
return totalTaskCount.get();
}
-
- /** count of tasks submitted but not ended */
+
+ /**
+ * count of tasks submitted but not ended
+ */
public long getNumIncompleteTasks() {
return incompleteTaskIds.size();
}
-
- /** count of tasks started but not ended */
+
+ /**
+ * count of tasks started but not ended
+ */
public long getNumActiveTasks() {
return activeTaskCount.get();
}
- /** count of tasks kept in memory, often including ended tasks */
+ /**
+ * count of tasks kept in memory, often including ended tasks
+ */
public long getNumInMemoryTasks() {
return tasksById.size();
}
@@ -444,7 +474,7 @@ public class BasicExecutionManager implements ExecutionManager {
Preconditions.checkNotNull(tag);
synchronized (tasksByTag) {
Set<Task<?>> result = tasksWithTagLiveOrNull(tag);
- if (result==null) {
+ if (result == null) {
result = Collections.synchronizedSet(new LinkedHashSet<Task<?>>());
tasksByTag.put(tag, result);
}
@@ -452,7 +482,9 @@ public class BasicExecutionManager implements ExecutionManager {
}
}
- /** exposes live view, for internal use only */
+ /**
+ * exposes live view, for internal use only
+ */
@Beta
public Set<Task<?>> tasksWithTagLiveOrNull(Object tag) {
synchronized (tasksByTag) {
@@ -464,8 +496,10 @@ public class BasicExecutionManager implements ExecutionManager {
public Task<?> getTask(String id) {
return tasksById.get(id);
}
-
- /** not on interface because potentially expensive */
+
+ /**
+ * not on interface because potentially expensive
+ */
public List<Task<?>> getAllTasks() {
// not sure if synching makes any difference; have not observed CME's yet
// (and so far this is only called when a CME was caught on a previous operation)
@@ -473,23 +507,23 @@ public class BasicExecutionManager implements ExecutionManager {
return MutableList.copyOf(tasksById.values());
}
}
-
+
@Override
public Set<Task<?>> getTasksWithTag(Object tag) {
Set<Task<?>> result = tasksWithTagLiveOrNull(tag);
- if (result==null) return Collections.emptySet();
+ if (result == null) return Collections.emptySet();
synchronized (result) {
return Collections.unmodifiableSet(new LinkedHashSet<Task<?>>(result));
}
}
-
+
@Override
public Set<Task<?>> getTasksWithAnyTag(Iterable<?> tags) {
Set<Task<?>> result = new LinkedHashSet<Task<?>>();
Iterator<?> ti = tags.iterator();
while (ti.hasNext()) {
Set<Task<?>> tasksForTag = tasksWithTagLiveOrNull(ti.next());
- if (tasksForTag!=null) {
+ if (tasksForTag != null) {
synchronized (tasksForTag) {
result.addAll(tasksForTag);
}
@@ -498,7 +532,9 @@ public class BasicExecutionManager implements ExecutionManager {
return Collections.unmodifiableSet(result);
}
- /** only works with at least one tag; returns empty if no tags */
+ /**
+ * only works with at least one tag; returns empty if no tags
+ */
@Override
public Set<Task<?>> getTasksWithAllTags(Iterable<?> tags) {
//NB: for this method retrieval for multiple tags could be made (much) more efficient (if/when it is used with multiple tags!)
@@ -509,7 +545,7 @@ public class BasicExecutionManager implements ExecutionManager {
Iterator<?> ti = tags.iterator();
while (ti.hasNext()) {
Object tag = ti.next();
- if (first) {
+ if (first) {
first = false;
result.addAll(getTasksWithTag(tag));
} else {
@@ -519,49 +555,83 @@ public class BasicExecutionManager implements ExecutionManager {
return Collections.unmodifiableSet(result);
}
- /** live view of all tasks, for internal use only */
+ /**
+ * live view of all tasks, for internal use only
+ */
@Beta
- public Collection<Task<?>> allTasksLive() { return tasksById.values(); }
-
+ public Collection<Task<?>> allTasksLive() {
+ return tasksById.values();
+ }
+
@Override
- public Set<Object> getTaskTags() {
+ public Set<Object> getTaskTags() {
synchronized (tasksByTag) {
- return Collections.unmodifiableSet(Sets.newLinkedHashSet(tasksByTag.keySet()));
+ return Collections.unmodifiableSet(Sets.newLinkedHashSet(tasksByTag.keySet()));
}
}
- @Override @Deprecated public Task<?> submit(Runnable r) { return submit(new LinkedHashMap<Object,Object>(1), r); }
- @Override public Task<?> submit(String displayName, Runnable r) { return submit(MutableMap.of("displayName", displayName), r); }
- @Override public Task<?> submit(Map<?,?> flags, Runnable r) { return submit(flags, new BasicTask<Void>(flags, r)); }
+ @Override
+ @Deprecated
+ public Task<?> submit(Runnable r) {
+ return submit(new LinkedHashMap<Object, Object>(1), r);
+ }
+
+ @Override
+ public Task<?> submit(String displayName, Runnable r) {
+ return submit(MutableMap.of("displayName", displayName), r);
+ }
+
+ @Override
+ public Task<?> submit(Map<?, ?> flags, Runnable r) {
+ return submit(flags, new BasicTask<Void>(flags, r));
+ }
+
+ @Override
+ @Deprecated
+ public <T> Task<T> submit(Callable<T> c) {
+ return submit(new LinkedHashMap<Object, Object>(1), c);
+ }
+
+ @Override
+ public <T> Task<T> submit(String displayName, Callable<T> c) {
+ return submit(MutableMap.of("displayName", displayName), c);
+ }
- @Override @Deprecated public <T> Task<T> submit(Callable<T> c) { return submit(new LinkedHashMap<Object,Object>(1), c); }
- @Override public <T> Task<T> submit(String displayName, Callable<T> c) { return submit(MutableMap.of("displayName", displayName), c); }
- @Override public <T> Task<T> submit(Map<?,?> flags, Callable<T> c) { return submit(flags, new BasicTask<T>(flags, c)); }
+ @Override
+ public <T> Task<T> submit(Map<?, ?> flags, Callable<T> c) {
+ return submit(flags, new BasicTask<T>(flags, c));
+ }
- @Override public <T> Task<T> submit(TaskAdaptable<T> t) { return submit(new LinkedHashMap<Object,Object>(1), t); }
+ @Override
+ public <T> Task<T> submit(TaskAdaptable<T> t) {
+ return submit(new LinkedHashMap<Object, Object>(1), t);
+ }
@Override
- public <T> Task<T> submit(Map<?,?> flags, TaskAdaptable<T> task) {
+ public <T> Task<T> submit(Map<?, ?> flags, TaskAdaptable<T> task) {
if (!(task instanceof Task))
task = task.asTask();
synchronized (task) {
- if (((TaskInternal<?>)task).getInternalFuture()!=null) return (Task<T>)task;
+ if (((TaskInternal<?>) task).getInternalFuture() != null) return (Task<T>) task;
return submitNewTask(flags, (Task<T>) task);
}
}
- public <T> Task<T> scheduleWith(Task<T> task) { return scheduleWith(Collections.emptyMap(), task); }
- public <T> Task<T> scheduleWith(Map<?,?> flags, Task<T> task) {
+ public <T> Task<T> scheduleWith(Task<T> task) {
+ return scheduleWith(Collections.emptyMap(), task);
+ }
+
+ public <T> Task<T> scheduleWith(Map<?, ?> flags, Task<T> task) {
synchronized (task) {
- if (((TaskInternal<?>)task).getInternalFuture()!=null) return task;
+ if (((TaskInternal<?>) task).getInternalFuture() != null) return task;
return submitNewTask(flags, task);
}
}
- protected Task<?> submitNewScheduledTask(final Map<?,?> flags, final ScheduledTask task) {
+ protected Task<?> submitNewScheduledTask(final Map<?, ?> flags, final ScheduledTask task) {
boolean result = false;
try {
- BrooklynTaskLoggingMdc.logEvent("Submitting scheduled task", task, BrooklynTaskTags.getTargetOrContextEntity(Tasks.current()));
+ BrooklynTaskLoggingMdc.logStartEvent("Submitting scheduled task", task, null);
result = submitSubsequentScheduledTask(flags, task);
} finally {
if (!result) {
@@ -570,11 +640,11 @@ public class BasicExecutionManager implements ExecutionManager {
}
return task;
}
-
- private boolean submitSubsequentScheduledTask(final Map<?,?> flags, final ScheduledTask task) {
+
+ private boolean submitSubsequentScheduledTask(final Map<?, ?> flags, final ScheduledTask task) {
if (!task.isDone()) {
task.internalFuture = delayedRunner.schedule(new ScheduledTaskCallable(task, flags),
- task.delay.toNanoseconds(), TimeUnit.NANOSECONDS);
+ task.delay.toNanoseconds(), TimeUnit.NANOSECONDS);
return true;
} else {
return false;
@@ -583,7 +653,7 @@ public class BasicExecutionManager implements ExecutionManager {
protected class ScheduledTaskCallable implements Callable<Object> {
public ScheduledTask task;
- public Map<?,?> flags;
+ public Map<?, ?> flags;
public ScheduledTaskCallable(ScheduledTask task, Map<?, ?> flags) {
this.task = task;
@@ -591,12 +661,12 @@ public class BasicExecutionManager implements ExecutionManager {
}
@Override
- @SuppressWarnings({ "rawtypes", "unchecked" })
+ @SuppressWarnings({"rawtypes", "unchecked"})
public Object call() {
TaskInternal<?> taskIteration = null;
Throwable error = null;
try {
- if (task.startTimeUtc==-1) {
+ if (task.startTimeUtc == -1) {
beforeSubmitScheduledTaskAllIterations(flags, task);
beforeStartScheduledTaskAllIterations(flags, task);
@@ -610,57 +680,60 @@ public class BasicExecutionManager implements ExecutionManager {
final Callable<?> oldJob = taskIteration.getJob();
final TaskInternal<?> taskIterationF = taskIteration;
- taskIteration.setJob(new Callable() { @Override public Object call() {
- if (task.isCancelled()) {
- CancellationException cancelDetected = new CancellationException("cancel detected");
- try {
- afterEndScheduledTaskSubmissionIteration(flags, task, taskIterationF, cancelDetected);
- } finally {
- // do in finally block so runs even if above throws cancelDetected
- afterEndScheduledTaskAllIterations(flags, task, cancelDetected);
- }
- throw cancelDetected;
- }
- Throwable lastError = null;
- boolean shouldResubmit = true;
- task.recentRun = taskIterationF;
- try (BrooklynTaskLoggingMdc mdc = BrooklynTaskLoggingMdc.create(taskIterationF).start()) {
- beforeStartScheduledTaskSubmissionIteration(flags, task, taskIterationF);
- synchronized (task) {
- task.notifyAll();
- }
- Object result;
- try {
- result = oldJob.call();
- task.lastThrownType = null;
- } catch (Exception e) {
- lastError = e;
- shouldResubmit = shouldResubmitOnException(oldJob, e);
- throw Exceptions.propagate(e);
+ taskIteration.setJob(new Callable() {
+ @Override
+ public Object call() {
+ if (task.isCancelled()) {
+ CancellationException cancelDetected = new CancellationException("cancel detected in scheduled job for " + task);
+ try {
+ afterEndScheduledTaskSubmissionIteration(flags, task, taskIterationF, cancelDetected);
+ } finally {
+ // do in finally block so runs even if above throws cancelDetected
+ afterEndScheduledTaskAllIterations(flags, task, cancelDetected);
+ }
+ throw cancelDetected;
}
- return result;
- } finally {
- if (!task.isCancelled() || task.getEndTimeUtc()<=0) {
- // don't re-run on cancellation
-
- afterEndScheduledTaskSubmissionIteration(flags, task, taskIterationF, lastError);
- // do in finally block in case we were interrupted
- if (shouldResubmit && resubmit()) {
- // resubmitted fine, no-op
- } else {
- // not resubmitted, note ending
- afterEndScheduledTaskAllIterations(flags, task, lastError);
+ Throwable lastError = null;
+ boolean shouldResubmit = true;
+ task.recentRun = taskIterationF;
+ try (BrooklynTaskLoggingMdc mdc = BrooklynTaskLoggingMdc.create(taskIterationF).start()) {
+ beforeStartScheduledTaskSubmissionIteration(flags, task, taskIterationF);
+ synchronized (task) {
+ task.notifyAll();
+ }
+ Object result;
+ try {
+ result = oldJob.call();
+ task.lastThrownType = null;
+ } catch (Exception e) {
+ lastError = e;
+ shouldResubmit = shouldResubmitOnException(oldJob, e);
+ throw Exceptions.propagate(e);
+ }
+ return result;
+ } finally {
+ if (!task.isCancelled() || task.getEndTimeUtc() <= 0) {
+ // don't re-run on cancellation
+
+ afterEndScheduledTaskSubmissionIteration(flags, task, taskIterationF, lastError);
+ // do in finally block in case we were interrupted
+ if (shouldResubmit && resubmit()) {
+ // resubmitted fine, no-op
+ } else {
+ // not resubmitted, note ending
+ afterEndScheduledTaskAllIterations(flags, task, lastError);
+ }
}
}
}
- }});
+ });
task.nextRun = taskIteration;
ExecutionContext ec =
// no longer associated the execution context on each execution;
// BasicExecutionContext.getCurrentExecutionContext();
// instead it is set on the task
task.executionContext;
- if (ec!=null) return ec.submit(taskIteration);
+ if (ec != null) return ec.submit(taskIteration);
else return submit(taskIteration);
} catch (Exception e) {
@@ -672,7 +745,7 @@ public class BasicExecutionManager implements ExecutionManager {
private boolean resubmit() {
task.runCount++;
- if (task.period!=null && !task.isCancelled()) {
+ if (task.period != null && !task.isCancelled()) {
task.delay = task.period;
return submitSubsequentScheduledTask(flags, task);
} else {
@@ -704,7 +777,7 @@ public class BasicExecutionManager implements ExecutionManager {
@Override
public String toString() {
- return "ScheduledTaskCallable["+task+","+flags+"]";
+ return "ScheduledTaskCallable[" + task + "," + flags + "]";
}
}
@@ -727,8 +800,8 @@ public class BasicExecutionManager implements ExecutionManager {
afterEndForCancelBeforeStart(flags, task, false);
throw new CancellationException();
}
- result = ((TaskInternal<T>)task).getJob().call();
- } catch(Throwable e) {
+ result = ((TaskInternal<T>) task).getJob().call();
+ } catch (Throwable e) {
error = e;
} finally {
afterEndAtomicTask(flags, task, error);
@@ -738,7 +811,7 @@ public class BasicExecutionManager implements ExecutionManager {
@Override
public String toString() {
- return "BEM.call("+task+","+flags+")";
+ return "BEM.call(" + task + "," + flags + ")";
}
}
@@ -746,7 +819,7 @@ public class BasicExecutionManager implements ExecutionManager {
private final Task<T> task;
private BasicExecutionManager execMgmt;
private final ExecutionList listeners;
-
+
private CancellingListenableForwardingFutureForTask(BasicExecutionManager execMgmt, Future<T> delegate, ExecutionList list, Task<T> task) {
super(delegate);
this.listeners = list;
@@ -767,48 +840,48 @@ public class BasicExecutionManager implements ExecutionManager {
public ExecutionList getListeners() {
return listeners;
}
-
+
@Override
public boolean cancel(TaskCancellationMode mode) {
boolean result = false;
if (log.isTraceEnabled()) {
- log.trace("CLFFT cancelling "+task+" mode "+mode);
+ log.trace("CLFFT cancelling " + task + " mode " + mode);
}
- if (!task.isCancelled()) result |= ((TaskInternal<T>)task).cancel(mode);
+ if (!task.isCancelled()) result |= ((TaskInternal<T>) task).cancel(mode);
result |= delegate().cancel(mode.isAllowedToInterruptTask());
-
+
if (mode.isAllowedToInterruptDependentSubmittedTasks()) {
- int subtasksFound=0;
- int subtasksReallyCancelled=0;
-
+ int subtasksFound = 0;
+ int subtasksReallyCancelled = 0;
+
if (task instanceof HasTaskChildren) {
// cancel tasks in reverse order --
// it should be the case that if child1 is cancelled,
// a parentTask should NOT call a subsequent child2,
// but just in case, we cancel child2 first
// NB: DST and others may apply their own recursive cancel behaviour
- MutableList<Task<?>> childrenReversed = MutableList.copyOf( ((HasTaskChildren)task).getChildren() );
+ MutableList<Task<?>> childrenReversed = MutableList.copyOf(((HasTaskChildren) task).getChildren());
Collections.reverse(childrenReversed);
-
- for (Task<?> child: childrenReversed) {
+
+ for (Task<?> child : childrenReversed) {
if (log.isTraceEnabled()) {
- log.trace("Cancelling "+child+" on recursive cancellation of "+task);
+ log.trace("Cancelling " + child + " on recursive cancellation of " + task);
}
subtasksFound++;
- if (((TaskInternal<?>)child).cancel(mode)) {
+ if (((TaskInternal<?>) child).cancel(mode)) {
result = true;
subtasksReallyCancelled++;
}
}
}
- for (Task<?> t: execMgmt.getAllTasks()) {
+ for (Task<?> t : execMgmt.getAllTasks()) {
if (task.equals(t.getSubmittedByTask())) {
if (mode.isAllowedToInterruptAllSubmittedTasks() || BrooklynTaskTags.isTransient(t)) {
if (log.isTraceEnabled()) {
- log.trace("Cancelling "+t+" on recursive cancellation of "+task);
+ log.trace("Cancelling " + t + " on recursive cancellation of " + task);
}
subtasksFound++;
- if (((TaskInternal<?>)t).cancel(mode)) {
+ if (((TaskInternal<?>) t).cancel(mode)) {
result = true;
subtasksReallyCancelled++;
}
@@ -816,10 +889,10 @@ public class BasicExecutionManager implements ExecutionManager {
}
}
if (log.isTraceEnabled()) {
- log.trace("On cancel of "+task+", applicable subtask count "+subtasksFound+", of which "+subtasksReallyCancelled+" were actively cancelled");
+ log.trace("On cancel of " + task + ", applicable subtask count " + subtasksFound + ", of which " + subtasksReallyCancelled + " were actively cancelled");
}
}
-
+
execMgmt.afterEndForCancelBeforeStart(null, task, true);
return result;
}
@@ -841,7 +914,7 @@ public class BasicExecutionManager implements ExecutionManager {
try {
listener.onTaskDone(task);
} catch (Exception e) {
- log.warn("Error running execution listener "+listener+" of task "+task+" done", e);
+ log.warn("Error running execution listener " + listener + " of task " + task + " done", e);
}
}
// run any listeners the task owner has added to its future
@@ -850,46 +923,47 @@ public class BasicExecutionManager implements ExecutionManager {
}
@SuppressWarnings("unchecked")
- protected <T> Task<T> submitNewTask(final Map<?,?> flags, final Task<T> task) {
+ protected <T> Task<T> submitNewTask(final Map<?, ?> flags, final Task<T> task) {
if (log.isTraceEnabled()) {
- log.trace("Submitting task {} ({}), with flags {}, and tags {}, job {}; caller {}",
- new Object[] {task.getId(), task, Sanitizer.sanitize(flags), BrooklynTaskTags.getTagsFast(task),
- (task instanceof TaskInternal ? ((TaskInternal<T>)task).getJob() : "<unavailable>"),
- Tasks.current() });
- if (Tasks.current()==null && BrooklynTaskTags.isTransient(task)) {
- log.trace("Stack trace for unparented submission of transient "+task, new Throwable("trace only (not an error)"));
+ log.trace("Submitting task {} ({}), with flags {}, and tags {}, job {}; caller {}",
+ new Object[]{task.getId(), task, Sanitizer.sanitize(flags), BrooklynTaskTags.getTagsFast(task),
+ (task instanceof TaskInternal ? ((TaskInternal<T>) task).getJob() : "<unavailable>"),
+ Tasks.current()});
+ if (Tasks.current() == null && BrooklynTaskTags.isTransient(task)) {
+ log.trace("Stack trace for unparented submission of transient " + task, new Throwable("trace only (not an error)"));
}
}
-
+
if (task instanceof ScheduledTask) {
return (Task<T>) submitNewScheduledTask(flags, (ScheduledTask) task);
}
-
+
beforeSubmitAtomicTask(flags, task);
-
- if (((TaskInternal<T>)task).getJob() == null)
- throw new NullPointerException("Task "+task+" submitted with with null job: job must be supplied.");
-
+
+ if (((TaskInternal<T>) task).getJob() == null)
+ throw new NullPointerException("Task " + task + " submitted with with null job: job must be supplied.");
+
Callable<T> job = new SubmissionCallable<T>(flags, task);
-
+
// If there's a scheduler then use that; otherwise execute it directly
Set<TaskScheduler> schedulers = null;
- for (Object tago: BrooklynTaskTags.getTagsFast(task)) {
+ for (Object tago : BrooklynTaskTags.getTagsFast(task)) {
TaskScheduler scheduler = getTaskSchedulerForTag(tago);
- if (scheduler!=null) {
- if (schedulers==null) schedulers = new LinkedHashSet<TaskScheduler>(2);
+ if (scheduler != null) {
+ if (schedulers == null) schedulers = new LinkedHashSet<TaskScheduler>(2);
schedulers.add(scheduler);
}
}
Future<T> future;
- if (schedulers!=null && !schedulers.isEmpty()) {
- if (schedulers.size()>1) log.warn("multiple schedulers detected, using only the first, for "+task+": "+schedulers);
+ if (schedulers != null && !schedulers.isEmpty()) {
+ if (schedulers.size() > 1)
+ log.warn("multiple schedulers detected, using only the first, for " + task + ": " + schedulers);
future = schedulers.iterator().next().submit(job);
} else {
future = runner.submit(job);
}
afterSubmitRecordFuture(task, future);
-
+
return task;
}
@@ -898,117 +972,136 @@ public class BasicExecutionManager implements ExecutionManager {
// this future allows a caller to add custom listeners
// (it does not notify the listeners; that's our job);
// except on cancel we want to listen
- CancellingListenableForwardingFutureForTask<T> listenableFuture = new CancellingListenableForwardingFutureForTask<T>(this, future, ((TaskInternal<T>)task).getListeners(), task);
+ CancellingListenableForwardingFutureForTask<T> listenableFuture = new CancellingListenableForwardingFutureForTask<T>(this, future, ((TaskInternal<T>) task).getListeners(), task);
// and we want to make sure *our* (manager) listeners are given suitable callback
- ((TaskInternal<T>)task).addListener(new SubmissionListenerToCallManagerListeners<T>(task, listenableFuture), runner);
+ ((TaskInternal<T>) task).addListener(new SubmissionListenerToCallManagerListeners<T>(task, listenableFuture), runner);
// NB: can the above mean multiple callbacks to TaskInternal#runListeners?
-
+
// finally expose the future to callers
- ((TaskInternal<T>)task).initInternalFuture(listenableFuture);
+ ((TaskInternal<T>) task).initInternalFuture(listenableFuture);
}
-
- protected void beforeSubmitScheduledTaskAllIterations(Map<?,?> flags, Task<?> task) {
+
+ protected void beforeSubmitScheduledTaskAllIterations(Map<?, ?> flags, Task<?> task) {
// for these, beforeSubmitAtomicTask is not called,
// but beforeStartAtomic and afterSubmitAtomic _are_ called
internalBeforeSubmit(flags, task);
}
- protected void beforeSubmitScheduledTaskSubmissionIteration(Map<?,?> flags, Task<?> task) {
+
+ protected void beforeSubmitScheduledTaskSubmissionIteration(Map<?, ?> flags, Task<?> task) {
internalBeforeSubmit(flags, task);
}
- protected void beforeSubmitAtomicTask(Map<?,?> flags, Task<?> task) {
+
+ protected void beforeSubmitAtomicTask(Map<?, ?> flags, Task<?> task) {
internalBeforeSubmit(flags, task);
}
- protected void beforeSubmitInSameThreadTask(Map<?,?> flags, Task<?> task) {
+
+ protected void beforeSubmitInSameThreadTask(Map<?, ?> flags, Task<?> task) {
internalBeforeSubmit(flags, task);
}
- /** invoked when a task is submitted */
- protected void internalBeforeSubmit(Map<?,?> flags, Task<?> task) {
+
+ /**
+ * invoked when a task is submitted
+ */
+ protected void internalBeforeSubmit(Map<?, ?> flags, Task<?> task) {
incompleteTaskIds.add(task.getId());
-
- if (task.getSubmittedByTaskId()==null) {
+
+ if (task.getSubmittedByTaskId() == null) {
Task<?> currentTask = Tasks.current();
- if (currentTask!=null) ((TaskInternal<?>)task).setSubmittedByTask(
+ if (currentTask != null) ((TaskInternal<?>) task).setSubmittedByTask(
// do this instead of soft reference (2017-09) as soft refs impact GC
Maybe.of(new TaskLookup(this, currentTask)),
currentTask.getId());
}
- ((TaskInternal<?>)task).setSubmitTimeUtc(System.currentTimeMillis());
-
- if (flags!=null && flags.get("tag")!=null) ((TaskInternal<?>)task).getMutableTags().add(flags.remove("tag"));
- if (flags!=null && flags.get("tags")!=null) ((TaskInternal<?>)task).getMutableTags().addAll((Collection<?>)flags.remove("tags"));
+ ((TaskInternal<?>) task).setSubmitTimeUtc(System.currentTimeMillis());
- for (Object tag: BrooklynTaskTags.getTagsFast(task)) {
+ if (flags != null && flags.get("tag") != null)
+ ((TaskInternal<?>) task).getMutableTags().add(flags.remove("tag"));
+ if (flags != null && flags.get("tags") != null)
+ ((TaskInternal<?>) task).getMutableTags().addAll((Collection<?>) flags.remove("tags"));
+
+ for (Object tag : BrooklynTaskTags.getTagsFast(task)) {
tasksWithTagCreating(tag).add(task);
}
-
+
tasksById.put(task.getId(), task);
totalTaskCount.incrementAndGet();
}
-
+
private static class TaskLookup implements Supplier<Task<?>> {
// this class is not meant to be serialized, but if it is, make sure exec mgr doesn't sneak in
transient BasicExecutionManager mgr;
-
+
String id;
String displayName;
+
public TaskLookup(BasicExecutionManager mgr, Task<?> t) {
this.mgr = mgr;
id = t.getId();
- if (mgr.getTask(id)==null) {
- log.warn("Created task lookup for task which is not registered: "+t);
+ if (mgr.getTask(id) == null) {
+ log.warn("Created task lookup for task which is not registered: " + t);
}
displayName = t.getDisplayName();
}
+
@Override
public Task<?> get() {
- if (mgr==null) return gone();
+ if (mgr == null) return gone();
Task<?> result = mgr.getTask(id);
- if (result!=null) return result;
+ if (result != null) return result;
return gone();
}
+
private <T> Task<T> gone() {
- Task<T> t = Tasks.<T>builder().dynamic(false).displayName(displayName+" (placeholder for "+id+")")
- .description("Details of the original task have been forgotten.")
- .body(Callables.returning((T)null)).build();
+ Task<T> t = Tasks.<T>builder().dynamic(false).displayName(displayName + " (placeholder for " + id + ")")
+ .description("Details of the original task have been forgotten.")
+ .body(Callables.returning((T) null)).build();
// don't really want anyone executing the "gone" task...
// also if we are GC'ing tasks then cancelled may help with cleanup
// of sub-tasks that have lost their submitted-by-task reference ?
// also don't want warnings when it's finalized, this means we don't need ignoreIfNotRun()
- ((BasicTask<T>)t).cancelled = true;
+ ((BasicTask<T>) t).cancelled = true;
return t;
}
}
- /** normally (if not interrupted) called once for each call to {@link #beforeSubmitScheduledTaskAllIterations(Map, Task)} */
- protected void beforeStartScheduledTaskAllIterations(Map<?,?> flags, ScheduledTask taskDoingTheInitialSchedule) {
- taskDoingTheInitialSchedule.mdc = BrooklynTaskLoggingMdc.create(taskDoingTheInitialSchedule).start();
+ /**
+ * normally (if not interrupted) called once for each call to {@link #beforeSubmitScheduledTaskAllIterations(Map, Task)}
+ */
+ protected void beforeStartScheduledTaskAllIterations(Map<?, ?> flags, ScheduledTask taskDoingTheInitialSchedule) {
+ BrooklynTaskLoggingMdc.logStartEvent("Starting scheduled task iterations", taskDoingTheInitialSchedule, null);
internalBeforeStart(flags, taskDoingTheInitialSchedule, !SCHEDULED_TASKS_COUNT_AS_ACTIVE, true, true);
}
- protected void beforeStartScheduledTaskSubmissionIteration(Map<?,?> flags, Task<?> taskDoingTheScheduling, Task<?> taskIteration) {
+
+ protected void beforeStartScheduledTaskSubmissionIteration(Map<?, ?> flags, Task<?> taskDoingTheScheduling, Task<?> taskIteration) {
// no-op, because handled as an atomic task
// internalBeforeStart(flags, taskIteration, true, false);
}
- protected void beforeStartAtomicTask(Map<?,?> flags, Task<?> task) {
+
+ protected void beforeStartAtomicTask(Map<?, ?> flags, Task<?> task) {
internalBeforeStart(flags, task, false, true, false);
}
- protected void beforeStartInSameThreadTask(Map<?,?> flags, Task<?> task) {
+
+ protected void beforeStartInSameThreadTask(Map<?, ?> flags, Task<?> task) {
internalBeforeStart(flags, task, false, false, false);
}
-
- /** invoked in a task's thread when a task is starting to run (may be some time after submitted),
- * but before doing any of the task's work, so that we can update bookkeeping and notify callbacks */
- protected void internalBeforeStart(Map<?,?> flags, Task<?> task, boolean skipIncrementCounter, boolean allowJitter, boolean startingThisThreadMightEndElsewhere) {
+
+ /**
+ * invoked in a task's thread when a task is starting to run (may be some time after submitted),
+ * but before doing any of the task's work, so that we can update bookkeeping and notify callbacks
+ */
+ protected void internalBeforeStart(Map<?, ?> flags, Task<?> task, boolean skipIncrementCounter, boolean allowJitter, boolean startingThisThreadMightEndElsewhere) {
int count = skipIncrementCounter ? activeTaskCount.get() : activeTaskCount.incrementAndGet();
- if (count % 1000==0 && count>0) {
- log.warn("High number of active tasks: task #"+count+" is "+task);
+ if (count % 1000 == 0 && count > 0) {
+ log.warn("High number of active tasks: task #" + count + " is " + task);
}
-
+
//set thread _before_ start time, so we won't get a null thread when there is a start-time
- if (log.isTraceEnabled()) log.trace(""+this+" beforeStart, task: "+task + " running on thread " + Thread.currentThread().getName());
+ if (log.isTraceEnabled())
+ log.trace("" + this + " beforeStart, task: " + task + " running on thread " + Thread.currentThread().getName());
if (!task.isCancelled()) {
Thread thread = Thread.currentThread();
- ((TaskInternal<?>)task).setThread(thread);
+ ((TaskInternal<?>) task).setThread(thread);
if (!startingThisThreadMightEndElsewhere) {
if (RENAME_THREADS) {
threadOriginalName.set(thread.getName());
@@ -1017,13 +1110,13 @@ public class BasicExecutionManager implements ExecutionManager {
}
PerThreadCurrentTaskHolder.perThreadCurrentTask.set(task);
}
- ((TaskInternal<?>)task).setStartTimeUtc(System.currentTimeMillis());
+ ((TaskInternal<?>) task).setStartTimeUtc(System.currentTimeMillis());
}
if (allowJitter) {
jitterThreadStart(task);
}
- if (flags!=null && !startingThisThreadMightEndElsewhere) {
+ if (flags != null && !startingThisThreadMightEndElsewhere) {
invokeCallback(flags.get("newTaskStartCallback"), task);
}
}
@@ -1039,7 +1132,7 @@ public class BasicExecutionManager implements ExecutionManager {
}
}
- @SuppressWarnings({ "unchecked", "rawtypes" })
+ @SuppressWarnings({"unchecked", "rawtypes"})
// not ideal, such loose typing on the callback -- should prefer Function<Task,Object>
// but at least it's package-private
static Object invokeCallback(Object callable, Task<?> task) {
@@ -1048,60 +1141,75 @@ public class BasicExecutionManager implements ExecutionManager {
log.warn("Use of groovy.lang.Closure is deprecated, in ExecutionManager.invokeCallback");
loggedClosureDeprecatedInInvokeCallback = true;
}
- return ((Closure<?>)callable).call(task);
+ return ((Closure<?>) callable).call(task);
}
if (callable instanceof Callable) {
try {
- return ((Callable<?>)callable).call();
+ return ((Callable<?>) callable).call();
} catch (Throwable t) {
throw Exceptions.propagate(t);
}
}
- if (callable instanceof Runnable) { ((Runnable)callable).run(); return null; }
- if (callable instanceof Function) { return ((Function)callable).apply(task); }
- if (callable==null) return null;
- throw new IllegalArgumentException("Cannot invoke unexpected callback object "+callable+" of type "+callable.getClass()+" on "+task);
+ if (callable instanceof Runnable) {
+ ((Runnable) callable).run();
+ return null;
+ }
+ if (callable instanceof Function) {
+ return ((Function) callable).apply(task);
+ }
+ if (callable == null) return null;
+ throw new IllegalArgumentException("Cannot invoke unexpected callback object " + callable + " of type " + callable.getClass() + " on " + task);
}
+
private static boolean loggedClosureDeprecatedInInvokeCallback;
-
- /** normally (if not interrupted) called once for each call to {@link #beforeStartScheduledTaskAllIterations(Map, ScheduledTask)} */
- protected void afterEndScheduledTaskAllIterations(Map<?,?> flags, ScheduledTask taskDoingTheInitialSchedule, Throwable error) {
+
+ /**
+ * normally (if not interrupted) called once for each call to {@link #beforeStartScheduledTaskAllIterations(Map, ScheduledTask)}
+ */
+ protected void afterEndScheduledTaskAllIterations(Map<?, ?> flags, ScheduledTask taskDoingTheInitialSchedule, Throwable error) {
boolean taskWasSubmittedAndNotYetEnded = true;
try {
taskWasSubmittedAndNotYetEnded = internalAfterEnd(flags, taskDoingTheInitialSchedule, !SCHEDULED_TASKS_COUNT_AS_ACTIVE, false, error);
} finally {
- if (taskDoingTheInitialSchedule.mdc!=null) {
- taskDoingTheInitialSchedule.mdc.close();
- taskDoingTheInitialSchedule.mdc = null;
+ BrooklynTaskLoggingMdc.logEndEvent("Ending scheduled task iterations", taskDoingTheInitialSchedule);
+ synchronized (taskDoingTheInitialSchedule) {
+ taskDoingTheInitialSchedule.notifyAll();
}
- synchronized (taskDoingTheInitialSchedule) { taskDoingTheInitialSchedule.notifyAll(); }
if (taskWasSubmittedAndNotYetEnded) {
// prevent from running twice on cancellation after start
((TaskInternal<?>) taskDoingTheInitialSchedule).runListeners();
}
}
}
- /** called once for each call to {@link #beforeStartScheduledTaskSubmissionIteration(Map, Task, Task)},
- * with a per-iteration task generated by the surrounding scheduled task */
- protected void afterEndScheduledTaskSubmissionIteration(Map<?,?> flags, Task<?> taskDoingTheInitialSchedule, Task<?> taskIteration, Throwable error) {
+
+ /**
+ * called once for each call to {@link #beforeStartScheduledTaskSubmissionIteration(Map, Task, Task)},
+ * with a per-iteration task generated by the surrounding scheduled task
+ */
+ protected void afterEndScheduledTaskSubmissionIteration(Map<?, ?> flags, Task<?> taskDoingTheInitialSchedule, Task<?> taskIteration, Throwable error) {
// no-op because handled as an atomic task
// internalAfterEnd(flags, taskIteration, false, true, error);
}
- /** called once for each task on which {@link #beforeStartAtomicTask(Map, Task)} is invoked,
- * and normally (if not interrupted prior to start)
- * called once for each task on which {@link #beforeSubmitAtomicTask(Map, Task)} */
- protected void afterEndAtomicTask(Map<?,?> flags, Task<?> task, Throwable error) {
+
+ /**
+ * called once for each task on which {@link #beforeStartAtomicTask(Map, Task)} is invoked,
+ * and normally (if not interrupted prior to start)
+ * called once for each task on which {@link #beforeSubmitAtomicTask(Map, Task)}
+ */
+ protected void afterEndAtomicTask(Map<?, ?> flags, Task<?> task, Throwable error) {
internalAfterEnd(flags, task, false, true, error);
}
- protected void afterEndInSameThreadTask(Map<?,?> flags, Task<?> task, Throwable error) {
+
+ protected void afterEndInSameThreadTask(Map<?, ?> flags, Task<?> task, Throwable error) {
internalAfterEnd(flags, task, false, true, error);
}
- protected void afterEndForCancelBeforeStart(Map<?,?> flags, Task<?> task, boolean calledFromCanceller) {
+
+ protected void afterEndForCancelBeforeStart(Map<?, ?> flags, Task<?> task, boolean calledFromCanceller) {
if (calledFromCanceller) {
if (task.isBegun()) {
// do nothing from canceller thread if task has begin;
// we don't want to set end time or listeners prematurely.
- return ;
+ return;
} else {
// normally task won't be submitted by executor, so do some of the end operations.
// there is a chance task has begun but not set start time,
@@ -1113,15 +1221,17 @@ public class BasicExecutionManager implements ExecutionManager {
}
internalAfterEnd(flags, task, true, !calledFromCanceller, null);
}
-
- /** normally (if not interrupted) called once for each call to {@link #internalBeforeSubmit(Map, Task)},
- * and, for atomic tasks and scheduled-task submission iterations where
+
+ /**
+ * normally (if not interrupted) called once for each call to {@link #internalBeforeSubmit(Map, Task)},
+ * and, for atomic tasks and scheduled-task submission iterations where
* always called once if {@link #internalBeforeStart(Map, Task, boolean, boolean, boolean)} is invoked and if possible
- * (but not possible for isEndingAllIterations) in the same thread as that method */
- protected boolean internalAfterEnd(Map<?,?> flags, Task<?> task, boolean skipDecrementCounter, boolean startedGuaranteedToEndInSameThreadAndEndingSameThread, Throwable error) {
+ * (but not possible for isEndingAllIterations) in the same thread as that method
+ */
+ protected boolean internalAfterEnd(Map<?, ?> flags, Task<?> task, boolean skipDecrementCounter, boolean startedGuaranteedToEndInSameThreadAndEndingSameThread, Throwable error) {
boolean taskWasSubmittedAndNotYetEnded = true;
try {
- if (log.isTraceEnabled()) log.trace(this+" afterEnd, task: "+task);
+ if (log.isTraceEnabled()) log.trace(this + " afterEnd, task: " + task);
taskWasSubmittedAndNotYetEnded = incompleteTaskIds.remove(task.getId());
// this method might be called more than once, eg if cancelled, so use the above as a guard where single invocation is needed (eg counts)
@@ -1129,13 +1239,13 @@ public class BasicExecutionManager implements ExecutionManager {
activeTaskCount.decrementAndGet();
}
- if (flags!=null && taskWasSubmittedAndNotYetEnded && startedGuaranteedToEndInSameThreadAndEndingSameThread) {
+ if (flags != null && taskWasSubmittedAndNotYetEnded && startedGuaranteedToEndInSameThreadAndEndingSameThread) {
invokeCallback(flags.get("newTaskEndCallback"), task);
}
- if (task.getEndTimeUtc()>0) {
+ if (task.getEndTimeUtc() > 0) {
if (taskWasSubmittedAndNotYetEnded) {
// shouldn't happen
- log.debug("Task "+task+" has end time "+task.getEndTimeUtc()+" but was marked as incomplete");
+ log.debug("Task " + task + " has end time " + task.getEndTimeUtc() + " but was marked as incomplete");
}
} else {
((TaskInternal<?>) task).setEndTimeUtc(System.currentTimeMillis());
@@ -1146,7 +1256,7 @@ public class BasicExecutionManager implements ExecutionManager {
//clear thread _after_ endTime set, so we won't get a null thread when there is no end-time
if (RENAME_THREADS) {
Thread thread = task.getThread();
- if (thread==null) {
+ if (thread == null) {
log.warn("BasicTask.afterEnd invoked without corresponding beforeStart");
} else {
thread.setName(threadOriginalName.get());
@@ -1154,7 +1264,7 @@ public class BasicExecutionManager implements ExecutionManager {
}
}
}
- ((TaskInternal<?>)task).setThread(null);
+ ((TaskInternal<?>) task).setThread(null);
// if uninteresting and transient and scheduled, go ahead and remove from task tags also, so it won't be reported as GC'd
if (BrooklynTaskTags.isTransient(task) && UNINTERESTING_TASK_NAMES.contains(task.getDisplayName()) && task.getSubmittedByTask() instanceof ScheduledTask) {
@@ -1163,7 +1273,7 @@ public class BasicExecutionManager implements ExecutionManager {
} finally {
try {
- if (error!=null) {
+ if (error != null) {
/* we throw, after logging debug.
* the throw means the error is available for task submitters to monitor.
* however it is possible no one is monitoring it, in which case we will have debug logging only for errors.
@@ -1174,26 +1284,28 @@ public class BasicExecutionManager implements ExecutionManager {
if (log.isDebugEnabled()) {
// debug only here, because most submitters will handle failures
if (error instanceof InterruptedException || error instanceof RuntimeInterruptedException) {
- log.debug("Detected interruption on task "+task+" (rethrowing)" +
- (Strings.isNonBlank(error.getMessage()) ? ": "+error.getMessage() : ""));
+ log.debug("Detected interruption on task " + task + " (rethrowing)" +
+ (Strings.isNonBlank(error.getMessage()) ? ": " + error.getMessage() : ""));
} else if (error instanceof NullPointerException ||
error instanceof IndexOutOfBoundsException ||
error instanceof ClassCastException) {
- log.debug("Exception running task "+task+" (rethrowing): "+error, error);
+ log.debug("Exception running task " + task + " (rethrowing): " + error, error);
} else {
- log.debug("Exception running task "+task+" (rethrowing): "+error);
+ log.debug("Exception running task " + task + " (rethrowing): " + error);
}
if (log.isTraceEnabled()) {
- log.trace("Trace for exception running task "+task+" (rethrowing): "+error, error);
+ log.trace("Trace for exception running task " + task + " (rethrowing): " + error, error);
}
}
throw Exceptions.propagate(error);
}
} finally {
- synchronized (task) { task.notifyAll(); }
+ synchronized (task) {
+ task.notifyAll();
+ }
if (taskWasSubmittedAndNotYetEnded) {
// prevent from running twice on cancellation after start
- ((TaskInternal<?>)task).runListeners();
+ ((TaskInternal<?>) task).runListeners();
}
}
}
@@ -1203,22 +1315,22 @@ public class BasicExecutionManager implements ExecutionManager {
public TaskScheduler getTaskSchedulerForTag(Object tag) {
return schedulerByTag.get(tag);
}
-
+
public void setTaskSchedulerForTag(Object tag, Class<? extends TaskScheduler> scheduler) {
synchronized (schedulerByTag) {
TaskScheduler old = getTaskSchedulerForTag(tag);
- if (old!=null) {
+ if (old != null) {
if (scheduler.isAssignableFrom(old.getClass())) {
/* already have such an instance */
return;
}
//might support multiple in future...
- throw new IllegalStateException("Not allowed to set multiple TaskSchedulers on ExecutionManager tag (tag "+tag+", has "+old+", setting new "+scheduler+")");
+ throw new IllegalStateException("Not allowed to set multiple TaskSchedulers on ExecutionManager tag (tag " + tag + ", has " + old + ", setting new " + scheduler + ")");
}
try {
TaskScheduler schedulerI = scheduler.newInstance();
// allow scheduler to have a nice name, for logging etc
- if (schedulerI instanceof CanSetName) ((CanSetName)schedulerI).setName(""+tag);
+ if (schedulerI instanceof CanSetName) ((CanSetName) schedulerI).setName("" + tag);
setTaskSchedulerForTag(tag, schedulerI);
} catch (InstantiationException e) {
throw Exceptions.propagate(e);
@@ -1227,10 +1339,10 @@ public class BasicExecutionManager implements ExecutionManager {
}
}
}
-
+
/**
* Defines a {@link TaskScheduler} to run on all subsequently submitted jobs with the given tag.
- *
+ * <p>
* Maximum of one allowed currently. Resubmissions of the same scheduler (or scheduler class)
* allowed. If changing, you must call {@link #clearTaskSchedulerForTag(Object)} between the two.
*
@@ -1241,9 +1353,9 @@ public class BasicExecutionManager implements ExecutionManager {
scheduler.injectExecutor(runner);
Object old = schedulerByTag.put(tag, scheduler);
- if (old!=null && old!=scheduler) {
+ if (old != null && old != scheduler) {
//might support multiple in future...
- throw new IllegalStateException("Not allowed to set multiple TaskSchedulers on ExecutionManager tag (tag "+tag+")");
+ throw new IllegalStateException("Not allowed to set multiple TaskSchedulers on ExecutionManager tag (tag " + tag + ")");
}
}
}
@@ -1257,10 +1369,10 @@ public class BasicExecutionManager implements ExecutionManager {
public boolean clearTaskSchedulerForTag(Object tag) {
synchronized (schedulerByTag) {
Object old = schedulerByTag.remove(tag);
- return (old!=null);
+ return (old != null);
}
}
-
+
@VisibleForTesting
public ConcurrentMap<Object, TaskScheduler> getSchedulerByTag() {
return schedulerByTag;
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/ScheduledTask.java b/core/src/main/java/org/apache/brooklyn/util/core/task/ScheduledTask.java
index 4422209..b6a7029 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/ScheduledTask.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/ScheduledTask.java
@@ -18,7 +18,10 @@
*/
package org.apache.brooklyn.util.core.task;
+import org.apache.brooklyn.api.internal.BrooklynLoggingCategories;
import org.apache.brooklyn.api.mgmt.ExecutionContext;
+import org.apache.brooklyn.core.BrooklynLogging;
+import org.apache.brooklyn.core.BrooklynLogging.LoggingLevel;
import static org.apache.brooklyn.util.JavaGroovyEquivalents.elvis;
import static org.apache.brooklyn.util.JavaGroovyEquivalents.groovyTruth;
@@ -53,8 +56,6 @@ public class ScheduledTask extends BasicTask<Object> {
final Callable<Task<?>> taskFactory;
- protected BrooklynTaskLoggingMdc mdc;
-
/**
* Initial delay before running, set as flag in constructor; defaults to 0
*/
@@ -266,6 +267,7 @@ public class ScheduledTask extends BasicTask<Object> {
@Override
protected boolean doCancel(org.apache.brooklyn.util.core.task.TaskInternal.TaskCancellationMode mode) {
+ BrooklynLogging.log(BrooklynLoggingCategories.TASK_LIFECYCLE_LOG, LoggingLevel.DEBUG, "Cancelling scheduled task "+this);
if (nextRun!=null) {
((TaskInternal<?>)nextRun).cancel(mode);
try {
diff --git a/utils/common/src/main/java/org/apache/brooklyn/util/exceptions/RuntimeInterruptedException.java b/utils/common/src/main/java/org/apache/brooklyn/util/exceptions/RuntimeInterruptedException.java
index c0cde50..f207001 100644
--- a/utils/common/src/main/java/org/apache/brooklyn/util/exceptions/RuntimeInterruptedException.java
+++ b/utils/common/src/main/java/org/apache/brooklyn/util/exceptions/RuntimeInterruptedException.java
@@ -33,6 +33,11 @@ public class RuntimeInterruptedException extends RuntimeException {
private static final long serialVersionUID = 915050245927866175L;
+ public RuntimeInterruptedException(String msg) {
+ super(msg);
+ Thread.currentThread().interrupt();
+ }
+
public RuntimeInterruptedException(InterruptedException cause) {
super(cause);
Thread.currentThread().interrupt();
[brooklyn-server] 03/27: more logging and error edge cases catches
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 0925e5d440e146bab504d2115b75361740229396
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Mon Sep 13 14:54:37 2021 +0100
more logging and error edge cases catches
---
.../catalog/internal/CatalogInitialization.java | 15 +-
.../core/catalog/internal/CatalogUtils.java | 8 +-
.../core/mgmt/ha/HighAvailabilityManagerImpl.java | 22 +-
.../mgmt/internal/LocalSubscriptionManager.java | 13 +-
.../brooklyn/core/mgmt/rebind/RebindIteration.java | 563 +++++++++++----------
.../core/mgmt/rebind/RebindManagerImpl.java | 3 +
6 files changed, 339 insertions(+), 285 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/core/catalog/internal/CatalogInitialization.java b/core/src/main/java/org/apache/brooklyn/core/catalog/internal/CatalogInitialization.java
index 669ab17..4928e8f 100644
--- a/core/src/main/java/org/apache/brooklyn/core/catalog/internal/CatalogInitialization.java
+++ b/core/src/main/java/org/apache/brooklyn/core/catalog/internal/CatalogInitialization.java
@@ -241,19 +241,14 @@ public class CatalogInitialization implements ManagementContextInjectable {
}
synchronized (populatingCatalogMutex) {
if (hasRunFinalInitialization()) {
- log.warn("Catalog initialization called to add persisted catalog, even though it has already run the final 'master' initialization; mode="+mode+" (perhaps previously demoted from master?)");
+ log.warn("Catalog initialization called to add persisted catalog, even though it has already run the final 'master' initialization; mode="+mode, new Throwable("Source of duplicate catalog initialization"));
hasRunFinalInitialization = false;
}
if (hasRunPersistenceInitialization()) {
// Multiple calls; will need to reset (only way to clear out the previous persisted state's catalog)
if (log.isDebugEnabled()) {
String message = "Catalog initialization repeated call to add persisted catalog, resetting catalog (including initial) to start from clean slate; mode="+mode;
- if (!ManagementNodeState.isHotProxy(mode)) {
- log.debug(message);
- } else {
- // in hot modes, make this message trace so we don't get too much output then
- log.trace(message);
- }
+ log.debug(message);
}
} else if (hasRunInitialCatalogInitialization()) {
throw new IllegalStateException("Catalog initialization already run for initial catalog by mechanism other than populating persisted state; mode="+mode);
@@ -433,6 +428,12 @@ public class CatalogInitialization implements ManagementContextInjectable {
confirmCatalog();
}
+ public void clearForSubsequentCatalogInit() {
+ hasRunInitialCatalogInitialization = false;
+ hasRunPersistenceInitialization = false;
+ hasRunFinalInitialization = false;
+ }
+
private void populateViaInitialBomImpl(BasicBrooklynCatalog catalog) {
// B1) look for --catalog-initial, if so read it, then go to C1
// B2) look for BrooklynServerConfig.BROOKLYN_CATALOG_URL, if so, read it, supporting YAML, then go to C1
diff --git a/core/src/main/java/org/apache/brooklyn/core/catalog/internal/CatalogUtils.java b/core/src/main/java/org/apache/brooklyn/core/catalog/internal/CatalogUtils.java
index 80c3f29..026c3c0 100644
--- a/core/src/main/java/org/apache/brooklyn/core/catalog/internal/CatalogUtils.java
+++ b/core/src/main/java/org/apache/brooklyn/core/catalog/internal/CatalogUtils.java
@@ -244,13 +244,13 @@ public class CatalogUtils {
if (itemBeingAdded.getCatalogItemId()==null) {
if (log.isDebugEnabled())
BrooklynLogging.log(log, BrooklynLogging.levelDebugOrTraceIfReadOnly(entity),
- "Catalog item addition: "+entity+" from "+entity.getCatalogItemId()+" applying its catalog item ID to "+itemBeingAdded);
+ "Entity "+entity+" from "+entity.getCatalogItemId()+" applying its catalog item ID to "+itemBeingAdded);
final BrooklynObjectInternal addInternal = (BrooklynObjectInternal) itemBeingAdded;
addInternal.setCatalogItemIdAndSearchPath(entity.getCatalogItemId(), entity.getCatalogItemIdSearchPath());
} else {
if (!itemBeingAdded.getCatalogItemId().equals(entity.getCatalogItemId())) {
- // not a problem, but something to watch out for
- log.debug("Cross-catalog item detected: "+entity+" from "+entity.getCatalogItemId()+" has "+itemBeingAdded+" from "+itemBeingAdded.getCatalogItemId());
+ // not a problem, but something worth noting
+ log.debug("Blueprint for "+entity+" from "+entity.getCatalogItemId()+" includes "+itemBeingAdded+" from other catalog "+itemBeingAdded.getCatalogItemId());
}
}
} else if (itemBeingAdded.getCatalogItemId()!=null) {
@@ -270,7 +270,7 @@ public class CatalogUtils {
/** @deprecated since 0.12.0 - the "version starts with number" test this does is hokey; use
* either {@link RegisteredTypeNaming#isUsableTypeColonVersion(String)} for weak enforcement
- * or {@link RegisteredTypeNaming#isGoodTypeColonVersion(String)} for OSGi enforcement. */
+ * or {@link RegisteredTypeNaming#isUsableTypeColonVersion(String)} for OSGi enforcement. */
// several references, but all deprecated, most safe to remove after a cycle or two and bad verisons have been warned
@Deprecated
public static boolean looksLikeVersionedId(String versionedId) {
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
index 2779f4f..2bd080d 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
@@ -288,6 +288,8 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
@VisibleForTesting
@Beta
public void changeMode(HighAvailabilityMode startMode, boolean preventElectionOnExplicitStandbyMode, boolean failOnExplicitModesIfUnusual) {
+ LOG.debug("Changing HA mode to "+startMode+", election prevention "+preventElectionOnExplicitStandbyMode+", fail on unusual "+failOnExplicitModesIfUnusual);
+
if (!running) {
// if was not running then start as disabled mode, then proceed as normal
LOG.info("HA changing mode to "+startMode+" from "+getInternalNodeState()+" when not running, forcing an intermediate start as DISABLED then will convert to "+startMode);
@@ -471,7 +473,9 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
throw Exceptions.propagate(e);
}
} else {
+ LOG.debug("Management node "+ownNodeId+" in "+managementContext.getManagementPlaneIdMaybe().or("<new-plane>")+" restarting read only on transition to "+startMode+" (from "+oldState+" / "+getNodeState()+")");
// transitioning among hot proxy states - tell the rebind manager
+ // TODO i think we might have previously started _in_ this method, if so we should set a flag and suppress the stop-then-restart
managementContext.getRebindManager().stopReadOnly();
managementContext.getRebindManager().startReadOnly(ManagementNodeState.of(startMode).get());
setNodeStateTransitionComplete(true);
@@ -534,6 +538,7 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
}
protected void setInternalNodeState(ManagementNodeState newState) {
+ LOG.debug("HA state internal "+nodeState+" -> "+newState);
ManagementNodeState oldState = getInternalNodeState();
synchronized (nodeStateHistory) {
if (this.nodeState != newState) {
@@ -554,6 +559,7 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
if (ManagementNodeState.isHotProxy(oldState) && !ManagementNodeState.isHotProxy(newState)) {
// could perhaps promote standby items on some transitions; but for now we stop the old read-only and re-load them
// TODO ideally there'd be an incremental rebind as well as an incremental persist
+ LOG.debug("Resetting rebind on hot->cold transition, from "+oldState+" to "+newState);
managementContext.getRebindManager().stopReadOnly();
clearManagedItems(ManagementTransitionMode.transitioning(BrooklynObjectManagementMode.LOADED_READ_ONLY, BrooklynObjectManagementMode.UNMANAGED_PERSISTED));
managementContext.getRebindManager().reset();
@@ -564,7 +570,8 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
private void setNodeStateTransitionComplete(boolean val) {
nodeStateTransitionComplete = val;
-
+ LOG.debug("HA state transition complete now "+val+" ("+nodeState+" / "+getNodeState()+")");
+
// Can cause getNodeState() value to change, so notify listener
stateListener.onStateChange(getNodeState());
}
@@ -873,6 +880,7 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
}
protected void promoteToMaster() {
+ LOG.debug("Promoting to master: "+this);
if (!running) {
LOG.warn("Ignoring promote-to-master request, as HighAvailabilityManager is not running");
return;
@@ -907,12 +915,20 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
}
protected void demoteTo(ManagementNodeState toState) {
+ LOG.debug("Management node "+ownNodeId+" in "+managementContext.getManagementPlaneIdMaybe().or("<new-plane>")+" demoting to "+toState+" (from "+nodeState+" / "+getNodeState()+")");
if (toState!=ManagementNodeState.FAILED && !running) {
LOG.warn("Ignoring demote-from-master request, as HighAvailabilityManager is no longer running");
return;
}
boolean wasMaster = (getInternalNodeState() == ManagementNodeState.MASTER);
- if (wasMaster) backupOnDemotionIfNeeded();
+ if (wasMaster) {
+ try {
+ backupOnDemotionIfNeeded();
+ } catch (Exception e) {
+ Exceptions.propagateIfFatal(e);
+ LOG.error("Unable to create backup on demotion (ignoring): "+e, e);
+ }
+ }
// TODO target may be RO ?
ManagementTransitionMode mode = ManagementTransitionMode.transitioning(
wasMaster ? BrooklynObjectManagementMode.MANAGED_PRIMARY : BrooklynObjectManagementMode.LOADED_READ_ONLY,
@@ -947,6 +963,7 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
protected void onDemotionStopItems(ManagementTransitionMode mode) {
// stop persistence and remove all apps etc
+ LOG.debug("Stopping rebind on demotion to "+mode+" (in state "+nodeState+")");
managementContext.getRebindManager().stopPersistence();
managementContext.getRebindManager().stopReadOnly();
clearManagedItems(mode);
@@ -1002,6 +1019,7 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
* (if not, errors should be stored elsewhere), callers may want to rethrow */
protected ReferenceWithError<Boolean> activateHotProxy(ManagementNodeState toState) {
try {
+ LOG.debug("Activating hot proxy for state "+toState);
Preconditions.checkState(nodeStateTransitionComplete==false, "Must be in transitioning state to go into "+toState);
setInternalNodeState(toState);
managementContext.getRebindManager().startReadOnly(toState);
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/LocalSubscriptionManager.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/LocalSubscriptionManager.java
index 9b2cd79..ca39bf8 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/LocalSubscriptionManager.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/LocalSubscriptionManager.java
@@ -331,7 +331,14 @@ public class LocalSubscriptionManager extends AbstractSubscriptionManager {
}
@Override
public void run() {
- BasicExecutionContext oldEC = ec instanceof BasicExecutionContext ? BasicExecutionContext.setPerThreadExecutionContext((BasicExecutionContext)ec) : null;
+ BasicExecutionContext oldEC = null;
+ boolean setEC;
+ if (ec instanceof BasicExecutionContext) {
+ oldEC = BasicExecutionContext.setPerThreadExecutionContext((BasicExecutionContext) ec);
+ setEC = true;
+ } else {
+ setEC = false;
+ }
try {
if (isEntityStarting) {
@@ -362,7 +369,9 @@ public class LocalSubscriptionManager extends AbstractSubscriptionManager {
LOG.warn("Error processing subscriptions to "+this+": "+t, t);
}
} finally {
- BasicExecutionContext.setPerThreadExecutionContext(oldEC);
+ if (setEC) {
+ BasicExecutionContext.setPerThreadExecutionContext(oldEC);
+ }
}
}};
if (!isInitialPublicationOfOldValueInCorrectScheduledThread) {
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
index 38f4af7..c3a7ab7 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
@@ -137,156 +137,158 @@ import com.google.common.collect.Maps;
import com.google.common.collect.Sets;
/**
-Multi-phase deserialization:
-
-<ul>
-<li> 1. load the manifest files and populate the summaries (ID+type) in {@link BrooklynMementoManifest}
-<li> 2. install bundles, instantiate and reconstruct catalog items
-<li> 3. instantiate entities+locations -- so that inter-entity references can subsequently
- be set during deserialize (and entity config/state is set).
-<li> 4. deserialize the manifests to instantiate the mementos
-<li> 5. instantiate policies+enrichers+feeds
- (could probably merge this with (3), depending how they are implemented)
-<li> 6. reconstruct the locations, policies, etc, then finally entities -- setting all fields and then calling
- {@link RebindSupport#reconstruct(RebindContext, Memento)}
-<li> 7. associate policies+enrichers+feeds to all the entities
-<li> 8. manage the entities
-</ul>
-
- If underlying data-store is changed between first and second manifest read (e.g. to add an
- entity), then second phase might try to reconstitute an entity that has not been put in
- the rebindContext. This should not affect normal production usage, because rebind is run
- against a data-store that is not being written to by other brooklyn instance(s).
- But clearly it would be desirable to have better locking possible against the backing store.
-
-<p>
- When rebinding to code in OSGi bundles, thecatalog item id context is inferred as follows:
- * most of the time the creator will be passing "my catalog item id"
- (or API could also take "BrooklynObject me" as a creation context and the
- receiver query the creator's catalog item id)
- * look at the context entity of Tasks.current() (if set)
- * propagate the catalog item id when doing setEntity, addChild
- * when none of the above work (or they are wrong) let the user specify the catalog item
-<p>
- Precedence of setting the catalog item ID:
- 1. User-supplied catalog item ID.
- 2. Creating from a catalog item - all items resolved during the creation of a spec
- from a catalog item receive the catalog item's ID as context.
- 3. When using the Java API for creating specs get the catalog item ID from the
- context entity of the Tasks.current() task.
- 4. Propagate the context catalog item ID to children, adjuncts if they don't have one already.
-*/
+ * Multi-phase deserialization:
+ *
+ * <ul>
+ * <li> 1. load the manifest files and populate the summaries (ID+type) in {@link BrooklynMementoManifest}
+ * <li> 2. install bundles, instantiate and reconstruct catalog items
+ * <li> 3. instantiate entities+locations -- so that inter-entity references can subsequently
+ * be set during deserialize (and entity config/state is set).
+ * <li> 4. deserialize the manifests to instantiate the mementos
+ * <li> 5. instantiate policies+enrichers+feeds
+ * (could probably merge this with (3), depending how they are implemented)
+ * <li> 6. reconstruct the locations, policies, etc, then finally entities -- setting all fields and then calling
+ * {@link RebindSupport#reconstruct(RebindContext, Memento)}
+ * <li> 7. associate policies+enrichers+feeds to all the entities
+ * <li> 8. manage the entities
+ * </ul>
+ * <p>
+ * If underlying data-store is changed between first and second manifest read (e.g. to add an
+ * entity), then second phase might try to reconstitute an entity that has not been put in
+ * the rebindContext. This should not affect normal production usage, because rebind is run
+ * against a data-store that is not being written to by other brooklyn instance(s).
+ * But clearly it would be desirable to have better locking possible against the backing store.
+ *
+ * <p>
+ * When rebinding to code in OSGi bundles, thecatalog item id context is inferred as follows:
+ * most of the time the creator will be passing "my catalog item id"
+ * (or API could also take "BrooklynObject me" as a creation context and the
+ * receiver query the creator's catalog item id)
+ * look at the context entity of Tasks.current() (if set)
+ * propagate the catalog item id when doing setEntity, addChild
+ * when none of the above work (or they are wrong) let the user specify the catalog item
+ * <p>
+ * Precedence of setting the catalog item ID:
+ * 1. User-supplied catalog item ID.
+ * 2. Creating from a catalog item - all items resolved during the creation of a spec
+ * from a catalog item receive the catalog item's ID as context.
+ * 3. When using the Java API for creating specs get the catalog item ID from the
+ * context entity of the Tasks.current() task.
+ * 4. Propagate the context catalog item ID to children, adjuncts if they don't have one already.
+ */
public abstract class RebindIteration {
private static final Logger LOG = LoggerFactory.getLogger(RebindIteration.class);
-
+
protected final RebindManagerImpl rebindManager;
-
+
protected final ClassLoader classLoader;
protected final RebindExceptionHandler exceptionHandler;
protected final ManagementNodeState mode;
protected final ManagementContextInternal managementContext;
- protected final Semaphore rebindActive;
+ protected final Semaphore rebindActive;
protected final AtomicInteger readOnlyRebindCount;
protected final PersistenceActivityMetrics rebindMetrics;
protected final BrooklynMementoPersister persistenceStoreAccess;
-
+
protected final AtomicBoolean iterationStarted = new AtomicBoolean();
protected final RebindContextImpl rebindContext;
protected final Reflections reflections;
protected final BrooklynObjectInstantiator instantiator;
-
+
// populated in the course of a run
-
+
// set on run start
-
+
protected Stopwatch timer;
- /** phase is used to ensure our steps are run as we've expected, and documented (in javadoc at top).
- * it's worth the extra effort due to the complication and the subtleties. */
+ /**
+ * phase is used to ensure our steps are run as we've expected, and documented (in javadoc at top).
+ * it's worth the extra effort due to the complication and the subtleties.
+ */
protected int phase = 0;
// set in first phase
-
+
protected BrooklynMementoRawData mementoRawData;
protected BrooklynMementoManifest mementoManifest;
protected Boolean overwritingMaster;
protected Boolean isEmpty;
// set later on
-
+
protected BrooklynMemento memento;
// set near the end
-
+
protected List<Application> applications;
- public RebindIteration(RebindManagerImpl rebindManager,
- ManagementNodeState mode,
- ClassLoader classLoader, RebindExceptionHandler exceptionHandler,
- Semaphore rebindActive, AtomicInteger readOnlyRebindCount, PersistenceActivityMetrics rebindMetrics, BrooklynMementoPersister persistenceStoreAccess
- ) {
+ public RebindIteration(RebindManagerImpl rebindManager,
+ ManagementNodeState mode,
+ ClassLoader classLoader, RebindExceptionHandler exceptionHandler,
+ Semaphore rebindActive, AtomicInteger readOnlyRebindCount, PersistenceActivityMetrics rebindMetrics, BrooklynMementoPersister persistenceStoreAccess
+ ) {
// NB: there is no particularly deep meaning in what is passed in vs what is looked up from the RebindManager which calls us
// (this is simply a refactoring of previous code to a new class)
-
+
this.rebindManager = rebindManager;
-
+
this.mode = mode;
this.classLoader = checkNotNull(classLoader, "classLoader");
this.exceptionHandler = checkNotNull(exceptionHandler, "exceptionHandler");
-
- this.rebindActive = rebindActive;
+
+ this.rebindActive = rebindActive;
this.readOnlyRebindCount = readOnlyRebindCount;
this.rebindMetrics = rebindMetrics;
this.persistenceStoreAccess = persistenceStoreAccess;
-
+
managementContext = rebindManager.getManagementContext();
rebindContext = new RebindContextImpl(managementContext, exceptionHandler, classLoader);
reflections = new Reflections(classLoader).applyClassRenames(DeserializingClassRenamesProvider.INSTANCE.loadDeserializingMapping());
instantiator = new BrooklynObjectInstantiator(classLoader, rebindContext, reflections);
-
- if (mode==ManagementNodeState.HOT_STANDBY || mode==ManagementNodeState.HOT_BACKUP) {
+
+ if (mode == ManagementNodeState.HOT_STANDBY || mode == ManagementNodeState.HOT_BACKUP) {
rebindContext.setAllReadOnly();
} else {
- Preconditions.checkState(mode==ManagementNodeState.MASTER, "Must be either master or read only to rebind (mode "+mode+")");
+ Preconditions.checkState(mode == ManagementNodeState.MASTER, "Must be either master or read only to rebind (mode " + mode + ")");
}
}
public List<Application> getApplications() {
return applications;
}
-
+
RebindContextImpl getRebindContext() {
return rebindContext;
}
-
+
protected void doRun() throws Exception {
- if (readOnlyRebindCount.get()>1) {
+ if (readOnlyRebindCount.get() > 1) {
// wait for tasks
Collection<Task<?>> entityTasks = ((BasicExecutionManager) managementContext.getExecutionManager()).allTasksLive()
- .stream().filter(t -> BrooklynTaskTags.getContextEntity(t)!=null).collect(Collectors.toList());
+ .stream().filter(t -> BrooklynTaskTags.getContextEntity(t) != null).collect(Collectors.toList());
List<Task<?>> openTasks;
CountdownTimer time = CountdownTimer.newInstanceStarted(Duration.seconds(15));
do {
openTasks = entityTasks.stream().filter(t -> !t.isDone()).collect(Collectors.toList());
if (openTasks.isEmpty()) break;
if (time.isExpired()) {
- LOG.warn("Aborting "+openTasks.size()+" incomplete task(s) before rebinding again: "+openTasks);
+ LOG.warn("Aborting " + openTasks.size() + " incomplete task(s) before rebinding again: " + openTasks);
openTasks.forEach(t -> t.cancel(true));
}
if (time.getDurationElapsed().isShorterThan(Duration.millis(200))) {
LOG.info("Waiting on " + openTasks.size() + " task(s) before rebinding again: " + openTasks);
- } else {
- LOG.debug("Waiting on " + openTasks.size() + " task(s) before rebinding again: " + openTasks);
}
+ LOG.debug("Waiting on " + openTasks.size() + " task(s) before rebinding again, details: " +
+ openTasks.stream().map(t -> ""+t+"("+BrooklynTaskTags.getContextEntity(t)+")").collect(Collectors.toList()));
Time.sleep(Duration.millis(200));
} while (true);
- entityTasks.forEach( ((BasicExecutionManager) managementContext.getExecutionManager())::deleteTask );
+ entityTasks.forEach(((BasicExecutionManager) managementContext.getExecutionManager())::deleteTask);
List<Task<?>> otherDoneTasks = ((BasicExecutionManager) managementContext.getExecutionManager()).allTasksLive()
.stream().filter(Task::isDone).collect(Collectors.toList());
- otherDoneTasks.forEach( ((BasicExecutionManager) managementContext.getExecutionManager())::deleteTask );
+ otherDoneTasks.forEach(((BasicExecutionManager) managementContext.getExecutionManager())::deleteTask);
}
loadManifestFiles();
@@ -302,63 +304,67 @@ public abstract class RebindIteration {
manageTheObjects();
finishingUp();
}
-
+
protected abstract void loadManifestFiles() throws Exception;
-
+
public void run() {
if (iterationStarted.getAndSet(true)) {
- throw new IllegalStateException("Iteration "+this+" has already run; create a new instance for another rebind pass.");
+ throw new IllegalStateException("Iteration " + this + " has already run; create a new instance for another rebind pass.");
}
try {
rebindActive.acquire();
- } catch (InterruptedException e) { Exceptions.propagate(e); }
+ } catch (InterruptedException e) {
+ Exceptions.propagate(e);
+ }
try {
RebindTracker.setRebinding();
- if (ManagementNodeState.isHotProxy(mode))
+ if (ManagementNodeState.isHotProxy(mode)) {
readOnlyRebindCount.incrementAndGet();
+ }
timer = Stopwatch.createStarted();
exceptionHandler.onStart(rebindContext);
doRun();
-
+
exceptionHandler.onDone();
-
+
rebindMetrics.noteSuccess(Duration.of(timer));
noteErrors(exceptionHandler, null);
-
+
} catch (Exception e) {
rebindMetrics.noteFailure(Duration.of(timer));
-
+
Exceptions.propagateIfFatal(e);
noteErrors(exceptionHandler, e);
throw exceptionHandler.onFailed(e);
-
+
} finally {
rebindActive.release();
RebindTracker.reset();
}
}
-
+
protected void checkEnteringPhase(int targetPhase) {
phase++;
checkContinuingPhase(targetPhase);
}
+
protected void checkContinuingPhase(int targetPhase) {
- if (targetPhase!=phase)
- throw new IllegalStateException("Phase mismatch: should be phase "+targetPhase+" but is currently "+phase);
+ if (targetPhase != phase)
+ throw new IllegalStateException("Phase mismatch: should be phase " + targetPhase + " but is currently " + phase);
}
-
+
protected void preprocessManifestFiles() throws Exception {
checkContinuingPhase(1);
- Preconditions.checkState(mementoRawData!=null, "Memento raw data should be set when calling this");
- Preconditions.checkState(mementoManifest==null, "Memento data should not yet be set when calling this");
-
+ Preconditions.checkState(mementoRawData != null, "Memento raw data should be set when calling this");
+ Preconditions.checkState(mementoManifest == null, "Memento data should not yet be set when calling this");
+
// TODO building the manifests should be part of this class (or parent)
// it does not have anything to do with the persistence store!
mementoManifest = persistenceStoreAccess.loadMementoManifest(mementoRawData, exceptionHandler);
-
+
overwritingMaster = false;
isEmpty = mementoManifest.isEmpty();
}
@@ -367,12 +373,13 @@ public abstract class RebindIteration {
// Build catalog early so we can load other things.
// Reads the persisted catalog contents, and passes it all to CatalogInitialization, which decides what to do with it.
checkEnteringPhase(2);
-
+
CatalogInitialization.RebindLogger rebindLogger = new CatalogInitialization.RebindLogger() {
@Override
public void debug(String message, Object... args) {
logRebindingDebug(message, args);
}
+
@Override
public void info(String message, Object... args) {
logRebindingInfo(message, args);
@@ -382,18 +389,20 @@ public abstract class RebindIteration {
class InstallableManagedBundleImpl implements CatalogInitialization.InstallableManagedBundle {
private final ManagedBundleMemento memento;
private final ManagedBundle managedBundle;
-
+
InstallableManagedBundleImpl(ManagedBundleMemento memento, ManagedBundle managedBundle) {
this.memento = memento;
this.managedBundle = managedBundle;
}
- @Override public ManagedBundle getManagedBundle() {
+ @Override
+ public ManagedBundle getManagedBundle() {
return managedBundle;
}
- @Override public Supplier<InputStream> getInputStreamSource() throws IOException {
- return InputStreamSource.ofRenewableSupplier("JAR for "+memento, () -> {
+ @Override
+ public Supplier<InputStream> getInputStreamSource() throws IOException {
+ return InputStreamSource.ofRenewableSupplier("JAR for " + memento, () -> {
try {
return memento.getJarContent().openStream();
} catch (IOException e) {
@@ -402,10 +411,10 @@ public abstract class RebindIteration {
});
}
}
-
+
Map<VersionedName, InstallableManagedBundle> bundles = new LinkedHashMap<>();
- Collection<CatalogItem<?,?>> legacyCatalogItems = new ArrayList<>();
-
+ Collection<CatalogItem<?, ?>> legacyCatalogItems = new ArrayList<>();
+
// Find the bundles
if (rebindManager.persistBundlesEnabled) {
for (ManagedBundleMemento bundleMemento : mementoManifest.getBundles().values()) {
@@ -416,10 +425,10 @@ public abstract class RebindIteration {
} else {
logRebindingDebug("Not rebinding bundles; feature disabled: {}", mementoManifest.getBundleIds());
}
-
+
// Construct the legacy catalog items; don't add them to the catalog here,
// but instead pass them to catalogInitialization.populateCatalog.
-
+
if (rebindManager.persistCatalogItemsEnabled) {
// Instantiate catalog items
logRebindingDebug("RebindManager instantiating catalog items: {}", mementoManifest.getCatalogItemIds());
@@ -433,7 +442,7 @@ public abstract class RebindIteration {
exceptionHandler.onCreateFailed(BrooklynObjectType.CATALOG_ITEM, catalogItemMemento.getId(), catalogItemMemento.getType(), e);
}
}
-
+
// Reconstruct catalog entries
logRebindingDebug("RebindManager reconstructing catalog items");
for (CatalogItemMemento catalogItemMemento : mementoManifest.getCatalogItemMementos().values()) {
@@ -460,22 +469,23 @@ public abstract class RebindIteration {
// Delegates to CatalogInitialization; see notes there.
CatalogInitialization.PersistedCatalogState persistedCatalogState = new CatalogInitialization.PersistedCatalogState(bundles, legacyCatalogItems);
-
+
CatalogInitialization catInit = managementContext.getCatalogInitialization();
+ catInit.clearForSubsequentCatalogInit();
catInit.populateInitialAndPersistedCatalog(mode, persistedCatalogState, exceptionHandler, rebindLogger);
}
protected void instantiateLocationsAndEntities() {
checkEnteringPhase(3);
-
+
// Instantiate locations
logRebindingDebug("RebindManager instantiating locations: {}", mementoManifest.getLocationIdToType().keySet());
for (Map.Entry<String, String> entry : mementoManifest.getLocationIdToType().entrySet()) {
String locId = entry.getKey();
String locType = entry.getValue();
if (LOG.isTraceEnabled()) LOG.trace("RebindManager instantiating location {}", locId);
-
+
try {
Location location = instantiator.newLocation(locId, locType);
rebindContext.registerLocation(locId, location);
@@ -483,18 +493,18 @@ public abstract class RebindIteration {
exceptionHandler.onCreateFailed(BrooklynObjectType.LOCATION, locId, locType, e);
}
}
-
+
// Instantiate entities
logRebindingDebug("RebindManager instantiating entities: {}", mementoManifest.getEntityIdToManifest().keySet());
for (Map.Entry<String, EntityMementoManifest> entry : mementoManifest.getEntityIdToManifest().entrySet()) {
String entityId = entry.getKey();
EntityMementoManifest entityManifest = entry.getValue();
-
+
if (LOG.isTraceEnabled()) LOG.trace("RebindManager instantiating entity {}", entityId);
-
+
try {
Entity entity = instantiator.newEntity(entityManifest);
- ((EntityInternal)entity).getManagementSupport().setReadOnly( rebindContext.isReadOnly(entity) );
+ ((EntityInternal) entity).getManagementSupport().setReadOnly(rebindContext.isReadOnly(entity));
rebindContext.registerEntity(entityId, entity);
} catch (Exception e) {
@@ -504,9 +514,9 @@ public abstract class RebindIteration {
}
protected void instantiateMementos() throws IOException {
-
+
checkEnteringPhase(4);
-
+
memento = persistenceStoreAccess.loadMemento(mementoRawData, rebindContext.lookup(), exceptionHandler);
}
@@ -518,23 +528,23 @@ public abstract class RebindIteration {
"Expected if this is the first rebind after upgrading to Brooklyn 0.12.0+");
}
if (managementContext.getManagementPlaneIdMaybe().isAbsent()) {
- ((LocalManagementContext)managementContext).generateManagementPlaneId();
+ ((LocalManagementContext) managementContext).generateManagementPlaneId();
}
} else {
- ((LocalManagementContext)managementContext).setManagementPlaneId(persistedPlaneId);
+ ((LocalManagementContext) managementContext).setManagementPlaneId(persistedPlaneId);
}
}
protected void instantiateAdjuncts(BrooklynObjectInstantiator instantiator) {
-
+
checkEnteringPhase(5);
-
+
// Instantiate policies
if (rebindManager.persistPoliciesEnabled) {
logRebindingDebug("RebindManager instantiating policies: {}", memento.getPolicyIds());
for (PolicyMemento policyMemento : memento.getPolicyMementos().values()) {
logRebindingDebug("RebindManager instantiating policy {}", policyMemento);
-
+
try {
Policy policy = instantiator.newPolicy(policyMemento);
rebindContext.registerPolicy(policyMemento.getId(), policy);
@@ -545,7 +555,7 @@ public abstract class RebindIteration {
} else {
logRebindingDebug("Not rebinding policies; feature disabled: {}", memento.getPolicyIds());
}
-
+
// Instantiate enrichers
if (rebindManager.persistEnrichersEnabled) {
logRebindingDebug("RebindManager instantiating enrichers: {}", memento.getEnricherIds());
@@ -561,8 +571,8 @@ public abstract class RebindIteration {
}
} else {
logRebindingDebug("Not rebinding enrichers; feature disabled: {}", memento.getEnricherIds());
- }
-
+ }
+
// Instantiate feeds
if (rebindManager.persistFeedsEnabled) {
logRebindingDebug("RebindManager instantiating feeds: {}", memento.getFeedIds());
@@ -582,9 +592,9 @@ public abstract class RebindIteration {
}
protected void reconstructEverything() {
-
+
checkEnteringPhase(6);
-
+
// Reconstruct locations
logRebindingDebug("RebindManager reconstructing locations");
for (LocationMemento locMemento : sortParentFirst(memento.getLocationMementos()).values()) {
@@ -595,7 +605,7 @@ public abstract class RebindIteration {
exceptionHandler.onNotFound(BrooklynObjectType.LOCATION, locMemento.getId());
} else {
try {
- ((LocationInternal)location).getRebindSupport().reconstruct(rebindContext, locMemento);
+ ((LocationInternal) location).getRebindSupport().reconstruct(rebindContext, locMemento);
} catch (Exception e) {
exceptionHandler.onRebindFailed(BrooklynObjectType.LOCATION, location, e);
}
@@ -608,7 +618,7 @@ public abstract class RebindIteration {
for (PolicyMemento policyMemento : memento.getPolicyMementos().values()) {
Policy policy = rebindContext.getPolicy(policyMemento.getId());
logRebindingDebug("RebindManager reconstructing policy {}", policyMemento);
-
+
if (policy == null) {
// usually because of creation-failure, when not using fail-fast
exceptionHandler.onNotFound(BrooklynObjectType.POLICY, policyMemento.getId());
@@ -629,7 +639,7 @@ public abstract class RebindIteration {
for (EnricherMemento enricherMemento : memento.getEnricherMementos().values()) {
Enricher enricher = rebindContext.getEnricher(enricherMemento.getId());
logRebindingDebug("RebindManager reconstructing enricher {}", enricherMemento);
-
+
if (enricher == null) {
// usually because of creation-failure, when not using fail-fast
exceptionHandler.onNotFound(BrooklynObjectType.ENRICHER, enricherMemento.getId());
@@ -643,14 +653,14 @@ public abstract class RebindIteration {
}
}
}
-
+
// Reconstruct feeds
if (rebindManager.persistFeedsEnabled) {
logRebindingDebug("RebindManager reconstructing feeds");
for (FeedMemento feedMemento : memento.getFeedMementos().values()) {
Feed feed = rebindContext.getFeed(feedMemento.getId());
logRebindingDebug("RebindManager reconstructing feed {}", feedMemento);
-
+
if (feed == null) {
// usually because of creation-failure, when not using fail-fast
exceptionHandler.onNotFound(BrooklynObjectType.FEED, feedMemento.getId());
@@ -665,20 +675,20 @@ public abstract class RebindIteration {
}
}
-
+
// Reconstruct entities
logRebindingDebug("RebindManager reconstructing entities");
for (EntityMemento entityMemento : sortParentFirst(memento.getEntityMementos()).values()) {
Entity entity = rebindContext.lookup().lookupEntity(entityMemento.getId());
logRebindingDebug("RebindManager reconstructing entity {}", entityMemento);
-
+
if (entity == null) {
// usually because of creation-failure, when not using fail-fast
exceptionHandler.onNotFound(BrooklynObjectType.ENTITY, entityMemento.getId());
} else {
try {
entityMemento.injectTypeClass(entity.getClass());
- ((EntityInternal)entity).getRebindSupport().reconstruct(rebindContext, entityMemento);
+ ((EntityInternal) entity).getRebindSupport().reconstruct(rebindContext, entityMemento);
} catch (Exception e) {
exceptionHandler.onRebindFailed(BrooklynObjectType.ENTITY, entity, e);
}
@@ -693,7 +703,7 @@ public abstract class RebindIteration {
for (EntityMemento entityMemento : sortParentFirst(memento.getEntityMementos()).values()) {
Entity entity = rebindContext.getEntity(entityMemento.getId());
logRebindingDebug("RebindManager associating adjuncts to entity {}", entityMemento);
-
+
if (entity == null) {
// usually because of creation-failure, when not using fail-fast
exceptionHandler.onNotFound(BrooklynObjectType.ENTITY, entityMemento.getId());
@@ -705,16 +715,16 @@ public abstract class RebindIteration {
entityMemento.injectTypeClass(entity.getClass());
// TODO these call to the entity which in turn sets the entity on the underlying feeds and enrichers;
// that is taken as the cue to start, but it should not be. start should be a separate call.
- ((EntityInternal)entity).getRebindSupport().addPolicies(rebindContext, entityMemento);
- ((EntityInternal)entity).getRebindSupport().addEnrichers(rebindContext, entityMemento);
- ((EntityInternal)entity).getRebindSupport().addFeeds(rebindContext, entityMemento);
+ ((EntityInternal) entity).getRebindSupport().addPolicies(rebindContext, entityMemento);
+ ((EntityInternal) entity).getRebindSupport().addEnrichers(rebindContext, entityMemento);
+ ((EntityInternal) entity).getRebindSupport().addFeeds(rebindContext, entityMemento);
} catch (Exception e) {
exceptionHandler.onRebindFailed(BrooklynObjectType.ENTITY, entity, e);
}
}
};
- ((EntityInternal)entity).getExecutionContext().get(Tasks.<Void>builder()
- .displayName("rebind-adjuncts-"+entity.getId())
+ ((EntityInternal) entity).getExecutionContext().get(Tasks.<Void>builder()
+ .displayName("rebind-adjuncts-" + entity.getId())
.dynamic(false)
.body(new RebindAdjuncts(entityMemento, entity, rebindContext, exceptionHandler))
.build());
@@ -741,9 +751,9 @@ public abstract class RebindIteration {
entityMemento.injectTypeClass(entity.getClass());
// TODO these call to the entity which in turn sets the entity on the underlying feeds and enrichers;
// that is taken as the cue to start, but it should not be. start should be a separate call.
- ((EntityInternal)entity).getRebindSupport().addPolicies(rebindContext, entityMemento);
- ((EntityInternal)entity).getRebindSupport().addEnrichers(rebindContext, entityMemento);
- ((EntityInternal)entity).getRebindSupport().addFeeds(rebindContext, entityMemento);
+ ((EntityInternal) entity).getRebindSupport().addPolicies(rebindContext, entityMemento);
+ ((EntityInternal) entity).getRebindSupport().addEnrichers(rebindContext, entityMemento);
+ ((EntityInternal) entity).getRebindSupport().addFeeds(rebindContext, entityMemento);
entityMemento = null;
entity = null;
@@ -756,20 +766,20 @@ public abstract class RebindIteration {
protected void manageTheObjects() {
checkEnteringPhase(8);
-
+
logRebindingDebug("RebindManager managing locations");
- LocationManagerInternal locationManager = (LocationManagerInternal)managementContext.getLocationManager();
+ LocationManagerInternal locationManager = (LocationManagerInternal) managementContext.getLocationManager();
Set<String> oldLocations = Sets.newLinkedHashSet(locationManager.getLocationIds());
- for (Location location: rebindContext.getLocations()) {
+ for (Location location : rebindContext.getLocations()) {
ManagementTransitionMode oldMode = updateTransitionMode(locationManager, location);
- if (oldMode!=null)
+ if (oldMode != null)
oldLocations.remove(location.getId());
}
- for (Location location: rebindContext.getLocations()) {
- if (location.getParent()==null) {
+ for (Location location : rebindContext.getLocations()) {
+ if (location.getParent() == null) {
// manage all root locations
try {
- ((LocationManagerInternal)managementContext.getLocationManager()).manageRebindedRoot(location);
+ ((LocationManagerInternal) managementContext.getLocationManager()).manageRebindedRoot(location);
} catch (Exception e) {
exceptionHandler.onManageFailed(BrooklynObjectType.LOCATION, location, e);
}
@@ -777,14 +787,14 @@ public abstract class RebindIteration {
}
// TODO could also see about purging unreferenced locations
cleanupOldLocations(oldLocations);
-
+
// Manage the top-level apps (causing everything under them to become managed)
logRebindingDebug("RebindManager managing entities");
- EntityManagerInternal entityManager = (EntityManagerInternal)managementContext.getEntityManager();
+ EntityManagerInternal entityManager = (EntityManagerInternal) managementContext.getEntityManager();
Set<String> oldEntities = Sets.newLinkedHashSet(entityManager.getEntityIds());
- for (Entity entity: rebindContext.getEntities()) {
+ for (Entity entity : rebindContext.getEntities()) {
ManagementTransitionMode oldMode = updateTransitionMode(entityManager, entity);
- if (oldMode!=null)
+ if (oldMode != null)
oldEntities.remove(entity.getId());
}
List<Application> apps = Lists.newArrayList();
@@ -800,7 +810,7 @@ public abstract class RebindIteration {
exceptionHandler.onManageFailed(BrooklynObjectType.ENTITY, entity, e);
}
if (entity instanceof Application)
- apps.add((Application)entity);
+ apps.add((Application) entity);
}
}
cleanupOldEntities(oldEntities);
@@ -810,10 +820,10 @@ public abstract class RebindIteration {
private <T extends BrooklynObject> ManagementTransitionMode updateTransitionMode(BrooklynObjectManagerInternal<T> boManager, T bo) {
ManagementTransitionMode oldTransitionMode = boManager.getLastManagementTransitionMode(bo.getId());
-
+
Boolean isNowReadOnly = rebindContext.isReadOnly(bo);
- BrooklynObjectManagementMode modeBefore, modeAfter;
- if (oldTransitionMode==null) {
+ BrooklynObjectManagementMode modeBefore, modeAfter;
+ if (oldTransitionMode == null) {
modeBefore = BrooklynObjectManagementMode.UNMANAGED_PERSISTED;
} else {
modeBefore = oldTransitionMode.getModeAfter();
@@ -821,14 +831,14 @@ public abstract class RebindIteration {
if (isRebindingActiveAgain()) {
Preconditions.checkState(!Boolean.TRUE.equals(isNowReadOnly));
- Preconditions.checkState(modeBefore==BrooklynObjectManagementMode.MANAGED_PRIMARY);
+ Preconditions.checkState(modeBefore == BrooklynObjectManagementMode.MANAGED_PRIMARY);
modeAfter = BrooklynObjectManagementMode.MANAGED_PRIMARY;
} else if (isNowReadOnly) {
modeAfter = BrooklynObjectManagementMode.LOADED_READ_ONLY;
} else {
modeAfter = BrooklynObjectManagementMode.MANAGED_PRIMARY;
}
-
+
ManagementTransitionMode newTransitionMode = ManagementTransitionMode.transitioning(modeBefore, modeAfter);
boManager.setManagementTransitionMode(bo, newTransitionMode);
return oldTransitionMode;
@@ -841,24 +851,25 @@ public abstract class RebindIteration {
}
protected abstract void cleanupOldLocations(Set<String> oldLocations);
+
protected abstract void cleanupOldEntities(Set<String> oldEntities);
protected void finishingUp() {
-
+
checkContinuingPhase(8);
-
+
if (!isEmpty) {
- BrooklynLogging.log(LOG, shouldLogRebinding() ? LoggingLevel.INFO : LoggingLevel.DEBUG,
- "Rebind complete " + "("+mode+(readOnlyRebindCount.get()>=0 ? ", iteration "+readOnlyRebindCount : "")+")" +
- " in {}: {} app{}, {} entit{}, {} location{}, {} polic{}, {} enricher{}, {} feed{}, {} catalog item{}, {} catalog bundle{}",
- Time.makeTimeStringRounded(timer), applications.size(), Strings.s(applications),
- rebindContext.getEntities().size(), Strings.ies(rebindContext.getEntities()),
- rebindContext.getLocations().size(), Strings.s(rebindContext.getLocations()),
- rebindContext.getPolicies().size(), Strings.ies(rebindContext.getPolicies()),
- rebindContext.getEnrichers().size(), Strings.s(rebindContext.getEnrichers()),
- rebindContext.getFeeds().size(), Strings.s(rebindContext.getFeeds()),
- rebindContext.getCatalogItems().size(), Strings.s(rebindContext.getCatalogItems()),
- rebindContext.getBundles().size(), Strings.s(rebindContext.getBundles())
+ BrooklynLogging.log(LOG, shouldLogRebinding() ? LoggingLevel.INFO : LoggingLevel.DEBUG,
+ "Rebind complete " + "(" + mode + (readOnlyRebindCount.get() >= 0 ? ", iteration " + readOnlyRebindCount : "") + ")" +
+ " in {}: {} app{}, {} entit{}, {} location{}, {} polic{}, {} enricher{}, {} feed{}, {} catalog item{}, {} catalog bundle{}",
+ Time.makeTimeStringRounded(timer), applications.size(), Strings.s(applications),
+ rebindContext.getEntities().size(), Strings.ies(rebindContext.getEntities()),
+ rebindContext.getLocations().size(), Strings.s(rebindContext.getLocations()),
+ rebindContext.getPolicies().size(), Strings.ies(rebindContext.getPolicies()),
+ rebindContext.getEnrichers().size(), Strings.s(rebindContext.getEnrichers()),
+ rebindContext.getFeeds().size(), Strings.s(rebindContext.getFeeds()),
+ rebindContext.getCatalogItems().size(), Strings.s(rebindContext.getCatalogItems()),
+ rebindContext.getBundles().size(), Strings.s(rebindContext.getBundles())
);
}
@@ -869,11 +880,11 @@ public abstract class RebindIteration {
protected void noteErrors(final RebindExceptionHandler exceptionHandler, Exception primaryException) {
List<Exception> exceptions = exceptionHandler.getExceptions();
List<String> warnings = exceptionHandler.getWarnings();
- if (primaryException!=null || !exceptions.isEmpty() || !warnings.isEmpty()) {
+ if (primaryException != null || !exceptions.isEmpty() || !warnings.isEmpty()) {
List<String> messages = MutableList.<String>of();
- if (primaryException!=null) messages.add(primaryException.toString());
- for (Exception e: exceptions) messages.add(e.toString());
- for (String w: warnings) messages.add(w);
+ if (primaryException != null) messages.add(primaryException.toString());
+ for (Exception e : exceptions) messages.add(e.toString());
+ for (String w : warnings) messages.add(w);
rebindMetrics.noteError(messages);
}
}
@@ -886,16 +897,22 @@ public abstract class RebindIteration {
this.catalogItemId = catalogItemId;
this.searchPath = searchPath;
}
- public String getCatalogItemId() { return catalogItemId; }
- public List<String> getSearchPath() { return searchPath; }
+
+ public String getCatalogItemId() {
+ return catalogItemId;
+ }
+
+ public List<String> getSearchPath() {
+ return searchPath;
+ }
}
protected CatalogItemIdAndSearchPath findCatalogItemIds(ClassLoader cl, Map<String,
- EntityMementoManifest> entityIdToManifest, EntityMementoManifest entityManifest) {
+ EntityMementoManifest> entityIdToManifest, EntityMementoManifest entityManifest) {
if (entityManifest.getCatalogItemId() != null) {
return new CatalogItemIdAndSearchPath(entityManifest.getCatalogItemId(),
- entityManifest.getCatalogItemIdSearchPath());
+ entityManifest.getCatalogItemIdSearchPath());
}
if (BrooklynFeatureEnablement.isEnabled(FEATURE_BACKWARDS_COMPATIBILITY_INFER_CATALOG_ITEM_ON_REBIND)) {
@@ -939,7 +956,7 @@ public abstract class RebindIteration {
while (ptr != null) {
RegisteredType t = types.get(ptr.getType(), BrooklynCatalog.DEFAULT_VERSION);
if (t != null) {
- LOG.debug("Inferred catalog item ID "+t.getId()+" for "+entityManifest+" from ancestor "+ptr);
+ LOG.debug("Inferred catalog item ID " + t.getId() + " for " + entityManifest + " from ancestor " + ptr);
return new CatalogItemIdAndSearchPath(t.getId(), ImmutableList.<String>of());
}
if (ptr.getParent() != null) {
@@ -961,7 +978,7 @@ public abstract class RebindIteration {
boolean canLoadClass = loader.tryLoadClass(entityManifest.getType()).isPresent();
if (canLoadClass) {
LOG.warn("Missing catalog item for " + entityManifest.getId() + " (" + entityManifest.getType()
- + "), inferring as " + item.getId() + " because that is able to load the item");
+ + "), inferring as " + item.getId() + " because that is able to load the item");
return new CatalogItemIdAndSearchPath(item.getId(), ImmutableList.<String>of());
}
}
@@ -986,7 +1003,7 @@ public abstract class RebindIteration {
protected final ClassLoader classLoader;
protected final RebindContextImpl rebindContext;
protected final Reflections reflections;
-
+
protected BrooklynObjectInstantiator(ClassLoader classLoader, RebindContextImpl rebindContext, Reflections reflections) {
this.classLoader = classLoader;
this.rebindContext = rebindContext;
@@ -996,15 +1013,15 @@ public abstract class RebindIteration {
protected Entity newEntity(EntityMementoManifest entityManifest) {
String entityId = entityManifest.getId();
CatalogItemIdAndSearchPath idPath =
- findCatalogItemIds(classLoader, mementoManifest.getEntityIdToManifest(), entityManifest);
+ findCatalogItemIds(classLoader, mementoManifest.getEntityIdToManifest(), entityManifest);
String entityType = entityManifest.getType();
LoadedClass<? extends Entity> loaded =
- load(Entity.class, entityType, idPath.getCatalogItemId(), idPath.getSearchPath(), entityId);
+ load(Entity.class, entityType, idPath.getCatalogItemId(), idPath.getSearchPath(), entityId);
Class<? extends Entity> entityClazz = loaded.clazz;
Entity entity;
-
+
if (InternalFactory.isNewStyle(entityClazz)) {
// Not using entityManager.createEntity(EntitySpec) because don't want init() to be called.
// Creates an uninitialized entity, but that has correct id + proxy.
@@ -1013,13 +1030,13 @@ public abstract class RebindIteration {
} else {
LOG.warn("Deprecated rebind of entity without no-arg constructor; " +
- "this may not be supported in future versions: id=" + entityId+"; type=" + entityType);
+ "this may not be supported in future versions: id=" + entityId + "; type=" + entityType);
// There are several possibilities for the constructor; find one that works.
// Prefer passing in the flags because required for Application to set the management context
// TODO Feels very hacky!
- Map<Object,Object> flags = Maps.newLinkedHashMap();
+ Map<Object, Object> flags = Maps.newLinkedHashMap();
flags.put("id", entityId);
if (AbstractApplication.class.isAssignableFrom(entityClazz)) flags.put("mgmt", managementContext);
@@ -1027,7 +1044,7 @@ public abstract class RebindIteration {
// supplying it as the flag
// (NB: merge reported conflict as the two things were added separately)
entity = invokeConstructor(null, entityClazz,
- new Object[] {flags}, new Object[] {flags, null}, new Object[] {null}, new Object[0]);
+ new Object[]{flags}, new Object[]{flags, null}, new Object[]{null}, new Object[0]);
// In case the constructor didn't take the Map arg, then also set it here.
// e.g. for top-level app instances such as WebClusterDatabaseExampleApp will (often?) not have
@@ -1038,7 +1055,7 @@ public abstract class RebindIteration {
if (entity instanceof AbstractApplication) {
FlagUtils.setFieldsFromFlags(ImmutableMap.of("mgmt", managementContext), entity);
}
- ((AbstractEntity)entity).setManagementContext(managementContext);
+ ((AbstractEntity) entity).setManagementContext(managementContext);
managementContext.prePreManage(entity);
}
@@ -1055,29 +1072,29 @@ public abstract class RebindIteration {
protected <T extends BrooklynObject> LoadedClass<? extends T> load(Class<T> bType, Memento memento) {
return load(bType, memento.getType(), memento.getCatalogItemId(), memento.getCatalogItemIdSearchPath(),
- memento.getId());
+ memento.getId());
}
-
+
@SuppressWarnings("unchecked")
// TODO should prefer a registered type as the type to load (in lieu of jType),
// but note some callers (enrichers etc) use catalogItemId to be the first entry in search path rather than their actual type,
// so until callers are all updated all we can do here is load the java type with no guarantee the catalogItemId should be the same.
// (yoml should help a lot with this.)
protected <T extends BrooklynObject> LoadedClass<? extends T> load(Class<T> bType, String jType,
- String catalogItemId, List<String> searchPath, String contextSuchAsId) {
+ String catalogItemId, List<String> searchPath, String contextSuchAsId) {
checkNotNull(jType, "Type of %s (%s) must not be null", contextSuchAsId, bType.getSimpleName());
CatalogUpgrades.markerForCodeThatLoadsJavaTypesButShouldLoadRegisteredType();
-
+
List<String> warnings = MutableList.of();
List<String> reboundSearchPath = MutableList.of();
if (searchPath != null && !searchPath.isEmpty()) {
for (String searchItemId : searchPath) {
String fixedSearchItemId = null;
OsgiManager osgi = managementContext.getOsgiManager().orNull();
- if (osgi!=null) {
+ if (osgi != null) {
ManagedBundle bundle = osgi.getManagedBundle(VersionedName.fromString(searchItemId));
- if (bundle!=null) {
+ if (bundle != null) {
// found as bundle
reboundSearchPath.add(searchItemId);
continue;
@@ -1086,56 +1103,56 @@ public abstract class RebindIteration {
// look for as a type now
RegisteredType t1 = managementContext.getTypeRegistry().get(searchItemId);
- if (t1==null) {
+ if (t1 == null) {
String newSearchItemId = CatalogUpgrades.getTypeUpgradedIfNecessary(managementContext, searchItemId);
if (!newSearchItemId.equals(searchItemId)) {
- logRebindingDebug("Upgrading search path entry of "+bType.getSimpleName().toLowerCase()+" "+contextSuchAsId+" from "+searchItemId+" to "+newSearchItemId);
+ logRebindingDebug("Upgrading search path entry of " + bType.getSimpleName().toLowerCase() + " " + contextSuchAsId + " from " + searchItemId + " to " + newSearchItemId);
searchItemId = newSearchItemId;
t1 = managementContext.getTypeRegistry().get(newSearchItemId);
}
}
- if (t1!=null) fixedSearchItemId = t1.getId();
- if (fixedSearchItemId==null) {
+ if (t1 != null) fixedSearchItemId = t1.getId();
+ if (fixedSearchItemId == null) {
CatalogItem<?, ?> ci = findCatalogItemInReboundCatalog(bType, searchItemId, contextSuchAsId);
- if (ci!=null) {
+ if (ci != null) {
fixedSearchItemId = ci.getCatalogItemId();
- logRebindingWarn("Needed rebind catalog to resolve search path entry "+searchItemId+" (now "+fixedSearchItemId+") for "+bType.getSimpleName().toLowerCase()+" "+contextSuchAsId+
- ", persistence should remove this in future but future versions will not support this and definitions should be fixed");
+ logRebindingWarn("Needed rebind catalog to resolve search path entry " + searchItemId + " (now " + fixedSearchItemId + ") for " + bType.getSimpleName().toLowerCase() + " " + contextSuchAsId +
+ ", persistence should remove this in future but future versions will not support this and definitions should be fixed");
} else {
- logRebindingWarn("Could not find search path entry "+searchItemId+" for "+bType.getSimpleName().toLowerCase()+" "+contextSuchAsId+", ignoring");
+ logRebindingWarn("Could not find search path entry " + searchItemId + " for " + bType.getSimpleName().toLowerCase() + " " + contextSuchAsId + ", ignoring");
}
}
if (fixedSearchItemId != null) {
reboundSearchPath.add(fixedSearchItemId);
} else {
- warnings.add("unable to resolve search path entry "+ searchItemId);
+ warnings.add("unable to resolve search path entry " + searchItemId);
}
}
}
if (catalogItemId != null) {
String transformedCatalogItemId = null;
-
+
Maybe<RegisteredType> contextRegisteredType = managementContext.getTypeRegistry().getMaybe(catalogItemId,
- // this is context RT, not item we are loading, so bType does not apply here
- // if we were instantiating from an RT instead of a JT (ideal) then we would use bType to filter
- null );
+ // this is context RT, not item we are loading, so bType does not apply here
+ // if we were instantiating from an RT instead of a JT (ideal) then we would use bType to filter
+ null);
if (contextRegisteredType.isAbsent()) {
transformedCatalogItemId = CatalogUpgrades.getTypeUpgradedIfNecessary(managementContext, catalogItemId);
if (!transformedCatalogItemId.equals(catalogItemId)) {
// catalog item id is sometimes the type of the item, but sometimes just the first part of the search path
- logRebindingInfo("Upgrading "+bType.getSimpleName().toLowerCase()+" "+contextSuchAsId+
- " stored catalog item context on rebind"+
- " from "+catalogItemId+" to "+transformedCatalogItemId);
-
+ logRebindingInfo("Upgrading " + bType.getSimpleName().toLowerCase() + " " + contextSuchAsId +
+ " stored catalog item context on rebind" +
+ " from " + catalogItemId + " to " + transformedCatalogItemId);
+
// again ignore bType
contextRegisteredType = managementContext.getTypeRegistry().getMaybe(transformedCatalogItemId, null);
-
+
} else {
transformedCatalogItemId = null;
}
}
-
+
if (contextRegisteredType.isPresent()) {
transformedCatalogItemId = contextRegisteredType.get().getId();
} else {
@@ -1144,20 +1161,20 @@ public abstract class RebindIteration {
transformedCatalogItemId = catalogItem.getCatalogItemId();
}
}
- if (transformedCatalogItemId!=null) {
+ if (transformedCatalogItemId != null) {
try {
BrooklynClassLoadingContextSequential loader =
- new BrooklynClassLoadingContextSequential(managementContext);
+ new BrooklynClassLoadingContextSequential(managementContext);
loader.add(newClassLoadingContextForCatalogItems(managementContext, transformedCatalogItemId,
- reboundSearchPath));
+ reboundSearchPath));
return new LoadedClass<T>(loader.loadClass(jType, bType), transformedCatalogItemId, reboundSearchPath);
} catch (Exception e) {
Exceptions.propagateIfFatal(e);
- warnings.add("unable to load class "+jType+" for resovled context type "+transformedCatalogItemId);
+ warnings.add("unable to load class " + jType + " for resovled context type " + transformedCatalogItemId);
}
} else {
// TODO fail, rather than fallback to java?
- warnings.add("unable to resolve context type "+catalogItemId);
+ warnings.add("unable to resolve context type " + catalogItemId);
}
} else {
// can happen for enrichers etc added by java, and for BasicApplication when things are deployed;
@@ -1165,9 +1182,9 @@ public abstract class RebindIteration {
}
try {
- Class<T> jTypeC = (Class<T>)loadClass(jType);
+ Class<T> jTypeC = (Class<T>) loadClass(jType);
if (!warnings.isEmpty()) {
- LOG.warn("Loaded java type "+jType+" for "+bType.getSimpleName().toLowerCase()+" "+contextSuchAsId+" but had errors: "+Strings.join(warnings, ";"));
+ LOG.warn("Loaded java type " + jType + " for " + bType.getSimpleName().toLowerCase() + " " + contextSuchAsId + " but had errors: " + Strings.join(warnings, ";"));
}
return new LoadedClass<T>(jTypeC, catalogItemId, reboundSearchPath);
} catch (Exception e) {
@@ -1175,10 +1192,10 @@ public abstract class RebindIteration {
}
if (catalogItemId != null) {
- String msg = "Class "+jType+" not found for "+bType.getSimpleName().toLowerCase()+" "+contextSuchAsId+" ("+catalogItemId+"): "+Strings.join(warnings, ";");
- LOG.warn(msg+" (rethrowing)");
+ String msg = "Class " + jType + " not found for " + bType.getSimpleName().toLowerCase() + " " + contextSuchAsId + " (" + catalogItemId + "): " + Strings.join(warnings, ";");
+ LOG.warn(msg + " (rethrowing)");
throw new IllegalStateException(msg);
-
+
} else if (BrooklynFeatureEnablement.isEnabled(FEATURE_BACKWARDS_COMPATIBILITY_INFER_CATALOG_ITEM_ON_REBIND)) {
//Try loading from whichever catalog bundle succeeds (legacy CI items only; also disabling this, as no longer needed 2017-09)
BrooklynCatalog catalog = managementContext.getCatalog();
@@ -1186,26 +1203,26 @@ public abstract class RebindIteration {
BrooklynClassLoadingContext catalogLoader = CatalogUtils.newClassLoadingContext(managementContext, item);
Maybe<Class<?>> catalogClass = catalogLoader.tryLoadClass(jType);
if (catalogClass.isPresent()) {
- LOG.warn("Falling back to java type "+jType+" for "+bType.getSimpleName().toLowerCase()+" "+contextSuchAsId+" using catalog search paths, found on "+item+
- (warnings.isEmpty() ? "" : ", after errors: "+Strings.join(warnings, ";")));
+ LOG.warn("Falling back to java type " + jType + " for " + bType.getSimpleName().toLowerCase() + " " + contextSuchAsId + " using catalog search paths, found on " + item +
+ (warnings.isEmpty() ? "" : ", after errors: " + Strings.join(warnings, ";")));
return new LoadedClass<T>((Class<? extends T>) catalogClass.get(), catalogItemId, reboundSearchPath);
}
}
- String msg = "Class "+jType+" not found for "+bType.getSimpleName().toLowerCase()+" "+contextSuchAsId+", even after legacy global classpath search"+
- (warnings.isEmpty() ? "" : ": "+Strings.join(warnings, ";"));
- LOG.warn(msg+" (rethrowing)");
+ String msg = "Class " + jType + " not found for " + bType.getSimpleName().toLowerCase() + " " + contextSuchAsId + ", even after legacy global classpath search" +
+ (warnings.isEmpty() ? "" : ": " + Strings.join(warnings, ";"));
+ LOG.warn(msg + " (rethrowing)");
throw new IllegalStateException(msg);
} else {
- String msg = "Class "+jType+" not found for "+bType.getSimpleName().toLowerCase()+" "+contextSuchAsId+
- (warnings.isEmpty() ? "" : ": "+Strings.join(warnings, ";"));
- LOG.warn(msg+" (rethrowing)");
+ String msg = "Class " + jType + " not found for " + bType.getSimpleName().toLowerCase() + " " + contextSuchAsId +
+ (warnings.isEmpty() ? "" : ": " + Strings.join(warnings, ";"));
+ LOG.warn(msg + " (rethrowing)");
throw new IllegalStateException(msg);
}
}
private <T extends BrooklynObject> CatalogItem<?, ?> findCatalogItemInReboundCatalog(Class<T> bType,
- String catalogItemId, String contextSuchAsId) {
+ String catalogItemId, String contextSuchAsId) {
CatalogItem<?, ?> catalogItem = rebindContext.lookup().lookupCatalogItem(catalogItemId);
if (catalogItem == null) {
if (BrooklynFeatureEnablement.isEnabled(FEATURE_AUTO_FIX_CATALOG_REF_ON_REBIND)) {
@@ -1220,8 +1237,8 @@ public abstract class RebindIteration {
if (catalogItem != null) {
LOG.warn("Unable to load catalog item " + catalogItemId + " for " + contextSuchAsId
- + " (" + bType.getSimpleName() + "); will auto-upgrade to "
- + catalogItem.getCatalogItemId() + ":" + catalogItem.getVersion());
+ + " (" + bType.getSimpleName() + "); will auto-upgrade to "
+ + catalogItem.getCatalogItemId() + ":" + catalogItem.getVersion());
}
}
}
@@ -1231,7 +1248,7 @@ public abstract class RebindIteration {
protected Class<?> loadClass(String jType) throws ClassNotFoundException {
try {
- return reflections.loadClass(jType);
+ return reflections.loadClass(jType);
} catch (Exception e) {
Exceptions.propagateIfFatal(e);
}
@@ -1262,19 +1279,19 @@ public abstract class RebindIteration {
Location location = locationFactory.constructLocation(locationClazz);
FlagUtils.setFieldsFromFlags(ImmutableMap.of("id", locationId), location);
managementContext.prePreManage(location);
- ((AbstractLocation)location).setManagementContext(managementContext);
+ ((AbstractLocation) location).setManagementContext(managementContext);
return location;
} else {
LOG.warn("Deprecated rebind of location without no-arg constructor; " +
- "this may not be supported in future versions: id=" + locationId + "; type="+locationType);
+ "this may not be supported in future versions: id=" + locationId + "; type=" + locationType);
// There are several possibilities for the constructor; find one that works.
// Prefer passing in the flags because required for Application to set the management context
// TODO Feels very hacky!
- Map<String,?> flags = MutableMap.of("id", locationId, "deferConstructionChecks", true);
+ Map<String, ?> flags = MutableMap.of("id", locationId, "deferConstructionChecks", true);
- return invokeConstructor(reflections, locationClazz, new Object[] {flags});
+ return invokeConstructor(reflections, locationClazz, new Object[]{flags});
}
// note 'used' config keys get marked in BasicLocationRebindSupport
}
@@ -1292,25 +1309,25 @@ public abstract class RebindIteration {
InternalPolicyFactory policyFactory = managementContext.getPolicyFactory();
policy = policyFactory.constructPolicy(policyClazz);
FlagUtils.setFieldsFromFlags(ImmutableMap.of("id", id), policy);
- ((AbstractPolicy)policy).setManagementContext(managementContext);
- ((AbstractPolicy)policy).setHighlights(memento.getHighlights());
+ ((AbstractPolicy) policy).setManagementContext(managementContext);
+ ((AbstractPolicy) policy).setHighlights(memento.getHighlights());
} else {
LOG.warn("Deprecated rebind of policy without no-arg constructor; " +
- "this may not be supported in future versions: id=" + id + "; type="+policyClazz);
+ "this may not be supported in future versions: id=" + id + "; type=" + policyClazz);
// There are several possibilities for the constructor; find one that works.
// Prefer passing in the flags because required for Application to set the management context
// TODO Feels very hacky!
Map<String, Object> flags = MutableMap.<String, Object>of(
- "id", id,
- "deferConstructionChecks", true,
- "noConstructionInit", true);
+ "id", id,
+ "deferConstructionChecks", true,
+ "noConstructionInit", true);
flags.putAll(memento.getConfig());
- policy = invokeConstructor(null, policyClazz, new Object[] {flags});
+ policy = invokeConstructor(null, policyClazz, new Object[]{flags});
}
-
+
setCatalogItemIds(policy, loaded.catalogItemId, loaded.searchPath);
return policy;
}
@@ -1328,24 +1345,24 @@ public abstract class RebindIteration {
InternalPolicyFactory policyFactory = managementContext.getPolicyFactory();
enricher = policyFactory.constructEnricher(enricherClazz);
FlagUtils.setFieldsFromFlags(ImmutableMap.of("id", id), enricher);
- ((AbstractEnricher)enricher).setManagementContext(managementContext);
+ ((AbstractEnricher) enricher).setManagementContext(managementContext);
} else {
LOG.warn("Deprecated rebind of enricher without no-arg constructor; " +
- "this may not be supported in future versions: id=" + id + "; type="+enricherClazz);
+ "this may not be supported in future versions: id=" + id + "; type=" + enricherClazz);
// There are several possibilities for the constructor; find one that works.
// Prefer passing in the flags because required for Application to set the management context
// TODO Feels very hacky!
Map<String, Object> flags = MutableMap.<String, Object>of(
- "id", id,
- "deferConstructionChecks", true,
- "noConstructionInit", true);
+ "id", id,
+ "deferConstructionChecks", true,
+ "noConstructionInit", true);
flags.putAll(memento.getConfig());
- enricher = invokeConstructor(reflections, enricherClazz, new Object[] {flags});
+ enricher = invokeConstructor(reflections, enricherClazz, new Object[]{flags});
}
-
+
setCatalogItemIds(enricher, loaded.catalogItemId, loaded.searchPath);
return enricher;
}
@@ -1363,18 +1380,18 @@ public abstract class RebindIteration {
InternalPolicyFactory policyFactory = managementContext.getPolicyFactory();
feed = policyFactory.constructFeed(feedClazz);
FlagUtils.setFieldsFromFlags(ImmutableMap.of("id", id), feed);
- ((AbstractFeed)feed).setManagementContext(managementContext);
+ ((AbstractFeed) feed).setManagementContext(managementContext);
} else {
throw new IllegalStateException("rebind of feed without no-arg constructor unsupported: id=" + id +
- "; type="+feedClazz);
+ "; type=" + feedClazz);
}
-
+
setCatalogItemIds(feed, loaded.catalogItemId, loaded.searchPath);
return feed;
}
- @SuppressWarnings({ "rawtypes" })
+ @SuppressWarnings({"rawtypes"})
protected CatalogItem<?, ?> newCatalogItem(CatalogItemMemento memento) {
String id = memento.getId();
// catalog item subtypes are internal to brooklyn, not in osgi
@@ -1389,7 +1406,7 @@ public abstract class RebindIteration {
FlagUtils.setFieldsFromFlags(ImmutableMap.of("id", memento.getId()), result);
return result;
}
-
+
protected <T> T invokeConstructor(Reflections reflections, Class<T> clazz, Object[]... possibleArgs) {
for (Object[] args : possibleArgs) {
try {
@@ -1402,29 +1419,31 @@ public abstract class RebindIteration {
}
}
StringBuilder args = new StringBuilder();
- if (possibleArgs.length<1) args.append("no possible argument sets supplied; error");
- else if (possibleArgs.length<2) args.append("args are "+Arrays.asList(possibleArgs[0]));
+ if (possibleArgs.length < 1) args.append("no possible argument sets supplied; error");
+ else if (possibleArgs.length < 2) args.append("args are " + Arrays.asList(possibleArgs[0]));
else {
- args.append("args are "+Arrays.asList(possibleArgs[0]));
- for (int i=1; i<possibleArgs.length; i++) {
+ args.append("args are " + Arrays.asList(possibleArgs[0]));
+ for (int i = 1; i < possibleArgs.length; i++) {
args.append(" or ");
args.append(Arrays.asList(possibleArgs[i]));
}
}
throw new IllegalStateException("Cannot instantiate instance of type " + clazz +
- "; expected constructor signature not found ("+args+")");
+ "; expected constructor signature not found (" + args + ")");
}
}
protected BrooklynMementoPersister getPersister() {
return rebindManager.getPersister();
}
-
+
protected <T extends TreeNode> Map<String, T> sortParentFirst(Map<String, T> nodes) {
return RebindManagerImpl.sortParentFirst(nodes);
}
- /** logs at debug, except during subsequent read-only rebinds, in which it logs trace */
+ /**
+ * logs at debug, except during subsequent read-only rebinds, in which it logs trace
+ */
protected void logRebindingDebug(String message, Object... args) {
if (shouldLogRebinding()) {
LOG.debug(message, args);
@@ -1432,8 +1451,10 @@ public abstract class RebindIteration {
LOG.trace(message, args);
}
}
-
- /** logs at info, except during subsequent read-only rebinds, in which it logs trace */
+
+ /**
+ * logs at info, except during subsequent read-only rebinds, in which it logs trace
+ */
protected void logRebindingInfo(String message, Object... args) {
if (shouldLogRebinding()) {
LOG.info(message, args);
@@ -1441,8 +1462,10 @@ public abstract class RebindIteration {
LOG.trace(message, args);
}
}
-
- /** logs at warn, except during subsequent read-only rebinds, in which it logs trace */
+
+ /**
+ * logs at warn, except during subsequent read-only rebinds, in which it logs trace
+ */
protected void logRebindingWarn(String message, Object... args) {
if (shouldLogRebinding()) {
LOG.warn(message, args);
@@ -1450,9 +1473,9 @@ public abstract class RebindIteration {
LOG.trace(message, args);
}
}
-
+
protected boolean shouldLogRebinding() {
- return (readOnlyRebindCount.get() < 5) || (readOnlyRebindCount.get()%1000==0);
+ return (readOnlyRebindCount.get() < 5) || (readOnlyRebindCount.get() % 1000 == 0);
}
}
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
index 3a597ab..8f6d6af 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
@@ -312,6 +312,7 @@ public class RebindManagerImpl implements RebindManager {
@Override
public void startReadOnly(final ManagementNodeState mode) {
+ LOG.debug("Starting RO rebind for "+mode+": "+this);
if (!ManagementNodeState.isHotProxy(mode)) {
throw new IllegalStateException("Read-only rebind thread only permitted for hot proxy modes; not "+mode);
}
@@ -370,6 +371,7 @@ public class RebindManagerImpl implements RebindManager {
}}).build();
}
};
+ LOG.debug("Submitted scheduled RO rebind task for "+mode+": "+this);
readOnlyTask = (ScheduledTask) managementContext.getServerExecutionContext().submit(
ScheduledTask.builder(taskFactory).displayName("scheduled:[periodic-read-only-rebind]").period(periodicPersistPeriod).build() );
}
@@ -409,6 +411,7 @@ public class RebindManagerImpl implements RebindManager {
@Override
public void stop() {
+ LOG.debug("Stopping rebind manager "+this);
stopReadOnly();
stopPersistence();
if (persistenceStoreAccess != null) persistenceStoreAccess.stop(true);
[brooklyn-server] 22/27: tidy up a few more places where we get
warns on unmanaging
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 8477a27eb2d24960b9340eb57a30e03b0c706a00
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 17:46:59 2021 +0100
tidy up a few more places where we get warns on unmanaging
add a new marker to tell us if we are unmanaging
---
.../main/java/org/apache/brooklyn/core/entity/Entities.java | 9 +++++++--
.../brooklyn/core/entity/lifecycle/ServiceStateLogic.java | 11 ++++++++---
.../brooklyn/core/mgmt/internal/EntityManagementSupport.java | 10 +++++++++-
.../apache/brooklyn/util/core/task/BasicExecutionManager.java | 7 ++++---
.../apache/brooklyn/policy/failover/ElectPrimaryPolicy.java | 2 +-
.../brooklyn/policy/failover/PrimaryRunningEnricher.java | 5 +++++
6 files changed, 34 insertions(+), 10 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/core/entity/Entities.java b/core/src/main/java/org/apache/brooklyn/core/entity/Entities.java
index 43a5104..52d611f 100644
--- a/core/src/main/java/org/apache/brooklyn/core/entity/Entities.java
+++ b/core/src/main/java/org/apache/brooklyn/core/entity/Entities.java
@@ -799,12 +799,17 @@ public class Entities {
/** If entity is under management and the management node is the primary for this entity, i.e. not read-only. */
public static boolean isManagedActive(Entity e) {
- return ((EntityInternal)e).getManagementSupport().isDeployed() && ((EntityInternal)e).getManagementContext().isRunning() && !isReadOnly(e);
+ return ((EntityInternal)e).getManagementSupport().isActive() && ((EntityInternal)e).getManagementContext().isRunning() && !isReadOnly(e);
+ }
+
+ /** If entity is under management or coming up and the management node is the primary for this entity, i.e. not read-only. */
+ public static boolean isManagedActiveOrComingUp(Entity e) {
+ return (isManagedActive(e) || !((EntityInternal)e).getManagementSupport().wasDeployed()) && !isReadOnly(e);
}
/** If entity is under management either as primary or in read-only (hot-standby) state. */
public static boolean isManagedOrReadOnly(Entity e) {
- return ((EntityInternal)e).getManagementSupport().isDeployed() && ((EntityInternal)e).getManagementContext().isRunning();
+ return ((EntityInternal)e).getManagementSupport().isActive() && ((EntityInternal)e).getManagementContext().isRunning();
}
public static boolean isNoLongerManaged(Entity e) {
diff --git a/core/src/main/java/org/apache/brooklyn/core/entity/lifecycle/ServiceStateLogic.java b/core/src/main/java/org/apache/brooklyn/core/entity/lifecycle/ServiceStateLogic.java
index 8ab3d27..2fc948f 100644
--- a/core/src/main/java/org/apache/brooklyn/core/entity/lifecycle/ServiceStateLogic.java
+++ b/core/src/main/java/org/apache/brooklyn/core/entity/lifecycle/ServiceStateLogic.java
@@ -185,7 +185,7 @@ public class ServiceStateLogic {
private static void waitBrieflyForServiceUpIfStateIsRunning(Entity entity, Lifecycle state) {
if (state==Lifecycle.RUNNING) {
Boolean up = entity.getAttribute(Attributes.SERVICE_UP);
- if (!Boolean.TRUE.equals(up) && !Entities.isReadOnly(entity)) {
+ if (!Boolean.TRUE.equals(up) && Entities.isManagedActive(entity)) {
// pause briefly to allow any recent problem-clearing processing to complete
Stopwatch timer = Stopwatch.createStarted();
boolean nowUp = Repeater.create()
@@ -196,8 +196,10 @@ public class ServiceStateLogic {
if (nowUp) {
log.debug("Had to wait "+Duration.of(timer)+" for "+entity+" "+Attributes.SERVICE_UP+" to be true before setting "+state);
} else {
- log.warn("Service is not up when setting "+state+" on "+entity+"; delayed "+Duration.of(timer)+" "
- + "but "+Attributes.SERVICE_UP+" did not recover from "+up+"; not-up-indicators="+entity.getAttribute(Attributes.SERVICE_NOT_UP_INDICATORS));
+ if (Entities.isManagedActive(entity)) {
+ log.warn("Service is not up when setting " + state + " on " + entity + "; delayed " + Duration.of(timer) + " "
+ + "but " + Attributes.SERVICE_UP + " did not recover from " + up + "; not-up-indicators=" + entity.getAttribute(Attributes.SERVICE_NOT_UP_INDICATORS));
+ }
}
}
}
@@ -322,6 +324,9 @@ public class ServiceStateLogic {
if (Boolean.TRUE.equals(serviceUp) && (problems==null || problems.isEmpty())) {
return Maybe.of(Lifecycle.RUNNING);
} else {
+ if (!Entities.isManagedActive(entity)) {
+ return Maybe.absent("entity not managed active");
+ }
if (!Lifecycle.ON_FIRE.equals(entity.getAttribute(SERVICE_STATE_ACTUAL))) {
Lifecycle.Transition serviceExpected = entity.getAttribute(SERVICE_STATE_EXPECTED);
// very occasional race here; might want to give a grace period if entity has just transitioned,
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/EntityManagementSupport.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/EntityManagementSupport.java
index 7bb36fd..7dd074e 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/EntityManagementSupport.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/EntityManagementSupport.java
@@ -91,6 +91,7 @@ public class EntityManagementSupport {
protected transient volatile ExecutionContext executionContext;
protected final AtomicBoolean managementContextUsable = new AtomicBoolean(false);
+ protected final AtomicBoolean currentlyStopping = new AtomicBoolean(false);
protected final AtomicBoolean currentlyDeployed = new AtomicBoolean(false);
protected final AtomicBoolean everDeployed = new AtomicBoolean(false);
protected Boolean readOnly = null;
@@ -107,12 +108,17 @@ public class EntityManagementSupport {
}
/**
- * Use this instead of !Entities.isManaged(entity) to avoid skipping publishing ov values that to be published before the entity starts.
+ * Use this instead of negating {@link Entities#isManaged(Entity)} to avoid skipping publishing of values that to be published before the entity is deployed;
+ * or (better) see {@link Entities#isManagedActiveOrComingUp(Entity)}
*/
public boolean isNoLongerManaged() {
return wasDeployed() && !isDeployed();
}
+ public boolean isActive() { return !isUnmanaging() && isDeployed(); }
+
+ public boolean isUnmanaging() { return currentlyStopping.get(); }
+
/** whether entity has ever been deployed (managed) */
public boolean wasDeployed() {
return everDeployed.get();
@@ -304,6 +310,7 @@ public class EntityManagementSupport {
@SuppressWarnings("deprecation")
public void onManagementStopping(ManagementTransitionInfo info, boolean wasDryRun) {
synchronized (this) {
+ currentlyStopping.set(true);
if (!wasDryRun) {
if (managementContext != info.getManagementContext()) {
throw new IllegalStateException("onManagementStopping encountered different management context for " + entity +
@@ -383,6 +390,7 @@ public class EntityManagementSupport {
entityChangeListener = EntityChangeListener.NOOP;
managementContextUsable.set(false);
currentlyDeployed.set(false);
+ currentlyStopping.set(false);
executionContext = null;
subscriptionContext = null;
}
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
index 88812e0..0498c4f 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
@@ -418,13 +418,14 @@ public class BasicExecutionManager implements ExecutionManager {
if (context != null && !Entities.isManaged(context)) {
log.debug("Deleting active task on unmanagement of " + context + ": " + removed);
} else {
- if (removed.isDone()) {
+ boolean debugOnly = removed.isDone();
+
+ if (debugOnly) {
log.debug("Deleting cancelled task before completion: " + removed + "; this task will continue to run in the background outwith " + this);
} else {
log.warn("Deleting submitted task before completion: " + removed + " (tags " + removed.getTags() + "); this task will continue to run in the background outwith " + this + ", but perhaps it should have been cancelled?");
-
+ log.debug("Active task deletion trace", new Throwable("Active task deletion trace"));
}
- log.debug("Active task deletion trace", new Throwable("Active task deletion trace"));
}
}
task.getTags().forEach(t -> {
diff --git a/policy/src/main/java/org/apache/brooklyn/policy/failover/ElectPrimaryPolicy.java b/policy/src/main/java/org/apache/brooklyn/policy/failover/ElectPrimaryPolicy.java
index ac9531d..35adfbf 100644
--- a/policy/src/main/java/org/apache/brooklyn/policy/failover/ElectPrimaryPolicy.java
+++ b/policy/src/main/java/org/apache/brooklyn/policy/failover/ElectPrimaryPolicy.java
@@ -261,7 +261,7 @@ public class ElectPrimaryPolicy extends AbstractPolicy implements ElectPrimaryCo
}
} catch (Throwable e) {
Exceptions.propagateIfFatal(e);
- if (Entities.isNoLongerManaged(entity)) throw Exceptions.propagate(e);
+ if (!Entities.isManagedActive(entity)) throw Exceptions.propagate(e);
Throwable root = Throwables.getRootCause(e);
if (root instanceof UserFacingException) {
diff --git a/policy/src/main/java/org/apache/brooklyn/policy/failover/PrimaryRunningEnricher.java b/policy/src/main/java/org/apache/brooklyn/policy/failover/PrimaryRunningEnricher.java
index 0353aa8..0c8b08c 100644
--- a/policy/src/main/java/org/apache/brooklyn/policy/failover/PrimaryRunningEnricher.java
+++ b/policy/src/main/java/org/apache/brooklyn/policy/failover/PrimaryRunningEnricher.java
@@ -24,6 +24,7 @@ import org.apache.brooklyn.api.sensor.SensorEventListener;
import org.apache.brooklyn.config.ConfigKey;
import org.apache.brooklyn.core.enricher.AbstractEnricher;
import org.apache.brooklyn.core.entity.Attributes;
+import org.apache.brooklyn.core.entity.Entities;
import org.apache.brooklyn.core.entity.lifecycle.Lifecycle;
import org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic;
import org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic.ServiceNotUpLogic;
@@ -55,6 +56,10 @@ public class PrimaryRunningEnricher extends AbstractEnricher implements SensorEv
@Override
public void onEvent(SensorEvent<Object> event) {
+ if (!Entities.isManagedActive(entity)) {
+ return;
+ }
+
Entity primary = entity.getAttribute( Sensors.newSensor(Entity.class, config().get(PRIMARY_SENSOR_NAME)) );
if (primary==null) {
ServiceNotUpLogic.updateNotUpIndicator(entity, "primary.enricher", "no primary found");
[brooklyn-server] 16/27: better cleanup on switch from RO/hot
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 751b7ff77919a46728faa98f8e203314cb9d870b
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 13:52:22 2021 +0100
better cleanup on switch from RO/hot
---
.../brooklyn/core/mgmt/rebind/RebindIteration.java | 28 ++-------------
.../core/mgmt/rebind/RebindManagerImpl.java | 42 +++++++++++++++++++++-
2 files changed, 43 insertions(+), 27 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
index cfbd400..69dd588 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
@@ -264,32 +264,8 @@ public abstract class RebindIteration {
protected void doRun() throws Exception {
if (readOnlyRebindCount.get() > 1) {
- // wait for tasks
- Collection<Task<?>> entityTasks = ((BasicExecutionManager) managementContext.getExecutionManager()).allTasksLive()
- .stream().filter(t -> BrooklynTaskTags.getContextEntity(t) != null).collect(Collectors.toList());
- List<Task<?>> openTasksIncludingCancelled;
- CountdownTimer time = CountdownTimer.newInstanceStarted(Duration.seconds(15));
- do {
- openTasksIncludingCancelled = entityTasks.stream().filter(t -> !t.isDone(true)).collect(Collectors.toList());
- List<Task<?>> openTasksCancellable = openTasksIncludingCancelled.stream().filter(t -> !t.isDone()).collect(Collectors.toList());
- if (openTasksIncludingCancelled.isEmpty()) break;
- if (time.isExpired() && !openTasksCancellable.isEmpty()) {
- LOG.warn("Aborting " + openTasksCancellable.size() + " incomplete task(s) before rebinding again: " + openTasksCancellable);
- openTasksCancellable.forEach(t -> t.cancel(true));
- }
- if (time.getDurationElapsed().isShorterThan(Duration.millis(200))) {
- LOG.info("Waiting on " + openTasksIncludingCancelled.size() + " task(s) before rebinding again: " + openTasksIncludingCancelled);
- }
- LOG.debug("Waiting on " + openTasksIncludingCancelled.size() + " task(s) before rebinding again, details: " +
- openTasksIncludingCancelled.stream().map(t -> ""+t+"("+BrooklynTaskTags.getContextEntity(t)+")").collect(Collectors.toList()));
- Time.sleep(Duration.millis(200));
- } while (true);
-
- entityTasks.forEach(((BasicExecutionManager) managementContext.getExecutionManager())::deleteTask);
-
- List<Task<?>> otherDoneTasks = ((BasicExecutionManager) managementContext.getExecutionManager()).allTasksLive()
- .stream().filter(t -> t.isDone(true)).collect(Collectors.toList());
- otherDoneTasks.forEach(((BasicExecutionManager) managementContext.getExecutionManager())::deleteTask);
+ // prevent leaking
+ rebindManager.stopEntityAndDoneTasksBeforeRebinding();
}
loadManifestFiles();
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
index 57395c7..2db7063 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
@@ -20,6 +20,7 @@ package org.apache.brooklyn.core.mgmt.rebind;
import static com.google.common.base.Preconditions.checkNotNull;
+import java.util.Collection;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
@@ -29,6 +30,7 @@ import java.util.concurrent.Semaphore;
import java.util.concurrent.TimeoutException;
import java.util.concurrent.atomic.AtomicInteger;
+import java.util.stream.Collectors;
import org.apache.brooklyn.api.entity.Application;
import org.apache.brooklyn.api.entity.Entity;
import org.apache.brooklyn.api.mgmt.ExecutionContext;
@@ -48,7 +50,9 @@ import org.apache.brooklyn.core.BrooklynFeatureEnablement;
import org.apache.brooklyn.core.config.ConfigKeys;
import org.apache.brooklyn.core.enricher.AbstractEnricher;
import org.apache.brooklyn.core.entity.Entities;
+import org.apache.brooklyn.core.mgmt.BrooklynTaskTags;
import org.apache.brooklyn.core.mgmt.ha.HighAvailabilityManagerImpl;
+import org.apache.brooklyn.core.mgmt.internal.LocalManagementContext;
import org.apache.brooklyn.core.mgmt.internal.ManagementContextInternal;
import org.apache.brooklyn.core.mgmt.persist.BrooklynMementoPersisterToObjectStore;
import org.apache.brooklyn.core.mgmt.persist.BrooklynPersistenceUtils;
@@ -61,11 +65,14 @@ import org.apache.brooklyn.util.collections.MutableMap;
import org.apache.brooklyn.util.collections.QuorumCheck;
import org.apache.brooklyn.util.collections.QuorumCheck.QuorumChecks;
import org.apache.brooklyn.util.core.task.BasicExecutionContext;
+import org.apache.brooklyn.util.core.task.BasicExecutionManager;
import org.apache.brooklyn.util.core.task.ScheduledTask;
import org.apache.brooklyn.util.core.task.Tasks;
import org.apache.brooklyn.util.exceptions.Exceptions;
import org.apache.brooklyn.util.exceptions.RuntimeInterruptedException;
+import org.apache.brooklyn.util.time.CountdownTimer;
import org.apache.brooklyn.util.time.Duration;
+import org.apache.brooklyn.util.time.Time;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -148,7 +155,7 @@ public class RebindManagerImpl implements RebindManager {
private PersistenceActivityMetrics persistMetrics = new PersistenceActivityMetrics();
Integer firstRebindAppCount, firstRebindEntityCount, firstRebindItemCount;
-
+
/**
* For tracking if rebinding, for {@link AbstractEnricher#isRebinding()} etc.
*
@@ -391,6 +398,39 @@ public class RebindManagerImpl implements RebindManager {
readOnlyTask = null;
LOG.debug("Stopped read-only rebinding ("+this+"), mgmt "+managementContext.getManagementNodeId());
}
+ stopEntityAndDoneTasksBeforeRebinding();
+ }
+
+ public void stopEntityAndDoneTasksBeforeRebinding() {
+ // wait for tasks
+ Collection<Task<?>> entityTasks = ((BasicExecutionManager) managementContext.getExecutionManager()).allTasksLive()
+ .stream().filter(t -> BrooklynTaskTags.getContextEntity(t) != null).collect(Collectors.toList());
+ List<Task<?>> openTasksIncludingCancelled;
+ CountdownTimer time = CountdownTimer.newInstanceStarted(Duration.seconds(15));
+ do {
+ openTasksIncludingCancelled = entityTasks.stream().filter(t -> !t.isDone(true)).collect(Collectors.toList());
+ List<Task<?>> openTasksCancellable = openTasksIncludingCancelled.stream().filter(t -> !t.isDone()).collect(Collectors.toList());
+ if (openTasksIncludingCancelled.isEmpty()) break;
+ if (time.isExpired() && !openTasksCancellable.isEmpty()) {
+ LOG.warn("Aborting " + openTasksCancellable.size() + " incomplete task(s) before rebinding again: " + openTasksCancellable);
+ openTasksCancellable.forEach(t -> t.cancel(true));
+ }
+ if (time.getDurationElapsed().isShorterThan(Duration.millis(200))) {
+ LOG.info("Waiting on " + openTasksIncludingCancelled.size() + " task(s) before rebinding again: " + openTasksIncludingCancelled);
+ }
+ LOG.debug("Waiting on " + openTasksIncludingCancelled.size() + " task(s) before rebinding again, details: " +
+ openTasksIncludingCancelled.stream().map(t -> ""+t+"("+BrooklynTaskTags.getContextEntity(t)+")").collect(Collectors.toList()));
+ Time.sleep(Duration.millis(200));
+ } while (true);
+
+ entityTasks.forEach(((BasicExecutionManager) managementContext.getExecutionManager())::deleteTask);
+
+ List<Task<?>> otherDoneTasks = ((BasicExecutionManager) managementContext.getExecutionManager()).allTasksLive()
+ .stream().filter(t -> t.isDone(true)).collect(Collectors.toList());
+ otherDoneTasks.forEach(((BasicExecutionManager) managementContext.getExecutionManager())::deleteTask);
+
+ // also collect tasks, so that unmanaged entities are cleared before next run
+ ((LocalManagementContext)managementContext).getGarbageCollector().gcTasks();
}
@Override
[brooklyn-server] 05/27: GC periodic-persistence tasks immediately
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 88fc1a57ce8144a8a19210385957b643ff7f146e
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Mon Sep 13 22:17:42 2021 +0100
GC periodic-persistence tasks immediately
cut down debug logging about GC when system is doing nothing
---
.../mgmt/internal/BrooklynGarbageCollector.java | 22 ++++++++++++++++------
.../util/core/task/BasicExecutionManager.java | 9 +++++++--
2 files changed, 23 insertions(+), 8 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/BrooklynGarbageCollector.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/BrooklynGarbageCollector.java
index 2d932fd..ca1e075 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/BrooklynGarbageCollector.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/BrooklynGarbageCollector.java
@@ -18,6 +18,7 @@
*/
package org.apache.brooklyn.core.mgmt.internal;
+import com.google.common.annotations.VisibleForTesting;
import java.util.Collection;
import java.util.Collections;
import java.util.Comparator;
@@ -35,6 +36,7 @@ import java.util.concurrent.ThreadFactory;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
+import java.util.stream.Collectors;
import org.apache.brooklyn.api.entity.Entity;
import org.apache.brooklyn.api.location.Location;
import org.apache.brooklyn.api.mgmt.HasTaskChildren;
@@ -154,6 +156,7 @@ public class BrooklynGarbageCollector {
return (end1 < end2) ? -1 : ((end1 == end2) ? 0 : 1);
}
};
+ protected final static Comparator<Task<?>> TASKS_NEWEST_FIRST_COMPARATOR = (t1,t2) -> -TASKS_OLDEST_FIRST_COMPARATOR.compare(t1, t2);
private final BasicExecutionManager executionManager;
@SuppressWarnings("unused") // TODO remove BrooklynStorage altogether?
@@ -330,7 +333,8 @@ public class BrooklynGarbageCollector {
* Deletes old tasks. The age/number of tasks to keep is controlled by fields like
* {@link #MAX_TASKS_PER_TAG} and {@link #MAX_TASKS_PER_TAG}.
*/
- protected synchronized int gcTasks() {
+ @VisibleForTesting
+ public synchronized int gcTasks() {
// NB: be careful with memory usage here: have seen OOME if we get crazy lots of tasks.
// hopefully the use new limits, filters, and use of live lists in some places (added Sep 2014) will help.
//
@@ -607,13 +611,19 @@ public class BrooklynGarbageCollector {
LOG.debug("Got CME inspecting tasks, with "+tasksToConsiderDeleting.size()+" found for deletion: "+e);
}
- if (LOG.isDebugEnabled())
- LOG.debug("brooklyn-gc detected "+taskTagsInCategoryOverCapacity.size()+" "+category+" "
+ Collections.sort(tasksToConsiderDeleting, TASKS_OLDEST_FIRST_COMPARATOR);
+
+ if (LOG.isDebugEnabled()) {
+ MutableList<Task<?>> tasksToConsiderDeletingNewest = MutableList.copyOf(tasksToConsiderDeleting);
+ Collections.sort(tasksToConsiderDeletingNewest, TASKS_NEWEST_FIRST_COMPARATOR);
+ LOG.debug("brooklyn-gc detected " + taskTagsInCategoryOverCapacity.size() + " " + category + " "
+ "tags over capacity, expiring old tasks; "
- + tasksToConsiderDeleting.size()+" tasks under consideration; categories are: "
- + taskTagsInCategoryOverCapacity);
+ + tasksToConsiderDeleting.size() + " tasks under consideration; categories are: "
+ + taskTagsInCategoryOverCapacity + "; including "
+ + tasksToConsiderDeleting.stream().limit(5).collect(Collectors.toList()) + " ... "
+ + tasksToConsiderDeletingNewest.stream().limit(5).collect(Collectors.toList()) );
+ }
- Collections.sort(tasksToConsiderDeleting, TASKS_OLDEST_FIRST_COMPARATOR);
// now try deleting tasks which are overcapacity for each (non-entity) tag
int deleted = 0;
for (Task<?> task: tasksToConsiderDeleting) {
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
index f1496fc..d7af2c0 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
@@ -108,7 +108,7 @@ public class BasicExecutionManager implements ExecutionManager {
public static final String LOGGING_MDC_KEY_ENTITY_IDS = "entity.ids";
public static final String LOGGING_MDC_KEY_TASK_ID = "task.id";
- private static final boolean SCHEDULED_TASKS_COUNT_AS_ACTIVE = false;
+ private static final boolean SCHEDULED_TASKS_COUNT_AS_ACTIVE = true;
private boolean jitterThreads = BrooklynFeatureEnablement.isEnabled(BrooklynFeatureEnablement.FEATURE_JITTER_THREADS);
private int jitterThreadsMaxDelay = Integer.getInteger(JITTER_THREADS_MAX_DELAY_PROPERTY, 200);
@@ -967,7 +967,7 @@ public class BasicExecutionManager implements ExecutionManager {
* but before doing any of the task's work, so that we can update bookkeeping and notify callbacks */
protected void internalBeforeStart(Map<?,?> flags, Task<?> task, boolean skipIncrementCounter, boolean allowJitter, boolean startingThisThreadMightEndElsewhere) {
int count = skipIncrementCounter ? activeTaskCount.get() : activeTaskCount.incrementAndGet();
- if (count % 1000==0) {
+ if (count % 1000==0 && count>0) {
log.warn("High number of active tasks: task #"+count+" is "+task);
}
@@ -1119,6 +1119,11 @@ public class BasicExecutionManager implements ExecutionManager {
}
((TaskInternal<?>)task).setThread(null);
+ // if uninteresting and transient and scheduled, go ahead and remove from task tags also, so it won't be reported as GC'd
+ if (BrooklynTaskTags.isTransient(task) && UNINTERESTING_TASK_NAMES.contains(task.getDisplayName()) && task.getSubmittedByTask() instanceof ScheduledTask) {
+ deleteTask(task);
+ }
+
} finally {
try {
if (error!=null) {
[brooklyn-server] 17/27: stop tasks when ending RO mode also,
more cleanly restart on rebind/promotion
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 109876b10579641964aea9627e67259e04dd13e7
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 14:08:40 2021 +0100
stop tasks when ending RO mode also, more cleanly restart on rebind/promotion
---
.../core/mgmt/ha/HighAvailabilityManagerImpl.java | 5 ++
.../brooklyn/core/mgmt/rebind/RebindIteration.java | 3 +-
.../core/mgmt/rebind/RebindManagerImpl.java | 54 +++++++++++++++++-----
.../core/mgmt/rebind/ManagementPlaneIdTest.java | 6 +--
4 files changed, 53 insertions(+), 15 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
index 00f511a..d3ad012 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/ha/HighAvailabilityManagerImpl.java
@@ -990,6 +990,9 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
* (e.g. during the periodic rebind as hot_stanby we will not repeatedly clear the brooklyn-managed-bundles).
*/
protected void clearManagedItems(ManagementTransitionMode mode) {
+ // note, tasks are cancelled prior to this, when coming from RO mode, via
+ // RebindManagerImpl.stopEntityAndDoneTasksBeforeRebinding
+
// log this because it may be surprising, it is just HA transitions,
// not symmetric with usual single-node start
LOG.info("Clearing all managed items on transition to "+mode);
@@ -1018,6 +1021,8 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
((BasicBrooklynCatalog)managementContext.getCatalog()).reset(CatalogDto.newEmptyInstance("<reset-by-ha-status-change>"));
((BasicBrooklynTypeRegistry)managementContext.getTypeRegistry()).clear();
managementContext.getCatalogInitialization().clearBrooklynManagedBundles();
+
+ ((LocalManagementContext)managementContext).getGarbageCollector().gcTasks();
}
/** Starts hot standby or hot backup, in foreground
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
index 69dd588..bf28d65 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
@@ -265,7 +265,7 @@ public abstract class RebindIteration {
protected void doRun() throws Exception {
if (readOnlyRebindCount.get() > 1) {
// prevent leaking
- rebindManager.stopEntityAndDoneTasksBeforeRebinding();
+ rebindManager.stopEntityAndDoneTasksBeforeRebinding("before next read-only rebind", Duration.seconds(10), Duration.seconds(20));
}
loadManifestFiles();
@@ -560,6 +560,7 @@ public abstract class RebindIteration {
try {
Feed feed = instantiator.newFeed(feedMemento);
rebindContext.registerFeed(feedMemento.getId(), feed);
+ // started during associateAdjunctsWithEntities by RebindAdjuncts
} catch (Exception e) {
exceptionHandler.onCreateFailed(BrooklynObjectType.FEED, feedMemento.getId(), feedMemento.getType(), e);
}
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
index 2db7063..781a8c1 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindManagerImpl.java
@@ -47,6 +47,7 @@ import org.apache.brooklyn.api.mgmt.rebind.mementos.TreeNode;
import org.apache.brooklyn.api.objs.BrooklynObject;
import org.apache.brooklyn.config.ConfigKey;
import org.apache.brooklyn.core.BrooklynFeatureEnablement;
+import org.apache.brooklyn.core.BrooklynVersion;
import org.apache.brooklyn.core.config.ConfigKeys;
import org.apache.brooklyn.core.enricher.AbstractEnricher;
import org.apache.brooklyn.core.entity.Entities;
@@ -398,29 +399,60 @@ public class RebindManagerImpl implements RebindManager {
readOnlyTask = null;
LOG.debug("Stopped read-only rebinding ("+this+"), mgmt "+managementContext.getManagementNodeId());
}
- stopEntityAndDoneTasksBeforeRebinding();
+ // short waits when promoting
+ stopEntityAndDoneTasksBeforeRebinding("when stopping hot proxy read-only mode",
+ Duration.seconds(2),
+ Duration.seconds(5));
+ // note, items are subsequently unmanaged via:
+ // HighAvailabilityManagerImpl.clearManagedItems
}
- public void stopEntityAndDoneTasksBeforeRebinding() {
+ public void stopEntityAndDoneTasksBeforeRebinding(String reason, Duration delayBeforeCancelling, Duration delayBeforeAbandoning) {
+ // TODO inputs should be configurable
+
+ if (!managementContext.isRunning() || managementContext.getExecutionManager().isShutdown()) {
+ return;
+ }
+
// wait for tasks
Collection<Task<?>> entityTasks = ((BasicExecutionManager) managementContext.getExecutionManager()).allTasksLive()
.stream().filter(t -> BrooklynTaskTags.getContextEntity(t) != null).collect(Collectors.toList());
List<Task<?>> openTasksIncludingCancelled;
- CountdownTimer time = CountdownTimer.newInstanceStarted(Duration.seconds(15));
+ CountdownTimer timeBeforeCancelling = CountdownTimer.newInstanceStarted(delayBeforeCancelling);
+ CountdownTimer timeBeforeAbandoning = CountdownTimer.newInstanceStarted(delayBeforeAbandoning);
+ Duration backoff = Duration.millis(10);
do {
openTasksIncludingCancelled = entityTasks.stream().filter(t -> !t.isDone(true)).collect(Collectors.toList());
- List<Task<?>> openTasksCancellable = openTasksIncludingCancelled.stream().filter(t -> !t.isDone()).collect(Collectors.toList());
if (openTasksIncludingCancelled.isEmpty()) break;
- if (time.isExpired() && !openTasksCancellable.isEmpty()) {
- LOG.warn("Aborting " + openTasksCancellable.size() + " incomplete task(s) before rebinding again: " + openTasksCancellable);
+
+ List<Task<?>> openTasksCancellable = openTasksIncludingCancelled.stream().filter(t -> !t.isDone()).collect(Collectors.toList());
+ List<Task<?>> openTasksScheduled = openTasksCancellable.stream().filter(t -> t instanceof ScheduledTask).collect(Collectors.toList());
+
+ if (!openTasksScheduled.isEmpty()) {
+ // stop scheduled tasks immediately
+ openTasksScheduled.forEach(t -> t.cancel(false));
+ continue;
+ }
+
+ if (timeBeforeCancelling!=null && timeBeforeCancelling.isExpired() && !openTasksCancellable.isEmpty()) {
+ LOG.warn("Aborting " + openTasksCancellable.size() + " incomplete task(s) "+reason+": " + openTasksCancellable);
openTasksCancellable.forEach(t -> t.cancel(true));
+ timeBeforeCancelling = null;
}
- if (time.getDurationElapsed().isShorterThan(Duration.millis(200))) {
- LOG.info("Waiting on " + openTasksIncludingCancelled.size() + " task(s) before rebinding again: " + openTasksIncludingCancelled);
+
+ if (timeBeforeAbandoning.isExpired()) break;
+
+ if (timeBeforeAbandoning.getDurationElapsed().isShorterThan(backoff)) {
+ LOG.info("Waiting on " + openTasksIncludingCancelled.size() + " task(s) "+reason+": " + openTasksIncludingCancelled);
}
- LOG.debug("Waiting on " + openTasksIncludingCancelled.size() + " task(s) before rebinding again, details: " +
- openTasksIncludingCancelled.stream().map(t -> ""+t+"("+BrooklynTaskTags.getContextEntity(t)+")").collect(Collectors.toList()));
- Time.sleep(Duration.millis(200));
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Waiting on " + openTasksIncludingCancelled.size() + " task(s) " + reason + ", details: " +
+ openTasksIncludingCancelled.stream().map(t -> "" + t + "(" + BrooklynTaskTags.getContextEntity(t) + ")").collect(Collectors.toList()));
+ }
+
+ Time.sleep(Duration.min(timeBeforeAbandoning.getDurationRemaining(), backoff));
+ backoff = Duration.min(backoff.multiply(2), Duration.millis(200));
+
} while (true);
entityTasks.forEach(((BasicExecutionManager) managementContext.getExecutionManager())::deleteTask);
diff --git a/core/src/test/java/org/apache/brooklyn/core/mgmt/rebind/ManagementPlaneIdTest.java b/core/src/test/java/org/apache/brooklyn/core/mgmt/rebind/ManagementPlaneIdTest.java
index ea3ee5e..c24edcd 100644
--- a/core/src/test/java/org/apache/brooklyn/core/mgmt/rebind/ManagementPlaneIdTest.java
+++ b/core/src/test/java/org/apache/brooklyn/core/mgmt/rebind/ManagementPlaneIdTest.java
@@ -85,7 +85,7 @@ public class ManagementPlaneIdTest {
checkPlaneIdPersisted(mgmt);
}
- @Test
+ @Test(groups="Integration") // because slow
public void testPlaneIdRolledBack() throws Exception {
final LocalManagementContext mgmt = createManagementContext(PersistMode.AUTO, HighAvailabilityMode.AUTO);
@@ -101,7 +101,7 @@ public class ManagementPlaneIdTest {
});
}
- @Test
+ @Test(groups="Integration") // because slow
public void testColdRebindInitialisesPlaneId() throws Exception {
final LocalManagementContext origMgmt = createManagementContext(PersistMode.AUTO, HighAvailabilityMode.DISABLED);
checkPlaneIdPersisted(origMgmt);
@@ -136,7 +136,7 @@ public class ManagementPlaneIdTest {
});
}
- @Test
+ @Test(groups="Integration") // because slow
public void testHaFailoverKeepsPlaneId() throws Exception {
final LocalManagementContext origMgmt = createManagementContext(PersistMode.AUTO, HighAvailabilityMode.MASTER);
final LocalManagementContext rebindMgmt = createManagementContext(PersistMode.AUTO, HighAvailabilityMode.STANDBY);
[brooklyn-server] 11/27: only delete tasks which are done including
completion if cancelled
Posted by he...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
heneveld pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/brooklyn-server.git
commit 783b243fdea0a70d97fc9db6a50d73c1dde44a4d
Author: Alex Heneveld <al...@cloudsoftcorp.com>
AuthorDate: Tue Sep 14 11:15:10 2021 +0100
only delete tasks which are done including completion if cancelled
---
.../mgmt/internal/BrooklynGarbageCollector.java | 11 +++--------
.../brooklyn/core/mgmt/rebind/RebindIteration.java | 22 ++++++++++++----------
.../util/core/task/BasicExecutionManager.java | 10 ++++++++--
3 files changed, 23 insertions(+), 20 deletions(-)
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/BrooklynGarbageCollector.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/BrooklynGarbageCollector.java
index b1bc870..74f89a4 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/BrooklynGarbageCollector.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/internal/BrooklynGarbageCollector.java
@@ -531,7 +531,7 @@ public class BrooklynGarbageCollector {
return false;
}
Task<?> submitter = task.getSubmittedByTask();
- if (submitter!=null && (!submitter.isDone() || executionManager.getTask(submitter.getId())!=null)) {
+ if (submitter!=null && (!submitter.isDone(true) || executionManager.getTask(submitter.getId())!=null)) {
return false;
}
// submitter task is GC'd
@@ -679,13 +679,8 @@ public class BrooklynGarbageCollector {
tasksLive = executionManager.getTasksWithAllTags(MutableList.of());
}
- MutableList<Task<?>> tasks = MutableList.of();
- for (Task<?> task: tasksLive) {
- if (task.isDone()) {
- tasks.add(task);
- }
- }
-
+ List<Task<?>> tasks = tasksLive.stream().filter(t -> t.isDone(true)).collect(Collectors.toList());
+
int numToDelete = tasks.size() - brooklynProperties.getConfig(MAX_TASKS_GLOBAL);
if (numToDelete <= 0) {
LOG.debug("brooklyn-gc detected only "+tasks.size()+" completed tasks in memory, not over global limit, so not deleting any");
diff --git a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
index c3a7ab7..ede24a9 100644
--- a/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
+++ b/core/src/main/java/org/apache/brooklyn/core/mgmt/rebind/RebindIteration.java
@@ -267,27 +267,28 @@ public abstract class RebindIteration {
// wait for tasks
Collection<Task<?>> entityTasks = ((BasicExecutionManager) managementContext.getExecutionManager()).allTasksLive()
.stream().filter(t -> BrooklynTaskTags.getContextEntity(t) != null).collect(Collectors.toList());
- List<Task<?>> openTasks;
+ List<Task<?>> openTasksIncludingCancelled;
CountdownTimer time = CountdownTimer.newInstanceStarted(Duration.seconds(15));
do {
- openTasks = entityTasks.stream().filter(t -> !t.isDone()).collect(Collectors.toList());
- if (openTasks.isEmpty()) break;
- if (time.isExpired()) {
- LOG.warn("Aborting " + openTasks.size() + " incomplete task(s) before rebinding again: " + openTasks);
- openTasks.forEach(t -> t.cancel(true));
+ openTasksIncludingCancelled = entityTasks.stream().filter(t -> !t.isDone(true)).collect(Collectors.toList());
+ List<Task<?>> openTasksCancellable = openTasksIncludingCancelled.stream().filter(t -> !t.isDone()).collect(Collectors.toList());
+ if (openTasksIncludingCancelled.isEmpty()) break;
+ if (time.isExpired() && !openTasksCancellable.isEmpty()) {
+ LOG.warn("Aborting " + openTasksCancellable.size() + " incomplete task(s) before rebinding again: " + openTasksCancellable);
+ openTasksCancellable.forEach(t -> t.cancel(true));
}
if (time.getDurationElapsed().isShorterThan(Duration.millis(200))) {
- LOG.info("Waiting on " + openTasks.size() + " task(s) before rebinding again: " + openTasks);
+ LOG.info("Waiting on " + openTasksIncludingCancelled.size() + " task(s) before rebinding again: " + openTasksIncludingCancelled);
}
- LOG.debug("Waiting on " + openTasks.size() + " task(s) before rebinding again, details: " +
- openTasks.stream().map(t -> ""+t+"("+BrooklynTaskTags.getContextEntity(t)+")").collect(Collectors.toList()));
+ LOG.debug("Waiting on " + openTasksIncludingCancelled.size() + " task(s) before rebinding again, details: " +
+ openTasksIncludingCancelled.stream().map(t -> ""+t+"("+BrooklynTaskTags.getContextEntity(t)+")").collect(Collectors.toList()));
Time.sleep(Duration.millis(200));
} while (true);
entityTasks.forEach(((BasicExecutionManager) managementContext.getExecutionManager())::deleteTask);
List<Task<?>> otherDoneTasks = ((BasicExecutionManager) managementContext.getExecutionManager()).allTasksLive()
- .stream().filter(Task::isDone).collect(Collectors.toList());
+ .stream().filter(t -> t.isDone(true)).collect(Collectors.toList());
otherDoneTasks.forEach(((BasicExecutionManager) managementContext.getExecutionManager())::deleteTask);
}
@@ -420,6 +421,7 @@ public abstract class RebindIteration {
for (ManagedBundleMemento bundleMemento : mementoManifest.getBundles().values()) {
ManagedBundle managedBundle = instantiator.newManagedBundle(bundleMemento);
bundles.put(managedBundle.getVersionedName(), new InstallableManagedBundleImpl(bundleMemento, managedBundle));
+ logRebindingDebug("Registering bundle "+bundleMemento.getId()+": "+managedBundle);
rebindContext.registerBundle(bundleMemento.getId(), managedBundle);
}
} else {
diff --git a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
index b7e5adf..995c49e 100644
--- a/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
+++ b/core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
@@ -390,9 +390,15 @@ public class BasicExecutionManager implements ExecutionManager {
if (removed!=null && removed.isSubmitted() && !removed.isDone(true)) {
Entity context = BrooklynTaskTags.getContextEntity(removed);
if (context!=null && !Entities.isManaged(context)) {
- log.debug("Forgetting about active task on unmanagement of "+context+": "+removed);
+ log.debug("Deleting active task on unmanagement of "+context+": "+removed);
} else {
- log.warn("Deleting submitted task before completion: "+removed+" (tags "+removed.getTags()+"); this task will continue to run in the background outwith "+this+", but perhaps it should have been cancelled?");
+ if (removed.isDone()) {
+ log.debug("Deleting cancelled task before completion: " + removed + "; this task will continue to run in the background outwith " + this);
+ } else {
+ log.warn("Deleting submitted task before completion: " + removed + " (tags " + removed.getTags() + "); this task will continue to run in the background outwith " + this + ", but perhaps it should have been cancelled?");
+
+ }
+ log.debug("Active task deletion trace", new Throwable("Active task deletion trace"));
}
}
task.getTags().forEach(t -> {