You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brooklyn.apache.org by ahgittin <gi...@git.apache.org> on 2015/01/29 12:27:10 UTC
[GitHub] incubator-brooklyn pull request: refactor RebindManager to have a ...
GitHub user ahgittin opened a pull request:
https://github.com/apache/incubator-brooklyn/pull/486
refactor RebindManager to have a RebindIteration
introduce new class and new methods for each phase. (no real changes.)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ahgittin/incubator-brooklyn rebind-refactor
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-brooklyn/pull/486.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #486
----
commit 848eac26663dc8ba3d3003d11c245a614bf915ed
Author: Alex Heneveld <al...@cloudsoftcorp.com>
Date: 2015-01-29T11:24:27Z
refactor RebindManager to have a RebindIteration
introduce new class and new methods for each phase. (no real changes.)
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: refactor RebindManager to have a ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-brooklyn/pull/486
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: refactor RebindManager to have a ...
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/486#discussion_r23836917
--- Diff: core/src/main/java/brooklyn/entity/rebind/RebindIteration.java ---
@@ -0,0 +1,1086 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package brooklyn.entity.rebind;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import java.io.IOException;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.Callable;
+import java.util.concurrent.Semaphore;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import brooklyn.basic.AbstractBrooklynObject;
+import brooklyn.basic.BrooklynObject;
+import brooklyn.basic.BrooklynObjectInternal;
+import brooklyn.catalog.BrooklynCatalog;
+import brooklyn.catalog.CatalogItem;
+import brooklyn.catalog.CatalogLoadMode;
+import brooklyn.catalog.internal.BasicBrooklynCatalog;
+import brooklyn.catalog.internal.CatalogUtils;
+import brooklyn.config.BrooklynLogging;
+import brooklyn.config.BrooklynLogging.LoggingLevel;
+import brooklyn.config.BrooklynServerConfig;
+import brooklyn.enricher.basic.AbstractEnricher;
+import brooklyn.entity.Application;
+import brooklyn.entity.Entity;
+import brooklyn.entity.Feed;
+import brooklyn.entity.basic.AbstractApplication;
+import brooklyn.entity.basic.AbstractEntity;
+import brooklyn.entity.basic.EntityInternal;
+import brooklyn.entity.proxying.InternalEntityFactory;
+import brooklyn.entity.proxying.InternalFactory;
+import brooklyn.entity.proxying.InternalLocationFactory;
+import brooklyn.entity.proxying.InternalPolicyFactory;
+import brooklyn.entity.rebind.RebindManagerImpl.RebindTracker;
+import brooklyn.entity.rebind.persister.PersistenceActivityMetrics;
+import brooklyn.event.feed.AbstractFeed;
+import brooklyn.internal.BrooklynFeatureEnablement;
+import brooklyn.location.Location;
+import brooklyn.location.basic.AbstractLocation;
+import brooklyn.location.basic.LocationInternal;
+import brooklyn.management.classloading.BrooklynClassLoadingContext;
+import brooklyn.management.ha.ManagementNodeState;
+import brooklyn.management.internal.EntityManagerInternal;
+import brooklyn.management.internal.LocationManagerInternal;
+import brooklyn.management.internal.ManagementContextInternal;
+import brooklyn.management.internal.ManagementTransitionInfo.ManagementTransitionMode;
+import brooklyn.mementos.BrooklynMemento;
+import brooklyn.mementos.BrooklynMementoManifest;
+import brooklyn.mementos.BrooklynMementoManifest.EntityMementoManifest;
+import brooklyn.mementos.BrooklynMementoPersister;
+import brooklyn.mementos.BrooklynMementoPersister.LookupContext;
+import brooklyn.mementos.BrooklynMementoRawData;
+import brooklyn.mementos.CatalogItemMemento;
+import brooklyn.mementos.EnricherMemento;
+import brooklyn.mementos.EntityMemento;
+import brooklyn.mementos.FeedMemento;
+import brooklyn.mementos.LocationMemento;
+import brooklyn.mementos.Memento;
+import brooklyn.mementos.PolicyMemento;
+import brooklyn.mementos.TreeNode;
+import brooklyn.policy.Enricher;
+import brooklyn.policy.Policy;
+import brooklyn.policy.basic.AbstractPolicy;
+import brooklyn.util.collections.MutableList;
+import brooklyn.util.collections.MutableMap;
+import brooklyn.util.exceptions.Exceptions;
+import brooklyn.util.flags.FlagUtils;
+import brooklyn.util.guava.Maybe;
+import brooklyn.util.javalang.Reflections;
+import brooklyn.util.text.Strings;
+import brooklyn.util.time.Duration;
+import brooklyn.util.time.Time;
+
+import com.google.common.base.Optional;
+import com.google.common.base.Preconditions;
+import com.google.common.base.Stopwatch;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Iterables;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+import com.google.common.collect.Sets;
+
+/**
+Multi-phase deserialization:
+
+<ul>
+<li> 1. load the manifest files and populate the summaries (ID+type) in {@link BrooklynMementoManifest}
+<li> 2. instantiate and reconstruct catalog items
+<li> 3. instantiate entities+locations -- so that inter-entity references can subsequently
+ be set during deserialize (and entity config/state is set).
+<li> 4. deserialize the manifests to instantiate the mementos
+<li> 5. instantiate policies+enrichers+feeds
+ (could probably merge this with (3), depending how they are implemented)
+<li> 6. reconstruct the locations, policies, etc, then finally entities -- setting all fields and then calling
+ {@link RebindSupport#reconstruct(RebindContext, Memento)}
+<li> 7. associate policies+enrichers+feeds to all the entities
+<li> 8. manage the entities
+</ul>
+
+ If underlying data-store is changed between first and second manifest read (e.g. to add an
+ entity), then second phase might try to reconstitute an entity that has not been put in
+ the rebindContext. This should not affect normal production usage, because rebind is run
+ against a data-store that is not being written to by other brooklyn instance(s).
+ But clearly it would be desirable to have better locking possible against the backing store.
+
+<p>
+ When rebinding to code in OSGi bundles, thecatalog item id context is inferred as follows:
+ * most of the time the creator will be passing "my catalog item id"
+ (or API could also take "BrooklynObject me" as a creation context and the
+ receiver query the creator's catalog item id)
+ * look at the context entity of Tasks.current() (if set)
+ * propagate the catalog item id when doing setEntity, addChild
+ * when none of the above work (or they are wrong) let the user specify the catalog item
+<p>
+ Precedence of setting the catalog item ID:
+ 1. User-supplied catalog item ID.
+ 2. Creating from a catalog item - all items resolved during the creation of a spec
+ from a catalog item receive the catalog item's ID as context.
+ 3. When using the Java API for creating specs get the catalog item ID from the
+ context entity of the Tasks.current() task.
+ 4. Propagate the context catalog item ID to children, adjuncts if they don't have one already.
+*/
+public class RebindIteration {
+
+ private static final Logger LOG = LoggerFactory.getLogger(RebindIteration.class);
+
+ private final RebindManagerImpl rebindManager;
+
+ private final ClassLoader classLoader;
+ private final RebindExceptionHandler exceptionHandler;
+ private final ManagementNodeState mode;
+ private final ManagementContextInternal managementContext;
+
+ private final Semaphore rebindActive;
+ private final AtomicInteger readOnlyRebindCount;
+ private final PersistenceActivityMetrics rebindMetrics;
+ private final BrooklynMementoPersister persistenceStoreAccess;
+
+ private final AtomicBoolean iterationStarted = new AtomicBoolean();
+ private final RebindContextImpl rebindContext;
+ private final Reflections reflections;
+ private final LookupContext lookupContext;
+ private final BrooklynObjectInstantiator instantiator;
+
+ // populated in the course of a run
+
+ // set on run start
+
+ private Stopwatch timer;
+ /** phase is used to ensure our steps are run as we've expected, and documented (in javadoc at top).
+ * it's worth the extra effort due to the complication and the subtleties. */
+ private int phase = 0;
+
+ // set in first phase
+
+ private BrooklynMementoRawData mementoRawData;
+ private BrooklynMementoManifest mementoManifest;
+ private Boolean overwritingMaster;
+ private Boolean isEmpty;
+
+ // set later on
+
+ private BrooklynMemento memento;
+
+ // set near the end
+
+ private List<Application> applications;
+
+
+ public RebindIteration(RebindManagerImpl rebindManager,
+ ManagementNodeState mode,
+ ClassLoader classLoader, RebindExceptionHandler exceptionHandler,
+ Semaphore rebindActive, AtomicInteger readOnlyRebindCount, PersistenceActivityMetrics rebindMetrics, BrooklynMementoPersister persistenceStoreAccess
+ ) {
+ // NB: there is no particularly deep meaning in what is passed in vs what is lookup up from the RebindManager which calls us
+ // (this is simply a refactoring of previous code to a new class)
+
+ this.rebindManager = rebindManager;
+
+ this.mode = mode;
+ this.classLoader = checkNotNull(classLoader, "classLoader");
+ this.exceptionHandler = checkNotNull(exceptionHandler, "exceptionHandler");
+
+ this.rebindActive = rebindActive;
+ this.readOnlyRebindCount = readOnlyRebindCount;
+ this.rebindMetrics = rebindMetrics;
+ this.persistenceStoreAccess = persistenceStoreAccess;
+
+ managementContext = rebindManager.getManagementContext();
+ rebindContext = new RebindContextImpl(exceptionHandler, classLoader);
+ reflections = new Reflections(classLoader);
+ lookupContext = new RebindContextLookupContext(managementContext, rebindContext, exceptionHandler);
+ rebindContext.setLookupContext(lookupContext);
+ instantiator = new BrooklynObjectInstantiator(classLoader, rebindContext, reflections);
+
+ if (mode==ManagementNodeState.HOT_STANDBY || mode==ManagementNodeState.HOT_BACKUP) {
+ rebindContext.setAllReadOnly();
+ } else {
+ Preconditions.checkState(mode==ManagementNodeState.MASTER, "Must be either master or read only to rebind (mode "+mode+")");
+ }
+ }
+
+ public List<Application> getApplications() {
+ return applications;
+ }
+
+ RebindContextImpl getRebindContext() {
+ return rebindContext;
+ }
+
+ public void runFullRebind() {
+ runWithLock(new Callable<Void>() {
+ public Void call() throws Exception {
+
+ loadManifestFiles();
+ rebuildCatalog();
+ instantiateLocationsAndEntities();
+ instantiateMementos();
+ instantiateAdjuncts(instantiator);
+ reconstructEverything();
+ associateAdjunctsWithEntities();
+ manageTheObjects();
+ finishingUp();
+
+ return null;
+ }
+ });
+ }
+
+ protected void runWithLock(Callable<?> target) {
+ if (iterationStarted.getAndSet(true)) {
+ throw new IllegalStateException("Iteration "+this+" has already run; create a new instance for another rebind pass.");
+ }
+ try {
+ rebindActive.acquire();
+ } catch (InterruptedException e) { Exceptions.propagate(e); }
+ try {
+ RebindTracker.setRebinding();
+ if (ManagementNodeState.isHotProxy(mode))
+ readOnlyRebindCount.incrementAndGet();
+
+ timer = Stopwatch.createStarted();
+ exceptionHandler.onStart(rebindContext);
+
+ target.call();
+
+ exceptionHandler.onDone();
+
+ rebindMetrics.noteSuccess(Duration.of(timer));
+ noteErrors(exceptionHandler, null);
+
+ } catch (Exception e) {
+ rebindMetrics.noteFailure(Duration.of(timer));
+
+ Exceptions.propagateIfFatal(e);
+ noteErrors(exceptionHandler, e);
+ throw exceptionHandler.onFailed(e);
+
+ } finally {
+ rebindActive.release();
+ RebindTracker.reset();
+ }
+ }
+
+ protected void checkEnteringPhase(int targetPhase) {
+ phase++;
--- End diff --
Maybe should use an enum, with ordinal for readability of these calls? No strong feelings.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: refactor RebindManager to have a ...
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/486#issuecomment-72188458
Looks good; much cleaner structure. I presume the body of the methods are the same as before (with just code moved around). I've really focused on just the structure during this review. Will merge now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: refactor RebindManager to have a ...
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/486#issuecomment-72197966
Great. Yes, the bodies are unchanged, apart from a few bits wrapping the start and catching exceptions, but I thought hard to confirm that.
I considered using an enum, but we'd need something like http://stackoverflow.com/questions/17664445/java-operator-for-enum -- probably worth doing it, putting a `<T extends Enumeration> T next(T item)` into `Enums`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---