You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geode.apache.org by GitBox <gi...@apache.org> on 2020/04/21 18:04:16 UTC
[GitHub] [geode] upthewaterspout commented on a change in pull request #4909: GEODE-7953: Restore Redundancy Internal API
upthewaterspout commented on a change in pull request #4909:
URL: https://github.com/apache/geode/pull/4909#discussion_r412362589
##########
File path: geode-core/src/main/java/org/apache/geode/cache/control/RestoreRedundancyResults.java
##########
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.cache.control;
+
+import java.util.Map;
+
+import org.apache.geode.cache.partition.PartitionRebalanceInfo;
+import org.apache.geode.internal.cache.control.RegionRedundancyStatus;
+import org.apache.geode.internal.cache.partitioned.PartitionedRegionRebalanceOp;
+
+/**
+ * A class to collect the results of restore redundancy operations for one or more regions and
+ * determine the success of failure of the operation.
+ */
+public interface RestoreRedundancyResults {
+
+ /**
+ * {@link #SUCCESS} is defined as every included region having fully satisfied redundancy.
+ * {@link #FAILURE} is defined as at least one region that is configured to have redundant copies
+ * having fewer than its configured number of redundant copies.
+ * {@link #ERROR} is for cases when the restore redundancy operation was unable to begin or threw
+ * an exception.
+ */
+ enum Status {
+ SUCCESS,
+ FAILURE,
+ ERROR
+ }
+
+ /**
+ * Adds the contents of another {@link RestoreRedundancyResults} object to this one, including
+ * both {@link RegionRedundancyStatus} objects and information on the number of primaries
+ * reassigned and the time taken to reassign them.
+ *
+ * @param results a {@link RestoreRedundancyResults} object whose contents will be added to this
+ * one.
+ */
+ void addRegionResults(RestoreRedundancyResults results);
+
+ /**
+ * Adds information regarding the number of primaries reassigned and the time taken to reassign
+ * them during a restore redundancy operation.
+ *
+ * @param details a {@link PartitionRebalanceInfo} generated by a
+ * {@link PartitionedRegionRebalanceOp} operation.
+ */
+ void addPrimaryReassignmentDetails(PartitionRebalanceInfo details);
+
+ /**
+ * Adds one {@link RegionRedundancyStatus} to the result set.
+ *
+ * @param regionResult The {@link RegionRedundancyStatus} to be added to the result set.
+ */
+ void addRegionResult(RegionRedundancyStatus regionResult);
Review comment:
This is leaking an internal class into the public API. The public API should not refer to internal classes.
In this case, I'm not clear why these mutation methods are part of the public API at all. Would it make more sense for RestoreRedundancyResults to be a read only view of the results?
##########
File path: geode-core/src/main/java/org/apache/geode/cache/control/RestoreRedundancyResults.java
##########
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.cache.control;
+
+import java.util.Map;
+
+import org.apache.geode.cache.partition.PartitionRebalanceInfo;
+import org.apache.geode.internal.cache.control.RegionRedundancyStatus;
+import org.apache.geode.internal.cache.partitioned.PartitionedRegionRebalanceOp;
+
+/**
+ * A class to collect the results of restore redundancy operations for one or more regions and
+ * determine the success of failure of the operation.
+ */
+public interface RestoreRedundancyResults {
+
+ /**
+ * {@link #SUCCESS} is defined as every included region having fully satisfied redundancy.
+ * {@link #FAILURE} is defined as at least one region that is configured to have redundant copies
+ * having fewer than its configured number of redundant copies.
+ * {@link #ERROR} is for cases when the restore redundancy operation was unable to begin or threw
+ * an exception.
+ */
+ enum Status {
+ SUCCESS,
+ FAILURE,
+ ERROR
+ }
+
+ /**
+ * Adds the contents of another {@link RestoreRedundancyResults} object to this one, including
+ * both {@link RegionRedundancyStatus} objects and information on the number of primaries
+ * reassigned and the time taken to reassign them.
+ *
+ * @param results a {@link RestoreRedundancyResults} object whose contents will be added to this
+ * one.
+ */
+ void addRegionResults(RestoreRedundancyResults results);
+
+ /**
+ * Adds information regarding the number of primaries reassigned and the time taken to reassign
+ * them during a restore redundancy operation.
+ *
+ * @param details a {@link PartitionRebalanceInfo} generated by a
+ * {@link PartitionedRegionRebalanceOp} operation.
Review comment:
Internal class.
##########
File path: geode-core/src/main/java/org/apache/geode/internal/cache/control/RegionRedundancyStatus.java
##########
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.internal.cache.control;
+
+import java.io.Serializable;
+
+import org.apache.geode.internal.cache.PartitionedRegion;
+
+/**
+ * Used to calculate and store the redundancy status for a {@link PartitionedRegion}.
+ */
+public class RegionRedundancyStatus implements Serializable {
Review comment:
I think this class probably should be DataSerializableFixedId if we intend to serialize it in production.
I saw RebalanceResults was Serializable as well. I'm not clear if we serialize these things in production or not, maybe for gfsh? For new classes we don't want to be using java serialization.
##########
File path: geode-core/src/main/java/org/apache/geode/cache/control/RestoreRedundancyResults.java
##########
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.cache.control;
+
+import java.util.Map;
+
+import org.apache.geode.cache.partition.PartitionRebalanceInfo;
+import org.apache.geode.internal.cache.control.RegionRedundancyStatus;
+import org.apache.geode.internal.cache.partitioned.PartitionedRegionRebalanceOp;
+
+/**
+ * A class to collect the results of restore redundancy operations for one or more regions and
+ * determine the success of failure of the operation.
+ */
+public interface RestoreRedundancyResults {
+
+ /**
+ * {@link #SUCCESS} is defined as every included region having fully satisfied redundancy.
+ * {@link #FAILURE} is defined as at least one region that is configured to have redundant copies
+ * having fewer than its configured number of redundant copies.
+ * {@link #ERROR} is for cases when the restore redundancy operation was unable to begin or threw
+ * an exception.
+ */
+ enum Status {
+ SUCCESS,
+ FAILURE,
+ ERROR
+ }
+
+ /**
+ * Adds the contents of another {@link RestoreRedundancyResults} object to this one, including
+ * both {@link RegionRedundancyStatus} objects and information on the number of primaries
+ * reassigned and the time taken to reassign them.
+ *
+ * @param results a {@link RestoreRedundancyResults} object whose contents will be added to this
+ * one.
+ */
+ void addRegionResults(RestoreRedundancyResults results);
+
+ /**
+ * Adds information regarding the number of primaries reassigned and the time taken to reassign
+ * them during a restore redundancy operation.
+ *
+ * @param details a {@link PartitionRebalanceInfo} generated by a
+ * {@link PartitionedRegionRebalanceOp} operation.
+ */
+ void addPrimaryReassignmentDetails(PartitionRebalanceInfo details);
+
+ /**
+ * Adds one {@link RegionRedundancyStatus} to the result set.
+ *
+ * @param regionResult The {@link RegionRedundancyStatus} to be added to the result set.
+ */
+ void addRegionResult(RegionRedundancyStatus regionResult);
+
+ /**
+ * Returns the {@link Status} of this restore redundancy operation. Possible statuses are
+ * {@link Status#SUCCESS}, {@link Status#FAILURE} and {@link Status#ERROR}.
+ *
+ * @return The {@link Status} of this restore redundancy operation.
+ */
+ Status getStatus();
+
+ /**
+ * Returns a message describing the results of this restore redundancy operation.
+ *
+ * @return A {@link String} describing the results of this restore redundancy operation.
+ */
+ String getMessage();
+
+ /**
+ * Returns the {@link RegionRedundancyStatus} for a specific region or null if that region
+ * is not present in this {@link RestoreRedundancyResults}.
+ *
+ * @param regionName The region to which the {@link RegionRedundancyStatus} to be returned
+ * belongs.
+ * @return A {@link RegionRedundancyStatus} for the specified region or null if that region is not
+ * present in this {@link RestoreRedundancyResults}.
+ */
+ RegionRedundancyStatus getRegionResult(String regionName);
Review comment:
Another reference to an internal class. Should RegionRedundancyStatus be part of the public API?
##########
File path: geode-core/src/main/java/org/apache/geode/internal/cache/control/RestoreRedundancyBuilderImpl.java
##########
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.internal.cache.control;
+
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+import org.apache.geode.cache.RegionDestroyedException;
+import org.apache.geode.cache.control.RestoreRedundancyBuilder;
+import org.apache.geode.cache.control.RestoreRedundancyResults;
+import org.apache.geode.cache.partition.PartitionRebalanceInfo;
+import org.apache.geode.internal.cache.InternalCache;
+import org.apache.geode.internal.cache.PartitionedRegion;
+import org.apache.geode.internal.cache.partitioned.PartitionedRegionRebalanceOp;
+import org.apache.geode.internal.cache.partitioned.rebalance.RestoreRedundancyDirector;
+import org.apache.geode.logging.internal.log4j.api.LogService;
+
+class RestoreRedundancyBuilderImpl implements RestoreRedundancyBuilder {
+
+ private final InternalCache cache;
+ private final InternalResourceManager manager;
+ private Set<String> includedRegions;
+ private Set<String> excludedRegions;
+ private boolean shouldReassign = true;
+ private ScheduledExecutorService executor;
+
+ public RestoreRedundancyBuilderImpl(InternalCache cache) {
+ this.cache = cache;
+ this.manager = cache.getInternalResourceManager();
+ this.executor = this.manager.getExecutor();
+ }
+
+ @Override
+ public RestoreRedundancyBuilder includeRegions(Set<String> regions) {
+ this.includedRegions = regions;
+ return this;
+ }
+
+ @Override
+ public RestoreRedundancyBuilder excludeRegions(Set<String> regions) {
+ this.excludedRegions = regions;
+ return this;
+ }
+
+ @Override
+ public RestoreRedundancyBuilder setReassignPrimaries(boolean shouldReassign) {
+ this.shouldReassign = shouldReassign;
+ return this;
+ }
+
+ @Override
+ public CompletableFuture<RestoreRedundancyResults> start() {
Review comment:
These seems to be sort of a reimplementation of RebalanceOperationImpl.start(). Both that and this method seem to have some complicated scheduling logic. Should we have two copies and two different ways of doing this complicated scheduling?
This seems to be missing some of the logic from RebalanceOperationImpl. For example, that method filters out child colocated regions. Seems like that should be done here as well.
Maybe we should just reuse/improve RebalanceOperationImpl?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org