You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/03/03 18:55:51 UTC

[GitHub] [ozone] errose28 commented on a change in pull request #3104: HDDS-6213. Pluggable OzoneManager request handling hooks

errose28 commented on a change in pull request #3104:
URL: https://github.com/apache/ozone/pull/3104#discussion_r817115572



##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/validation/ValidationCondition.java
##########
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership.  The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.hadoop.ozone.om.request.validation;
+
+import org.apache.hadoop.ozone.ClientVersions;
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.OMRequest;
+
+import java.util.function.Predicate;
+
+/**
+ * Defines conditions for which validators can be assigned to.
+ *
+ * These conditions describe a situation where special request handling might
+ * be necessary. In these cases we do not override the actual request handling
+ * code, but based on certain request properties we might reject a request
+ * early, or we might modify the request, or the response received/sent from/to
+ * the client.
+ */
+public enum ValidationCondition {
+  /**
+   * Classifies validations that has to run after an upgrade until the cluster
+   * is in a pre-finalized state.
+   */
+  CLUSTER_NEEDS_FINALIZATION(r -> true,
+      ctx -> ctx.versionManager().needsFinalization()),
+  /**
+   * Classifies validations that has to run, when the client uses an older
+   * protocol version than the server.
+   */
+  OLDER_CLIENT_REQUESTS(r -> r.getVersion() < ClientVersions.CURRENT_VERSION,
+      ctx -> true),
+  /**
+   * Classifies validations that has to run, when the client uses a newer
+   * protocol version than the server.
+   */
+  NEWER_CLIENT_REQUESTS(r -> r.getVersion() > ClientVersions.CURRENT_VERSION,
+      ctx -> true),
+  /**
+   * Classifies validations that has to run for every request.
+   * If you plan to use this, please justify why the validation code should not
+   * be part of the actual request handling code.
+   */
+  UNCONDITIONAL(r -> true, ctx -> true);

Review comment:
       I don't think we need these last two right now. Maybe it would be better to remove them to reduce confusion? I can see the unconditional validator being misused since there isn't really a clearly defined use case. I think most newer client handling is going to happen on the client side based on the service info call, so having it here might be misleading for devs working on something with forward compatibility concerns.

##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/validation/ValidatorRegistry.java
##########
@@ -0,0 +1,201 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership.  The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.hadoop.ozone.om.request.validation;
+
+import org.apache.commons.lang3.tuple.Pair;
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.Type;
+import org.reflections.Reflections;
+import org.reflections.scanners.MethodAnnotationsScanner;
+import org.reflections.util.ClasspathHelper;
+import org.reflections.util.ConfigurationBuilder;
+
+import java.lang.reflect.Method;
+import java.net.URL;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.EnumMap;
+import java.util.HashSet;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Registry that loads and stores the request validators to be applied by
+ * a service.
+ */
+public class ValidatorRegistry {
+
+  private final EnumMap<
+      ValidationCondition, EnumMap<Type, Pair<List<Method>, List<Method>>>>
+      validators = new EnumMap<>(ValidationCondition.class);
+
+  /**
+   * Creates a {@link ValidatorRegistry} instance that discovers validation
+   * methods in the provided package and the packages in the same resource.
+   * A validation method is recognized by the {@link RequestFeatureValidator}
+   * annotation that contains important information about how and when to use
+   * the validator.
+   * @param validatorPackage the main package inside which validatiors should
+   *                         be discovered.
+   */
+  ValidatorRegistry(String validatorPackage) {
+    this(ClasspathHelper.forPackage(validatorPackage));
+  }
+
+  /**
+   * Creates a {@link ValidatorRegistry} instance that discovers validation
+   * methods under the provided URL.
+   * A validation method is recognized by the {@link RequestFeatureValidator}
+   * annotation that contains important information about how and when to use
+   * the validator.
+   * @param searchUrls the path in which the annotated methods are searched.
+   */
+  ValidatorRegistry(Collection<URL> searchUrls) {
+    Reflections reflections = new Reflections(new ConfigurationBuilder()
+        .setUrls(searchUrls)
+        .setScanners(new MethodAnnotationsScanner())
+        .useParallelExecutor()
+    );
+
+    Set<Method> describedValidators =
+        reflections.getMethodsAnnotatedWith(RequestFeatureValidator.class);
+    initMaps(describedValidators);
+  }
+
+  /**
+   * Get the validators that has to be run in the given list of
+   * {@link ValidationCondition}s, for the given requestType and
+   * {@link RequestProcessingPhase}.
+   *
+   * @param conditions conditions that are present for the request
+   * @param requestType the type of the protocol message
+   * @param phase the request processing phase
+   * @return the list of validation methods that has to run.
+   */
+  List<Method> validationsFor(
+      List<ValidationCondition> conditions,
+      Type requestType,
+      RequestProcessingPhase phase) {
+
+    if (conditions.isEmpty() || validators.isEmpty()) {
+      return Collections.emptyList();
+    }
+
+    Set<Method> returnValue =
+        new HashSet<>(validationsFor(conditions.get(0), requestType, phase));

Review comment:
       How could we end up with duplicate method entries here that the set needs to remove?

##########
File path: dev-support/annotations/src/main/java/org/apache/ozone/annotations/RequestFeatureValidatorProcessor.java
##########
@@ -0,0 +1,277 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership.  The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.ozone.annotations;
+
+import javax.annotation.processing.AbstractProcessor;
+import javax.annotation.processing.RoundEnvironment;
+import javax.annotation.processing.SupportedAnnotationTypes;
+import javax.annotation.processing.SupportedSourceVersion;
+import javax.lang.model.SourceVersion;
+import javax.lang.model.element.AnnotationMirror;
+import javax.lang.model.element.AnnotationValue;
+import javax.lang.model.element.AnnotationValueVisitor;
+import javax.lang.model.element.Element;
+import javax.lang.model.element.ElementKind;
+import javax.lang.model.element.ExecutableElement;
+import javax.lang.model.element.Modifier;
+import javax.lang.model.element.TypeElement;
+import javax.lang.model.element.VariableElement;
+import javax.lang.model.type.ExecutableType;
+import javax.lang.model.type.TypeMirror;
+import javax.lang.model.util.SimpleAnnotationValueVisitor8;
+import javax.tools.Diagnostic;
+import java.util.List;
+import java.util.Map.Entry;
+import java.util.Set;
+
+@SupportedAnnotationTypes(
+    "org.apache.hadoop.ozone.om.request.validation.RequestFeatureValidator")
+@SupportedSourceVersion(SourceVersion.RELEASE_8)
+public class RequestFeatureValidatorProcessor extends AbstractProcessor {

Review comment:
       Can we add a javadoc to this class explaining it is used to validate annotations are constructed correctly, which annotations it is operating on, and that it is picked up from the javax.annotations.processing.Processor file?

##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/validation/ValidationCondition.java
##########
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership.  The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.hadoop.ozone.om.request.validation;
+
+import org.apache.hadoop.ozone.ClientVersions;
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.OMRequest;
+
+import java.util.function.Predicate;
+
+/**
+ * Defines conditions for which validators can be assigned to.
+ *
+ * These conditions describe a situation where special request handling might
+ * be necessary. In these cases we do not override the actual request handling
+ * code, but based on certain request properties we might reject a request
+ * early, or we might modify the request, or the response received/sent from/to
+ * the client.
+ */
+public enum ValidationCondition {
+  /**
+   * Classifies validations that has to run after an upgrade until the cluster
+   * is in a pre-finalized state.
+   */
+  CLUSTER_NEEDS_FINALIZATION(r -> true,
+      ctx -> ctx.versionManager().needsFinalization()),
+  /**
+   * Classifies validations that has to run, when the client uses an older
+   * protocol version than the server.
+   */
+  OLDER_CLIENT_REQUESTS(r -> r.getVersion() < ClientVersions.CURRENT_VERSION,
+      ctx -> true),
+  /**
+   * Classifies validations that has to run, when the client uses a newer
+   * protocol version than the server.
+   */
+  NEWER_CLIENT_REQUESTS(r -> r.getVersion() > ClientVersions.CURRENT_VERSION,
+      ctx -> true),
+  /**
+   * Classifies validations that has to run for every request.
+   * If you plan to use this, please justify why the validation code should not
+   * be part of the actual request handling code.
+   */
+  UNCONDITIONAL(r -> true, ctx -> true);
+
+  private final Predicate<OMRequest> shouldApplyTo;
+  private final Predicate<ValidationContext> shouldApplyIn;
+
+  ValidationCondition(
+      Predicate<OMRequest> shouldApplyTo,
+      Predicate<ValidationContext> shouldApplyIn) {
+    this.shouldApplyTo = shouldApplyTo;
+    this.shouldApplyIn = shouldApplyIn;
+  }
+
+  boolean shouldApply(OMRequest req, ValidationContext ctx) {
+    return shouldApplyTo.test(req) && shouldApplyIn.test(ctx);

Review comment:
       Since we are just ANDing the conditions together, would it be easier to have ValidationConditions take just one predicate, and let the predicate's implementation define what conditions to act on. The current implementation seems to frequently require these `true` placeholders. I think it would read clearer if ValidationConditions were declared like:
   ```java
     CLUSTER_NEEDS_FINALIZATION(ctx -> ctx.versionManager().needsFinalization()),
     OLDER_CLIENT_REQUESTS(r -> r.getVersion() < ClientVersions.CURRENT_VERSION),
     NEWER_CLIENT_REQUESTS(r -> r.getVersion() > ClientVersions.CURRENT_VERSION),
     UNCONDITIONAL(r -> true);
   ```

##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/validation/ValidatorRegistry.java
##########
@@ -0,0 +1,201 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership.  The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.hadoop.ozone.om.request.validation;
+
+import org.apache.commons.lang3.tuple.Pair;
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.Type;
+import org.reflections.Reflections;
+import org.reflections.scanners.MethodAnnotationsScanner;
+import org.reflections.util.ClasspathHelper;
+import org.reflections.util.ConfigurationBuilder;
+
+import java.lang.reflect.Method;
+import java.net.URL;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.EnumMap;
+import java.util.HashSet;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Registry that loads and stores the request validators to be applied by
+ * a service.
+ */
+public class ValidatorRegistry {
+
+  private final EnumMap<
+      ValidationCondition, EnumMap<Type, Pair<List<Method>, List<Method>>>>
+      validators = new EnumMap<>(ValidationCondition.class);

Review comment:
       It looks like the pair in this structure is being used to determine the processing phase, with left meaning pre and right meaning post. However it looks like each method will only ever be a pre or post processor, so only one half of the pair is used for each entry.
   
   My understanding of the usage of this method is that given a ValidationCondition, RequestType, and RequestProcessingPhase we can get a list of corresponding methods to do the validation. I think `ValidationCondition, EnumMap<Type, EnumMap<RequestProcessingPhase, List<Method>>>>` would be a little more intuitive.

##########
File path: hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/validation/testvalidatorset1/GeneralValidatorsForTesting.java
##########
@@ -0,0 +1,233 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership.  The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.hadoop.ozone.om.request.validation.testvalidatorset1;
+
+import org.apache.hadoop.ozone.om.request.validation.RequestFeatureValidator;
+import org.apache.hadoop.ozone.om.request.validation.TestRequestValidations;
+import org.apache.hadoop.ozone.om.request.validation.ValidationContext;
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.OMRequest;
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.OMResponse;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+
+import static org.apache.hadoop.ozone.om.request.validation.RequestProcessingPhase.POST_PROCESS;
+import static org.apache.hadoop.ozone.om.request.validation.RequestProcessingPhase.PRE_PROCESS;
+import static org.apache.hadoop.ozone.om.request.validation.ValidationCondition.CLUSTER_NEEDS_FINALIZATION;
+import static org.apache.hadoop.ozone.om.request.validation.ValidationCondition.NEWER_CLIENT_REQUESTS;
+import static org.apache.hadoop.ozone.om.request.validation.ValidationCondition.OLDER_CLIENT_REQUESTS;
+import static org.apache.hadoop.ozone.om.request.validation.ValidationCondition.UNCONDITIONAL;
+import static org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.Type.CreateKey;
+import static org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.Type.CreateVolume;
+import static org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.Type.DeleteKeys;
+
+/**
+ * Some annotated request validator method, and facilities to help check if
+ * validations were properly called from tests where applicable.
+ */
+public final class GeneralValidatorsForTesting {
+
+  /**
+   * As the validators written here does not override any request or response
+   * but throw exceptions for specific tests, a test that wants to directly
+   * use a validator here, has to turn on this boolean, and the method that
+   * the test relies on has to check for this value.
+   *
+   * This is necessary to do not affect other tests that are testing requests
+   * processing, as for some of those tests this package is on the classpath
+   * and therefore the annotated validations are loadede for them.
+   */
+  private static boolean validatorTestsRunning = false;
+
+  public static void startValidatorTest() {
+    validatorTestsRunning = true;
+  }
+
+  public static void finishValidatorTest() {
+    validatorTestsRunning = false;
+  }
+
+  private GeneralValidatorsForTesting() { }
+
+  /**
+   * Interface to easily add listeners that get notified if a certain validator
+   * method defined in this class was called.
+   *
+   * @see TestRequestValidations for more details on how this intercace is
+   *      being used.
+   */
+  @FunctionalInterface
+  public interface ValidationListener {
+    void validationCalled(String calledMethodName);
+  }
+
+  private static List<ValidationListener> listeners = new ArrayList<>();
+
+  public static void addListener(ValidationListener listener) {
+    listeners.add(listener);
+  }
+
+  public static void removeListener(ValidationListener listener) {
+    listeners.remove(listener);
+  }
+
+  private static void fireValidationEvent(String calledMethodName) {
+    listeners.forEach(l -> l.validationCalled(calledMethodName));
+  }
+
+  @RequestFeatureValidator(
+      conditions = { CLUSTER_NEEDS_FINALIZATION },
+      processingPhase = PRE_PROCESS,
+      requestType = CreateKey)
+  public static OMRequest preFinalizePreProcessCreateKeyValidator(
+      OMRequest req, ValidationContext ctx) {
+    fireValidationEvent("preFinalizePreProcessCreateKeyValidator");
+    return req;
+  }
+
+  @RequestFeatureValidator(
+      conditions = { CLUSTER_NEEDS_FINALIZATION },
+      processingPhase = POST_PROCESS,
+      requestType = CreateKey)
+  public static OMResponse preFinalizePostProcessCreateKeyValidator(

Review comment:
       Annotating the validator instead of the request isn't bad, but would it be possible to rework it so the annotation goes on the request, and the validator is passed in as a functional interface to the annotation? Right now request annotations like `@DisallowedUntilLayoutVersion` go on the request itself, so if this set of annotations is not on the request, it may be difficult to figure out what processing each request has. Even if I know where to look as an upgrade dev, I think annotating the request would be more obvious/intuitive for the rest of the community who may need to use this feature.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org