You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/09/04 16:56:23 UTC
[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #1392: HDDS-4173. Implement HDDS Version management using the LayoutVersion…
avijayanhwx commented on a change in pull request #1392:
URL: https://github.com/apache/hadoop-ozone/pull/1392#discussion_r483365229
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/upgrade/SCMLayoutFeatureAPI.java
##########
@@ -0,0 +1,35 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hdds.scm.upgrade;
+
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+import org.apache.hadoop.hdds.scm.upgrade.SCMLayoutFeatureCatalog.SCMLayoutFeature;
+
+/**
+ * Annotation to specify if an API is backed up by a Layout Feature.
+ */
+@Target(ElementType.METHOD)
+@Retention(RetentionPolicy.RUNTIME)
+public @interface SCMLayoutFeatureAPI {
Review comment:
Can be removed.
##########
File path: hadoop-hdds/server-scm/pom.xml
##########
@@ -29,6 +29,19 @@ https://maven.apache.org/xsd/maven-4.0.0.xsd">
<packaging>jar</packaging>
<dependencies>
+
+ <dependency>
Review comment:
Let's hold off on introducing Aspectj in HDDS layer, since we have not fully understood the impact of bringing it in. Moreover, it suits a '**client facing**' component like OzoneManager better since the point cut that we have is used for disallowing '**pre-finalize state unsupported client APIs**'. Once there are concrete API use cases for SCM, we can bring it here by moving the OM Aspect to a common maven module.
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/upgrade/SCMLayoutFeatureCatalog.java
##########
@@ -0,0 +1,93 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hdds.scm.upgrade;
+
+import java.util.Optional;
+
+import org.apache.hadoop.ozone.upgrade.LayoutFeature;
+
+/**
+ * Catalog of SCM features.
+ */
+public class SCMLayoutFeatureCatalog {
+
+ /**
+ * List of SCM Features.
+ */
+ public enum SCMLayoutFeature implements LayoutFeature {
Review comment:
Should be "HDDSLayoutFeature" since it is common across SCM and DN.
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/upgrade/SCMLayoutFeatureCatalog.java
##########
@@ -0,0 +1,93 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hdds.scm.upgrade;
+
+import java.util.Optional;
+
+import org.apache.hadoop.ozone.upgrade.LayoutFeature;
+
+/**
+ * Catalog of SCM features.
+ */
+public class SCMLayoutFeatureCatalog {
+
+ /**
+ * List of SCM Features.
+ */
+ public enum SCMLayoutFeature implements LayoutFeature {
+ INITIAL_VERSION(0, "Initial Layout Version"),
+ CREATE_EC(1, ""),
+ NEW_FEATURE(2, "new feature", new NewSCMFeatureUpgradeAction());
+
+
+ private int layoutVersion;
+ private String description;
+ private Optional<SCMUpgradeAction> scmUpgradeAction = Optional.empty();
Review comment:
We may need an 'HDDSUpgradeAction' from which SCMUpgradeAction is extended. Then, in this class we can have something like _Optional<? extends HDDSUpgradeAction>_.
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/upgrade/SCMLayoutFeatureAspect.java
##########
@@ -0,0 +1,54 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hdds.scm.upgrade;
+
+import static org.apache.hadoop.hdds.scm.exceptions.SCMException.ResultCodes.NOT_SUPPORTED_OPERATION;
+
+import org.apache.hadoop.hdds.scm.exceptions.SCMException;
+import org.apache.hadoop.ozone.upgrade.LayoutFeature;
+import org.apache.hadoop.ozone.upgrade.LayoutVersionManager;
+import org.aspectj.lang.JoinPoint;
+import org.aspectj.lang.annotation.Aspect;
+import org.aspectj.lang.annotation.Before;
+import org.aspectj.lang.reflect.MethodSignature;
+
+/**
+ * 'Aspect' for SCM Layout Feature API. All methods annotated with the
+ * specific annotation will have pre-processing done here to check layout
+ * version compatibility.
+ */
+@Aspect
+public class SCMLayoutFeatureAspect {
Review comment:
Can be removed.
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/upgrade/SCMLayoutFeatureCatalog.java
##########
@@ -0,0 +1,93 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hdds.scm.upgrade;
+
+import java.util.Optional;
+
+import org.apache.hadoop.ozone.upgrade.LayoutFeature;
+
+/**
+ * Catalog of SCM features.
+ */
+public class SCMLayoutFeatureCatalog {
+
+ /**
+ * List of SCM Features.
+ */
+ public enum SCMLayoutFeature implements LayoutFeature {
+ INITIAL_VERSION(0, "Initial Layout Version"),
+ CREATE_EC(1, ""),
Review comment:
We already have "test" features in OM Catalog to support testing. HDDS can be left with just 'INITIAL_VERSION'.
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/upgrade/SCMLayoutVersionManager.java
##########
@@ -0,0 +1,91 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hdds.scm.upgrade;
+
+import static org.apache.hadoop.hdds.scm.exceptions.SCMException.ResultCodes.NOT_SUPPORTED_OPERATION;
+import org.apache.hadoop.ozone.common.Storage;
+import org.apache.hadoop.hdds.scm.server.SCMStorageConfig;
+import org.apache.hadoop.hdds.scm.exceptions.SCMException;
+import org.apache.hadoop.hdds.scm.upgrade.SCMLayoutFeatureCatalog.SCMLayoutFeature;
+import org.apache.hadoop.ozone.upgrade.AbstractLayoutVersionManager;
+import org.apache.hadoop.ozone.upgrade.LayoutVersionManager;
+
+import com.google.common.annotations.VisibleForTesting;
+
+/**
+ * Class to manage layout versions and features for Storage Container Manager.
+ */
+public final class SCMLayoutVersionManager extends AbstractLayoutVersionManager {
Review comment:
Can we rename to HDDSLayoutVersionManager to reuse between SCM and DN?
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/upgrade/SCMLayoutFeatureCatalog.java
##########
@@ -0,0 +1,93 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hdds.scm.upgrade;
+
+import java.util.Optional;
+
+import org.apache.hadoop.ozone.upgrade.LayoutFeature;
+
+/**
+ * Catalog of SCM features.
+ */
+public class SCMLayoutFeatureCatalog {
+
+ /**
+ * List of SCM Features.
+ */
+ public enum SCMLayoutFeature implements LayoutFeature {
+ INITIAL_VERSION(0, "Initial Layout Version"),
+ CREATE_EC(1, ""),
+ NEW_FEATURE(2, "new feature", new NewSCMFeatureUpgradeAction());
+
+
+ private int layoutVersion;
+ private String description;
+ private Optional<SCMUpgradeAction> scmUpgradeAction = Optional.empty();
+
+ SCMLayoutFeature(final int layoutVersion, String description) {
+ this.layoutVersion = layoutVersion;
+ this.description = description;
+ }
+
+ SCMLayoutFeature(final int layoutVersion, String description,
+ SCMUpgradeAction upgradeAction) {
+ this.layoutVersion = layoutVersion;
+ this.description = description;
+ scmUpgradeAction = Optional.of(upgradeAction);
+ }
+
+ @Override
+ public int layoutVersion() {
+ return layoutVersion;
+ }
+
+ @Override
+ public String description() {
+ return description;
+ }
+
+ @Override
+ public Optional<SCMUpgradeAction> onFinalizeAction() {
+ return scmUpgradeAction;
+ }
+ }
+
+ /**
+ * This is an example of an "API" that uses a new Layout feature (EC) that is
+ * not yet supported by the current layout version. The following can be
+ * "guarded" by just adding the following annotation, thereby keeping the
+ * method logic and upgrade logic separate.
+ */
+ @SCMLayoutFeatureAPI(SCMLayoutFeature.CREATE_EC)
Review comment:
Can be removed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org