[GitHub] [ozone] rakeshadr commented on a change in pull request #2357: HDDS-5362. [FSO] Support bucket types in OM

[GitBox <gi...@apache.org>]

rakeshadr commented on a change in pull request #2357:
URL: https://github.com/apache/ozone/pull/2357#discussion_r656746156



##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/BucketType.java
##########
@@ -0,0 +1,56 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.helpers;
+
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos;
+
+/**
+ * The replication type to be used while writing key into ozone.

Review comment:
       Javadoc is wrong, please modify it.

##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmBucketInfo.java
##########
@@ -102,6 +107,7 @@
    * @param usedBytes - Bucket Quota Usage in bytes.
    * @param quotaInBytes Bucket quota in bytes.
    * @param quotaInNamespace Bucket quota in counts.
+   * @param bucketType Bucket Types.

Review comment:
       typo: `Bucket Types.` -> `Bucket Type.`

##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestObjectStoreWithFSO.java
##########
@@ -622,6 +621,40 @@ public void testRenameToAnExistingKey() throws Exception {
     }
   }
 
+  @Test
+  public void testCreateBucketWithBucketType() throws Exception {
+    String sampleVolumeName = UUID.randomUUID().toString();
+    String sampleBucketName = UUID.randomUUID().toString();
+    OzoneClient client = cluster.getClient();
+    ObjectStore store = client.getObjectStore();
+    store.createVolume(sampleVolumeName);
+    OzoneVolume volume = store.getVolume(sampleVolumeName);
+
+    // Case 1: Bucket Type: FSO
+    BucketArgs.Builder builder = BucketArgs.newBuilder();
+    builder.setBucketType(BucketType.FSO);
+    volume.createBucket(sampleBucketName, builder.build());
+    OzoneBucket bucket = volume.getBucket(sampleBucketName);
+    Assert.assertEquals(sampleBucketName, bucket.getName());
+    Assert.assertEquals(BucketType.FSO, bucket.getBucketType());
+
+    // Case 2: Bucket Type: OBJECT_STORE
+    sampleBucketName = UUID.randomUUID().toString();
+    builder.setBucketType(BucketType.OBJECT_STORE);
+    volume.createBucket(sampleBucketName, builder.build());
+    bucket = volume.getBucket(sampleBucketName);
+    Assert.assertEquals(sampleBucketName, bucket.getName());
+    Assert.assertEquals(BucketType.OBJECT_STORE, bucket.getBucketType());
+
+    // Case 3: Bucket Type: Empty
+    sampleBucketName = UUID.randomUUID().toString();
+    builder.setBucketType(BucketType.DEFAULT);
+    volume.createBucket(sampleBucketName, builder.build());
+    bucket = volume.getBucket(sampleBucketName);
+    Assert.assertEquals(sampleBucketName, bucket.getName());
+    Assert.assertEquals(BucketType.DEFAULT, bucket.getBucketType());

Review comment:
       Please point me to a test case where user passes an argument with LEGACY type. It shouldn't allow to create a LEGACY bucket and throws exception. If there is no test then please add it a 5th case like,
   
   // case 5: Bucket Type: LEGACY
       builder.setBucketType(BucketType.LEGACY);
       volume.createBucket(sampleBucketName, builder.build());

##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestObjectStoreWithFSO.java
##########
@@ -622,6 +621,40 @@ public void testRenameToAnExistingKey() throws Exception {
     }
   }
 
+  @Test
+  public void testCreateBucketWithBucketType() throws Exception {
+    String sampleVolumeName = UUID.randomUUID().toString();
+    String sampleBucketName = UUID.randomUUID().toString();
+    OzoneClient client = cluster.getClient();
+    ObjectStore store = client.getObjectStore();
+    store.createVolume(sampleVolumeName);
+    OzoneVolume volume = store.getVolume(sampleVolumeName);
+
+    // Case 1: Bucket Type: FSO
+    BucketArgs.Builder builder = BucketArgs.newBuilder();
+    builder.setBucketType(BucketType.FSO);
+    volume.createBucket(sampleBucketName, builder.build());
+    OzoneBucket bucket = volume.getBucket(sampleBucketName);
+    Assert.assertEquals(sampleBucketName, bucket.getName());
+    Assert.assertEquals(BucketType.FSO, bucket.getBucketType());
+
+    // Case 2: Bucket Type: OBJECT_STORE
+    sampleBucketName = UUID.randomUUID().toString();
+    builder.setBucketType(BucketType.OBJECT_STORE);
+    volume.createBucket(sampleBucketName, builder.build());
+    bucket = volume.getBucket(sampleBucketName);
+    Assert.assertEquals(sampleBucketName, bucket.getName());
+    Assert.assertEquals(BucketType.OBJECT_STORE, bucket.getBucketType());
+
+    // Case 3: Bucket Type: Empty

Review comment:
       Here, it says `Empty` but it is passing `DEFAULT` in the builder `builder.setBucketType(BucketType.DEFAULT);`.
   Please remove `builder.setBucketType(BucketType.DEFAULT); `for verifying `Empty` case and you can add one more case for DEFAULT.
   
   `// Case 4: Bucket Type: DEFAULT`

##########
File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/CreateBucketHandler.java
##########
@@ -60,9 +67,18 @@
   public void execute(OzoneClient client, OzoneAddress address)
       throws IOException {
 
-    BucketArgs.Builder bb = new BucketArgs.Builder()
-        .setStorageType(StorageType.DEFAULT)
-        .setVersioning(false);
+    BucketArgs.Builder bb;
+    if (bucketType.equals(BucketType.OBJECT_STORE) || bucketType
+        .equals(BucketType.FSO)) {
+      bb = new BucketArgs.Builder().setStorageType(StorageType.DEFAULT)
+          .setVersioning(false).setBucketType(bucketType);
+    } else {
+      throw new IllegalArgumentException(

Review comment:
       Hope we have test validating this exception. If not then please add it. 

##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/BucketType.java
##########
@@ -0,0 +1,56 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.helpers;
+
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos;
+
+/**
+ * The replication type to be used while writing key into ozone.
+ */
+public enum BucketType {
+  FSO, OBJECT_STORE, LEGACY;
+  public static final BucketType DEFAULT = OBJECT_STORE;
+  public static BucketType fromProto(
+      OzoneManagerProtocolProtos.BucketTypeProto bucketType) {
+    if (bucketType == null) {
+      return BucketType.LEGACY;
+    }
+    switch (bucketType) {
+    case FSO:
+      return BucketType.FSO;
+    case OBJECT_STORE:
+    default:
+      return DEFAULT;
+    }
+  }
+
+  public OzoneManagerProtocolProtos.BucketTypeProto toProto() {
+    switch (this) {
+    case FSO:
+      return OzoneManagerProtocolProtos.BucketTypeProto.FSO;
+    case OBJECT_STORE:
+      return OzoneManagerProtocolProtos.BucketTypeProto.OBJECT_STORE;
+    case LEGACY:
+      return OzoneManagerProtocolProtos.BucketTypeProto.LEGACY;
+    default:
+      throw new IllegalStateException(

Review comment:
       As this is not state, can we`throw new IllegalArgumentException("Error: BucketType not found, type=" + this)`?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org