You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/10/05 05:44:12 UTC
[GitHub] [ozone] JyotinderSingh opened a new pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
JyotinderSingh opened a new pull request #2707:
URL: https://github.com/apache/ozone/pull/2707
## What changes were proposed in this pull request?
Introduce OM configuration “ozone.bucket.layout.default” in OM*.* OM will use this default value during bucket creation if the client(S3 or o3fs) is not passing the bucket type argument.
Defaulting to “ozone.bucket.layout.default=OBS”. Presently, we have a default value “ozone.om.enable.filesystem.paths=false”, which is exactly equivalent to OBS bucket type behavior.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-5373
## How was this patch tested?
work in progress
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724791021
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##########
@@ -452,6 +455,9 @@ private OzoneManager(OzoneConfiguration conf, StartupOption startupOption)
isRatisEnabled = configuration.getBoolean(
OMConfigKeys.OZONE_OM_RATIS_ENABLE_KEY,
OMConfigKeys.OZONE_OM_RATIS_ENABLE_DEFAULT);
+ this.defaultBucketLayout =
+ configuration.getTrimmed(OZONE_DEFAULT_BUCKET_LAYOUT,
+ OZONE_DEFAULT_BUCKET_LAYOUT_DEFAULT);
Review comment:
I have added the validation check
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 removed a comment on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 removed a comment on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-935492211
/pending
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r723464254
##########
File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/CreateBucketHandler.java
##########
@@ -53,11 +53,11 @@
// TODO: LEGACY should be removed and should not be exposed to the end user
Review comment:
Can you please remove these comment because we are implementing the default behavior in this patch, isn't it?
```
// TODO: LEGACY should be removed and should not be exposed to the end user
// we will revisit during the client side defaulting behaviour
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r721922631
##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMConfigKeys.java
##########
@@ -270,6 +271,11 @@ private OMConfigKeys() {
public static final String OZONE_OM_METADATA_LAYOUT_PREFIX = "PREFIX";
+ public static final String OZONE_BUCKET_LAYOUT =
+ "ozone.bucket.layout";
Review comment:
Understood, will add this to the patch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 removed a comment on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 removed a comment on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-935492879
/pending Need to fix review comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-935492879
/pending Need to fix review comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724623031
##########
File path: hadoop-hdds/common/src/main/resources/ozone-default.xml
##########
@@ -2813,7 +2813,7 @@
<tag>OZONE, SCM</tag>
<description>Thread pool size configured to process container reports.
</description>
- </property>
+ </property>
Review comment:
Minor NIT: unnecessary change
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##########
@@ -452,6 +455,9 @@ private OzoneManager(OzoneConfiguration conf, StartupOption startupOption)
isRatisEnabled = configuration.getBoolean(
OMConfigKeys.OZONE_OM_RATIS_ENABLE_KEY,
OMConfigKeys.OZONE_OM_RATIS_ENABLE_DEFAULT);
+ this.defaultBucketLayout =
+ configuration.getTrimmed(OZONE_DEFAULT_BUCKET_LAYOUT,
+ OZONE_DEFAULT_BUCKET_LAYOUT_DEFAULT);
Review comment:
Do we want to validate it here? If it is one of the supported values and fail startup here?
##########
File path: hadoop-hdds/common/src/main/resources/ozone-default.xml
##########
@@ -2108,7 +2108,7 @@
not be renewed.
</description>
</property>
-
+
Review comment:
Minor NIT: unnecessary change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724690390
##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/BucketLayout.java
##########
@@ -59,4 +60,10 @@ public static BucketLayout fromProto(
"Error: BucketLayout not found, type=" + this);
}
}
+
+ public static BucketLayout fromString(String value) {
+ // Todo: should we throw error if user configured unsupported value
+ // during OM startup or bucket creation time.
+ return StringUtils.isBlank(value) ? LEGACY : BucketLayout.valueOf(value);
Review comment:
We mainly needed the check here for sake of testing, because sometimes tests do not initialize the OM properly and leave some fields uninitialized.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r723051136
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##########
@@ -4072,6 +4074,11 @@ public String getOMMetadataLayout() {
.getTrimmed(OZONE_OM_METADATA_LAYOUT, OZONE_OM_METADATA_LAYOUT_DEFAULT);
}
+ public String getOMDefaultBucketLayout() {
Review comment:
I am now initializing an instance variable in the constructor for this value, please let me know if that's okay
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r723509326
##########
File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/CreateBucketHandler.java
##########
@@ -53,11 +53,11 @@
// TODO: LEGACY should be removed and should not be exposed to the end user
Review comment:
Removed these comments in the latest patch
##########
File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/CreateBucketHandler.java
##########
@@ -53,11 +53,11 @@
// TODO: LEGACY should be removed and should not be exposed to the end user
Review comment:
Removing these comments in the latest patch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724792048
##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/BucketLayout.java
##########
@@ -59,4 +60,10 @@ public static BucketLayout fromProto(
"Error: BucketLayout not found, type=" + this);
}
}
+
+ public static BucketLayout fromString(String value) {
+ // Todo: should we throw error if user configured unsupported value
+ // during OM startup or bucket creation time.
+ return StringUtils.isBlank(value) ? LEGACY : BucketLayout.valueOf(value);
Review comment:
Updated the comment and added validation logic to OM start up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r725222907
##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestObjectStore.java
##########
@@ -0,0 +1,103 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.hadoop.ozone.om;
+
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ozone.MiniOzoneCluster;
+import org.apache.hadoop.ozone.client.*;
+import org.apache.hadoop.ozone.om.helpers.BucketLayout;
+import org.junit.*;
+import org.junit.rules.Timeout;
+
+import java.io.IOException;
+import java.util.UUID;
+
+/**
+ * Tests to verify Object store without prefix enabled.
+ */
+public class TestObjectStore {
+ private static MiniOzoneCluster cluster = null;
+ private static OzoneConfiguration conf;
+ private static String clusterId;
+ private static String scmId;
+ private static String omId;
+
+ @Rule
+ public Timeout timeout = new Timeout(1200000);
+
+ /**
+ * Create a MiniOzoneCluster for testing.
+ * <p>
+ *
+ * @throws IOException
+ */
+ @BeforeClass
+ public static void init() throws Exception {
+ conf = new OzoneConfiguration();
+ clusterId = UUID.randomUUID().toString();
+ scmId = UUID.randomUUID().toString();
+ omId = UUID.randomUUID().toString();
+ cluster = MiniOzoneCluster.newBuilder(conf).setClusterId(clusterId)
+ .setScmId(scmId).setOmId(omId).build();
+ cluster.waitForClusterToBeReady();
+ }
+
+ /**
+ * Shutdown MiniOzoneCluster.
+ */
+ @AfterClass
+ public static void shutdown() {
+ if (cluster != null) {
+ cluster.shutdown();
+ }
+ }
+
+ @Test
+ public void testCreateBucketWithBucketLayout() throws Exception {
+ String sampleVolumeName = UUID.randomUUID().toString();
+ String sampleBucketName = UUID.randomUUID().toString();
+ OzoneClient client = cluster.getClient();
+ ObjectStore store = client.getObjectStore();
+ store.createVolume(sampleVolumeName);
+ OzoneVolume volume = store.getVolume(sampleVolumeName);
+
+ // Case 1: Bucket layout: Empty and OM default bucket layout: OBJECT_STORE
+ BucketArgs.Builder builder = BucketArgs.newBuilder();
+ volume.createBucket(sampleBucketName, builder.build());
+ OzoneBucket bucket = volume.getBucket(sampleBucketName);
+ Assert.assertEquals(sampleBucketName, bucket.getName());
+ Assert.assertEquals(BucketLayout.OBJECT_STORE,
+ bucket.getBucketLayout());
+
+ // Case 2: Bucket layout: DEFAULT
+ sampleBucketName = UUID.randomUUID().toString();
+ builder.setBucketLayout(BucketLayout.DEFAULT);
+ volume.createBucket(sampleBucketName, builder.build());
+ bucket = volume.getBucket(sampleBucketName);
+ Assert.assertEquals(sampleBucketName, bucket.getName());
+ Assert.assertEquals(BucketLayout.OBJECT_STORE,
+ bucket.getBucketLayout());
+
+ // Case 3: Bucket layout: LEGACY
+ sampleBucketName = UUID.randomUUID().toString();
+ builder.setBucketLayout(BucketLayout.LEGACY);
+ volume.createBucket(sampleBucketName, builder.build());
Review comment:
As anyway we have BucketType.DEFAULT to let users create bucket type which is matching with server. So, I think we can avoid allowing LEGACY from createBucket Java API also.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh removed a comment on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh removed a comment on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-939226810
/retest
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724625214
##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/BucketLayout.java
##########
@@ -59,4 +60,10 @@ public static BucketLayout fromProto(
"Error: BucketLayout not found, type=" + this);
}
}
+
+ public static BucketLayout fromString(String value) {
+ // Todo: should we throw error if user configured unsupported value
+ // during OM startup or bucket creation time.
+ return StringUtils.isBlank(value) ? LEGACY : BucketLayout.valueOf(value);
Review comment:
If we validate the value from config from startup, we don't need this isBlank check here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r723470380
##########
File path: hadoop-hdds/docs/content/feature/PrefixFSO.md
##########
@@ -72,4 +72,17 @@ By default the feature is disabled. It can be enabled with the following
<name>ozone.om.metadata.layout</name>
<value>PREFIX</value>
</property>
+```
+
+In reference to efforts towards supporting protocol aware buckets within an
+Ozone cluster, the following configuration can be used to define the default
+value for bucket layout during bucket creation if the client has not specified
+the bucket layout argument.
Review comment:
Please include this sentence also.
`Supported values are OBJECT_STORE and FILE_SYSTEM_OPTIMIZED.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r721922734
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketCreateRequest.java
##########
@@ -145,12 +145,7 @@ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager,
OmBucketInfo omBucketInfo = null;
if (bucketInfo.getBucketLayout() == null || bucketInfo.getBucketLayout()
.equals(BucketLayoutProto.LEGACY)) {
- BucketLayout defaultType = BucketLayout.LEGACY;
- if (StringUtils
- .equalsIgnoreCase(OMConfigKeys.OZONE_OM_METADATA_LAYOUT_PREFIX,
- omLayout)) {
- defaultType = BucketLayout.FILE_SYSTEM_OPTIMIZED;
- }
+ BucketLayout defaultType = OMConfigKeys.OZONE_BUCKET_LAYOUT_DEFAULT;
Review comment:
Thanks for the review, working on this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] github-actions[bot] commented on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-935492482
No such command. `pending
` Available commands:
* /**close** : Close pending pull request temporary
* /**help** : Show all the available comment commands
* /**label** : add new label to the issue: `/label <label>`
* /**pending** : Add a REQUESTED_CHANGE type review to mark issue non-mergeable: `/pending <reason>`
* /**ready** : Dismiss all the blocking reviews by github-actions bot
* /**retest** : provide help on how to trigger new CI build
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r723615006
##########
File path: hadoop-hdds/docs/content/feature/PrefixFSO.md
##########
@@ -72,4 +72,17 @@ By default the feature is disabled. It can be enabled with the following
<name>ozone.om.metadata.layout</name>
<value>PREFIX</value>
</property>
+```
+
+In reference to efforts towards supporting protocol aware buckets within an
+Ozone cluster, the following configuration can be used to define the default
+value for bucket layout during bucket creation if the client has not specified
+the bucket layout argument.
Review comment:
Added these in the latest patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-938659653
@bharatviswa504 could you review these changes? I hope I've addressed your concerns :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh edited a comment on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh edited a comment on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-934311602
@rakeshadr with these changes in `OMBucketCreateRequest.java` we no longer respect the cluster level configurations like `ozone.om.metadata.layout` - which causes checks under `TestObjectStoreWithFSO#testCreateBucketWithBucketLayout` to fail.
Are these tests supposed to change or is there some inconsistency in my implementation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r722281528
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketCreateRequest.java
##########
@@ -145,10 +145,14 @@ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager,
OmBucketInfo omBucketInfo = null;
if (bucketInfo.getBucketLayout() == null || bucketInfo.getBucketLayout()
.equals(BucketLayoutProto.LEGACY)) {
- BucketLayout defaultType = BucketLayout.LEGACY;
- if (StringUtils
- .equalsIgnoreCase(OMConfigKeys.OZONE_OM_METADATA_LAYOUT_PREFIX,
- omLayout)) {
+ // Bucket Layout argument was not passed during bucket creation.
Review comment:
I've addressed the reviews in my latest patch, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r723521878
##########
File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/CreateBucketHandler.java
##########
@@ -53,11 +53,11 @@
// TODO: LEGACY should be removed and should not be exposed to the end user
// we will revisit during the client side defaulting behaviour
- enum AllowedBucketLayouts { FILE_SYSTEM_OPTIMIZED, OBJECT_STORE, LEGACY }
+ enum AllowedBucketLayouts { FILE_SYSTEM_OPTIMIZED, OBJECT_STORE }
Review comment:
Thank you for the help, I'm adding this test to the suite.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr edited a comment on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
rakeshadr edited a comment on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-938654415
> And a question? In further Jiras we will be changing the buckets created from (OFS) be FILE_SYSTEM_OPTIMIZED Buckets?
yes, @bharatviswa504. Thanks for bringing that point, I've raised HDDS-5839 to track this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-939226810
/retest
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r725306333
##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestObjectStore.java
##########
@@ -0,0 +1,103 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.hadoop.ozone.om;
+
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ozone.MiniOzoneCluster;
+import org.apache.hadoop.ozone.client.*;
+import org.apache.hadoop.ozone.om.helpers.BucketLayout;
+import org.junit.*;
+import org.junit.rules.Timeout;
+
+import java.io.IOException;
+import java.util.UUID;
+
+/**
+ * Tests to verify Object store without prefix enabled.
+ */
+public class TestObjectStore {
+ private static MiniOzoneCluster cluster = null;
+ private static OzoneConfiguration conf;
+ private static String clusterId;
+ private static String scmId;
+ private static String omId;
+
+ @Rule
+ public Timeout timeout = new Timeout(1200000);
+
+ /**
+ * Create a MiniOzoneCluster for testing.
+ * <p>
+ *
+ * @throws IOException
+ */
+ @BeforeClass
+ public static void init() throws Exception {
+ conf = new OzoneConfiguration();
+ clusterId = UUID.randomUUID().toString();
+ scmId = UUID.randomUUID().toString();
+ omId = UUID.randomUUID().toString();
+ cluster = MiniOzoneCluster.newBuilder(conf).setClusterId(clusterId)
+ .setScmId(scmId).setOmId(omId).build();
+ cluster.waitForClusterToBeReady();
+ }
+
+ /**
+ * Shutdown MiniOzoneCluster.
+ */
+ @AfterClass
+ public static void shutdown() {
+ if (cluster != null) {
+ cluster.shutdown();
+ }
+ }
+
+ @Test
+ public void testCreateBucketWithBucketLayout() throws Exception {
+ String sampleVolumeName = UUID.randomUUID().toString();
+ String sampleBucketName = UUID.randomUUID().toString();
+ OzoneClient client = cluster.getClient();
+ ObjectStore store = client.getObjectStore();
+ store.createVolume(sampleVolumeName);
+ OzoneVolume volume = store.getVolume(sampleVolumeName);
+
+ // Case 1: Bucket layout: Empty and OM default bucket layout: OBJECT_STORE
+ BucketArgs.Builder builder = BucketArgs.newBuilder();
+ volume.createBucket(sampleBucketName, builder.build());
+ OzoneBucket bucket = volume.getBucket(sampleBucketName);
+ Assert.assertEquals(sampleBucketName, bucket.getName());
+ Assert.assertEquals(BucketLayout.OBJECT_STORE,
+ bucket.getBucketLayout());
+
+ // Case 2: Bucket layout: DEFAULT
+ sampleBucketName = UUID.randomUUID().toString();
+ builder.setBucketLayout(BucketLayout.DEFAULT);
+ volume.createBucket(sampleBucketName, builder.build());
+ bucket = volume.getBucket(sampleBucketName);
+ Assert.assertEquals(sampleBucketName, bucket.getName());
+ Assert.assertEquals(BucketLayout.OBJECT_STORE,
+ bucket.getBucketLayout());
+
+ // Case 3: Bucket layout: LEGACY
+ sampleBucketName = UUID.randomUUID().toString();
+ builder.setBucketLayout(BucketLayout.LEGACY);
+ volume.createBucket(sampleBucketName, builder.build());
Review comment:
I have raised [HDDS-5841](https://issues.apache.org/jira/browse/HDDS-5841) for tracking this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r725304471
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##########
@@ -453,6 +457,18 @@ private OzoneManager(OzoneConfiguration conf, StartupOption startupOption)
OMConfigKeys.OZONE_OM_RATIS_ENABLE_KEY,
OMConfigKeys.OZONE_OM_RATIS_ENABLE_DEFAULT);
+ this.defaultBucketLayout =
+ configuration.getTrimmed(OZONE_DEFAULT_BUCKET_LAYOUT,
+ OZONE_DEFAULT_BUCKET_LAYOUT_DEFAULT);
+ // Make sure defaultBucketLayout is set to a valid value
+ if (!defaultBucketLayout.equals(
+ BucketLayout.FILE_SYSTEM_OPTIMIZED.name()) &&
+ !defaultBucketLayout.equals(BucketLayout.OBJECT_STORE.name())
+ ) {
+ throw new IllegalArgumentException(
Review comment:
Sure, I've replaced this with ConfigurationException
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724690390
##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/BucketLayout.java
##########
@@ -59,4 +60,10 @@ public static BucketLayout fromProto(
"Error: BucketLayout not found, type=" + this);
}
}
+
+ public static BucketLayout fromString(String value) {
+ // Todo: should we throw error if user configured unsupported value
+ // during OM startup or bucket creation time.
+ return StringUtils.isBlank(value) ? LEGACY : BucketLayout.valueOf(value);
Review comment:
We mainly needed the check here for sake of testing, since they sometimes do not initialize the OM properly and leave some fields uninitialized.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-936603469
@JyotinderSingh Please include below code snippet to fix the test case failures. Those are failing especially because "ozone.default.bucket.layout" should be set of `FILE_SYSTEM_OPTIMIZED`. Presently, it is defaulting to `OBJECT_STORE` and is causing the failure.
Please go to[ TestOMRequestUtils#configureFSOptimizedPaths](https://github.com/apache/ozone/blob/master/hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/TestOMRequestUtils.java#L1038)method and set default to FSO like below:
```
public static void configureFSOptimizedPaths(Configuration conf,
boolean enableFileSystemPaths, String version) {
conf.setBoolean(OMConfigKeys.OZONE_OM_ENABLE_FILESYSTEM_PATHS,
enableFileSystemPaths);
conf.set(OMConfigKeys.OZONE_OM_METADATA_LAYOUT, version);
if (StringUtils.equalsIgnoreCase(OMConfigKeys.OZONE_OM_METADATA_LAYOUT_PREFIX, version)) {
conf.set(OMConfigKeys.OZONE_DEFAULT_BUCKET_LAYOUT, BucketLayout.FILE_SYSTEM_OPTIMIZED.name());
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr merged pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
rakeshadr merged pull request #2707:
URL: https://github.com/apache/ozone/pull/2707
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh edited a comment on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh edited a comment on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-935806252
Thanks for the reviews @bharatviswa504 . I have addressed your feedback in my patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r723030634
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketCreateRequest.java
##########
@@ -145,12 +143,9 @@ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager,
OmBucketInfo omBucketInfo = null;
if (bucketInfo.getBucketLayout() == null || bucketInfo.getBucketLayout()
.equals(BucketLayoutProto.LEGACY)) {
Review comment:
I have removed LEGACY from `AllowedBucketLayouts` and ran a few tests that seemed to be using that value. I currently do not see any errors coming up. I am including this change in my patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-934311602
@rakeshadr with these changes in `OMBucketCreateRequest.java` we no longer respect the cluster level configurations like `ozone.om.metadata.layout` - which causes checks under `TestObjectStoreWithFSO#testCreateBucketWithBucketLayout` to fail.
Should I change these tests or is there some inconsistency in my implementation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r725243245
##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestObjectStore.java
##########
@@ -0,0 +1,103 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.hadoop.ozone.om;
+
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ozone.MiniOzoneCluster;
+import org.apache.hadoop.ozone.client.*;
+import org.apache.hadoop.ozone.om.helpers.BucketLayout;
+import org.junit.*;
+import org.junit.rules.Timeout;
+
+import java.io.IOException;
+import java.util.UUID;
+
+/**
+ * Tests to verify Object store without prefix enabled.
+ */
+public class TestObjectStore {
+ private static MiniOzoneCluster cluster = null;
+ private static OzoneConfiguration conf;
+ private static String clusterId;
+ private static String scmId;
+ private static String omId;
+
+ @Rule
+ public Timeout timeout = new Timeout(1200000);
+
+ /**
+ * Create a MiniOzoneCluster for testing.
+ * <p>
+ *
+ * @throws IOException
+ */
+ @BeforeClass
+ public static void init() throws Exception {
+ conf = new OzoneConfiguration();
+ clusterId = UUID.randomUUID().toString();
+ scmId = UUID.randomUUID().toString();
+ omId = UUID.randomUUID().toString();
+ cluster = MiniOzoneCluster.newBuilder(conf).setClusterId(clusterId)
+ .setScmId(scmId).setOmId(omId).build();
+ cluster.waitForClusterToBeReady();
+ }
+
+ /**
+ * Shutdown MiniOzoneCluster.
+ */
+ @AfterClass
+ public static void shutdown() {
+ if (cluster != null) {
+ cluster.shutdown();
+ }
+ }
+
+ @Test
+ public void testCreateBucketWithBucketLayout() throws Exception {
+ String sampleVolumeName = UUID.randomUUID().toString();
+ String sampleBucketName = UUID.randomUUID().toString();
+ OzoneClient client = cluster.getClient();
+ ObjectStore store = client.getObjectStore();
+ store.createVolume(sampleVolumeName);
+ OzoneVolume volume = store.getVolume(sampleVolumeName);
+
+ // Case 1: Bucket layout: Empty and OM default bucket layout: OBJECT_STORE
+ BucketArgs.Builder builder = BucketArgs.newBuilder();
+ volume.createBucket(sampleBucketName, builder.build());
+ OzoneBucket bucket = volume.getBucket(sampleBucketName);
+ Assert.assertEquals(sampleBucketName, bucket.getName());
+ Assert.assertEquals(BucketLayout.OBJECT_STORE,
+ bucket.getBucketLayout());
+
+ // Case 2: Bucket layout: DEFAULT
+ sampleBucketName = UUID.randomUUID().toString();
+ builder.setBucketLayout(BucketLayout.DEFAULT);
+ volume.createBucket(sampleBucketName, builder.build());
+ bucket = volume.getBucket(sampleBucketName);
+ Assert.assertEquals(sampleBucketName, bucket.getName());
+ Assert.assertEquals(BucketLayout.OBJECT_STORE,
+ bucket.getBucketLayout());
+
+ // Case 3: Bucket layout: LEGACY
+ sampleBucketName = UUID.randomUUID().toString();
+ builder.setBucketLayout(BucketLayout.LEGACY);
+ volume.createBucket(sampleBucketName, builder.build());
Review comment:
I am fine with either.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r723052037
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketCreateRequest.java
##########
@@ -145,12 +143,9 @@ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager,
OmBucketInfo omBucketInfo = null;
if (bucketInfo.getBucketLayout() == null || bucketInfo.getBucketLayout()
.equals(BucketLayoutProto.LEGACY)) {
Review comment:
Sure, I'll see if there's any failing tests and make required changes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-937893678
/ready
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r723035249
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketCreateRequest.java
##########
@@ -145,12 +143,9 @@ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager,
OmBucketInfo omBucketInfo = null;
if (bucketInfo.getBucketLayout() == null || bucketInfo.getBucketLayout()
.equals(BucketLayoutProto.LEGACY)) {
Review comment:
@JyotinderSingh Make sense to me. Can you check for any existing test cases with invalid values or unsupported value and assert validation exception. If not then we need to add new test cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-935492211
/pending
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724752691
##########
File path: hadoop-hdds/common/src/main/resources/ozone-default.xml
##########
@@ -2108,7 +2108,7 @@
not be renewed.
</description>
</property>
-
+
Review comment:
Interesting. Just check might be some spaces introduced in some commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724733340
##########
File path: hadoop-hdds/common/src/main/resources/ozone-default.xml
##########
@@ -2108,7 +2108,7 @@
not be renewed.
</description>
</property>
-
+
Review comment:
I'm not sure why this change is showing up here. This line shows up as unchanged in my IDE with the annotation showing Elek's name who originally wrote it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-938654415
> And a question? In further Jiras we will be changing the buckets created from (OFS) be FILE_SYSTEM_OPTIMIZED Buckets?
yes, @bharatviswa504. Thanks for bringing that point, I've raised HDDS-5672 to track this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r723461754
##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMConfigKeys.java
##########
@@ -270,6 +270,15 @@ private OMConfigKeys() {
public static final String OZONE_OM_METADATA_LAYOUT_PREFIX = "PREFIX";
+ // Default bucket layout used by Ozone Manager during bucket creation
+ // when a client does not specify the bucket layout option.
+ public static final String OZONE_DEFAULT_BUCKET_LAYOUT =
+ "ozone.default.bucket.layout";
+ public static final String OZONE_DEFAULT_BUCKET_LAYOUT_DEFAULT =
+ "OBJECT_STORE";
Review comment:
@JyotinderSingh Please reuse `BucketLayout.OBJECT_STORE.name()`, instead of hardcoding` "OBJECT_STORE"`.
Also, the same applicable to `"FILE_SYSTEM_OPTIMIZED"` hard code. Please use `BucketLayout.FILE_SYSTEM_OPTIMIZED.name()`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r723508130
##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMConfigKeys.java
##########
@@ -270,6 +270,15 @@ private OMConfigKeys() {
public static final String OZONE_OM_METADATA_LAYOUT_PREFIX = "PREFIX";
+ // Default bucket layout used by Ozone Manager during bucket creation
+ // when a client does not specify the bucket layout option.
+ public static final String OZONE_DEFAULT_BUCKET_LAYOUT =
+ "ozone.default.bucket.layout";
+ public static final String OZONE_DEFAULT_BUCKET_LAYOUT_DEFAULT =
+ "OBJECT_STORE";
Review comment:
Yes, that makes sense. I'll add this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r725247716
##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestObjectStore.java
##########
@@ -0,0 +1,103 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.hadoop.ozone.om;
+
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ozone.MiniOzoneCluster;
+import org.apache.hadoop.ozone.client.*;
+import org.apache.hadoop.ozone.om.helpers.BucketLayout;
+import org.junit.*;
+import org.junit.rules.Timeout;
+
+import java.io.IOException;
+import java.util.UUID;
+
+/**
+ * Tests to verify Object store without prefix enabled.
+ */
+public class TestObjectStore {
+ private static MiniOzoneCluster cluster = null;
+ private static OzoneConfiguration conf;
+ private static String clusterId;
+ private static String scmId;
+ private static String omId;
+
+ @Rule
+ public Timeout timeout = new Timeout(1200000);
+
+ /**
+ * Create a MiniOzoneCluster for testing.
+ * <p>
+ *
+ * @throws IOException
+ */
+ @BeforeClass
+ public static void init() throws Exception {
+ conf = new OzoneConfiguration();
+ clusterId = UUID.randomUUID().toString();
+ scmId = UUID.randomUUID().toString();
+ omId = UUID.randomUUID().toString();
+ cluster = MiniOzoneCluster.newBuilder(conf).setClusterId(clusterId)
+ .setScmId(scmId).setOmId(omId).build();
+ cluster.waitForClusterToBeReady();
+ }
+
+ /**
+ * Shutdown MiniOzoneCluster.
+ */
+ @AfterClass
+ public static void shutdown() {
+ if (cluster != null) {
+ cluster.shutdown();
+ }
+ }
+
+ @Test
+ public void testCreateBucketWithBucketLayout() throws Exception {
+ String sampleVolumeName = UUID.randomUUID().toString();
+ String sampleBucketName = UUID.randomUUID().toString();
+ OzoneClient client = cluster.getClient();
+ ObjectStore store = client.getObjectStore();
+ store.createVolume(sampleVolumeName);
+ OzoneVolume volume = store.getVolume(sampleVolumeName);
+
+ // Case 1: Bucket layout: Empty and OM default bucket layout: OBJECT_STORE
+ BucketArgs.Builder builder = BucketArgs.newBuilder();
+ volume.createBucket(sampleBucketName, builder.build());
+ OzoneBucket bucket = volume.getBucket(sampleBucketName);
+ Assert.assertEquals(sampleBucketName, bucket.getName());
+ Assert.assertEquals(BucketLayout.OBJECT_STORE,
+ bucket.getBucketLayout());
+
+ // Case 2: Bucket layout: DEFAULT
+ sampleBucketName = UUID.randomUUID().toString();
+ builder.setBucketLayout(BucketLayout.DEFAULT);
+ volume.createBucket(sampleBucketName, builder.build());
+ bucket = volume.getBucket(sampleBucketName);
+ Assert.assertEquals(sampleBucketName, bucket.getName());
+ Assert.assertEquals(BucketLayout.OBJECT_STORE,
+ bucket.getBucketLayout());
+
+ // Case 3: Bucket layout: LEGACY
+ sampleBucketName = UUID.randomUUID().toString();
+ builder.setBucketLayout(BucketLayout.LEGACY);
+ volume.createBucket(sampleBucketName, builder.build());
Review comment:
Sure, I'll raise a new Jira for this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r721919109
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketCreateRequest.java
##########
@@ -145,12 +145,7 @@ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager,
OmBucketInfo omBucketInfo = null;
if (bucketInfo.getBucketLayout() == null || bucketInfo.getBucketLayout()
.equals(BucketLayoutProto.LEGACY)) {
- BucketLayout defaultType = BucketLayout.LEGACY;
- if (StringUtils
- .equalsIgnoreCase(OMConfigKeys.OZONE_OM_METADATA_LAYOUT_PREFIX,
- omLayout)) {
- defaultType = BucketLayout.FILE_SYSTEM_OPTIMIZED;
- }
+ BucketLayout defaultType = OMConfigKeys.OZONE_BUCKET_LAYOUT_DEFAULT;
Review comment:
Please read the value from OMConfiguration instead of reading a static value.
Below is sample code reference.
[OMFileCreateRequest.java#L90](https://github.com/apache/ozone/blob/master/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileCreateRequest.java#L90)
In your patch, you can do `ozoneMgr.getConfiguration().getTrimmed(OMConfigKeys.OZONE_DEFAULT_BUCKET_LAYOUT OMConfigKeys.OZONE_BUCKET_LAYOUT_DEFAULT);`
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketCreateRequest.java
##########
@@ -145,12 +145,7 @@ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager,
OmBucketInfo omBucketInfo = null;
if (bucketInfo.getBucketLayout() == null || bucketInfo.getBucketLayout()
.equals(BucketLayoutProto.LEGACY)) {
- BucketLayout defaultType = BucketLayout.LEGACY;
- if (StringUtils
- .equalsIgnoreCase(OMConfigKeys.OZONE_OM_METADATA_LAYOUT_PREFIX,
- omLayout)) {
- defaultType = BucketLayout.FILE_SYSTEM_OPTIMIZED;
- }
+ BucketLayout defaultType = OMConfigKeys.OZONE_BUCKET_LAYOUT_DEFAULT;
Review comment:
@JyotinderSingh Please include test case to verify the behavior.
##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMConfigKeys.java
##########
@@ -270,6 +271,11 @@ private OMConfigKeys() {
public static final String OZONE_OM_METADATA_LAYOUT_PREFIX = "PREFIX";
+ public static final String OZONE_BUCKET_LAYOUT =
+ "ozone.bucket.layout";
Review comment:
Can you please rename the config name to`ozone.default.bucket.type` or a better name conveying that the configurations meant for setting default value. Also, please add this config to the following files:
```
1) hadoop-hdds/common/src/main/resources/ozone-default.xml
2) hadoop-hdds/docs/content/feature/PrefixFSO.md
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] github-actions[bot] commented on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-939226838
To re-run CI checks, please follow these steps with the source branch checked out:
```
git commit --allow-empty -m 'trigger new CI check'
git push
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r725200404
##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestObjectStore.java
##########
@@ -0,0 +1,103 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.hadoop.ozone.om;
+
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ozone.MiniOzoneCluster;
+import org.apache.hadoop.ozone.client.*;
+import org.apache.hadoop.ozone.om.helpers.BucketLayout;
+import org.junit.*;
+import org.junit.rules.Timeout;
+
+import java.io.IOException;
+import java.util.UUID;
+
+/**
+ * Tests to verify Object store without prefix enabled.
+ */
+public class TestObjectStore {
+ private static MiniOzoneCluster cluster = null;
+ private static OzoneConfiguration conf;
+ private static String clusterId;
+ private static String scmId;
+ private static String omId;
+
+ @Rule
+ public Timeout timeout = new Timeout(1200000);
+
+ /**
+ * Create a MiniOzoneCluster for testing.
+ * <p>
+ *
+ * @throws IOException
+ */
+ @BeforeClass
+ public static void init() throws Exception {
+ conf = new OzoneConfiguration();
+ clusterId = UUID.randomUUID().toString();
+ scmId = UUID.randomUUID().toString();
+ omId = UUID.randomUUID().toString();
+ cluster = MiniOzoneCluster.newBuilder(conf).setClusterId(clusterId)
+ .setScmId(scmId).setOmId(omId).build();
+ cluster.waitForClusterToBeReady();
+ }
+
+ /**
+ * Shutdown MiniOzoneCluster.
+ */
+ @AfterClass
+ public static void shutdown() {
+ if (cluster != null) {
+ cluster.shutdown();
+ }
+ }
+
+ @Test
+ public void testCreateBucketWithBucketLayout() throws Exception {
+ String sampleVolumeName = UUID.randomUUID().toString();
+ String sampleBucketName = UUID.randomUUID().toString();
+ OzoneClient client = cluster.getClient();
+ ObjectStore store = client.getObjectStore();
+ store.createVolume(sampleVolumeName);
+ OzoneVolume volume = store.getVolume(sampleVolumeName);
+
+ // Case 1: Bucket layout: Empty and OM default bucket layout: OBJECT_STORE
+ BucketArgs.Builder builder = BucketArgs.newBuilder();
+ volume.createBucket(sampleBucketName, builder.build());
+ OzoneBucket bucket = volume.getBucket(sampleBucketName);
+ Assert.assertEquals(sampleBucketName, bucket.getName());
+ Assert.assertEquals(BucketLayout.OBJECT_STORE,
+ bucket.getBucketLayout());
+
+ // Case 2: Bucket layout: DEFAULT
+ sampleBucketName = UUID.randomUUID().toString();
+ builder.setBucketLayout(BucketLayout.DEFAULT);
+ volume.createBucket(sampleBucketName, builder.build());
+ bucket = volume.getBucket(sampleBucketName);
+ Assert.assertEquals(sampleBucketName, bucket.getName());
+ Assert.assertEquals(BucketLayout.OBJECT_STORE,
+ bucket.getBucketLayout());
+
+ // Case 3: Bucket layout: LEGACY
+ sampleBucketName = UUID.randomUUID().toString();
+ builder.setBucketLayout(BucketLayout.LEGACY);
+ volume.createBucket(sampleBucketName, builder.build());
Review comment:
Question: From Shell we donot allow LEGACY, and do we need similar handling in Java API's also? (To be consistent from all interfaces)
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##########
@@ -453,6 +457,18 @@ private OzoneManager(OzoneConfiguration conf, StartupOption startupOption)
OMConfigKeys.OZONE_OM_RATIS_ENABLE_KEY,
OMConfigKeys.OZONE_OM_RATIS_ENABLE_DEFAULT);
+ this.defaultBucketLayout =
+ configuration.getTrimmed(OZONE_DEFAULT_BUCKET_LAYOUT,
+ OZONE_DEFAULT_BUCKET_LAYOUT_DEFAULT);
+ // Make sure defaultBucketLayout is set to a valid value
+ if (!defaultBucketLayout.equals(
+ BucketLayout.FILE_SYSTEM_OPTIMIZED.name()) &&
+ !defaultBucketLayout.equals(BucketLayout.OBJECT_STORE.name())
+ ) {
+ throw new IllegalArgumentException(
Review comment:
Can we use Configuration Exception, and in the message can we add supported values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724690390
##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/BucketLayout.java
##########
@@ -59,4 +60,10 @@ public static BucketLayout fromProto(
"Error: BucketLayout not found, type=" + this);
}
}
+
+ public static BucketLayout fromString(String value) {
+ // Todo: should we throw error if user configured unsupported value
+ // during OM startup or bucket creation time.
+ return StringUtils.isBlank(value) ? LEGACY : BucketLayout.valueOf(value);
Review comment:
We mainly needed the check here for sake of testing, because sometimes tests do not initialize the OM via the constructor and leave some fields uninitialized.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724724752
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##########
@@ -452,6 +455,9 @@ private OzoneManager(OzoneConfiguration conf, StartupOption startupOption)
isRatisEnabled = configuration.getBoolean(
OMConfigKeys.OZONE_OM_RATIS_ENABLE_KEY,
OMConfigKeys.OZONE_OM_RATIS_ENABLE_DEFAULT);
+ this.defaultBucketLayout =
+ configuration.getTrimmed(OZONE_DEFAULT_BUCKET_LAYOUT,
+ OZONE_DEFAULT_BUCKET_LAYOUT_DEFAULT);
Review comment:
@bharatviswa504, IMHO to validate and fail startup if it is configured with unsupported value. Hope it makes sense to you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724752164
##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/BucketLayout.java
##########
@@ -59,4 +60,10 @@ public static BucketLayout fromProto(
"Error: BucketLayout not found, type=" + this);
}
}
+
+ public static BucketLayout fromString(String value) {
+ // Todo: should we throw error if user configured unsupported value
+ // during OM startup or bucket creation time.
+ return StringUtils.isBlank(value) ? LEGACY : BucketLayout.valueOf(value);
Review comment:
If we are addressing this, we donot need Todo, refer above comments which we are discussing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724696182
##########
File path: hadoop-hdds/common/src/main/resources/ozone-default.xml
##########
@@ -2988,4 +2988,18 @@
OM/SCM/DN/S3GATEWAY Server connection timeout in milliseconds.
</description>
</property>
+
+ <property>
+ <name>ozone.default.bucket.layout</name>
+ <value>OBJECT_STORE</value>
+ <tag>OZONE, MANAGEMENT</tag>
+ <description>
+ Default bucket layout used by Ozone Manager during bucket creation when a client does not specify the
+ bucket layout option. Supported values are OBJECT_STORE and FILE_SYSTEM_OPTIMIZED. OBJECT_STORE
+ layout allows the bucket to behave as a pure object store and will not allow interoperability
Review comment:
Sure, I'll format it properly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724722487
##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/BucketLayout.java
##########
@@ -59,4 +60,10 @@ public static BucketLayout fromProto(
"Error: BucketLayout not found, type=" + this);
}
}
+
+ public static BucketLayout fromString(String value) {
+ // Todo: should we throw error if user configured unsupported value
+ // during OM startup or bucket creation time.
+ return StringUtils.isBlank(value) ? LEGACY : BucketLayout.valueOf(value);
Review comment:
@JyotinderSingh ok, you meant it can be null in unit tests where it uses mocking. IMHO, please do :
1) Just add a java comment conveying that, this `isBlank` check is added for testing purpose and value won't be null in production.
```
// This will never be null in production but can be null in mocked unit test cases.
// Added safer `isBlank` check for unit test cases.
```
2) Move the existing todo comment to startup.
```
// Todo: need to validate then throw error or set to OM default layout if user configured unsupported value
// during OM startup.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724751979
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##########
@@ -452,6 +455,9 @@ private OzoneManager(OzoneConfiguration conf, StartupOption startupOption)
isRatisEnabled = configuration.getBoolean(
OMConfigKeys.OZONE_OM_RATIS_ENABLE_KEY,
OMConfigKeys.OZONE_OM_RATIS_ENABLE_DEFAULT);
+ this.defaultBucketLayout =
+ configuration.getTrimmed(OZONE_DEFAULT_BUCKET_LAYOUT,
+ OZONE_DEFAULT_BUCKET_LAYOUT_DEFAULT);
Review comment:
Yes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-938984419
> > And a question? In further Jiras we will be changing the buckets created from (OFS) be FILE_SYSTEM_OPTIMIZED Buckets?
>
> yes, @bharatviswa504. Thanks for bringing that point, I've raised [HDDS-5839](https://issues.apache.org/jira/browse/HDDS-5839) to track this.
One more point, do we also throw error not supported when we figure out bucket type is of kind OBJECT_STORE?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724752691
##########
File path: hadoop-hdds/common/src/main/resources/ozone-default.xml
##########
@@ -2108,7 +2108,7 @@
not be renewed.
</description>
</property>
-
+
Review comment:
Interesting. Just check might be some spaces introduced in some commit. (If not, i am not sure some issue with github)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-937893678
/ready
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724752164
##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/BucketLayout.java
##########
@@ -59,4 +60,10 @@ public static BucketLayout fromProto(
"Error: BucketLayout not found, type=" + this);
}
}
+
+ public static BucketLayout fromString(String value) {
+ // Todo: should we throw error if user configured unsupported value
+ // during OM startup or bucket creation time.
+ return StringUtils.isBlank(value) ? LEGACY : BucketLayout.valueOf(value);
Review comment:
If we are addressing this, we donot need Todo.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724773019
##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/BucketLayout.java
##########
@@ -59,4 +60,10 @@ public static BucketLayout fromProto(
"Error: BucketLayout not found, type=" + this);
}
}
+
+ public static BucketLayout fromString(String value) {
+ // Todo: should we throw error if user configured unsupported value
+ // during OM startup or bucket creation time.
+ return StringUtils.isBlank(value) ? LEGACY : BucketLayout.valueOf(value);
Review comment:
yes, agreed @bharatviswa504
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r725230972
##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestObjectStore.java
##########
@@ -0,0 +1,103 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.hadoop.ozone.om;
+
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ozone.MiniOzoneCluster;
+import org.apache.hadoop.ozone.client.*;
+import org.apache.hadoop.ozone.om.helpers.BucketLayout;
+import org.junit.*;
+import org.junit.rules.Timeout;
+
+import java.io.IOException;
+import java.util.UUID;
+
+/**
+ * Tests to verify Object store without prefix enabled.
+ */
+public class TestObjectStore {
+ private static MiniOzoneCluster cluster = null;
+ private static OzoneConfiguration conf;
+ private static String clusterId;
+ private static String scmId;
+ private static String omId;
+
+ @Rule
+ public Timeout timeout = new Timeout(1200000);
+
+ /**
+ * Create a MiniOzoneCluster for testing.
+ * <p>
+ *
+ * @throws IOException
+ */
+ @BeforeClass
+ public static void init() throws Exception {
+ conf = new OzoneConfiguration();
+ clusterId = UUID.randomUUID().toString();
+ scmId = UUID.randomUUID().toString();
+ omId = UUID.randomUUID().toString();
+ cluster = MiniOzoneCluster.newBuilder(conf).setClusterId(clusterId)
+ .setScmId(scmId).setOmId(omId).build();
+ cluster.waitForClusterToBeReady();
+ }
+
+ /**
+ * Shutdown MiniOzoneCluster.
+ */
+ @AfterClass
+ public static void shutdown() {
+ if (cluster != null) {
+ cluster.shutdown();
+ }
+ }
+
+ @Test
+ public void testCreateBucketWithBucketLayout() throws Exception {
+ String sampleVolumeName = UUID.randomUUID().toString();
+ String sampleBucketName = UUID.randomUUID().toString();
+ OzoneClient client = cluster.getClient();
+ ObjectStore store = client.getObjectStore();
+ store.createVolume(sampleVolumeName);
+ OzoneVolume volume = store.getVolume(sampleVolumeName);
+
+ // Case 1: Bucket layout: Empty and OM default bucket layout: OBJECT_STORE
+ BucketArgs.Builder builder = BucketArgs.newBuilder();
+ volume.createBucket(sampleBucketName, builder.build());
+ OzoneBucket bucket = volume.getBucket(sampleBucketName);
+ Assert.assertEquals(sampleBucketName, bucket.getName());
+ Assert.assertEquals(BucketLayout.OBJECT_STORE,
+ bucket.getBucketLayout());
+
+ // Case 2: Bucket layout: DEFAULT
+ sampleBucketName = UUID.randomUUID().toString();
+ builder.setBucketLayout(BucketLayout.DEFAULT);
+ volume.createBucket(sampleBucketName, builder.build());
+ bucket = volume.getBucket(sampleBucketName);
+ Assert.assertEquals(sampleBucketName, bucket.getName());
+ Assert.assertEquals(BucketLayout.OBJECT_STORE,
+ bucket.getBucketLayout());
+
+ // Case 3: Bucket layout: LEGACY
+ sampleBucketName = UUID.randomUUID().toString();
+ builder.setBucketLayout(BucketLayout.LEGACY);
+ volume.createBucket(sampleBucketName, builder.build());
Review comment:
Should I remove the Java API support for LEGACY as a part of this patch itself or track it via a separate PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r722897250
##########
File path: hadoop-hdds/common/src/main/resources/ozone-default.xml
##########
@@ -2988,4 +2988,14 @@
OM/SCM/DN/S3GATEWAY Server connection timeout in milliseconds.
</description>
</property>
+
+ <property>
+ <name>ozone.default.bucket.layout</name>
+ <value>OBJECT_STORE</value>
+ <tag>OZONE, MANAGEMENT</tag>
+ <description>
+ Default bucket layout used by Ozone Manager during bucket creation when a client does not specify the
Review comment:
Supported values are `OBJECT_STORE` and `FILE_SYSTEM_OPTIMIZED`.
@JyotinderSingh , please include these into the description section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r722747544
##########
File path: hadoop-hdds/docs/content/feature/PrefixFSO.md
##########
@@ -72,4 +72,17 @@ By default the feature is disabled. It can be enabled with the following
<name>ozone.om.metadata.layout</name>
<value>PREFIX</value>
</property>
+```
+
+In reference to efforts towards supporting protocol aware buckets within a
+single OM, the following configurations can be used to define the default value
Review comment:
remove single here. it is causing confusion here.
We can instead say in a ozone cluster
##########
File path: hadoop-hdds/common/src/main/resources/ozone-default.xml
##########
@@ -2988,4 +2988,14 @@
OM/SCM/DN/S3GATEWAY Server connection timeout in milliseconds.
</description>
</property>
+
+ <property>
+ <name>ozone.default.bucket.layout</name>
+ <value>OBJECT_STORE</value>
+ <tag>OZONE, MANAGEMENT</tag>
+ <description>
+ Default bucket layout used by Ozone Manager during bucket creation when a client does not specify the
Review comment:
Can you also specify what are the supported values here, and little bit explanation about each value?
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##########
@@ -4072,6 +4074,11 @@ public String getOMMetadataLayout() {
.getTrimmed(OZONE_OM_METADATA_LAYOUT, OZONE_OM_METADATA_LAYOUT_DEFAULT);
}
+ public String getOMDefaultBucketLayout() {
Review comment:
Instead of reading every time from config during bucket creation, we can read once validate it.
This will make easy during bucket creation, the string value we have is a appropriate value.
##########
File path: hadoop-hdds/docs/content/feature/PrefixFSO.md
##########
@@ -72,4 +72,17 @@ By default the feature is disabled. It can be enabled with the following
<name>ozone.om.metadata.layout</name>
<value>PREFIX</value>
</property>
+```
+
+In reference to efforts towards supporting protocol aware buckets within a
+single OM, the following configurations can be used to define the default value
Review comment:
if the client is not
passing the bucket layout argument -> if the client has not specified the bucket layout argument
##########
File path: hadoop-hdds/docs/content/feature/PrefixFSO.md
##########
@@ -72,4 +72,17 @@ By default the feature is disabled. It can be enabled with the following
<name>ozone.om.metadata.layout</name>
<value>PREFIX</value>
</property>
+```
+
+In reference to efforts towards supporting protocol aware buckets within a
+single OM, the following configurations can be used to define the default value
Review comment:
configurations -> configuration
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketCreateRequest.java
##########
@@ -145,12 +143,9 @@ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager,
OmBucketInfo omBucketInfo = null;
if (bucketInfo.getBucketLayout() == null || bucketInfo.getBucketLayout()
.equals(BucketLayoutProto.LEGACY)) {
Review comment:
Not understood why when it is legacy also we use default type here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r722924739
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketCreateRequest.java
##########
@@ -145,12 +143,9 @@ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager,
OmBucketInfo omBucketInfo = null;
if (bucketInfo.getBucketLayout() == null || bucketInfo.getBucketLayout()
.equals(BucketLayoutProto.LEGACY)) {
Review comment:
Since we use enum in the proto, it will assign the default value automatically when the bucket layout is not specified or passed. It will use the first indexed enum value as default, which is LEGACY [Proto Reference](https://developers.google.com/protocol-buffers/docs/proto3#default) We have added the null check for the safer side, which is actually not required.
Supported values are only OBJECT_STORE and FILE_SYSTEM_OPTIMIZED. Client/User won't pass LEGACY so the LEGACY can occur when the argument is missing.
I think, [allowed values](https://github.com/apache/ozone/blob/master/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/CreateBucketHandler.java#L56) has to be revisited. @JyotinderSingh can you please check it quickly by removing LEGACY from `AllowedBucketLayouts`. I think to raise a separate sub-task to address client side validation for the SUPPORTED_VALUES, if it requires more code changes.
@bharatviswa504 Does this make sense to you?
```
enum BucketLayoutProto {
LEGACY = 1;
FILE_SYSTEM_OPTIMIZED = 2;
OBJECT_STORE = 3;
}
```
[OmClientProtocol.proto#L556 BucketLayoutProto](https://github.com/apache/ozone/blob/master/hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto#L556)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#issuecomment-935806252
Thanks for the reviews @bharatviswa504 . I have addressed your review suggestions in my patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r723506127
##########
File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/CreateBucketHandler.java
##########
@@ -53,11 +53,11 @@
// TODO: LEGACY should be removed and should not be exposed to the end user
// we will revisit during the client side defaulting behaviour
- enum AllowedBucketLayouts { FILE_SYSTEM_OPTIMIZED, OBJECT_STORE, LEGACY }
+ enum AllowedBucketLayouts { FILE_SYSTEM_OPTIMIZED, OBJECT_STORE }
Review comment:
Please add a test case to verify the client side allowed layout validation. One idea is to add test case in TestOzoneShellHA.java.
Sample test case snippet:
String [] args1 = new String[]{
"bucket", "create", "o3://" + omServiceId +
"/volume4" + "/buckettest1", "--type",
"LEGACY"};
try {
execute(ozoneShell, args1);
Assert.fail("Should throw exception on unsupported bucket layouts!");
} catch (Exception e) {
GenericTestUtils.assertExceptionContains("expected one of [FILE_SYSTEM_OPTIMIZED, OBJECT_STORE] ", e);
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r723050125
##########
File path: hadoop-hdds/docs/content/feature/PrefixFSO.md
##########
@@ -72,4 +72,17 @@ By default the feature is disabled. It can be enabled with the following
<name>ozone.om.metadata.layout</name>
<value>PREFIX</value>
</property>
+```
+
+In reference to efforts towards supporting protocol aware buckets within a
+single OM, the following configurations can be used to define the default value
Review comment:
I have made these changes in my patch, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JyotinderSingh commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
JyotinderSingh commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r725230972
##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestObjectStore.java
##########
@@ -0,0 +1,103 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.hadoop.ozone.om;
+
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ozone.MiniOzoneCluster;
+import org.apache.hadoop.ozone.client.*;
+import org.apache.hadoop.ozone.om.helpers.BucketLayout;
+import org.junit.*;
+import org.junit.rules.Timeout;
+
+import java.io.IOException;
+import java.util.UUID;
+
+/**
+ * Tests to verify Object store without prefix enabled.
+ */
+public class TestObjectStore {
+ private static MiniOzoneCluster cluster = null;
+ private static OzoneConfiguration conf;
+ private static String clusterId;
+ private static String scmId;
+ private static String omId;
+
+ @Rule
+ public Timeout timeout = new Timeout(1200000);
+
+ /**
+ * Create a MiniOzoneCluster for testing.
+ * <p>
+ *
+ * @throws IOException
+ */
+ @BeforeClass
+ public static void init() throws Exception {
+ conf = new OzoneConfiguration();
+ clusterId = UUID.randomUUID().toString();
+ scmId = UUID.randomUUID().toString();
+ omId = UUID.randomUUID().toString();
+ cluster = MiniOzoneCluster.newBuilder(conf).setClusterId(clusterId)
+ .setScmId(scmId).setOmId(omId).build();
+ cluster.waitForClusterToBeReady();
+ }
+
+ /**
+ * Shutdown MiniOzoneCluster.
+ */
+ @AfterClass
+ public static void shutdown() {
+ if (cluster != null) {
+ cluster.shutdown();
+ }
+ }
+
+ @Test
+ public void testCreateBucketWithBucketLayout() throws Exception {
+ String sampleVolumeName = UUID.randomUUID().toString();
+ String sampleBucketName = UUID.randomUUID().toString();
+ OzoneClient client = cluster.getClient();
+ ObjectStore store = client.getObjectStore();
+ store.createVolume(sampleVolumeName);
+ OzoneVolume volume = store.getVolume(sampleVolumeName);
+
+ // Case 1: Bucket layout: Empty and OM default bucket layout: OBJECT_STORE
+ BucketArgs.Builder builder = BucketArgs.newBuilder();
+ volume.createBucket(sampleBucketName, builder.build());
+ OzoneBucket bucket = volume.getBucket(sampleBucketName);
+ Assert.assertEquals(sampleBucketName, bucket.getName());
+ Assert.assertEquals(BucketLayout.OBJECT_STORE,
+ bucket.getBucketLayout());
+
+ // Case 2: Bucket layout: DEFAULT
+ sampleBucketName = UUID.randomUUID().toString();
+ builder.setBucketLayout(BucketLayout.DEFAULT);
+ volume.createBucket(sampleBucketName, builder.build());
+ bucket = volume.getBucket(sampleBucketName);
+ Assert.assertEquals(sampleBucketName, bucket.getName());
+ Assert.assertEquals(BucketLayout.OBJECT_STORE,
+ bucket.getBucketLayout());
+
+ // Case 3: Bucket layout: LEGACY
+ sampleBucketName = UUID.randomUUID().toString();
+ builder.setBucketLayout(BucketLayout.LEGACY);
+ volume.createBucket(sampleBucketName, builder.build());
Review comment:
Should I remove the Java API support for LEGACY as a part of this patch itself or track it via a separate Jira/PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r722924739
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketCreateRequest.java
##########
@@ -145,12 +143,9 @@ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager,
OmBucketInfo omBucketInfo = null;
if (bucketInfo.getBucketLayout() == null || bucketInfo.getBucketLayout()
.equals(BucketLayoutProto.LEGACY)) {
Review comment:
Since we use enum in the proto, it will assign the default value automatically when the bucket layout is not specified or passed. It will use the first indexed enum value as default, which is LEGACY. We have added the null check for the safer side, which is actually not required.
Supported values are only OBJECT_STORE and FILE_SYSTEM_OPTIMIZED. Client/User won't pass LEGACY so the LEGACY can occur when the argument is missing.
I think, [allowed values](https://github.com/apache/ozone/blob/master/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/CreateBucketHandler.java#L56) has to be revisited. @JyotinderSingh can you please check it quickly by removing LEGACY from `AllowedBucketLayouts`. I think to raise a separate sub-task to address client side validation for the SUPPORTED_VALUES, if it requires more code changes.
@bharatviswa504 Does this make sense to you?
```
enum BucketLayoutProto {
LEGACY = 1;
FILE_SYSTEM_OPTIMIZED = 2;
OBJECT_STORE = 3;
}
```
[OmClientProtocol.proto#L556 BucketLayoutProto](https://github.com/apache/ozone/blob/master/hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto#L556)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r722179962
##########
File path: hadoop-hdds/common/src/main/resources/ozone-default.xml
##########
@@ -2988,4 +2988,13 @@
OM/SCM/DN/S3GATEWAY Server connection timeout in milliseconds.
</description>
</property>
+
+ <property>
+ <name>ozone.default.bucket.layout</name>
+ <value>OBS</value>
Review comment:
Please use full name `OBJECT_STORE` instead of OBS.
[BucketLayout.java#L30](https://github.com/apache/ozone/blob/master/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/BucketLayout.java#L30)
##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMConfigKeys.java
##########
@@ -270,6 +270,13 @@ private OMConfigKeys() {
public static final String OZONE_OM_METADATA_LAYOUT_PREFIX = "PREFIX";
+ // Defines the default bucket behaviour when OM does not pass the
Review comment:
please modify comment to:
`Default bucket layout used by Ozone Manager during bucket creation when a client does not specify the bucket layout option.`
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketCreateRequest.java
##########
@@ -145,10 +145,14 @@ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager,
OmBucketInfo omBucketInfo = null;
if (bucketInfo.getBucketLayout() == null || bucketInfo.getBucketLayout()
.equals(BucketLayoutProto.LEGACY)) {
- BucketLayout defaultType = BucketLayout.LEGACY;
- if (StringUtils
- .equalsIgnoreCase(OMConfigKeys.OZONE_OM_METADATA_LAYOUT_PREFIX,
- omLayout)) {
+ // Bucket Layout argument was not passed during bucket creation.
Review comment:
We can simply read the default value and assign to the bucket. Please remove the below if/else, which has some confusion.
```
// Bucket Layout argument was not passed during bucket creation.
BucketLayout defaultType;
String omDefaultBucketLayout = ozoneManager.getOMDefaultBucketLayout();
if (StringUtils.equalsIgnoreCase(
OMConfigKeys.OZONE_DEFAULT_BUCKET_LAYOUT_DEFAULT,
omDefaultBucketLayout)) {
defaultType = BucketLayout.OBJECT_STORE;
} else {
defaultType = BucketLayout.FILE_SYSTEM_OPTIMIZED;
```
Here the code can be as simple like,
```
if (bucketInfo.getBucketLayout() == null || bucketInfo.getBucketLayout()
.equals(BucketLayoutProto.LEGACY)) {
// Bucket Layout argument is not passed during bucket creation, will assign OM default layout.
String omDefaultBucketLayout = ozoneManager.getOMDefaultBucketLayout();
BucketLayout defaultType = BucketLayout.fromString(String value);
omBucketInfo = OmBucketInfo.getFromProtobuf(bucketInfo, defaultType);
}
```
Note: You have to implement `BucketLayout.fromString(String value);`, where you can do a string comparison and return matching enum type.
if (StringUtils.equalsIgnoreCase(
defaultBucketLayout, BucketLayout.OBJECT_STORE.name())) {
defaultType = BucketLayout.OBJECT_STORE;
} else if (StringUtils.equalsIgnoreCase(
defaultBucketLayout, BucketLayout.FILE_SYSTEM_OPTIMIZED.name())) {
defaultType = BucketLayout.FILE_SYSTEM_OPTIMIZED;
} else {
// Todo: need to discuss about throwing a validation error during OM startup or bucket creation time.
defaultType = BucketLayout.LEGACY;
}
##########
File path: hadoop-hdds/common/src/main/resources/ozone-default.xml
##########
@@ -2988,4 +2988,13 @@
OM/SCM/DN/S3GATEWAY Server connection timeout in milliseconds.
</description>
</property>
+
+ <property>
+ <name>ozone.default.bucket.layout</name>
+ <value>OBS</value>
+ <tag>OZONE, MANAGEMENT</tag>
+ <description>
+ Default bucket layout used during bucket creation when OM does not pass the bucket type argument.
Review comment:
small typo:
Please modify description to ->
`Default bucket layout used by Ozone Manager during bucket creation when a client does not specify the bucket layout option.`
##########
File path: hadoop-hdds/docs/content/feature/PrefixFSO.md
##########
@@ -72,4 +72,17 @@ By default the feature is disabled. It can be enabled with the following
<name>ozone.om.metadata.layout</name>
<value>PREFIX</value>
</property>
+```
+
+In reference to efforts towards supporting protocol aware buckets within a
+single OM the following configurations can be used to define the default value
+for bucket layout during bucket creation if the client (S3 or o3fs) is not
Review comment:
small typo:
I think to remove `(S3 or o3fs)`, let it be generic.
##########
File path: hadoop-hdds/docs/content/feature/PrefixFSO.md
##########
@@ -72,4 +72,17 @@ By default the feature is disabled. It can be enabled with the following
<name>ozone.om.metadata.layout</name>
<value>PREFIX</value>
</property>
+```
+
+In reference to efforts towards supporting protocol aware buckets within a
+single OM the following configurations can be used to define the default value
+for bucket layout during bucket creation if the client (S3 or o3fs) is not
+passing the bucket layout argument.
+
+By default, the buckets will default to OBS behaviour.
+```XML
+<property>
+ <name>ozone.default.bucket.layout</name>
+ <value>OBS</value>
Review comment:
Like my earlier comment, please use full name OBJECT_STORE instead of OBS.
##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMConfigKeys.java
##########
@@ -270,6 +270,13 @@ private OMConfigKeys() {
public static final String OZONE_OM_METADATA_LAYOUT_PREFIX = "PREFIX";
+ // Defines the default bucket behaviour when OM does not pass the
+ // bucket layout argument during bucket creation.
+ public static final String OZONE_DEFAULT_BUCKET_LAYOUT =
+ "ozone.default.bucket.layout";
+ public static final String OZONE_DEFAULT_BUCKET_LAYOUT_DEFAULT = "OBS";
Review comment:
Like my earlier comment, please use full name OBJECT_STORE instead of OBS and FILE_SYSTEM_OPTIMIZED instead of OBS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on a change in pull request #2707: HDDS-5373. [FSO] Define default bucket layout.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #2707:
URL: https://github.com/apache/ozone/pull/2707#discussion_r724624645
##########
File path: hadoop-hdds/common/src/main/resources/ozone-default.xml
##########
@@ -2988,4 +2988,18 @@
OM/SCM/DN/S3GATEWAY Server connection timeout in milliseconds.
</description>
</property>
+
+ <property>
+ <name>ozone.default.bucket.layout</name>
+ <value>OBJECT_STORE</value>
+ <tag>OZONE, MANAGEMENT</tag>
+ <description>
+ Default bucket layout used by Ozone Manager during bucket creation when a client does not specify the
+ bucket layout option. Supported values are OBJECT_STORE and FILE_SYSTEM_OPTIMIZED. OBJECT_STORE
+ layout allows the bucket to behave as a pure object store and will not allow interoperability
Review comment:
Can we format this properly like
OBJECT_STORE: << explanation>>
FILE_SYSTEM_OPTIMIZED: << explanation>>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org