You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/04/14 11:24:04 UTC
[GitHub] [ozone] tanvipenumudy opened a new pull request, #3310: HDDS-6583. Introduce lock strategy pattern implementations based on configuration flags
tanvipenumudy opened a new pull request, #3310:
URL: https://github.com/apache/ozone/pull/3310
## What changes were proposed in this pull request?
This task is intended to introduce lock strategy pattern implementations for the regular `BUCKET_LOCK` and `OBJECT_STORE("OBS")` `KEY_PATH_LOCK`.
The `OBS` `KEY_PATH_LOCK` feature shall be disabled by default using the configuration `ozone.om.enable.key.path.lock` (= false).
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-6583
## How was this patch tested?
NA
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] tanvipenumudy commented on a diff in pull request #3310: HDDS-6583. Introduce lock strategy pattern implementations based on configuration flags
Posted by GitBox <gi...@apache.org>.
tanvipenumudy commented on code in PR #3310:
URL: https://github.com/apache/ozone/pull/3310#discussion_r855019957
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/lock/OzoneLockProvider.java:
##########
@@ -0,0 +1,52 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.lock;
+
+import org.apache.hadoop.ozone.om.helpers.BucketLayout;
+
+/**
+ * OzoneLockProvider class returns the appropriate lock strategy pattern
+ * implementation based on the configuration flags passed.
+ */
+public class OzoneLockProvider {
Review Comment:
Added a UT for OzoneLockProvider, thank you for the comment!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] tanvipenumudy commented on a diff in pull request #3310: HDDS-6583. Introduce lock strategy pattern implementations based on configuration flags
Posted by GitBox <gi...@apache.org>.
tanvipenumudy commented on code in PR #3310:
URL: https://github.com/apache/ozone/pull/3310#discussion_r855018442
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/lock/OzoneLockProvider.java:
##########
@@ -0,0 +1,52 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.lock;
+
+import org.apache.hadoop.ozone.om.helpers.BucketLayout;
+
+/**
+ * OzoneLockProvider class returns the appropriate lock strategy pattern
+ * implementation based on the configuration flags passed.
+ */
+public class OzoneLockProvider {
+
+ private boolean enableKeyPathLock;
+ private boolean enableFileSystemPaths;
+
+ public OzoneLockProvider(boolean enableKeyPathLock,
Review Comment:
Added a UT for `OzoneLockProvider`, thank you for the comment!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] tanvipenumudy commented on a diff in pull request #3310: HDDS-6583. Introduce lock strategy pattern implementations based on configuration flags
Posted by GitBox <gi...@apache.org>.
tanvipenumudy commented on code in PR #3310:
URL: https://github.com/apache/ozone/pull/3310#discussion_r855137100
##########
hadoop-hdds/common/src/main/resources/ozone-default.xml:
##########
@@ -2819,6 +2819,15 @@
</description>
</property>
+ <property>
+ <name>ozone.om.enable.key.path.lock</name>
Review Comment:
I have made the changes, thank you for the comment!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kerneltime commented on a diff in pull request #3310: HDDS-6583. Introduce lock strategy pattern implementations based on configuration flags
Posted by GitBox <gi...@apache.org>.
kerneltime commented on code in PR #3310:
URL: https://github.com/apache/ozone/pull/3310#discussion_r852413181
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/lock/OzoneLockStrategy.java:
##########
@@ -0,0 +1,44 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.lock;
+
+import org.apache.hadoop.ozone.om.OMMetadataManager;
+
+import java.io.IOException;
+
+/**
+ * Common interface for lock strategies.
+ */
+public interface OzoneLockStrategy {
Review Comment:
Please add detailed Java doc for what this interface guarantees.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr merged pull request #3310: HDDS-6583. Introduce lock strategy pattern implementations based on configuration flags
Posted by GitBox <gi...@apache.org>.
rakeshadr merged PR #3310:
URL: https://github.com/apache/ozone/pull/3310
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a diff in pull request #3310: HDDS-6583. Introduce lock strategy pattern implementations based on configuration flags
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on code in PR #3310:
URL: https://github.com/apache/ozone/pull/3310#discussion_r854252924
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/lock/OzoneLockProvider.java:
##########
@@ -0,0 +1,52 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.lock;
+
+import org.apache.hadoop.ozone.om.helpers.BucketLayout;
+
+/**
+ * OzoneLockProvider class returns the appropriate lock strategy pattern
+ * implementation based on the configuration flags passed.
+ */
+public class OzoneLockProvider {
+
+ private boolean enableKeyPathLock;
+ private boolean enableFileSystemPaths;
+
+ public OzoneLockProvider(boolean enableKeyPathLock,
Review Comment:
Add another test case where you can check the combinations enableKeyPathLock & enableFileSystemPaths & BucketLayout, then `Assert.assertTrue( lockStrategy instanceof OBSKeyPathLockStrategy)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a diff in pull request #3310: HDDS-6583. Introduce lock strategy pattern implementations based on configuration flags
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on code in PR #3310:
URL: https://github.com/apache/ozone/pull/3310#discussion_r854247423
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/lock/OzoneLockProvider.java:
##########
@@ -0,0 +1,52 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.lock;
+
+import org.apache.hadoop.ozone.om.helpers.BucketLayout;
+
+/**
+ * OzoneLockProvider class returns the appropriate lock strategy pattern
+ * implementation based on the configuration flags passed.
+ */
+public class OzoneLockProvider {
Review Comment:
Please add a test case to verify the different strategies and acquire/release locks.
OBS - two locks BUCKET_LOCK and KEY_PATH_LOCK
Regular - only BUCKET_LOCK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a diff in pull request #3310: HDDS-6583. Introduce lock strategy pattern implementations based on configuration flags
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on code in PR #3310:
URL: https://github.com/apache/ozone/pull/3310#discussion_r855085479
##########
hadoop-hdds/common/src/main/resources/ozone-default.xml:
##########
@@ -2819,6 +2819,15 @@
</description>
</property>
+ <property>
+ <name>ozone.om.enable.key.path.lock</name>
+ <tag>OZONE, OM</tag>
+ <value>false</value>
+ <description>If true, the fine-grained KEY_PATH_LOCK functionality is enabled.
+ If false, it is disabled.
Review Comment:
You can say,
Defaults to false.
##########
hadoop-hdds/common/src/main/resources/ozone-default.xml:
##########
@@ -2819,6 +2819,15 @@
</description>
</property>
+ <property>
+ <name>ozone.om.enable.key.path.lock</name>
Review Comment:
Please rename this property `ozone.om.enable.key.path.lock` to` ozone.om.key.path.lock.enabled`.
Also, please rename the corresponding items
-> `OZONE_OM_ENABLE_KEY_PATH_LOCK` to `OZONE_OM_KEY_PATH_LOCK_ENABLED`.
-> `getKeyPathLock() `to `getKeyPathLockEnabled()`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] tanvipenumudy commented on a diff in pull request #3310: HDDS-6583. Introduce lock strategy pattern implementations based on configuration flags
Posted by GitBox <gi...@apache.org>.
tanvipenumudy commented on code in PR #3310:
URL: https://github.com/apache/ozone/pull/3310#discussion_r855134865
##########
hadoop-hdds/common/src/main/resources/ozone-default.xml:
##########
@@ -2819,6 +2819,15 @@
</description>
</property>
+ <property>
+ <name>ozone.om.enable.key.path.lock</name>
+ <tag>OZONE, OM</tag>
+ <value>false</value>
+ <description>If true, the fine-grained KEY_PATH_LOCK functionality is enabled.
+ If false, it is disabled.
Review Comment:
Thank you, I have incorporated the change!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] tanvipenumudy commented on a diff in pull request #3310: HDDS-6583. Introduce lock strategy pattern implementations based on configuration flags
Posted by GitBox <gi...@apache.org>.
tanvipenumudy commented on code in PR #3310:
URL: https://github.com/apache/ozone/pull/3310#discussion_r852575676
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/lock/OzoneLockStrategy.java:
##########
@@ -0,0 +1,44 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.lock;
+
+import org.apache.hadoop.ozone.om.OMMetadataManager;
+
+import java.io.IOException;
+
+/**
+ * Common interface for lock strategies.
+ */
+public interface OzoneLockStrategy {
Review Comment:
Thanks for the suggestion - will be adding more details to the Java doc!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org