You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "TaiJuWu (via GitHub)" <gi...@apache.org> on 2023/12/09 02:09:36 UTC
[PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
TaiJuWu opened a new pull request, #5753:
URL: https://github.com/apache/ozone/pull/5753
## What changes were proposed in this pull request?
Provide a one-liner summary of the changes in the PR **Title** field above.
Remove duplicate helper methods for getting FSO open key.
Remove duplicated code in S3MultipartUploadCommitPartRequestWithFSO.getOpenKey() and OMAllocateBlockRequestWithFSO.getOpenKeyName().
Please describe your PR in detail:
A new class OmGetKey for get key from OMMetadataManager.
## What is the link to the Apache JIRA
HDDS-8650
## How was this patch tested?
CI test: https://github.com/TaiJuWu/ozone/actions/runs/7144668299
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "kerneltime (via GitHub)" <gi...@apache.org>.
kerneltime commented on PR #5753:
URL: https://github.com/apache/ozone/pull/5753#issuecomment-1850516767
@TaiJuWu please close this PR if you plan to open a new one, as discussed in community sync?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on PR #5753:
URL: https://github.com/apache/ozone/pull/5753#issuecomment-1868558893
@ivandika3 @jojochuang Please take a look, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #5753:
URL: https://github.com/apache/ozone/pull/5753#issuecomment-1864965024
@jojochuang would you like to take another look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on PR #5753:
URL: https://github.com/apache/ozone/pull/5753#issuecomment-1869399840
@TaiJuWu , thanks for resolving the conflicts. LGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "nandakumar131 (via GitHub)" <gi...@apache.org>.
nandakumar131 commented on PR #5753:
URL: https://github.com/apache/ozone/pull/5753#issuecomment-1885099261
Thanks @TaiJuWu for this improvement.
Thanks to @adoroszlai @jojochuang @ivandika3 @kerneltime for the reviews.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu closed pull request #5753: HDDS-8650. Remove duplicate helper methods for getting FSO open key
URL: https://github.com/apache/ozone/pull/5753
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on code in PR #5753:
URL: https://github.com/apache/ozone/pull/5753#discussion_r1424872118
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMRecoverLeaseRequest.java:
##########
@@ -212,8 +211,7 @@ private String doWork(OzoneManager ozoneManager, long transactionLogIndex)
LOG.warn("Key:" + keyName + " is already closed");
return null;
}
- String openFileDBKey = omMetadataManager.getOpenFileName(
- volumeId, bucketId, parentID, fileName, Long.parseLong(clientId));
+ String openFileDBKey = getKey.getOpenFileName();
Review Comment:
Correct me if I'm wrong, but since `clientID` is not set in `OmGetKey` class, the `openFileDBKey` will take null as the `clientID`?
You can refer to the earlier review on passing the `clientID` instead of storing it in the `OmGetKey` class.
Also you can refer to https://github.com/ivandika3/ozone/commit/4803bb27faa4b7790763d9952520fa62fd057d39#diff-4a6940e76f86d73b1112150209e4f32b9fb9c08aa5b9557e75ccb31b671ceafd .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on code in PR #5753:
URL: https://github.com/apache/ozone/pull/5753#discussion_r1429489159
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/helpers/OmGetKey.java:
##########
@@ -0,0 +1,175 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.hadoop.ozone.om.helpers;
+
+import org.apache.hadoop.ozone.om.OMMetadataManager;
+import org.apache.hadoop.ozone.om.request.file.OMFileRequest;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Iterator;
+
+/**
+ * This class help to get metadata keys.
+ */
+public final class OmGetKey {
+
+ private String volumeName;
+ private String bucketName;
+ private String keyName;
+ private OMMetadataManager omMetadataManager;
+ private long clientID;
+
+ private String fileName;
+ private Iterator<Path> pathComponents;
+ private long volumeId;
+ private long bucketId;
+ private long parentID;
+
+
+ @SuppressWarnings("checkstyle:parameternumber")
+ private OmGetKey(String volumeName, String bucketName, String keyName,
+ OMMetadataManager omMetadataManager, long clientID,
+ String fileName, Iterator<Path> pathComponents, long volumeId,
+ long bucketId, long parentID) {
+ this.volumeName = volumeName;
+ this.bucketName = bucketName;
+ this.keyName = keyName;
+ this.omMetadataManager = omMetadataManager;
+ this.clientID = clientID;
+
+ this.fileName = fileName;
+ this.pathComponents = pathComponents;
+ this.volumeId = volumeId;
+ this.bucketId = bucketId;
+ this.parentID = parentID;
+ }
+
+ /**
+ * Builder class for OmGetKey.
+ */
+ public static class Builder {
+ private String volumeName;
+ private String bucketName;
+ private String keyName;
+ private OMMetadataManager omMetadataManager;
+ private long clientID;
+ private String errMsg;
+
+ public Builder() {
+ this.errMsg = null;
+ }
+
+ public Builder setVolumeName(String volumeName) {
+ this.volumeName = volumeName;
+ return this;
+ }
+
+ public Builder setBucketName(String bucketName) {
+ this.bucketName = bucketName;
+ return this;
+ }
+
+ public Builder setKeyName(String keyName) {
+ this.keyName = keyName;
+ return this;
+ }
+
+ public Builder setOmMetadataManager(OMMetadataManager omMetadataManager) {
+ this.omMetadataManager = omMetadataManager;
+ return this;
+ }
+
+ public Builder setClientID(long clientID) {
+ this.clientID = clientID;
+ return this;
+ }
+
+ public Builder setErrMsg(String errMsg) {
+ this.errMsg = errMsg;
+ return this;
+ }
+
+ public OmGetKey build() throws IOException {
+ String fileName = OzoneFSUtils.getFileName(this.keyName);
+ Iterator<Path> pathComponents = Paths.get(this.keyName).iterator();
+ final long volumeId = omMetadataManager.getVolumeId(this.volumeName);
+ final long bucketId = omMetadataManager.getBucketId(this.volumeName,
+ this.bucketName);
+ long parentID = OMFileRequest
+ .getParentID(volumeId, bucketId, pathComponents, this.keyName,
+ this.omMetadataManager, this.errMsg);
Review Comment:
@TaiJuWu I have created https://issues.apache.org/jira/browse/HDDS-9946 and assigned it to you. Please let me know if you have any questions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on code in PR #5753:
URL: https://github.com/apache/ozone/pull/5753#discussion_r1429484856
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMRecoverLeaseRequest.java:
##########
@@ -212,8 +211,7 @@ private String doWork(OzoneManager ozoneManager, long transactionLogIndex)
LOG.warn("Key:" + keyName + " is already closed");
return null;
}
- String openFileDBKey = omMetadataManager.getOpenFileName(
- volumeId, bucketId, parentID, fileName, Long.parseLong(clientId));
+ String openFileDBKey = getKey.getOpenFileName();
Review Comment:
Thank you for the update.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on code in PR #5753:
URL: https://github.com/apache/ozone/pull/5753#discussion_r1424861368
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/helpers/OmGetKey.java:
##########
@@ -0,0 +1,175 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.hadoop.ozone.om.helpers;
+
+import org.apache.hadoop.ozone.om.OMMetadataManager;
+import org.apache.hadoop.ozone.om.request.file.OMFileRequest;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Iterator;
+
+/**
+ * This class help to get metadata keys.
+ */
+public final class OmGetKey {
+
+ private String volumeName;
+ private String bucketName;
+ private String keyName;
+ private OMMetadataManager omMetadataManager;
+ private long clientID;
+
+ private String fileName;
+ private Iterator<Path> pathComponents;
+ private long volumeId;
+ private long bucketId;
+ private long parentID;
+
+
+ @SuppressWarnings("checkstyle:parameternumber")
+ private OmGetKey(String volumeName, String bucketName, String keyName,
+ OMMetadataManager omMetadataManager, long clientID,
+ String fileName, Iterator<Path> pathComponents, long volumeId,
+ long bucketId, long parentID) {
+ this.volumeName = volumeName;
+ this.bucketName = bucketName;
+ this.keyName = keyName;
+ this.omMetadataManager = omMetadataManager;
+ this.clientID = clientID;
+
+ this.fileName = fileName;
+ this.pathComponents = pathComponents;
+ this.volumeId = volumeId;
+ this.bucketId = bucketId;
+ this.parentID = parentID;
+ }
+
+ /**
+ * Builder class for OmGetKey.
+ */
+ public static class Builder {
+ private String volumeName;
+ private String bucketName;
+ private String keyName;
+ private OMMetadataManager omMetadataManager;
+ private long clientID;
+ private String errMsg;
+
+ public Builder() {
+ this.errMsg = null;
+ }
+
+ public Builder setVolumeName(String volumeName) {
+ this.volumeName = volumeName;
+ return this;
+ }
+
+ public Builder setBucketName(String bucketName) {
+ this.bucketName = bucketName;
+ return this;
+ }
+
+ public Builder setKeyName(String keyName) {
+ this.keyName = keyName;
+ return this;
+ }
+
+ public Builder setOmMetadataManager(OMMetadataManager omMetadataManager) {
+ this.omMetadataManager = omMetadataManager;
+ return this;
+ }
+
+ public Builder setClientID(long clientID) {
+ this.clientID = clientID;
+ return this;
+ }
+
+ public Builder setErrMsg(String errMsg) {
+ this.errMsg = errMsg;
+ return this;
+ }
+
+ public OmGetKey build() throws IOException {
+ String fileName = OzoneFSUtils.getFileName(this.keyName);
+ Iterator<Path> pathComponents = Paths.get(this.keyName).iterator();
+ final long volumeId = omMetadataManager.getVolumeId(this.volumeName);
+ final long bucketId = omMetadataManager.getBucketId(this.volumeName,
+ this.bucketName);
+ long parentID = OMFileRequest
+ .getParentID(volumeId, bucketId, pathComponents, this.keyName,
+ this.omMetadataManager, this.errMsg);
Review Comment:
This can be addressed in another Jira ticket, but from what I see since the `pathComponents` is derived from the `keyName`. We can remove this `pathComponents` argument from the `OMFileRequest#getParentID` method and instead call the `Iterator<Path> pathComponents = Paths.get(keyName).iterator()` inside the `OMFileRequest#getParentID` instead.
You can refer to https://github.com/ivandika3/ozone/commit/4803bb27faa4b7790763d9952520fa62fd057d39#diff-4ea3716be41b5b8969a751fdc62db55f498b23ddb6f4d1085605dc974057dd1e
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on PR #5753:
URL: https://github.com/apache/ozone/pull/5753#issuecomment-1859571233
> Hi @TaiJuWu, the patch looks good. Let's wait until there is a clean CI in your fork.
Got it. Thanks for your review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on code in PR #5753:
URL: https://github.com/apache/ozone/pull/5753#discussion_r1424872118
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMRecoverLeaseRequest.java:
##########
@@ -212,8 +211,7 @@ private String doWork(OzoneManager ozoneManager, long transactionLogIndex)
LOG.warn("Key:" + keyName + " is already closed");
return null;
}
- String openFileDBKey = omMetadataManager.getOpenFileName(
- volumeId, bucketId, parentID, fileName, Long.parseLong(clientId));
+ String openFileDBKey = getKey.getOpenFileName();
Review Comment:
Correct me if I'm wrong, but since `clientID` is not set in `OmGetKey` class, the `openFileDBKey` will take null as the `clientID`?
You can refer to the earlier review on passing the `clientID` instead of storing it in the `OmGetKey` class as a possible solution.
Also you can refer to https://github.com/ivandika3/ozone/commit/4803bb27faa4b7790763d9952520fa62fd057d39#diff-4a6940e76f86d73b1112150209e4f32b9fb9c08aa5b9557e75ccb31b671ceafd .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on PR #5753:
URL: https://github.com/apache/ozone/pull/5753#issuecomment-1850543416
> @TaiJuWu please close this PR if you plan to open a new one, as discussed in community sync? cc @jojochuang
The PR already is new version. Thank you for your reply.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on PR #5753:
URL: https://github.com/apache/ozone/pull/5753#issuecomment-1848093002
I don't have good idea to merge OMRecoverLeaseRequest.doWork() and OMKeyCommitRequestWithFSO.validateAndUpdateCache().
They call different method (getOzonePathKey) in omMetadataManger and the number of parameters is different.
If anyone has suggestions for me, I can continue to do this work.
Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on code in PR #5753:
URL: https://github.com/apache/ozone/pull/5753#discussion_r1424664901
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/helpers/OmGetKey.java:
##########
@@ -0,0 +1,175 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.hadoop.ozone.om.helpers;
+
+import org.apache.hadoop.ozone.om.OMMetadataManager;
+import org.apache.hadoop.ozone.om.request.file.OMFileRequest;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Iterator;
+
+/**
+ * This class help to get metadata keys.
Review Comment:
"used exclusively in FSO buckets."
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/helpers/package-info.java:
##########
@@ -0,0 +1,23 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.helpers;
+
+/**
+ * This package contains classes and interfaces for OM Lock Strategy Patterns.
Review Comment:
change this description.
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/helpers/OmGetKey.java:
##########
@@ -0,0 +1,175 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.hadoop.ozone.om.helpers;
+
+import org.apache.hadoop.ozone.om.OMMetadataManager;
+import org.apache.hadoop.ozone.om.request.file.OMFileRequest;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Iterator;
+
+/**
+ * This class help to get metadata keys.
+ */
+public final class OmGetKey {
+
+ private String volumeName;
+ private String bucketName;
+ private String keyName;
+ private OMMetadataManager omMetadataManager;
+ private long clientID;
+
+ private String fileName;
+ private Iterator<Path> pathComponents;
+ private long volumeId;
+ private long bucketId;
+ private long parentID;
+
+
+ @SuppressWarnings("checkstyle:parameternumber")
+ private OmGetKey(String volumeName, String bucketName, String keyName,
Review Comment:
IMO this helper is exclusively used by request handlers dealing with FSO buckets, so should rename this helper method accordingly. How about renaming it as "FSOFile"?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on code in PR #5753:
URL: https://github.com/apache/ozone/pull/5753#discussion_r1424872118
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMRecoverLeaseRequest.java:
##########
@@ -212,8 +211,7 @@ private String doWork(OzoneManager ozoneManager, long transactionLogIndex)
LOG.warn("Key:" + keyName + " is already closed");
return null;
}
- String openFileDBKey = omMetadataManager.getOpenFileName(
- volumeId, bucketId, parentID, fileName, Long.parseLong(clientId));
+ String openFileDBKey = getKey.getOpenFileName();
Review Comment:
Correct me if I'm wrong, but since `clientID` is not set in `OmGetKey` class, the `openFileDBKey` will take null as the `clientID`?
You can refer to the earlier review on passing the `clientID` instead of storing it in the `OmGetKey` class.
Also you can refer to https://github.com/ivandika3/ozone/commit/4803bb27faa4b7790763d9952520fa62fd057d39#diff-4a6940e76f86d73b1112150209e4f32b9fb9c08aa5b9557e75ccb31b671ceafd (still a rough work).
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/helpers/OmGetKey.java:
##########
@@ -0,0 +1,175 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.hadoop.ozone.om.helpers;
+
+import org.apache.hadoop.ozone.om.OMMetadataManager;
+import org.apache.hadoop.ozone.om.request.file.OMFileRequest;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Iterator;
+
+/**
+ * This class help to get metadata keys.
+ */
+public final class OmGetKey {
+
+ private String volumeName;
+ private String bucketName;
+ private String keyName;
+ private OMMetadataManager omMetadataManager;
+ private long clientID;
Review Comment:
How about we remove `clientID` from the helper class? Instead, we can pass the `clientID` in the `getOpenFileName(long clientID)`.
This makes the helper class reusable for getting different FSO-related DB keys, for example for FSO multipart open key (`OMMetadataManager#getMultipartKey`), which has the DB key format `/{volumeId}/{bucketId}/{parentId}/{fileName}/{uploadId}`, we can pass `uploadId` instead.
To see what I meant, you can refer to https://github.com/ivandika3/ozone/commit/4803bb27faa4b7790763d9952520fa62fd057d39#diff-fae9ce494b51c92df1ea1465582a7ce2ea29afaf3aa06008ce5f0a7b41a17cbcR140-R153
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/helpers/OmGetKey.java:
##########
@@ -0,0 +1,175 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.hadoop.ozone.om.helpers;
+
+import org.apache.hadoop.ozone.om.OMMetadataManager;
+import org.apache.hadoop.ozone.om.request.file.OMFileRequest;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Iterator;
+
+/**
+ * This class help to get metadata keys.
+ */
+public final class OmGetKey {
+
+ private String volumeName;
+ private String bucketName;
+ private String keyName;
+ private OMMetadataManager omMetadataManager;
+ private long clientID;
+
+ private String fileName;
+ private Iterator<Path> pathComponents;
+ private long volumeId;
+ private long bucketId;
+ private long parentID;
+
+
+ @SuppressWarnings("checkstyle:parameternumber")
+ private OmGetKey(String volumeName, String bucketName, String keyName,
+ OMMetadataManager omMetadataManager, long clientID,
+ String fileName, Iterator<Path> pathComponents, long volumeId,
+ long bucketId, long parentID) {
+ this.volumeName = volumeName;
+ this.bucketName = bucketName;
+ this.keyName = keyName;
+ this.omMetadataManager = omMetadataManager;
+ this.clientID = clientID;
+
+ this.fileName = fileName;
+ this.pathComponents = pathComponents;
+ this.volumeId = volumeId;
+ this.bucketId = bucketId;
+ this.parentID = parentID;
+ }
+
+ /**
+ * Builder class for OmGetKey.
+ */
+ public static class Builder {
+ private String volumeName;
+ private String bucketName;
+ private String keyName;
+ private OMMetadataManager omMetadataManager;
+ private long clientID;
+ private String errMsg;
+
+ public Builder() {
+ this.errMsg = null;
+ }
+
+ public Builder setVolumeName(String volumeName) {
+ this.volumeName = volumeName;
+ return this;
+ }
+
+ public Builder setBucketName(String bucketName) {
+ this.bucketName = bucketName;
+ return this;
+ }
+
+ public Builder setKeyName(String keyName) {
+ this.keyName = keyName;
+ return this;
+ }
+
+ public Builder setOmMetadataManager(OMMetadataManager omMetadataManager) {
+ this.omMetadataManager = omMetadataManager;
+ return this;
+ }
+
+ public Builder setClientID(long clientID) {
+ this.clientID = clientID;
+ return this;
+ }
+
+ public Builder setErrMsg(String errMsg) {
+ this.errMsg = errMsg;
+ return this;
+ }
+
+ public OmGetKey build() throws IOException {
+ String fileName = OzoneFSUtils.getFileName(this.keyName);
+ Iterator<Path> pathComponents = Paths.get(this.keyName).iterator();
+ final long volumeId = omMetadataManager.getVolumeId(this.volumeName);
+ final long bucketId = omMetadataManager.getBucketId(this.volumeName,
+ this.bucketName);
+ long parentID = OMFileRequest
+ .getParentID(volumeId, bucketId, pathComponents, this.keyName,
+ this.omMetadataManager, this.errMsg);
Review Comment:
This can be addressed in another Jira ticket, but from what I see since the `pathComponents` is derived from the `keyName`. We can remove this in `pathComponents` from the `OMFileRequest#getParentID` method arguments and instead call the `Iterator<Path> pathComponents = Paths.get(this.keyName).iterator();` inside the `OMFileRequest#getParentID` instead.
I did some rough work on this: https://github.com/ivandika3/ozone/commit/4803bb27faa4b7790763d9952520fa62fd057d39#diff-4ea3716be41b5b8969a751fdc62db55f498b23ddb6f4d1085605dc974057dd1e
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on code in PR #5753:
URL: https://github.com/apache/ozone/pull/5753#discussion_r1424872118
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMRecoverLeaseRequest.java:
##########
@@ -212,8 +211,7 @@ private String doWork(OzoneManager ozoneManager, long transactionLogIndex)
LOG.warn("Key:" + keyName + " is already closed");
return null;
}
- String openFileDBKey = omMetadataManager.getOpenFileName(
- volumeId, bucketId, parentID, fileName, Long.parseLong(clientId));
+ String openFileDBKey = getKey.getOpenFileName();
Review Comment:
Correct me if I'm wrong, but since `clientID` is not set in `OmGetKey` class, the `openFileDBKey` will take null as the `clientID`?
You can refer to the earlier review on passing the `clientID` instead of storing it in the `OmGetKey` class. This is just one possible solution.
Also you can refer to https://github.com/ivandika3/ozone/commit/4803bb27faa4b7790763d9952520fa62fd057d39#diff-4a6940e76f86d73b1112150209e4f32b9fb9c08aa5b9557e75ccb31b671ceafd .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on PR #5753:
URL: https://github.com/apache/ozone/pull/5753#issuecomment-1868525259
@jojochuang @ivandika3 @adoroszlai Please take a look, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on code in PR #5753:
URL: https://github.com/apache/ozone/pull/5753#discussion_r1424861368
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/helpers/OmGetKey.java:
##########
@@ -0,0 +1,175 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.hadoop.ozone.om.helpers;
+
+import org.apache.hadoop.ozone.om.OMMetadataManager;
+import org.apache.hadoop.ozone.om.request.file.OMFileRequest;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Iterator;
+
+/**
+ * This class help to get metadata keys.
+ */
+public final class OmGetKey {
+
+ private String volumeName;
+ private String bucketName;
+ private String keyName;
+ private OMMetadataManager omMetadataManager;
+ private long clientID;
+
+ private String fileName;
+ private Iterator<Path> pathComponents;
+ private long volumeId;
+ private long bucketId;
+ private long parentID;
+
+
+ @SuppressWarnings("checkstyle:parameternumber")
+ private OmGetKey(String volumeName, String bucketName, String keyName,
+ OMMetadataManager omMetadataManager, long clientID,
+ String fileName, Iterator<Path> pathComponents, long volumeId,
+ long bucketId, long parentID) {
+ this.volumeName = volumeName;
+ this.bucketName = bucketName;
+ this.keyName = keyName;
+ this.omMetadataManager = omMetadataManager;
+ this.clientID = clientID;
+
+ this.fileName = fileName;
+ this.pathComponents = pathComponents;
+ this.volumeId = volumeId;
+ this.bucketId = bucketId;
+ this.parentID = parentID;
+ }
+
+ /**
+ * Builder class for OmGetKey.
+ */
+ public static class Builder {
+ private String volumeName;
+ private String bucketName;
+ private String keyName;
+ private OMMetadataManager omMetadataManager;
+ private long clientID;
+ private String errMsg;
+
+ public Builder() {
+ this.errMsg = null;
+ }
+
+ public Builder setVolumeName(String volumeName) {
+ this.volumeName = volumeName;
+ return this;
+ }
+
+ public Builder setBucketName(String bucketName) {
+ this.bucketName = bucketName;
+ return this;
+ }
+
+ public Builder setKeyName(String keyName) {
+ this.keyName = keyName;
+ return this;
+ }
+
+ public Builder setOmMetadataManager(OMMetadataManager omMetadataManager) {
+ this.omMetadataManager = omMetadataManager;
+ return this;
+ }
+
+ public Builder setClientID(long clientID) {
+ this.clientID = clientID;
+ return this;
+ }
+
+ public Builder setErrMsg(String errMsg) {
+ this.errMsg = errMsg;
+ return this;
+ }
+
+ public OmGetKey build() throws IOException {
+ String fileName = OzoneFSUtils.getFileName(this.keyName);
+ Iterator<Path> pathComponents = Paths.get(this.keyName).iterator();
+ final long volumeId = omMetadataManager.getVolumeId(this.volumeName);
+ final long bucketId = omMetadataManager.getBucketId(this.volumeName,
+ this.bucketName);
+ long parentID = OMFileRequest
+ .getParentID(volumeId, bucketId, pathComponents, this.keyName,
+ this.omMetadataManager, this.errMsg);
Review Comment:
This can be addressed in another Jira ticket, but from what I see since the `pathComponents` is derived from the `keyName`. We can remove this in `pathComponents` from the `OMFileRequest#getParentID` method arguments and instead call the `Iterator<Path> pathComponents = Paths.get(this.keyName).iterator()` inside the `OMFileRequest#getParentID` instead.
You can refer to https://github.com/ivandika3/ozone/commit/4803bb27faa4b7790763d9952520fa62fd057d39#diff-4ea3716be41b5b8969a751fdc62db55f498b23ddb6f4d1085605dc974057dd1e
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on PR #5753:
URL: https://github.com/apache/ozone/pull/5753#issuecomment-1858178084
> Thank you @TaiJuWu for the patch. This clean up would be very helpful. I have left a few comments.
>
> I did some very rough work on this cleanup in [ivandika3@4803bb2](https://github.com/ivandika3/ozone/commit/4803bb27faa4b7790763d9952520fa62fd057d39). It's still in a very rough form and failed tests, but I hope it would help.
>
> As @jojochuang said, there are still some duplications left, like in `OMMultipartUploadUtils#getMultipartOpenKeyFSO`, `S3MultipartUploadCompleteRequestWithFSO`, etc. My rough draft covered some of these classes, but I might miss some. We can revisit it in future patches.
> Thank you @TaiJuWu for the patch. This clean up would be very helpful. I have left a few comments.
>
> I did some very rough work on this cleanup in [ivandika3@4803bb2](https://github.com/ivandika3/ozone/commit/4803bb27faa4b7790763d9952520fa62fd057d39). It's still in a very rough form and failed tests, but I hope it would help.
>
> As @jojochuang said, there are still some duplications left, like in `OMMultipartUploadUtils#getMultipartOpenKeyFSO`, `S3MultipartUploadCompleteRequestWithFSO`, etc. My rough draft covered some of these classes, but I might miss some. We can revisit it in future patches.
Im not sure our CI fail is same or not.
But I fail CI test on org.apache.hadoop.ozone.debug.TestLeaseRecoverer.testCLI(TestLeaseRecoverer.java:140.
I still try to find this reason.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on PR #5753:
URL: https://github.com/apache/ozone/pull/5753#issuecomment-1859551482
Hi @TaiJuWu, the patch looks good. Let's wait until there is a clean CI in your fork.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on PR #5753:
URL: https://github.com/apache/ozone/pull/5753#issuecomment-1863757629
Looks like the CI failures are not related. @jojochuang Could you help trigger the CI?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on code in PR #5753:
URL: https://github.com/apache/ozone/pull/5753#discussion_r1431426564
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/helpers/OmGetKey.java:
##########
@@ -0,0 +1,175 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.hadoop.ozone.om.helpers;
+
+import org.apache.hadoop.ozone.om.OMMetadataManager;
+import org.apache.hadoop.ozone.om.request.file.OMFileRequest;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Iterator;
+
+/**
+ * This class help to get metadata keys.
+ */
+public final class OmGetKey {
+
+ private String volumeName;
+ private String bucketName;
+ private String keyName;
+ private OMMetadataManager omMetadataManager;
+ private long clientID;
+
+ private String fileName;
+ private Iterator<Path> pathComponents;
+ private long volumeId;
+ private long bucketId;
+ private long parentID;
+
+
+ @SuppressWarnings("checkstyle:parameternumber")
+ private OmGetKey(String volumeName, String bucketName, String keyName,
+ OMMetadataManager omMetadataManager, long clientID,
+ String fileName, Iterator<Path> pathComponents, long volumeId,
+ long bucketId, long parentID) {
+ this.volumeName = volumeName;
+ this.bucketName = bucketName;
+ this.keyName = keyName;
+ this.omMetadataManager = omMetadataManager;
+ this.clientID = clientID;
+
+ this.fileName = fileName;
+ this.pathComponents = pathComponents;
+ this.volumeId = volumeId;
+ this.bucketId = bucketId;
+ this.parentID = parentID;
+ }
+
+ /**
+ * Builder class for OmGetKey.
+ */
+ public static class Builder {
+ private String volumeName;
+ private String bucketName;
+ private String keyName;
+ private OMMetadataManager omMetadataManager;
+ private long clientID;
+ private String errMsg;
+
+ public Builder() {
+ this.errMsg = null;
+ }
+
+ public Builder setVolumeName(String volumeName) {
+ this.volumeName = volumeName;
+ return this;
+ }
+
+ public Builder setBucketName(String bucketName) {
+ this.bucketName = bucketName;
+ return this;
+ }
+
+ public Builder setKeyName(String keyName) {
+ this.keyName = keyName;
+ return this;
+ }
+
+ public Builder setOmMetadataManager(OMMetadataManager omMetadataManager) {
+ this.omMetadataManager = omMetadataManager;
+ return this;
+ }
+
+ public Builder setClientID(long clientID) {
+ this.clientID = clientID;
+ return this;
+ }
+
+ public Builder setErrMsg(String errMsg) {
+ this.errMsg = errMsg;
+ return this;
+ }
+
+ public OmGetKey build() throws IOException {
+ String fileName = OzoneFSUtils.getFileName(this.keyName);
+ Iterator<Path> pathComponents = Paths.get(this.keyName).iterator();
+ final long volumeId = omMetadataManager.getVolumeId(this.volumeName);
+ final long bucketId = omMetadataManager.getBucketId(this.volumeName,
+ this.bucketName);
+ long parentID = OMFileRequest
+ .getParentID(volumeId, bucketId, pathComponents, this.keyName,
+ this.omMetadataManager, this.errMsg);
Review Comment:
> @TaiJuWu I have created https://issues.apache.org/jira/browse/HDDS-9946 and assigned it to you. Please let me know if you have any questions.
Thanks, I will start working this weekend.
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMRecoverLeaseRequest.java:
##########
@@ -212,8 +211,7 @@ private String doWork(OzoneManager ozoneManager, long transactionLogIndex)
LOG.warn("Key:" + keyName + " is already closed");
return null;
}
- String openFileDBKey = omMetadataManager.getOpenFileName(
- volumeId, bucketId, parentID, fileName, Long.parseLong(clientId));
+ String openFileDBKey = getKey.getOpenFileName();
Review Comment:
> Correct me if I'm wrong, but since `clientID` is not set in `OmGetKey` class, the `openFileDBKey` will take null as the `clientID`?
>
Yes, you're right. This is an error. Maybe this is the reason that CI failed.
> You can refer to the earlier review on passing the `clientID` instead of storing it in the `OmGetKey` class as a possible solution.
>
> Also you can refer to [ivandika3@4803bb2#diff-4a6940e76f86d73b1112150209e4f32b9fb9c08aa5b9557e75ccb31b671ceafd](https://github.com/ivandika3/ozone/commit/4803bb27faa4b7790763d9952520fa62fd057d39#diff-4a6940e76f86d73b1112150209e4f32b9fb9c08aa5b9557e75ccb31b671ceafd) .
Thanks for your suggestion, I think clientID as parameter is great idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on code in PR #5753:
URL: https://github.com/apache/ozone/pull/5753#discussion_r1428118975
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/helpers/OmGetKey.java:
##########
@@ -0,0 +1,175 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.hadoop.ozone.om.helpers;
+
+import org.apache.hadoop.ozone.om.OMMetadataManager;
+import org.apache.hadoop.ozone.om.request.file.OMFileRequest;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Iterator;
+
+/**
+ * This class help to get metadata keys.
+ */
+public final class OmGetKey {
+
+ private String volumeName;
+ private String bucketName;
+ private String keyName;
+ private OMMetadataManager omMetadataManager;
+ private long clientID;
+
+ private String fileName;
+ private Iterator<Path> pathComponents;
+ private long volumeId;
+ private long bucketId;
+ private long parentID;
+
+
+ @SuppressWarnings("checkstyle:parameternumber")
+ private OmGetKey(String volumeName, String bucketName, String keyName,
Review Comment:
Thanks for your review.
OmFSOFile is great.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "nandakumar131 (via GitHub)" <gi...@apache.org>.
nandakumar131 commented on PR #5753:
URL: https://github.com/apache/ozone/pull/5753#issuecomment-1885093456
+1, LGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8650. Remove duplicate helper methods for getting FSO open key [ozone]
Posted by "nandakumar131 (via GitHub)" <gi...@apache.org>.
nandakumar131 merged PR #5753:
URL: https://github.com/apache/ozone/pull/5753
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org