You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/12/04 07:13:43 UTC
[GitHub] [ozone] rakeshadr opened a new pull request #1656: HDDS-4514. AllocateBlock : lookup and update open file table for the given path
rakeshadr opened a new pull request #1656:
URL: https://github.com/apache/ozone/pull/1656
## What changes were proposed in this pull request?
This task is to use open file table for the allocate block operations. This has been identified as part of Hive TPCDS benchmark test.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-4514
## How was this patch tested?
Added UT cases. Also, verified by running hive tpcds test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a change in pull request #1656: HDDS-4514. AllocateBlock : lookup and update open file table for the given path
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on a change in pull request #1656:
URL: https://github.com/apache/ozone/pull/1656#discussion_r539264335
##########
File path: hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/key/TestOMAllocateBlockResponseV1.java
##########
@@ -0,0 +1,69 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.response.key;
+
+import org.apache.hadoop.hdds.protocol.proto.HddsProtos;
+import org.apache.hadoop.ozone.OzoneConsts;
+import org.apache.hadoop.ozone.om.helpers.OmBucketInfo;
+import org.apache.hadoop.ozone.om.helpers.OmKeyInfo;
+import org.apache.hadoop.ozone.om.helpers.OmVolumeArgs;
+import org.apache.hadoop.ozone.om.request.TestOMRequestUtils;
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.OMResponse;
+import org.apache.hadoop.util.Time;
+import org.jetbrains.annotations.NotNull;
+
+/**
+ * Tests OMAllocateBlockResponse layout version V1.
+ */
+public class TestOMAllocateBlockResponseV1
+ extends TestOMAllocateBlockResponse {
+
Review comment:
Thanks @linyiqun , configuring V1 version will be required to pick new tables, otherwise all the db entries will go to old table with new format.
In new PR https://github.com/apache/ozone/pull/1679, I've updated existing response test cases as well to follow same pattern. Please review it again. Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on pull request #1656: HDDS-4514. AllocateBlock : lookup and update open file table for the given path
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on pull request #1656:
URL: https://github.com/apache/ozone/pull/1656#issuecomment-741737363
Current PR has multiple changes and looks like stale. I am closing this.
@linyiqun @bharatviswa504 , I've created new PR# https://github.com/apache/ozone/pull/1679 on top of the branch. Kindly review the same. Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on pull request #1656: HDDS-4514. AllocateBlock : lookup and update open file table for the given path
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on pull request #1656:
URL: https://github.com/apache/ozone/pull/1656#issuecomment-740272671
@rakeshadr can you rebase this PR, it has multiple changes looks like it.
Can you post a PR related to this Jira, which will make it easier to review?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr closed pull request #1656: HDDS-4514. AllocateBlock : lookup and update open file table for the given path
Posted by GitBox <gi...@apache.org>.
rakeshadr closed pull request #1656:
URL: https://github.com/apache/ozone/pull/1656
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] rakeshadr commented on a change in pull request #1656: HDDS-4514. AllocateBlock : lookup and update open file table for the given path
Posted by GitBox <gi...@apache.org>.
rakeshadr commented on a change in pull request #1656:
URL: https://github.com/apache/ozone/pull/1656#discussion_r536829353
##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystem.java
##########
@@ -822,26 +849,26 @@ protected void testRenameDirToFile() throws Exception {
ContractTestUtils.touch(fs, file1Destin);
Path abcRootPath = new Path(fs.getUri().toString() + "/a/b/c");
fs.mkdirs(abcRootPath);
- try {
- fs.rename(abcRootPath, file1Destin);
- Assert.fail("key already exists /root_dir/file1");
- } catch (FileAlreadyExistsException faee) {
- // expected
- }
+ Assert.assertFalse("key already exists /root_dir/file1",
+ fs.rename(abcRootPath, file1Destin));
}
Review comment:
@linyiqun yes, you are right. Since that is in progress, I just added to make clean build report. We can push #1607 push/upstream it first and then I will rebase this PR on top of that. Hope that would be fine.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] linyiqun commented on a change in pull request #1656: HDDS-4514. AllocateBlock : lookup and update open file table for the given path
Posted by GitBox <gi...@apache.org>.
linyiqun commented on a change in pull request #1656:
URL: https://github.com/apache/ozone/pull/1656#discussion_r536100525
##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystem.java
##########
@@ -822,26 +849,26 @@ protected void testRenameDirToFile() throws Exception {
ContractTestUtils.touch(fs, file1Destin);
Path abcRootPath = new Path(fs.getUri().toString() + "/a/b/c");
fs.mkdirs(abcRootPath);
- try {
- fs.rename(abcRootPath, file1Destin);
- Assert.fail("key already exists /root_dir/file1");
- } catch (FileAlreadyExistsException faee) {
- // expected
- }
+ Assert.assertFalse("key already exists /root_dir/file1",
+ fs.rename(abcRootPath, file1Destin));
}
Review comment:
Above rename related change already contained in PR #1607.
##########
File path: hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/key/TestOMAllocateBlockResponseV1.java
##########
@@ -0,0 +1,69 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.response.key;
+
+import org.apache.hadoop.hdds.protocol.proto.HddsProtos;
+import org.apache.hadoop.ozone.OzoneConsts;
+import org.apache.hadoop.ozone.om.helpers.OmBucketInfo;
+import org.apache.hadoop.ozone.om.helpers.OmKeyInfo;
+import org.apache.hadoop.ozone.om.helpers.OmVolumeArgs;
+import org.apache.hadoop.ozone.om.request.TestOMRequestUtils;
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.OMResponse;
+import org.apache.hadoop.util.Time;
+import org.jetbrains.annotations.NotNull;
+
+/**
+ * Tests OMAllocateBlockResponse layout version V1.
+ */
+public class TestOMAllocateBlockResponseV1
+ extends TestOMAllocateBlockResponse {
+
Review comment:
Do we need to override getOzoneConfiguration method in this class?
```java
protected OzoneConfiguration getOzoneConfiguration() {
OzoneConfiguration config = super.getOzoneConfiguration();
config.set(OMConfigKeys.OZONE_OM_LAYOUT_VERSION, "V1");
return config;
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org