You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/08/22 14:23:13 UTC
[GitHub] [ozone] ChenSammi opened a new pull request, #3705: HDDS-7158. ldb cli command supports to scan container V3.
ChenSammi opened a new pull request, #3705:
URL: https://github.com/apache/ozone/pull/3705
https://issues.apache.org/jira/browse/HDDS-7158
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ChenSammi commented on a diff in pull request #3705: HDDS-7158. ldb cli command supports to scan container V3.
Posted by GitBox <gi...@apache.org>.
ChenSammi commented on code in PR #3705:
URL: https://github.com/apache/ozone/pull/3705#discussion_r953302735
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedSlice.java:
##########
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.hadoop.hdds.utils.db.managed;
+
+import org.rocksdb.Slice;
+
+/**
+ * Managed Options.
+ */
+public class ManagedSlice extends Slice {
+
+ public ManagedSlice(byte[] var1) {
+ super(var1);
+ }
+
+ @Override
+ protected void finalize() throws Throwable {
+ ManagedRocksObjectMetrics.INSTANCE.increaseManagedObject();
Review Comment:
@duongnguyen0 , thanks for the code review. RocksMutableObject.isOwningHandle has a protected access privilege, can cannot be called in either ManagedRocksObjectMetrics or ManagedRocksObjectUtils(I think you assume this class).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ChenSammi commented on a diff in pull request #3705: HDDS-7158. ldb cli command supports to scan container V3.
Posted by GitBox <gi...@apache.org>.
ChenSammi commented on code in PR #3705:
URL: https://github.com/apache/ozone/pull/3705#discussion_r953302735
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedSlice.java:
##########
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.hadoop.hdds.utils.db.managed;
+
+import org.rocksdb.Slice;
+
+/**
+ * Managed Options.
+ */
+public class ManagedSlice extends Slice {
+
+ public ManagedSlice(byte[] var1) {
+ super(var1);
+ }
+
+ @Override
+ protected void finalize() throws Throwable {
+ ManagedRocksObjectMetrics.INSTANCE.increaseManagedObject();
Review Comment:
@duongnguyen0 , thanks for the code review. RocksMutableObject.isOwningHandle has a protected access privilege, can cannot be called in either ManagedRocksObjectMetrics or ManagedRocksObjectUtils(I assume you mean this class).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kerneltime commented on pull request #3705: HDDS-7158. ldb cli command supports to scan container V3.
Posted by GitBox <gi...@apache.org>.
kerneltime commented on PR #3705:
URL: https://github.com/apache/ozone/pull/3705#issuecomment-1222586268
@duongnguyen0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] duongnguyen0 commented on a diff in pull request #3705: HDDS-7158. ldb cli command supports to scan container V3.
Posted by GitBox <gi...@apache.org>.
duongnguyen0 commented on code in PR #3705:
URL: https://github.com/apache/ozone/pull/3705#discussion_r953008171
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedSlice.java:
##########
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.hadoop.hdds.utils.db.managed;
+
+import org.rocksdb.Slice;
+
+/**
+ * Managed Options.
+ */
+public class ManagedSlice extends Slice {
+
+ public ManagedSlice(byte[] var1) {
+ super(var1);
+ }
+
+ @Override
+ protected void finalize() throws Throwable {
+ ManagedRocksObjectMetrics.INSTANCE.increaseManagedObject();
Review Comment:
Let's make it an util in `ManagedRocksObjectMetrics`.
```
static void assertClosed(RocksMutableObject rocksObject) {
ManagedRocksObjectMetrics.INSTANCE.increaseManagedObject();
if (rocksObject.isOwningHandle()) {
ManagedRocksObjectMetrics.INSTANCE.increaseLeakObject();
LOG.warn("{} is not closed properly",
rocksObject.getClass().getSimpleName());
}
}
```
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedSlice.java:
##########
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.hadoop.hdds.utils.db.managed;
+
+import org.rocksdb.Slice;
+
+/**
+ * Managed Options.
Review Comment:
```suggestion
* Managed Slice.
```
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/DBScanner.java:
##########
@@ -100,13 +109,25 @@ private static List<Object> displayTable(ManagedRocksIterator iterator,
new FileOutputStream(fileName), StandardCharsets.UTF_8);
printWriter = new PrintWriter(fileWriter);
}
+
+ boolean schemaV3 = dnDBSchemaVersion != null &&
+ dnDBSchemaVersion.equals("V3");
while (iterator.get().isValid()) {
StringBuilder result = new StringBuilder();
if (withKey) {
Object key = dbColumnFamilyDefinition.getKeyCodec()
.fromPersistedFormat(iterator.get().key());
Gson gson = new GsonBuilder().setPrettyPrinting().create();
- result.append(gson.toJson(key));
+ if (schemaV3) {
+ int index =
+ DatanodeSchemaThreeDBDefinition.getContainerKeyPrefixLength();
+ String cid = ((String)key).substring(0, index);
Review Comment:
```suggestion
String cid = key.toString().substring(0, index);
```
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/DBScanner.java:
##########
@@ -100,13 +109,25 @@ private static List<Object> displayTable(ManagedRocksIterator iterator,
new FileOutputStream(fileName), StandardCharsets.UTF_8);
printWriter = new PrintWriter(fileWriter);
}
+
+ boolean schemaV3 = dnDBSchemaVersion != null &&
+ dnDBSchemaVersion.equals("V3");
while (iterator.get().isValid()) {
StringBuilder result = new StringBuilder();
if (withKey) {
Object key = dbColumnFamilyDefinition.getKeyCodec()
.fromPersistedFormat(iterator.get().key());
Gson gson = new GsonBuilder().setPrettyPrinting().create();
- result.append(gson.toJson(key));
+ if (schemaV3) {
+ int index =
+ DatanodeSchemaThreeDBDefinition.getContainerKeyPrefixLength();
+ String cid = ((String)key).substring(0, index);
+ String blockId = ((String)key).substring(index);
Review Comment:
```suggestion
String blockId = key.toString().substring(index);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ChenSammi commented on pull request #3705: HDDS-7158. ldb cli command supports to scan container V3.
Posted by GitBox <gi...@apache.org>.
ChenSammi commented on PR #3705:
URL: https://github.com/apache/ozone/pull/3705#issuecomment-1228048224
Thanks @duongnguyen0 and @guihecheng for the code review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ChenSammi commented on a diff in pull request #3705: HDDS-7158. ldb cli command supports to scan container V3.
Posted by GitBox <gi...@apache.org>.
ChenSammi commented on code in PR #3705:
URL: https://github.com/apache/ozone/pull/3705#discussion_r953302735
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedSlice.java:
##########
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.hadoop.hdds.utils.db.managed;
+
+import org.rocksdb.Slice;
+
+/**
+ * Managed Options.
+ */
+public class ManagedSlice extends Slice {
+
+ public ManagedSlice(byte[] var1) {
+ super(var1);
+ }
+
+ @Override
+ protected void finalize() throws Throwable {
+ ManagedRocksObjectMetrics.INSTANCE.increaseManagedObject();
Review Comment:
@duongnguyen0 , thanks for the code review. RocksMutableObject.isOwningHandle has a protected access privilege, can cannot be directly called in either ManagedRocksObjectMetrics or ManagedRocksObjectUtils(I assume you mean this class).
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedSlice.java:
##########
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.hadoop.hdds.utils.db.managed;
+
+import org.rocksdb.Slice;
+
+/**
+ * Managed Options.
+ */
+public class ManagedSlice extends Slice {
+
+ public ManagedSlice(byte[] var1) {
+ super(var1);
+ }
+
+ @Override
+ protected void finalize() throws Throwable {
+ ManagedRocksObjectMetrics.INSTANCE.increaseManagedObject();
Review Comment:
@duongnguyen0 , thanks for the code review. RocksMutableObject.isOwningHandle has a protected access privilege, and cannot be directly called in either ManagedRocksObjectMetrics or ManagedRocksObjectUtils(I assume you mean this class).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ChenSammi merged pull request #3705: HDDS-7158. ldb cli command supports to scan container V3.
Posted by GitBox <gi...@apache.org>.
ChenSammi merged PR #3705:
URL: https://github.com/apache/ozone/pull/3705
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ChenSammi commented on a diff in pull request #3705: HDDS-7158. ldb cli command supports to scan container V3.
Posted by GitBox <gi...@apache.org>.
ChenSammi commented on code in PR #3705:
URL: https://github.com/apache/ozone/pull/3705#discussion_r953302735
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedSlice.java:
##########
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.hadoop.hdds.utils.db.managed;
+
+import org.rocksdb.Slice;
+
+/**
+ * Managed Options.
+ */
+public class ManagedSlice extends Slice {
+
+ public ManagedSlice(byte[] var1) {
+ super(var1);
+ }
+
+ @Override
+ protected void finalize() throws Throwable {
+ ManagedRocksObjectMetrics.INSTANCE.increaseManagedObject();
Review Comment:
@duongnguyen0 , thanks for the code review. RocksMutableObject.isOwningHandle has a protected access privilege, and cannot be directly called in either ManagedRocksObjectMetrics or ManagedRocksObjectUtils(I assume you mean this class). That's why move the check into the final method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] duongnguyen0 commented on a diff in pull request #3705: HDDS-7158. ldb cli command supports to scan container V3.
Posted by GitBox <gi...@apache.org>.
duongnguyen0 commented on code in PR #3705:
URL: https://github.com/apache/ozone/pull/3705#discussion_r953326372
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedSlice.java:
##########
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.hadoop.hdds.utils.db.managed;
+
+import org.rocksdb.Slice;
+
+/**
+ * Managed Options.
+ */
+public class ManagedSlice extends Slice {
+
+ public ManagedSlice(byte[] var1) {
+ super(var1);
+ }
+
+ @Override
+ protected void finalize() throws Throwable {
+ ManagedRocksObjectMetrics.INSTANCE.increaseManagedObject();
Review Comment:
Sounds good, thanks for the clarification, Sammi.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org