You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/10/28 06:16:19 UTC
[GitHub] [ozone] sadanand48 opened a new pull request #2781: HDDS-5906. Fix DBScanner to support Datanode DB
sadanand48 opened a new pull request #2781:
URL: https://github.com/apache/ozone/pull/2781
## What changes were proposed in this pull request?
Currently the LDB DBScanner Tool only supports DatanodeDBDefinition Schema V2. The aim here is to support both schemas V1 and V2.
## What is the link to the Apache JIRA?
https://issues.apache.org/jira/browse/HDDS-5906
## How was this patch tested?
`bin/ozone debug ldb --db=/Users/sadanand.shenoy/4-dn-container.db scan --column_family=default -d=V1
`
<img width="1785" alt="Screenshot 2021-10-28 at 11 42 15 AM" src="https://user-images.githubusercontent.com/31859223/139197477-3b46498c-5a81-4322-bc01-a604873e15d3.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sadanand48 commented on pull request #2781: HDDS-5906. Fix DBScanner to support Datanode DB
Posted by GitBox <gi...@apache.org>.
sadanand48 commented on pull request #2781:
URL: https://github.com/apache/ozone/pull/2781#issuecomment-963933809
@avijayanhwx Could you please take a look .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sadanand48 commented on a change in pull request #2781: HDDS-5906. Fix DBScanner to support Datanode DB
Posted by GitBox <gi...@apache.org>.
sadanand48 commented on a change in pull request #2781:
URL: https://github.com/apache/ozone/pull/2781#discussion_r738054059
##########
File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeSchemaOneDBDefinition.java
##########
@@ -91,7 +91,7 @@ protected DatanodeSchemaOneDBDefinition(String dbPath) {
@Override
public DBColumnFamilyDefinition[] getColumnFamilies() {
- return new DBColumnFamilyDefinition[] {getBlockDataColumnFamily(),
- getMetadataColumnFamily(), getDeletedBlocksColumnFamily() };
+ return new DBColumnFamilyDefinition[] {getMetadataColumnFamily(),
Review comment:
In DBScanner we maintain a 1-1 map for tableName to ColumFamilyDefinition but in case of V1 , we just have a single table DEFAULT mapped to multiple columnFamilies, so whichever put happens last in the map, it will pick that columnFamily, hence the order is changed here so that the defaultTable is mapped to BlockDataColumnFamily
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] avijayanhwx commented on a change in pull request #2781: HDDS-5906. Fix DBScanner to support Datanode DB
Posted by GitBox <gi...@apache.org>.
avijayanhwx commented on a change in pull request #2781:
URL: https://github.com/apache/ozone/pull/2781#discussion_r746151052
##########
File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeSchemaOneDBDefinition.java
##########
@@ -91,7 +91,7 @@ protected DatanodeSchemaOneDBDefinition(String dbPath) {
@Override
public DBColumnFamilyDefinition[] getColumnFamilies() {
- return new DBColumnFamilyDefinition[] {getBlockDataColumnFamily(),
- getMetadataColumnFamily(), getDeletedBlocksColumnFamily() };
+ return new DBColumnFamilyDefinition[] {getMetadataColumnFamily(),
Review comment:
Does this mean we cannot use this tool to access metadata and deletedblocks data from Schema V1 DB?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sadanand48 commented on a change in pull request #2781: HDDS-5906. Fix DBScanner to support Datanode DB
Posted by GitBox <gi...@apache.org>.
sadanand48 commented on a change in pull request #2781:
URL: https://github.com/apache/ozone/pull/2781#discussion_r746302375
##########
File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeSchemaOneDBDefinition.java
##########
@@ -91,7 +91,7 @@ protected DatanodeSchemaOneDBDefinition(String dbPath) {
@Override
public DBColumnFamilyDefinition[] getColumnFamilies() {
- return new DBColumnFamilyDefinition[] {getBlockDataColumnFamily(),
- getMetadataColumnFamily(), getDeletedBlocksColumnFamily() };
+ return new DBColumnFamilyDefinition[] {getMetadataColumnFamily(),
Review comment:
With respect to the current patch yes , it can only access blockData. It's also possible to access metadata if needed. I faced an error while trying to access the DeletedBlocks i.e `Invalid chunk information. This data may have been written using datanode schema version one, which did not save chunk information.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sadanand48 commented on a change in pull request #2781: HDDS-5906. Fix DBScanner to support Datanode DB
Posted by GitBox <gi...@apache.org>.
sadanand48 commented on a change in pull request #2781:
URL: https://github.com/apache/ozone/pull/2781#discussion_r738054059
##########
File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeSchemaOneDBDefinition.java
##########
@@ -91,7 +91,7 @@ protected DatanodeSchemaOneDBDefinition(String dbPath) {
@Override
public DBColumnFamilyDefinition[] getColumnFamilies() {
- return new DBColumnFamilyDefinition[] {getBlockDataColumnFamily(),
- getMetadataColumnFamily(), getDeletedBlocksColumnFamily() };
+ return new DBColumnFamilyDefinition[] {getMetadataColumnFamily(),
Review comment:
FYI: In DBScanner we maintain a 1-1 map for tableName to ColumFamilyDefinition but in case of V1 , we just have a single table DEFAULT mapped to multiple columnFamilies, so whichever put happens last in the map, it will pick that columnFamily, hence the order is changed here so that the defaultTable is mapped to BlockDataColumnFamily
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] mukul1987 merged pull request #2781: HDDS-5906. Fix DBScanner to support Datanode DB
Posted by GitBox <gi...@apache.org>.
mukul1987 merged pull request #2781:
URL: https://github.com/apache/ozone/pull/2781
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org