You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/01/20 09:30:05 UTC
[GitHub] [ozone] GlenGeng opened a new pull request #1824: HDDS-4724. Revert column family names of Datanode db to avoid compatibility issue.
GlenGeng opened a new pull request #1824:
URL: https://github.com/apache/ozone/pull/1824
## What changes were proposed in this pull request?
Tencent monthly deploy the latest master to our production environment.
During latest upgrade test, we found that new Ozone(version at Jan 14 2021) can not read the data written by old Ozone(version at Dec 14 2020).
We found following traces
```
2021-01-20 16:36:39,575 INFO org.apache.hadoop.ozone.container.common.volume.MutableVolumeSet: Added Volume : /data4/hdds/hdds to VolumeSet
2021-01-20 16:36:39,582 INFO org.apache.hadoop.ozone.container.common.volume.ThrottledAsyncChecker: Scheduling a check for /data4/hdds/hdds
2021-01-20 16:36:39,594 INFO org.apache.hadoop.ozone.container.common.volume.HddsVolumeChecker: Scheduled health check for volume /data4/hdds/hdds
2021-01-20 16:36:39,595 INFO org.apache.hadoop.ozone.container.common.volume.ThrottledAsyncChecker: Scheduling a check for /data1/hdds/hdds
2021-01-20 16:36:39,595 INFO org.apache.hadoop.ozone.container.common.volume.HddsVolumeChecker: Scheduled health check for volume /data1/hdds/hdds
2021-01-20 16:36:39,595 INFO org.apache.hadoop.ozone.container.common.volume.ThrottledAsyncChecker: Scheduling a check for /data2/hdds/hdds
2021-01-20 16:36:39,596 INFO org.apache.hadoop.ozone.container.common.volume.HddsVolumeChecker: Scheduled health check for volume /data2/hdds/hdds
2021-01-20 16:36:39,596 INFO org.apache.hadoop.ozone.container.common.volume.ThrottledAsyncChecker: Scheduling a check for /data3/hdds/hdds
2021-01-20 16:36:39,596 INFO org.apache.hadoop.ozone.container.common.volume.HddsVolumeChecker: Scheduled health check for volume /data3/hdds/hdds
2021-01-20 16:36:39,793 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found the following extra column families in existing DB : [block_data, deleted_blocks]
2021-01-20 16:36:39,794 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found the following extra column families in existing DB : [block_data, deleted_blocks]
2021-01-20 16:36:39,794 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found the following extra column families in existing DB : [block_data, deleted_blocks]
2021-01-20 16:36:39,797 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found the following extra column families in existing DB : [block_data, deleted_blocks]
2021-01-20 16:36:39,916 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found the following extra column families in existing DB : [block_data, deleted_blocks]
2021-01-20 16:36:39,916 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found the following extra column families in existing DB : [block_data, deleted_blocks]
2021-01-20 16:36:39,916 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found the following extra column families in existing DB : [block_data, deleted_blocks]
2021-01-20 16:36:39,927 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found the following extra column families in existing DB : [block_data, deleted_blocks]
2021-01-20 16:36:40,001 WARN org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerUtil: Attempt to get an uncached RocksDB handle failed and an instance was retrieved from the cache. This should only happen in tests
2021-01-20 16:36:40,006 WARN org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerUtil: Attempt to get an uncached RocksDB handle failed and an instance was retrieved from the cache. This should only happen in tests
2021-01-20 16:36:40,013 WARN org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerUtil: Attempt to get an uncached RocksDB handle failed and an instance was retrieved from the cache. This should only happen in tests
2021-01-20 16:36:40,018 WARN org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerUtil: Attempt to get an uncached RocksDB handle failed and an instance was retrieved from the cache. This should only happen in tests
```
The name changing of the datanode column families done in HDDS-4369 is actually not backward compatible.
This PR aims to revert the name from deletedBlocks, deleteTxns, blockData to block_data, deleted_blocks, delete_txns.
## What is the link to the Apache JIRA
(Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HDDS-XXXX. Fix a typo in YYY.)
Please replace this section with the link to the Apache JIRA)
## How was this patch tested?
(Please explain how this patch was tested. Ex: unit tests, manual tests)
(If this patch involves UI changes, please attach a screen-shot; otherwise, remove this)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] amaliujia commented on pull request #1824: HDDS-4724. Revert column family names of Datanode db to avoid compatibility issue.
Posted by GitBox <gi...@apache.org>.
amaliujia commented on pull request #1824:
URL: https://github.com/apache/ozone/pull/1824#issuecomment-763915438
LGTM. Thanks for catching this up @GlenGeng
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] runzhiwang commented on pull request #1824: HDDS-4724. Revert column family names of Datanode db to avoid compatibility issue.
Posted by GitBox <gi...@apache.org>.
runzhiwang commented on pull request #1824:
URL: https://github.com/apache/ozone/pull/1824#issuecomment-764191139
@GlenGeng Thanks the patch. @lokeshj1703 @amaliujia Thanks for review. I have merged the patch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] GlenGeng commented on pull request #1824: HDDS-4724. Revert column family names of Datanode db to avoid compatibility issue.
Posted by GitBox <gi...@apache.org>.
GlenGeng commented on pull request #1824:
URL: https://github.com/apache/ozone/pull/1824#issuecomment-764190559
Thanks @errose28 @lokeshj1703 @amaliujia for the review!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] amaliujia commented on pull request #1824: HDDS-4724. Revert column family names of Datanode db to avoid compatibility issue.
Posted by GitBox <gi...@apache.org>.
amaliujia commented on pull request #1824:
URL: https://github.com/apache/ozone/pull/1824#issuecomment-763915438
LGTM. Thanks for catching this up @GlenGeng
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] GlenGeng commented on pull request #1824: HDDS-4724. Revert column family names of Datanode db to avoid compatibility issue.
Posted by GitBox <gi...@apache.org>.
GlenGeng commented on pull request #1824:
URL: https://github.com/apache/ozone/pull/1824#issuecomment-764190559
Thanks @errose28 @lokeshj1703 @amaliujia for the review!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] runzhiwang commented on pull request #1824: HDDS-4724. Revert column family names of Datanode db to avoid compatibility issue.
Posted by GitBox <gi...@apache.org>.
runzhiwang commented on pull request #1824:
URL: https://github.com/apache/ozone/pull/1824#issuecomment-764191139
@GlenGeng Thanks the patch. @lokeshj1703 @amaliujia Thanks for review. I have merged the patch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] runzhiwang merged pull request #1824: HDDS-4724. Revert column family names of Datanode db to avoid compatibility issue.
Posted by GitBox <gi...@apache.org>.
runzhiwang merged pull request #1824:
URL: https://github.com/apache/ozone/pull/1824
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] runzhiwang merged pull request #1824: HDDS-4724. Revert column family names of Datanode db to avoid compatibility issue.
Posted by GitBox <gi...@apache.org>.
runzhiwang merged pull request #1824:
URL: https://github.com/apache/ozone/pull/1824
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] errose28 commented on pull request #1824: HDDS-4724. Revert column family names of Datanode db to avoid compatibility issue.
Posted by GitBox <gi...@apache.org>.
errose28 commented on pull request #1824:
URL: https://github.com/apache/ozone/pull/1824#issuecomment-763553947
Thanks @GlenGeng +1 lgtm. This was my mistake. I am going to investigate why the upgrade acceptance test did not catch this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] GlenGeng commented on pull request #1824: HDDS-4724. Revert column family names of Datanode db to avoid compatibility issue.
Posted by GitBox <gi...@apache.org>.
GlenGeng commented on pull request #1824:
URL: https://github.com/apache/ozone/pull/1824#issuecomment-763468507
cc @lokeshj1703 cc @errose28 Please take a look, thanks !
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org