You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/12/11 00:44:33 UTC
[GitHub] [ozone] bharatviswa504 opened a new pull request #1688: HDDS-3378. OzoneManager group init failed because of incorrect snapshot directory location.
bharatviswa504 opened a new pull request #1688:
URL: https://github.com/apache/ozone/pull/1688
## What changes were proposed in this pull request?
Make ratis.snapshot.dir do not depend on ozone.om.ratis.storage.dir, so that ozone.om.ratis.storage.dir does not have any other directories.
And also if ratis.storage.dir is not defined, the default we fall back to ozone.metadata.dirs, then we will have ratis.storage.dir as ozone.metadata.dirs + "/ratis".
And for older clusters, the directory will be moved to new ratis.snapshot.dir, if snapshot directory exists in ozone.om.ratis.storage.dir.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-3378
Please replace this section with the link to the Apache JIRA)
## How was this patch tested?
Existing tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 edited a comment on pull request #1688: HDDS-3378. OzoneManager group init failed because of incorrect snapshot directory location.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 edited a comment on pull request #1688:
URL: https://github.com/apache/ozone/pull/1688#issuecomment-742911350
Thank You @hanishakoneru for the offline discussion.
I have posted the PR, as discussed to solve the issue for fresh insatllation and also for upgrades.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] hanishakoneru commented on pull request #1688: HDDS-3378. OzoneManager group init failed because of incorrect snapshot directory location.
Posted by GitBox <gi...@apache.org>.
hanishakoneru commented on pull request #1688:
URL: https://github.com/apache/ozone/pull/1688#issuecomment-749123722
Thanks Bharat for updating the patch.
We can use the Const OM_RATIS_SNAPSHOT_DIR in OzoneManagerRatisServer#getOMRatisSnapshotDirectory() also.
I am +1 with that fixed. Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on pull request #1688: HDDS-3378. OzoneManager group init failed because of incorrect snapshot directory location.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on pull request #1688:
URL: https://github.com/apache/ozone/pull/1688#issuecomment-749742137
Thank You @hanishakoneru for the review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on a change in pull request #1688: HDDS-3378. OzoneManager group init failed because of incorrect snapshot directory location.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #1688:
URL: https://github.com/apache/ozone/pull/1688#discussion_r544518541
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##########
@@ -464,16 +465,33 @@ private OzoneManager(OzoneConfiguration conf) throws IOException,
" must be defined.");
}
OmUtils.createOMDir(omRatisDirectory);
+
// Create Ratis snapshot dir
omRatisSnapshotDir = OmUtils.createOMDir(
OzoneManagerRatisServer.getOMRatisSnapshotDirectory(configuration));
+ // Before starting ratis server check, if previous installation has
+ // snapshot directory in Ratis storage directory if so, move it to
+ // snapshot directory.
+ File[] dirs = new File(omRatisDirectory).listFiles();
+
+ if (dirs != null) {
+ for (File dir : dirs) {
+ if (dir.isDirectory() && dir.getName().equals("snapshot")) {
+ FileUtils.moveDirectory(dir.toPath(), omRatisSnapshotDir.toPath());
+ break;
Review comment:
Yes, it can be, thanks for the suggestion updated it.
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##########
@@ -464,16 +465,33 @@ private OzoneManager(OzoneConfiguration conf) throws IOException,
" must be defined.");
}
OmUtils.createOMDir(omRatisDirectory);
+
// Create Ratis snapshot dir
omRatisSnapshotDir = OmUtils.createOMDir(
OzoneManagerRatisServer.getOMRatisSnapshotDirectory(configuration));
+ // Before starting ratis server check, if previous installation has
+ // snapshot directory in Ratis storage directory if so, move it to
+ // snapshot directory.
+ File[] dirs = new File(omRatisDirectory).listFiles();
+
+ if (dirs != null) {
+ for (File dir : dirs) {
+ if (dir.isDirectory() && dir.getName().equals("snapshot")) {
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] hanishakoneru commented on a change in pull request #1688: HDDS-3378. OzoneManager group init failed because of incorrect snapshot directory location.
Posted by GitBox <gi...@apache.org>.
hanishakoneru commented on a change in pull request #1688:
URL: https://github.com/apache/ozone/pull/1688#discussion_r543736892
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##########
@@ -464,16 +465,33 @@ private OzoneManager(OzoneConfiguration conf) throws IOException,
" must be defined.");
}
OmUtils.createOMDir(omRatisDirectory);
+
// Create Ratis snapshot dir
omRatisSnapshotDir = OmUtils.createOMDir(
OzoneManagerRatisServer.getOMRatisSnapshotDirectory(configuration));
+ // Before starting ratis server check, if previous installation has
+ // snapshot directory in Ratis storage directory if so, move it to
+ // snapshot directory.
+ File[] dirs = new File(omRatisDirectory).listFiles();
+
+ if (dirs != null) {
+ for (File dir : dirs) {
+ if (dir.isDirectory() && dir.getName().equals("snapshot")) {
+ FileUtils.moveDirectory(dir.toPath(), omRatisSnapshotDir.toPath());
+ break;
Review comment:
Instead of listing the files and trying to find if snapshot subdir exists, why don't we construct the snapshot subdir path and check it that exists and move it.
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##########
@@ -464,16 +465,33 @@ private OzoneManager(OzoneConfiguration conf) throws IOException,
" must be defined.");
}
OmUtils.createOMDir(omRatisDirectory);
+
// Create Ratis snapshot dir
omRatisSnapshotDir = OmUtils.createOMDir(
OzoneManagerRatisServer.getOMRatisSnapshotDirectory(configuration));
+ // Before starting ratis server check, if previous installation has
+ // snapshot directory in Ratis storage directory if so, move it to
+ // snapshot directory.
+ File[] dirs = new File(omRatisDirectory).listFiles();
+
+ if (dirs != null) {
+ for (File dir : dirs) {
+ if (dir.isDirectory() && dir.getName().equals("snapshot")) {
Review comment:
Can we define "snapshot" as a constant in OzoneConsts and use that here and in getOMRatisSnapshotDirectory().
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on pull request #1688: HDDS-3378. OzoneManager group init failed because of incorrect snapshot directory location.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on pull request #1688:
URL: https://github.com/apache/ozone/pull/1688#issuecomment-749251584
Thank You @hanishakoneru for the review. Fixed it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 merged pull request #1688: HDDS-3378. OzoneManager group init failed because of incorrect snapshot directory location.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 merged pull request #1688:
URL: https://github.com/apache/ozone/pull/1688
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on pull request #1688: HDDS-3378. OzoneManager group init failed because of incorrect snapshot directory location.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on pull request #1688:
URL: https://github.com/apache/ozone/pull/1688#issuecomment-742911350
Thank You @hanishakoneru for the offline discussion.
I have posted the PR, as discussed to solve the issue for fresh instllation and also for upgrades.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org