You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/06/17 09:09:04 UTC
[GitHub] [hbase] WenFeiYi opened a new pull request #1916: HBASE-24546 CloneSnapshotProcedure unlimited retry
WenFeiYi opened a new pull request #1916:
URL: https://github.com/apache/hbase/pull/1916
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] infraio commented on a change in pull request #1916: HBASE-24546 CloneSnapshotProcedure unlimited retry
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1916:
URL: https://github.com/apache/hbase/pull/1916#discussion_r441422204
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CloneSnapshotProcedure.java
##########
@@ -457,6 +457,10 @@ private void postCloneSnapshot(final MasterProcedureEnv env)
// 1. Create Table Descriptor
// using a copy of descriptor, table will be created enabling first
final Path tempTableDir = FSUtils.getTableDir(tempdir, tableDescriptor.getTableName());
+ if (FSUtils.isExists(mfs.getFileSystem(), tempTableDir)) {
+ LOG.warn("temp table dir already exists on disk: {}, will be deleted.", tempTableDir);
+ FSUtils.deleteDirectory(mfs.getFileSystem(), tempTableDir);
+ }
Review comment:
The problem is: the retry will never succeed if the region dirs exist? Add comment here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1916: HBASE-24546 CloneSnapshotProcedure unlimited retry
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1916:
URL: https://github.com/apache/hbase/pull/1916#issuecomment-645360823
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 6s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -0 :warning: | test4tests | 0m 1s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ branch-2.2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 6s | branch-2.2 passed |
| +1 :green_heart: | compile | 0m 55s | branch-2.2 passed |
| +1 :green_heart: | checkstyle | 1m 22s | branch-2.2 passed |
| +1 :green_heart: | shadedjars | 4m 6s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | branch-2.2 passed |
| +0 :ok: | spotbugs | 3m 20s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 17s | branch-2.2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 42s | the patch passed |
| +1 :green_heart: | compile | 0m 54s | the patch passed |
| +1 :green_heart: | javac | 0m 54s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 19s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 4m 17s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 26m 24s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 2.10.0 or 3.1.2 3.2.1. |
| +1 :green_heart: | javadoc | 0m 35s | the patch passed |
| +1 :green_heart: | findbugs | 3m 28s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 168m 31s | hbase-server in the patch failed. |
| +1 :green_heart: | asflicense | 0m 37s | The patch does not generate ASF License warnings. |
| | | 234m 26s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hbase.quotas.TestClusterScopeQuotaThrottle |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1916/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1916 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux d9bb4412386f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1916/out/precommit/personality/provided.sh |
| git revision | branch-2.2 / f3e912b7d8 |
| Default Java | 1.8.0_181 |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1916/1/artifact/out/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1916/1/testReport/ |
| Max. process+thread count | 4380 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1916/1/console |
| versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] WenFeiYi commented on a change in pull request #1916: HBASE-24546 CloneSnapshotProcedure unlimited retry
Posted by GitBox <gi...@apache.org>.
WenFeiYi commented on a change in pull request #1916:
URL: https://github.com/apache/hbase/pull/1916#discussion_r441931716
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CloneSnapshotProcedure.java
##########
@@ -457,6 +457,10 @@ private void postCloneSnapshot(final MasterProcedureEnv env)
// 1. Create Table Descriptor
// using a copy of descriptor, table will be created enabling first
final Path tempTableDir = FSUtils.getTableDir(tempdir, tableDescriptor.getTableName());
+ if (FSUtils.isExists(mfs.getFileSystem(), tempTableDir)) {
+ LOG.warn("temp table dir already exists on disk: {}, will be deleted.", tempTableDir);
+ FSUtils.deleteDirectory(mfs.getFileSystem(), tempTableDir);
+ }
Review comment:
yes, RestoreSnapshotHelper.restoreHdfsRegions will remove the region dirs, but CloneSnapshotProcedure.createFilesystemLayout not allow exist remove. retrying will have the same effect
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1916: HBASE-24546 CloneSnapshotProcedure unlimited retry
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1916:
URL: https://github.com/apache/hbase/pull/1916#issuecomment-654051757
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 11s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ branch-2.2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 15s | branch-2.2 passed |
| +1 :green_heart: | compile | 0m 56s | branch-2.2 passed |
| +1 :green_heart: | checkstyle | 1m 19s | branch-2.2 passed |
| +1 :green_heart: | shadedjars | 4m 0s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | branch-2.2 passed |
| +0 :ok: | spotbugs | 3m 18s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 16s | branch-2.2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 41s | the patch passed |
| +1 :green_heart: | compile | 0m 56s | the patch passed |
| +1 :green_heart: | javac | 0m 56s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 18s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 4m 0s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 25m 12s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 2.10.0 or 3.1.2 3.2.1. |
| +1 :green_heart: | javadoc | 0m 35s | the patch passed |
| +1 :green_heart: | findbugs | 3m 17s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 179m 25s | hbase-server in the patch passed. |
| +1 :green_heart: | asflicense | 0m 33s | The patch does not generate ASF License warnings. |
| | | 243m 29s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1916/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1916 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux ebc0fd97eb30 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1916/out/precommit/personality/provided.sh |
| git revision | branch-2.2 / 4be5fe3154 |
| Default Java | 1.8.0_181 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1916/3/testReport/ |
| Max. process+thread count | 4531 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1916/3/console |
| versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] infraio commented on a change in pull request #1916: HBASE-24546 CloneSnapshotProcedure unlimited retry
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1916:
URL: https://github.com/apache/hbase/pull/1916#discussion_r442589333
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CloneSnapshotProcedure.java
##########
@@ -457,6 +457,10 @@ private void postCloneSnapshot(final MasterProcedureEnv env)
// 1. Create Table Descriptor
// using a copy of descriptor, table will be created enabling first
final Path tempTableDir = FSUtils.getTableDir(tempdir, tableDescriptor.getTableName());
+ if (FSUtils.isExists(mfs.getFileSystem(), tempTableDir)) {
+ LOG.warn("temp table dir already exists on disk: {}, will be deleted.", tempTableDir);
+ FSUtils.deleteDirectory(mfs.getFileSystem(), tempTableDir);
+ }
Review comment:
Sorry. I didn't follow here is why region dir changed? Add more comments to code too?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] infraio merged pull request #1916: HBASE-24546 CloneSnapshotProcedure unlimited retry
Posted by GitBox <gi...@apache.org>.
infraio merged pull request #1916:
URL: https://github.com/apache/hbase/pull/1916
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1916: HBASE-24546 CloneSnapshotProcedure unlimited retry
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1916:
URL: https://github.com/apache/hbase/pull/1916#issuecomment-646641201
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 37s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ branch-2.2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 12s | branch-2.2 passed |
| +1 :green_heart: | compile | 0m 56s | branch-2.2 passed |
| +1 :green_heart: | checkstyle | 1m 20s | branch-2.2 passed |
| +1 :green_heart: | shadedjars | 4m 5s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | branch-2.2 passed |
| +0 :ok: | spotbugs | 3m 23s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 22s | branch-2.2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 40s | the patch passed |
| +1 :green_heart: | compile | 0m 55s | the patch passed |
| +1 :green_heart: | javac | 0m 55s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 15s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 4m 3s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 25m 15s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 2.10.0 or 3.1.2 3.2.1. |
| +1 :green_heart: | javadoc | 0m 34s | the patch passed |
| +1 :green_heart: | findbugs | 3m 16s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 166m 30s | hbase-server in the patch passed. |
| +1 :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. |
| | | 229m 21s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1916/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1916 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 019766351e58 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1916/out/precommit/personality/provided.sh |
| git revision | branch-2.2 / e7f17fe68d |
| Default Java | 1.8.0_181 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1916/2/testReport/ |
| Max. process+thread count | 4484 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1916/2/console |
| versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] WenFeiYi commented on a change in pull request #1916: HBASE-24546 CloneSnapshotProcedure unlimited retry
Posted by GitBox <gi...@apache.org>.
WenFeiYi commented on a change in pull request #1916:
URL: https://github.com/apache/hbase/pull/1916#discussion_r442743034
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CloneSnapshotProcedure.java
##########
@@ -457,6 +457,10 @@ private void postCloneSnapshot(final MasterProcedureEnv env)
// 1. Create Table Descriptor
// using a copy of descriptor, table will be created enabling first
final Path tempTableDir = FSUtils.getTableDir(tempdir, tableDescriptor.getTableName());
+ if (FSUtils.isExists(mfs.getFileSystem(), tempTableDir)) {
+ LOG.warn("temp table dir already exists on disk: {}, will be deleted.", tempTableDir);
+ FSUtils.deleteDirectory(mfs.getFileSystem(), tempTableDir);
+ }
Review comment:
if ok, I will update other commit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] WenFeiYi commented on a change in pull request #1916: HBASE-24546 CloneSnapshotProcedure unlimited retry
Posted by GitBox <gi...@apache.org>.
WenFeiYi commented on a change in pull request #1916:
URL: https://github.com/apache/hbase/pull/1916#discussion_r442743034
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CloneSnapshotProcedure.java
##########
@@ -457,6 +457,10 @@ private void postCloneSnapshot(final MasterProcedureEnv env)
// 1. Create Table Descriptor
// using a copy of descriptor, table will be created enabling first
final Path tempTableDir = FSUtils.getTableDir(tempdir, tableDescriptor.getTableName());
+ if (FSUtils.isExists(mfs.getFileSystem(), tempTableDir)) {
+ LOG.warn("temp table dir already exists on disk: {}, will be deleted.", tempTableDir);
+ FSUtils.deleteDirectory(mfs.getFileSystem(), tempTableDir);
+ }
Review comment:
if ok, I will update other commit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org