You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/06/16 18:46:47 UTC
[GitHub] [hbase] saintstack opened a new pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
saintstack opened a new pull request #1912:
URL: https://github.com/apache/hbase/pull/1912
hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
Don't register as a chore on construction if no coordination state
manager instance (there is no instance when procv2 WAL splitter).
hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitWALManager.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Edit logs.
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionRemoteProcedureBase.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALRemoteProcedure.java
Add proc name rather than rely on default behavior. Add detail to the
toString.
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
Factoring
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
Print the maxLogs... we don't do it any where.
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java
Utility method to strip prefix from wal path.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#discussion_r441837300
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitWALManager.java
##########
@@ -96,7 +97,7 @@ public SplitWALManager(MasterServices master) {
// 2. create corresponding procedures
return createSplitWALProcedures(splittingFiles, crashedServer);
} catch (IOException e) {
- LOG.error("failed to create procedures for splitting logs of {}", crashedServer, e);
+ LOG.error("Failed to create procedures for splitting logs of {}", crashedServer, e);
Review comment:
Let me do.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack merged pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
saintstack merged pull request #1912:
URL: https://github.com/apache/hbase/pull/1912
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#discussion_r441782616
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionRemoteProcedureBase.java
##########
@@ -365,4 +363,17 @@ protected void deserializeStateData(ProcedureStateSerializer serializer) throws
protected void afterReplay(MasterProcedureEnv env) {
getParent(env).attachRemoteProc(this);
}
+
+ @Override public String getProcName() {
Review comment:
proc name should be short. stringclassdetails should be all parameters. Sometimes the name and the stringclassdetails are the same if there is nothing much to show for a procedure.
In this case, I think target server is a fundamental attribute of RegionRemoteProcedure so thought it should be in the proc name. Could move it to the string details.....Let me do that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#issuecomment-645756082
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. |
| -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 14s | branch-2 passed |
| +1 :green_heart: | compile | 0m 59s | branch-2 passed |
| +1 :green_heart: | shadedjars | 5m 32s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | branch-2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 40s | the patch passed |
| +1 :green_heart: | compile | 1m 2s | the patch passed |
| +1 :green_heart: | javac | 1m 2s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 33s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 212m 56s | hbase-server in the patch passed. |
| | | 237m 34s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1912 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 6407fd031f2f 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / 812d1e2bb5 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/3/testReport/ |
| Max. process+thread count | 2342 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/3/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] infraio commented on a change in pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#discussion_r441211036
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionRemoteProcedureBase.java
##########
@@ -365,4 +363,17 @@ protected void deserializeStateData(ProcedureStateSerializer serializer) throws
protected void afterReplay(MasterProcedureEnv env) {
getParent(env).attachRemoteProc(this);
}
+
+ @Override public String getProcName() {
Review comment:
The difference of ProcName and StringClassDetails is? The targetServer should add to toStringClassDetails?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] infraio commented on a change in pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#discussion_r441209011
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitWALManager.java
##########
@@ -96,7 +97,7 @@ public SplitWALManager(MasterServices master) {
// 2. create corresponding procedures
return createSplitWALProcedures(splittingFiles, crashedServer);
} catch (IOException e) {
- LOG.error("failed to create procedures for splitting logs of {}", crashedServer, e);
+ LOG.error("Failed to create procedures for splitting logs of {}", crashedServer, e);
Review comment:
logs => WALs?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#issuecomment-645686333
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
| -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 37s | branch-2 passed |
| +1 :green_heart: | compile | 0m 55s | branch-2 passed |
| +1 :green_heart: | shadedjars | 5m 1s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 38s | branch-2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 14s | the patch passed |
| +1 :green_heart: | compile | 0m 55s | the patch passed |
| +1 :green_heart: | javac | 0m 55s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 59s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 35s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 132m 12s | hbase-server in the patch passed. |
| | | 154m 58s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1912 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 7e573f915928 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / e202c98fb0 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/testReport/ |
| Max. process+thread count | 4416 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#issuecomment-645697980
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 34s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 40s | branch-2 passed |
| +1 :green_heart: | checkstyle | 1m 11s | branch-2 passed |
| +1 :green_heart: | spotbugs | 1m 56s | branch-2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 11s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 8s | hbase-server: The patch generated 0 new + 85 unchanged - 2 fixed = 85 total (was 87) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 37s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. |
| +1 :green_heart: | spotbugs | 2m 7s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. |
| | | 32m 56s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1912 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux f2d8554a8b85 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / 812d1e2bb5 |
| Max. process+thread count | 94 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/3/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] infraio commented on a change in pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#discussion_r441211301
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALProcedure.java
##########
@@ -192,6 +189,21 @@ protected void afterReplay(MasterProcedureEnv env){
if(worker != null){
env.getMasterServices().getSplitWALManager().addUsedSplitWALWorker(worker);
}
+ }
+
+ @Override protected void toStringClassDetails(StringBuilder builder) {
+ builder.append(getProcName());
+ if (this.worker != null) {
+ builder.append(", splitWorker=");
+ builder.append(this.worker);
+ }
+ if (this.retryCounter != null) {
+ builder.append(", retry=");
+ builder.append(this.retryCounter);
+ }
+ }
+ @Override public String getProcName() {
+ return getClass().getSimpleName() + " " + WAL.stripWALsDirPrefix(getWAL());
Review comment:
Why only prefix? How about use the WAL directly?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#issuecomment-645046531
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 33s | Docker mode activated. |
| -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 10s | branch-2 passed |
| +1 :green_heart: | compile | 1m 2s | branch-2 passed |
| +1 :green_heart: | shadedjars | 5m 39s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 38s | branch-2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 39s | the patch passed |
| +1 :green_heart: | compile | 1m 0s | the patch passed |
| +1 :green_heart: | javac | 1m 0s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 29s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 201m 6s | hbase-server in the patch passed. |
| | | 225m 50s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1912 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 70d5164b8356 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / c49df36e85 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/1/testReport/ |
| Max. process+thread count | 2838 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#issuecomment-644968171
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 33s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 13s | branch-2 passed |
| +1 :green_heart: | checkstyle | 1m 19s | branch-2 passed |
| +1 :green_heart: | spotbugs | 2m 14s | branch-2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 38s | the patch passed |
| -0 :warning: | checkstyle | 1m 19s | hbase-server: The patch generated 1 new + 90 unchanged - 2 fixed = 91 total (was 92) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 13m 8s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. |
| +1 :green_heart: | spotbugs | 2m 54s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. |
| | | 38m 26s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1912 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 89e9ef7b7ae3 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / c49df36e85 |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 84 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#discussion_r441837527
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitWALManager.java
##########
@@ -122,7 +123,9 @@ public void deleteSplitWAL(String wal) throws IOException {
public void deleteWALDir(ServerName serverName) throws IOException {
Path splitDir = getWALSplitDir(serverName);
- fs.delete(splitDir, false);
+ if (!fs.delete(splitDir, false)) {
+ LOG.warn("Failed delete {}", splitDir);
Review comment:
Let me fix.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#discussion_r441773327
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALProcedure.java
##########
@@ -192,6 +189,21 @@ protected void afterReplay(MasterProcedureEnv env){
if(worker != null){
env.getMasterServices().getSplitWALManager().addUsedSplitWALWorker(worker);
}
+ }
+
+ @Override protected void toStringClassDetails(StringBuilder builder) {
+ builder.append(getProcName());
+ if (this.worker != null) {
+ builder.append(", splitWorker=");
+ builder.append(this.worker);
+ }
+ if (this.retryCounter != null) {
+ builder.append(", retry=");
+ builder.append(this.retryCounter);
+ }
+ }
+ @Override public String getProcName() {
+ return getClass().getSimpleName() + " " + WAL.stripWALsDirPrefix(getWAL());
Review comment:
hmm... I only have a string... but let me just output the file name instead.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#issuecomment-645647017
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 13s | branch-2 passed |
| +1 :green_heart: | checkstyle | 1m 25s | branch-2 passed |
| +1 :green_heart: | spotbugs | 2m 32s | branch-2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 8s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 25s | hbase-server: The patch generated 0 new + 85 unchanged - 2 fixed = 85 total (was 87) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 12m 48s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. |
| +1 :green_heart: | spotbugs | 2m 14s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. |
| | | 37m 14s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1912 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 7e98a43abb06 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / e202c98fb0 |
| Max. process+thread count | 84 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] infraio commented on a change in pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#discussion_r441209202
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitWALManager.java
##########
@@ -122,7 +123,9 @@ public void deleteSplitWAL(String wal) throws IOException {
public void deleteWALDir(ServerName serverName) throws IOException {
Path splitDir = getWALSplitDir(serverName);
- fs.delete(splitDir, false);
+ if (!fs.delete(splitDir, false)) {
+ LOG.warn("Failed delete {}", splitDir);
Review comment:
Failed to delete WAL dir {}?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#issuecomment-645018005
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 34s | Docker mode activated. |
| -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 14s | branch-2 passed |
| +1 :green_heart: | compile | 1m 5s | branch-2 passed |
| +1 :green_heart: | shadedjars | 5m 46s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 41s | hbase-server in branch-2 failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 51s | the patch passed |
| +1 :green_heart: | compile | 1m 2s | the patch passed |
| +1 :green_heart: | javac | 1m 2s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 44s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 38s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 123m 41s | hbase-server in the patch passed. |
| | | 149m 35s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1912 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 57b5c2b4f455 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / c49df36e85 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/1/testReport/ |
| Max. process+thread count | 4341 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#issuecomment-645684435
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 33s | Docker mode activated. |
| -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 11s | branch-2 passed |
| +1 :green_heart: | compile | 1m 4s | branch-2 passed |
| +1 :green_heart: | shadedjars | 5m 48s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 42s | hbase-server in branch-2 failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 56s | the patch passed |
| +1 :green_heart: | compile | 1m 3s | the patch passed |
| +1 :green_heart: | javac | 1m 3s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 44s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 122m 11s | hbase-server in the patch passed. |
| | | 148m 9s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1912 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 6e5ea84c884e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / e202c98fb0 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/testReport/ |
| Max. process+thread count | 3931 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#discussion_r441769423
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALProcedure.java
##########
@@ -192,6 +189,21 @@ protected void afterReplay(MasterProcedureEnv env){
if(worker != null){
env.getMasterServices().getSplitWALManager().addUsedSplitWALWorker(worker);
}
+ }
+
+ @Override protected void toStringClassDetails(StringBuilder builder) {
+ builder.append(getProcName());
+ if (this.worker != null) {
+ builder.append(", splitWorker=");
+ builder.append(this.worker);
+ }
+ if (this.retryCounter != null) {
+ builder.append(", retry=");
+ builder.append(this.retryCounter);
+ }
+ }
+ @Override public String getProcName() {
+ return getClass().getSimpleName() + " " + WAL.stripWALsDirPrefix(getWAL());
Review comment:
You idea is better. Let me do it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org