You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/06/16 23:12:12 UTC
[GitHub] [hbase] saintstack opened a new pull request #1913: HBASE-24577 Doc WALSplitter classes
saintstack opened a new pull request #1913:
URL: https://github.com/apache/hbase/pull/1913
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ramkrish86 commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
ramkrish86 commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441966355
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/RecoveredEditsOutputSink.java
##########
@@ -81,6 +83,7 @@ private RecoveredEditsWriter getRecoveredEditsWriter(TableName tableName, byte[]
if (ret == null) {
return null;
}
+ LOG.trace("Created {}", ret.path);
Review comment:
Just asking. this can be in DEBUG mode right rather than trace?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441745855
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
##########
@@ -99,12 +102,13 @@ public void append(RegionEntryBuffer buffer) throws IOException {
}
}
- // The key point is create a new writer for each column family, write edits then close writer.
+ // Create a new hfile writer for each column family, write edits then close writer.
String regionName = Bytes.toString(buffer.encodedRegionName);
for (Map.Entry<String, CellSet> cellsEntry : familyCells.entrySet()) {
String familyName = cellsEntry.getKey();
StoreFileWriter writer = createRecoveredHFileWriter(buffer.tableName, regionName,
familySeqIds.get(familyName), familyName, isMetaTable);
+ LOG.trace("Created {}", writer.getPath());
Review comment:
This is to avoid the cost of parameter construction: http://logging.apache.org/log4j/1.2/manual.html#performance
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#issuecomment-645065138
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
| -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 36s | branch-2 passed |
| +1 :green_heart: | compile | 0m 52s | branch-2 passed |
| +1 :green_heart: | shadedjars | 5m 0s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | branch-2 passed |
||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 1m 48s | root in the patch failed. |
| -1 :x: | compile | 0m 51s | hbase-server in the patch failed. |
| -0 :warning: | javac | 0m 51s | hbase-server in the patch failed. |
| -1 :x: | shadedjars | 4m 1s | patch has 20 errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 34s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 0m 52s | hbase-server in the patch failed. |
| | | 20m 3s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1913 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 7d3e2aa17d51 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / c49df36e85 |
| Default Java | 1.8.0_232 |
| mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-jdk8-hadoop2-check/output/patch-mvninstall-root.txt |
| compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-jdk8-hadoop2-check/output/patch-compile-hbase-server.txt |
| javac | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-jdk8-hadoop2-check/output/patch-compile-hbase-server.txt |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-jdk8-hadoop2-check/output/patch-shadedjars.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/testReport/ |
| Max. process+thread count | 88 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441746876
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
##########
@@ -99,12 +102,13 @@ public void append(RegionEntryBuffer buffer) throws IOException {
}
}
- // The key point is create a new writer for each column family, write edits then close writer.
+ // Create a new hfile writer for each column family, write edits then close writer.
String regionName = Bytes.toString(buffer.encodedRegionName);
for (Map.Entry<String, CellSet> cellsEntry : familyCells.entrySet()) {
String familyName = cellsEntry.getKey();
StoreFileWriter writer = createRecoveredHFileWriter(buffer.tableName, regionName,
familySeqIds.get(familyName), familyName, isMetaTable);
+ LOG.trace("Created {}", writer.getPath());
Review comment:
My bad, this is old doc for apache log4j, not slf4j
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441750599
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
##########
@@ -99,12 +102,13 @@ public void append(RegionEntryBuffer buffer) throws IOException {
}
}
- // The key point is create a new writer for each column family, write edits then close writer.
+ // Create a new hfile writer for each column family, write edits then close writer.
String regionName = Bytes.toString(buffer.encodedRegionName);
for (Map.Entry<String, CellSet> cellsEntry : familyCells.entrySet()) {
String familyName = cellsEntry.getKey();
StoreFileWriter writer = createRecoveredHFileWriter(buffer.tableName, regionName,
familySeqIds.get(familyName), familyName, isMetaTable);
+ LOG.trace("Created {}", writer.getPath());
Review comment:
You are right, I just looked into one of the implementors of slf4j (Log4jLoggerAdapter) and it already covers `isTraceEnabled` check:
```
public void trace(String format, Object arg) {
if (isTraceEnabled()) {
FormattingTuple ft = MessageFormatter.format(format, arg);
logger.log(FQCN, traceCapable ? Level.TRACE : Level.DEBUG, ft.getMessage(), ft.getThrowable());
}
}
```
We are following explicit guarding for the cases of Trace and Debug level log at multiple places, all are redundant checks since library already does it for us.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] anoopsjohn commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
anoopsjohn commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441312577
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
##########
@@ -115,7 +119,18 @@
private final String tmpDirName;
+ /**
+ * Split WAL directly to hfiles instead of into intermediary 'recovered.edits' files.
+ */
+ public static final String WAL_SPLIT_TO_HFILE = "hbase.wal.split.to.hfile";
+ public static final boolean DEFAULT_WAL_SPLIT_TO_HFILE = false;
+
+ /**
+ * True if we are to run with bounded amount of writers rather than let the count blossom.
+ * Bounded writing tends to have higher throughput.
+ */
public final static String SPLIT_WRITER_CREATION_BOUNDED = "hbase.split.writer.creation.bounded";
Review comment:
This defaults to false right?
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/RecoveredEditsOutputSink.java
##########
@@ -26,20 +26,20 @@
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Future;
-
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.io.MultipleIOException;
import org.apache.yetus.audience.InterfaceAudience;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-
import org.apache.hbase.thirdparty.com.google.common.collect.Lists;
/**
* Class that manages the output streams from the log splitting process.
- * Every region only has one recovered edits.
+ * Every region only has one recovered edits file.
Review comment:
Per one WAL file split, every region will have one recovered edits file. If another WAL file also get split, we will have one more right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#issuecomment-645714547
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 53s | Docker mode activated. |
| -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 32s | branch-2 passed |
| +1 :green_heart: | compile | 1m 4s | branch-2 passed |
| +1 :green_heart: | shadedjars | 5m 44s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 40s | branch-2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 50s | the patch passed |
| +1 :green_heart: | compile | 1m 1s | the patch passed |
| +1 :green_heart: | javac | 1m 1s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 38s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 38s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 199m 35s | hbase-server in the patch passed. |
| | | 226m 41s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1913 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 459e52911708 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / e202c98fb0 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/testReport/ |
| Max. process+thread count | 2541 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#issuecomment-645066622
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 12s | Docker mode activated. |
| -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 52s | branch-2 passed |
| +1 :green_heart: | compile | 1m 10s | branch-2 passed |
| +1 :green_heart: | shadedjars | 6m 29s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 42s | hbase-server in branch-2 failed. |
||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 2m 31s | root in the patch failed. |
| -1 :x: | compile | 1m 4s | hbase-server in the patch failed. |
| -0 :warning: | javac | 1m 4s | hbase-server in the patch failed. |
| -1 :x: | shadedjars | 5m 5s | patch has 20 errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 41s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| -1 :x: | unit | 1m 4s | hbase-server in the patch failed. |
| | | 26m 3s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1913 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux e2da9475ab97 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / c49df36e85 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-jdk11-hadoop3-check/output/patch-mvninstall-root.txt |
| compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt |
| javac | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/testReport/ |
| Max. process+thread count | 91 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#issuecomment-645710536
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 16s | Docker mode activated. |
| -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 43s | branch-2 passed |
| +1 :green_heart: | compile | 1m 9s | branch-2 passed |
| +1 :green_heart: | shadedjars | 6m 25s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 42s | hbase-server in branch-2 failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 23s | the patch passed |
| +1 :green_heart: | compile | 1m 8s | the patch passed |
| +1 :green_heart: | javac | 1m 8s | the patch passed |
| +1 :green_heart: | shadedjars | 6m 24s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 40s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 183m 34s | hbase-server in the patch passed. |
| | | 212m 21s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1913 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 438b448c14c7 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / e202c98fb0 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/testReport/ |
| Max. process+thread count | 3206 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441725369
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
##########
@@ -115,7 +119,18 @@
private final String tmpDirName;
+ /**
+ * Split WAL directly to hfiles instead of into intermediary 'recovered.edits' files.
+ */
+ public static final String WAL_SPLIT_TO_HFILE = "hbase.wal.split.to.hfile";
+ public static final boolean DEFAULT_WAL_SPLIT_TO_HFILE = false;
+
+ /**
+ * True if we are to run with bounded amount of writers rather than let the count blossom.
+ * Bounded writing tends to have higher throughput.
+ */
public final static String SPLIT_WRITER_CREATION_BOUNDED = "hbase.split.writer.creation.bounded";
Review comment:
I note that it defaults to 'false' but man, this should be 'true'. Let me test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441720655
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
##########
@@ -99,12 +102,13 @@ public void append(RegionEntryBuffer buffer) throws IOException {
}
}
- // The key point is create a new writer for each column family, write edits then close writer.
+ // Create a new hfile writer for each column family, write edits then close writer.
String regionName = Bytes.toString(buffer.encodedRegionName);
for (Map.Entry<String, CellSet> cellsEntry : familyCells.entrySet()) {
String familyName = cellsEntry.getKey();
StoreFileWriter writer = createRecoveredHFileWriter(buffer.tableName, regionName,
familySeqIds.get(familyName), familyName, isMetaTable);
+ LOG.trace("Created {}", writer.getPath());
Review comment:
Why you say? The slf4j doesn't do any work creating log string if we are not at log TRACE level.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack merged pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
saintstack merged pull request #1913:
URL: https://github.com/apache/hbase/pull/1913
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441447292
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
##########
@@ -118,6 +122,7 @@ public void append(RegionEntryBuffer buffer) throws IOException {
openingWritersNum.decrementAndGet();
} finally {
writer.close();
+ LOG.trace("Closed {}, edits={}", writer.getPath(), familyCells.size());
Review comment:
same here
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
##########
@@ -99,12 +102,13 @@ public void append(RegionEntryBuffer buffer) throws IOException {
}
}
- // The key point is create a new writer for each column family, write edits then close writer.
+ // Create a new hfile writer for each column family, write edits then close writer.
String regionName = Bytes.toString(buffer.encodedRegionName);
for (Map.Entry<String, CellSet> cellsEntry : familyCells.entrySet()) {
String familyName = cellsEntry.getKey();
StoreFileWriter writer = createRecoveredHFileWriter(buffer.tableName, regionName,
familySeqIds.get(familyName), familyName, isMetaTable);
+ LOG.trace("Created {}", writer.getPath());
Review comment:
Good to guard with `LOG.isTraceEnabled()`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441721127
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/RecoveredEditsOutputSink.java
##########
@@ -26,20 +26,20 @@
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Future;
-
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.io.MultipleIOException;
import org.apache.yetus.audience.InterfaceAudience;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-
import org.apache.hbase.thirdparty.com.google.common.collect.Lists;
/**
* Class that manages the output streams from the log splitting process.
- * Every region only has one recovered edits.
+ * Every region only has one recovered edits file.
Review comment:
Let me add qualification here.
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
##########
@@ -233,7 +247,7 @@ public static boolean splitLogFile(Path walDir, FileStatus logfile, FileSystem w
}
/**
- * log splitting implementation, splits one log file.
+ * Log splitting implementation, splits one log file.
Review comment:
Let me change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#issuecomment-645658012
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 53s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 32s | branch-2 passed |
| +1 :green_heart: | checkstyle | 1m 20s | branch-2 passed |
| +1 :green_heart: | spotbugs | 2m 17s | branch-2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 43s | the patch passed |
| -0 :warning: | checkstyle | 1m 19s | hbase-server: The patch generated 1 new + 0 unchanged - 1 fixed = 1 total (was 1) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 13m 28s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. |
| +1 :green_heart: | spotbugs | 2m 54s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| | | 40m 53s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1913 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux b7e71b0e79c1 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / e202c98fb0 |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 84 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#issuecomment-645612480
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
| -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 12s | branch-2 passed |
| +1 :green_heart: | compile | 1m 3s | branch-2 passed |
| +1 :green_heart: | shadedjars | 5m 47s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 42s | hbase-server in branch-2 failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 53s | the patch passed |
| +1 :green_heart: | compile | 1m 3s | the patch passed |
| +1 :green_heart: | javac | 1m 3s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 45s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 40s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 123m 31s | hbase-server in the patch passed. |
| | | 149m 25s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1913 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux a5b94087c197 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / e202c98fb0 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/testReport/ |
| Max. process+thread count | 4801 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#issuecomment-645640215
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 29s | Docker mode activated. |
| -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 52s | branch-2 passed |
| +1 :green_heart: | compile | 0m 56s | branch-2 passed |
| +1 :green_heart: | shadedjars | 5m 27s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 35s | branch-2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 27s | the patch passed |
| +1 :green_heart: | compile | 0m 57s | the patch passed |
| +1 :green_heart: | javac | 0m 57s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 25s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 35s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 187m 27s | hbase-server in the patch passed. |
| | | 211m 0s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1913 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux ee6fc56bbbb5 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / e202c98fb0 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/testReport/ |
| Max. process+thread count | 2575 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441840561
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
##########
@@ -99,12 +102,13 @@ public void append(RegionEntryBuffer buffer) throws IOException {
}
}
- // The key point is create a new writer for each column family, write edits then close writer.
+ // Create a new hfile writer for each column family, write edits then close writer.
String regionName = Bytes.toString(buffer.encodedRegionName);
for (Map.Entry<String, CellSet> cellsEntry : familyCells.entrySet()) {
String familyName = cellsEntry.getKey();
StoreFileWriter writer = createRecoveredHFileWriter(buffer.tableName, regionName,
familySeqIds.get(familyName), familyName, isMetaTable);
+ LOG.trace("Created {}", writer.getPath());
Review comment:
Thanks @virajjasani
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] infraio commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441314474
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/RecoveredEditsOutputSink.java
##########
@@ -26,20 +26,20 @@
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Future;
-
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.io.MultipleIOException;
import org.apache.yetus.audience.InterfaceAudience;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-
import org.apache.hbase.thirdparty.com.google.common.collect.Lists;
/**
* Class that manages the output streams from the log splitting process.
- * Every region only has one recovered edits.
+ * Every region only has one recovered edits file.
Review comment:
Yes. Right.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] bsglz commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
bsglz commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441929197
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
##########
@@ -99,12 +102,13 @@ public void append(RegionEntryBuffer buffer) throws IOException {
}
}
- // The key point is create a new writer for each column family, write edits then close writer.
+ // Create a new hfile writer for each column family, write edits then close writer.
String regionName = Bytes.toString(buffer.encodedRegionName);
for (Map.Entry<String, CellSet> cellsEntry : familyCells.entrySet()) {
String familyName = cellsEntry.getKey();
StoreFileWriter writer = createRecoveredHFileWriter(buffer.tableName, regionName,
familySeqIds.get(familyName), familyName, isMetaTable);
+ LOG.trace("Created {}", writer.getPath());
Review comment:
IIRC, this extra check could be used only if the parameter self will cause a heavy call.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] infraio commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441208503
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
##########
@@ -233,7 +247,7 @@ public static boolean splitLogFile(Path walDir, FileStatus logfile, FileSystem w
}
/**
- * log splitting implementation, splits one log file.
+ * Log splitting implementation, splits one log file.
Review comment:
Log => WAL?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#issuecomment-645552689
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 37s | branch-2 passed |
| +1 :green_heart: | checkstyle | 1m 9s | branch-2 passed |
| +1 :green_heart: | spotbugs | 1m 59s | branch-2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 10s | the patch passed |
| -0 :warning: | checkstyle | 1m 7s | hbase-server: The patch generated 1 new + 0 unchanged - 1 fixed = 1 total (was 1) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 26s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. |
| +1 :green_heart: | spotbugs | 2m 4s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| | | 32m 35s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1913 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux ff480c5257e3 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / e202c98fb0 |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 94 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#issuecomment-645064782
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 11s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 57s | branch-2 passed |
| +1 :green_heart: | checkstyle | 1m 16s | branch-2 passed |
| +1 :green_heart: | spotbugs | 2m 9s | branch-2 passed |
||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 2m 0s | root in the patch failed. |
| -0 :warning: | checkstyle | 1m 13s | hbase-server: The patch generated 1 new + 0 unchanged - 1 fixed = 1 total (was 1) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| -1 :x: | hadoopcheck | 2m 22s | The patch causes 20 errors with Hadoop v3.1.2. |
| -1 :x: | hadoopcheck | 4m 42s | The patch causes 20 errors with Hadoop v3.2.1. |
| -1 :x: | spotbugs | 0m 40s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. |
| | | 18m 55s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1913 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux c61f8505d325 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / c49df36e85 |
| mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-general-check/output/patch-mvninstall-root.txt |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-general-check/output/patch-javac-3.1.2.txt |
| hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-general-check/output/patch-javac-3.2.1.txt |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/artifact/yetus-general-check/output/patch-spotbugs-hbase-server.txt |
| Max. process+thread count | 84 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441720952
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
##########
@@ -115,7 +119,18 @@
private final String tmpDirName;
+ /**
+ * Split WAL directly to hfiles instead of into intermediary 'recovered.edits' files.
+ */
+ public static final String WAL_SPLIT_TO_HFILE = "hbase.wal.split.to.hfile";
+ public static final boolean DEFAULT_WAL_SPLIT_TO_HFILE = false;
+
+ /**
+ * True if we are to run with bounded amount of writers rather than let the count blossom.
+ * Bounded writing tends to have higher throughput.
+ */
public final static String SPLIT_WRITER_CREATION_BOUNDED = "hbase.split.writer.creation.bounded";
Review comment:
Let me add this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org