You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2022/09/28 18:10:50 UTC
[GitHub] [hbase] chrajeshbabu opened a new pull request, #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
chrajeshbabu opened a new pull request, #4805:
URL: https://github.com/apache/hbase/pull/4805
…t cache writers queue by introducing wait time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] wchevreuil commented on a diff in pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
wchevreuil commented on code in PR #4805:
URL: https://github.com/apache/hbase/pull/4805#discussion_r984753778
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java:
##########
@@ -288,6 +292,8 @@ public BucketCache(String ioEngineName, long capacity, int blockSize, int[] buck
this.singleFactor = conf.getFloat(SINGLE_FACTOR_CONFIG_NAME, DEFAULT_SINGLE_FACTOR);
this.multiFactor = conf.getFloat(MULTI_FACTOR_CONFIG_NAME, DEFAULT_MULTI_FACTOR);
this.memoryFactor = conf.getFloat(MEMORY_FACTOR_CONFIG_NAME, DEFAULT_MEMORY_FACTOR);
+ this.queueAdditionWaitTime = conf.getLong(QUEUE_ADDITION_WAIT_TIME,
+ DEFAULT_QUEUE_ADDITION_WAIT_TIME);
Review Comment:
Fair enough.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] chrajeshbabu merged pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
chrajeshbabu merged PR #4805:
URL: https://github.com/apache/hbase/pull/4805
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] wchevreuil commented on a diff in pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
wchevreuil commented on code in PR #4805:
URL: https://github.com/apache/hbase/pull/4805#discussion_r983418390
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java:
##########
@@ -288,6 +292,8 @@ public BucketCache(String ioEngineName, long capacity, int blockSize, int[] buck
this.singleFactor = conf.getFloat(SINGLE_FACTOR_CONFIG_NAME, DEFAULT_SINGLE_FACTOR);
this.multiFactor = conf.getFloat(MULTI_FACTOR_CONFIG_NAME, DEFAULT_MULTI_FACTOR);
this.memoryFactor = conf.getFloat(MEMORY_FACTOR_CONFIG_NAME, DEFAULT_MEMORY_FACTOR);
+ this.queueAdditionWaitTime = conf.getLong(QUEUE_ADDITION_WAIT_TIME,
+ DEFAULT_QUEUE_ADDITION_WAIT_TIME);
Review Comment:
Any cons on setting a default value > 0?
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruAdaptiveBlockCache.java:
##########
@@ -552,6 +552,11 @@ public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean inMemory)
}
}
+ @Override public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean inMemory,
Review Comment:
nit: Make this the default impl in the interface?
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java:
##########
@@ -1341,8 +1341,10 @@ public HFileBlock readBlock(long dataBlockOffset, long onDiskBlockSize, final bo
// Cache the block if necessary
cacheConf.getBlockCache().ifPresent(cache -> {
if (cacheBlock && cacheConf.shouldCacheBlockOnRead(category)) {
+ // Using the wait on cache during compaction and prefetching.
cache.cacheBlock(cacheKey, cacheCompressed ? hfileBlock : unpacked,
- cacheConf.isInMemory());
+ cacheConf.isInMemory(), isCompaction ||
+ (pread && expectedBlockType == null && expectedDataBlockEncoding == null));
Review Comment:
Mind explain the logic behind these particular flags to determine when we are going to trigger the wait? And we don't need this when caching compressed blocks in line#1331 above?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4805:
URL: https://github.com/apache/hbase/pull/4805#issuecomment-1264204439
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 13s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 42s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 17s | master passed |
| +1 :green_heart: | compile | 1m 0s | master passed |
| +1 :green_heart: | shadedjars | 4m 29s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 5s | the patch passed |
| +1 :green_heart: | compile | 0m 56s | the patch passed |
| +1 :green_heart: | javac | 0m 56s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 28s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 35s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 285m 31s | hbase-server in the patch failed. |
| +1 :green_heart: | unit | 0m 33s | hbase-external-blockcache in the patch passed. |
| | | 308m 40s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4805 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux cf4e37636fc2 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 23a56331db |
| Default Java | Temurin-1.8.0_345-b01 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/4/testReport/ |
| Max. process+thread count | 2582 (vs. ulimit of 30000) |
| modules | C: hbase-server hbase-external-blockcache U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/4/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4805:
URL: https://github.com/apache/hbase/pull/4805#issuecomment-1262129627
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 18s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 51s | master passed |
| +1 :green_heart: | compile | 1m 9s | master passed |
| +1 :green_heart: | shadedjars | 4m 29s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 41s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 22s | the patch passed |
| +1 :green_heart: | compile | 1m 9s | the patch passed |
| +1 :green_heart: | javac | 1m 9s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 17s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 43s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 275m 41s | hbase-server in the patch failed. |
| +1 :green_heart: | unit | 1m 19s | hbase-external-blockcache in the patch passed. |
| | | 300m 40s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4805 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux ef9390c37072 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 23a56331db |
| Default Java | Eclipse Adoptium-11.0.16.1+1 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/2/testReport/ |
| Max. process+thread count | 2657 (vs. ulimit of 30000) |
| modules | C: hbase-server hbase-external-blockcache U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4805:
URL: https://github.com/apache/hbase/pull/4805#issuecomment-1264024611
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 28s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 24s | master passed |
| +1 :green_heart: | compile | 1m 7s | master passed |
| +1 :green_heart: | shadedjars | 4m 18s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 40s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 17s | the patch passed |
| +1 :green_heart: | compile | 1m 6s | the patch passed |
| +1 :green_heart: | javac | 1m 6s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 16s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 44s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 271m 59s | hbase-server in the patch passed. |
| +1 :green_heart: | unit | 0m 44s | hbase-external-blockcache in the patch passed. |
| | | 296m 17s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4805 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 259869aec49c 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 23a56331db |
| Default Java | Eclipse Adoptium-11.0.16.1+1 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/3/testReport/ |
| Max. process+thread count | 2578 (vs. ulimit of 30000) |
| modules | C: hbase-server hbase-external-blockcache U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/3/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4805:
URL: https://github.com/apache/hbase/pull/4805#issuecomment-1264201786
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 34s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 37s | master passed |
| +1 :green_heart: | compile | 1m 7s | master passed |
| +1 :green_heart: | shadedjars | 4m 22s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 43s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 21s | the patch passed |
| +1 :green_heart: | compile | 1m 9s | the patch passed |
| +1 :green_heart: | javac | 1m 9s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 19s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 40s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 271m 25s | hbase-server in the patch failed. |
| +1 :green_heart: | unit | 1m 0s | hbase-external-blockcache in the patch passed. |
| | | 296m 3s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4805 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux c1778b20554f 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 23a56331db |
| Default Java | Eclipse Adoptium-11.0.16.1+1 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/4/testReport/ |
| Max. process+thread count | 2442 (vs. ulimit of 30000) |
| modules | C: hbase-server hbase-external-blockcache U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/4/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4805:
URL: https://github.com/apache/hbase/pull/4805#issuecomment-1264033601
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 35s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 7s | master passed |
| +1 :green_heart: | compile | 0m 54s | master passed |
| +1 :green_heart: | shadedjars | 4m 33s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 3s | the patch passed |
| +1 :green_heart: | compile | 0m 53s | the patch passed |
| +1 :green_heart: | javac | 0m 53s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 29s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 35s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 285m 17s | hbase-server in the patch failed. |
| +1 :green_heart: | unit | 0m 35s | hbase-external-blockcache in the patch passed. |
| | | 308m 28s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4805 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux cec12d5e4d14 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 23a56331db |
| Default Java | Temurin-1.8.0_345-b01 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/3/testReport/ |
| Max. process+thread count | 2546 (vs. ulimit of 30000) |
| modules | C: hbase-server hbase-external-blockcache U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/3/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] chrajeshbabu commented on a diff in pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
chrajeshbabu commented on code in PR #4805:
URL: https://github.com/apache/hbase/pull/4805#discussion_r984750812
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java:
##########
@@ -288,6 +292,8 @@ public BucketCache(String ioEngineName, long capacity, int blockSize, int[] buck
this.singleFactor = conf.getFloat(SINGLE_FACTOR_CONFIG_NAME, DEFAULT_SINGLE_FACTOR);
this.multiFactor = conf.getFloat(MULTI_FACTOR_CONFIG_NAME, DEFAULT_MULTI_FACTOR);
this.memoryFactor = conf.getFloat(MEMORY_FACTOR_CONFIG_NAME, DEFAULT_MEMORY_FACTOR);
+ this.queueAdditionWaitTime = conf.getLong(QUEUE_ADDITION_WAIT_TIME,
+ DEFAULT_QUEUE_ADDITION_WAIT_TIME);
Review Comment:
By default don't want to add wait time because memory based offheap will be faster. So it can be completely depending up on the use case like if the file based offheap used eg. cloud ephemeral storages then configuring wait time would be helpful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] chrajeshbabu commented on a diff in pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
chrajeshbabu commented on code in PR #4805:
URL: https://github.com/apache/hbase/pull/4805#discussion_r984747792
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruAdaptiveBlockCache.java:
##########
@@ -552,6 +552,11 @@ public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean inMemory)
}
}
+ @Override public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean inMemory,
Review Comment:
That's good idea and done in new changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4805:
URL: https://github.com/apache/hbase/pull/4805#issuecomment-1261836730
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 6s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 26s | master passed |
| +1 :green_heart: | compile | 2m 30s | master passed |
| +1 :green_heart: | checkstyle | 0m 41s | master passed |
| +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 1m 38s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 7s | the patch passed |
| +1 :green_heart: | compile | 2m 35s | the patch passed |
| +1 :green_heart: | javac | 2m 35s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 38s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | hadoopcheck | 8m 2s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. |
| -1 :x: | spotless | 0m 34s | patch has 69 errors when running spotless:check, run spotless:apply to fix. |
| +1 :green_heart: | spotbugs | 1m 51s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 20s | The patch does not generate ASF License warnings. |
| | | 30m 45s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4805 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile xml |
| uname | Linux 926ac1c64bae 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 23a56331db |
| Default Java | Temurin-1.8.0_345-b01 |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/2/artifact/yetus-general-check/output/patch-spotless.txt |
| Max. process+thread count | 64 (vs. ulimit of 30000) |
| modules | C: hbase-server hbase-external-blockcache U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/2/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.7.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4805:
URL: https://github.com/apache/hbase/pull/4805#issuecomment-1263789938
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 23s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 7s | master passed |
| +1 :green_heart: | compile | 2m 28s | master passed |
| +1 :green_heart: | checkstyle | 0m 38s | master passed |
| +1 :green_heart: | spotless | 0m 38s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 1m 34s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 8s | the patch passed |
| +1 :green_heart: | compile | 2m 29s | the patch passed |
| +1 :green_heart: | javac | 2m 29s | the patch passed |
| -0 :warning: | checkstyle | 0m 30s | hbase-server: The patch generated 4 new + 6 unchanged - 0 fixed = 10 total (was 6) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 0s | The patch has no ill-formed XML file. |
| +1 :green_heart: | hadoopcheck | 7m 47s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. |
| -1 :x: | spotless | 0m 34s | patch has 66 errors when running spotless:check, run spotless:apply to fix. |
| +1 :green_heart: | spotbugs | 1m 48s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 18s | The patch does not generate ASF License warnings. |
| | | 29m 8s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4805 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile xml |
| uname | Linux e23568ce3222 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 23a56331db |
| Default Java | Temurin-1.8.0_345-b01 |
| checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/3/artifact/yetus-general-check/output/patch-spotless.txt |
| Max. process+thread count | 64 (vs. ulimit of 30000) |
| modules | C: hbase-server hbase-external-blockcache U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/3/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.7.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4805:
URL: https://github.com/apache/hbase/pull/4805#issuecomment-1264056296
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 22s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 12s | master passed |
| +1 :green_heart: | compile | 2m 32s | master passed |
| +1 :green_heart: | checkstyle | 0m 36s | master passed |
| +1 :green_heart: | spotless | 0m 39s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 1m 35s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 10s | the patch passed |
| +1 :green_heart: | compile | 2m 30s | the patch passed |
| +1 :green_heart: | javac | 2m 30s | the patch passed |
| -0 :warning: | checkstyle | 0m 32s | hbase-server: The patch generated 4 new + 6 unchanged - 0 fixed = 10 total (was 6) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | hadoopcheck | 7m 42s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. |
| -1 :x: | spotless | 0m 34s | patch has 66 errors when running spotless:check, run spotless:apply to fix. |
| +1 :green_heart: | spotbugs | 1m 50s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 19s | The patch does not generate ASF License warnings. |
| | | 29m 15s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/4/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4805 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile xml |
| uname | Linux 02856156f46a 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 23a56331db |
| Default Java | Temurin-1.8.0_345-b01 |
| checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/4/artifact/yetus-general-check/output/patch-spotless.txt |
| Max. process+thread count | 64 (vs. ulimit of 30000) |
| modules | C: hbase-server hbase-external-blockcache U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/4/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.7.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4805:
URL: https://github.com/apache/hbase/pull/4805#issuecomment-1261328261
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 19s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 33s | master passed |
| +1 :green_heart: | compile | 2m 33s | master passed |
| +1 :green_heart: | checkstyle | 0m 40s | master passed |
| +1 :green_heart: | spotless | 0m 42s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 1m 44s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 10s | the patch passed |
| +1 :green_heart: | compile | 2m 26s | the patch passed |
| +1 :green_heart: | javac | 2m 26s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 38s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 0s | The patch has no ill-formed XML file. |
| +1 :green_heart: | hadoopcheck | 8m 5s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. |
| -1 :x: | spotless | 0m 34s | patch has 69 errors when running spotless:check, run spotless:apply to fix. |
| +1 :green_heart: | spotbugs | 1m 47s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 19s | The patch does not generate ASF License warnings. |
| | | 32m 26s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4805 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile xml |
| uname | Linux ca78f7b7fc7b 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 23a56331db |
| Default Java | Temurin-1.8.0_345-b01 |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/1/artifact/yetus-general-check/output/patch-spotless.txt |
| Max. process+thread count | 64 (vs. ulimit of 30000) |
| modules | C: hbase-server hbase-external-blockcache U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.7.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4805:
URL: https://github.com/apache/hbase/pull/4805#issuecomment-1264240663
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 24s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 11s | master passed |
| +1 :green_heart: | compile | 2m 29s | master passed |
| +1 :green_heart: | checkstyle | 0m 38s | master passed |
| +1 :green_heart: | spotless | 0m 39s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 1m 34s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 7s | the patch passed |
| +1 :green_heart: | compile | 2m 28s | the patch passed |
| +1 :green_heart: | javac | 2m 28s | the patch passed |
| -0 :warning: | checkstyle | 0m 30s | hbase-server: The patch generated 1 new + 6 unchanged - 0 fixed = 7 total (was 6) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | hadoopcheck | 7m 47s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. |
| +1 :green_heart: | spotless | 0m 39s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 1m 48s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 18s | The patch does not generate ASF License warnings. |
| | | 29m 11s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/5/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4805 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile xml |
| uname | Linux 71731c7780be 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 23a56331db |
| Default Java | Temurin-1.8.0_345-b01 |
| checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 64 (vs. ulimit of 30000) |
| modules | C: hbase-server hbase-external-blockcache U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/5/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.7.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4805:
URL: https://github.com/apache/hbase/pull/4805#issuecomment-1261508735
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 24s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 1m 3s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 16s | master passed |
| +1 :green_heart: | compile | 0m 43s | master passed |
| +1 :green_heart: | shadedjars | 3m 59s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 31s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 0s | the patch passed |
| +1 :green_heart: | compile | 0m 44s | the patch passed |
| +1 :green_heart: | javac | 0m 44s | the patch passed |
| +1 :green_heart: | shadedjars | 3m 56s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 30s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 205m 26s | hbase-server in the patch failed. |
| +1 :green_heart: | unit | 0m 19s | hbase-external-blockcache in the patch passed. |
| | | 225m 42s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4805 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux e350439e86ce 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 23a56331db |
| Default Java | Temurin-1.8.0_345-b01 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/1/testReport/ |
| Max. process+thread count | 2375 (vs. ulimit of 30000) |
| modules | C: hbase-server hbase-external-blockcache U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] chrajeshbabu commented on a diff in pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
chrajeshbabu commented on code in PR #4805:
URL: https://github.com/apache/hbase/pull/4805#discussion_r984817428
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java:
##########
@@ -1341,8 +1342,9 @@ public HFileBlock readBlock(long dataBlockOffset, long onDiskBlockSize, final bo
// Cache the block if necessary
cacheConf.getBlockCache().ifPresent(cache -> {
if (cacheBlock && cacheConf.shouldCacheBlockOnRead(category)) {
+ // Using the wait on cache during compaction and prefetching.
cache.cacheBlock(cacheKey, cacheCompressed ? hfileBlock : unpacked,
- cacheConf.isInMemory());
+ cacheConf.isInMemory(), isCompaction || cacheOnly);
Review Comment:
Removed the compaction flog from the check. Yeah it's better not to cache blocks reading during compaction or evict soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4805:
URL: https://github.com/apache/hbase/pull/4805#issuecomment-1264318156
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 29s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 36s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 32s | master passed |
| +1 :green_heart: | compile | 1m 7s | master passed |
| +1 :green_heart: | shadedjars | 4m 24s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 20s | the patch passed |
| +1 :green_heart: | compile | 1m 7s | the patch passed |
| +1 :green_heart: | javac | 1m 7s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 16s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 49s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 328m 24s | hbase-server in the patch failed. |
| +1 :green_heart: | unit | 9m 11s | hbase-external-blockcache in the patch passed. |
| | | 368m 43s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4805 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 386f5719bd8d 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 23a56331db |
| Default Java | Eclipse Adoptium-11.0.16.1+1 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/5/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/5/testReport/ |
| Max. process+thread count | 2686 (vs. ulimit of 30000) |
| modules | C: hbase-server hbase-external-blockcache U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/5/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4805:
URL: https://github.com/apache/hbase/pull/4805#issuecomment-1261533453
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 52s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 56s | master passed |
| +1 :green_heart: | compile | 1m 23s | master passed |
| +1 :green_heart: | shadedjars | 5m 25s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 46s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 19s | the patch passed |
| +1 :green_heart: | compile | 1m 32s | the patch passed |
| +1 :green_heart: | javac | 1m 32s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 40s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 53s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 233m 39s | hbase-server in the patch failed. |
| +1 :green_heart: | unit | 0m 20s | hbase-external-blockcache in the patch passed. |
| | | 261m 33s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4805 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux d89da0505213 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 23a56331db |
| Default Java | Eclipse Adoptium-11.0.16.1+1 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/1/testReport/ |
| Max. process+thread count | 2686 (vs. ulimit of 30000) |
| modules | C: hbase-server hbase-external-blockcache U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4805:
URL: https://github.com/apache/hbase/pull/4805#issuecomment-1262138743
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 11s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 34s | master passed |
| +1 :green_heart: | compile | 0m 56s | master passed |
| +1 :green_heart: | shadedjars | 4m 37s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 40s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 4s | the patch passed |
| +1 :green_heart: | compile | 0m 57s | the patch passed |
| +1 :green_heart: | javac | 0m 57s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 31s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 33s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 286m 37s | hbase-server in the patch failed. |
| +1 :green_heart: | unit | 0m 50s | hbase-external-blockcache in the patch passed. |
| | | 309m 50s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4805 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 5aee39281f23 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 23a56331db |
| Default Java | Temurin-1.8.0_345-b01 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/2/testReport/ |
| Max. process+thread count | 2653 (vs. ulimit of 30000) |
| modules | C: hbase-server hbase-external-blockcache U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] wchevreuil commented on a diff in pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
wchevreuil commented on code in PR #4805:
URL: https://github.com/apache/hbase/pull/4805#discussion_r984787042
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java:
##########
@@ -1341,8 +1342,9 @@ public HFileBlock readBlock(long dataBlockOffset, long onDiskBlockSize, final bo
// Cache the block if necessary
cacheConf.getBlockCache().ifPresent(cache -> {
if (cacheBlock && cacheConf.shouldCacheBlockOnRead(category)) {
+ // Using the wait on cache during compaction and prefetching.
cache.cacheBlock(cacheKey, cacheCompressed ? hfileBlock : unpacked,
- cacheConf.isInMemory());
+ cacheConf.isInMemory(), isCompaction || cacheOnly);
Review Comment:
Sorry to come back/forth here, but I guess it's better not to cache blocks we are reading during compaction, as these may soon require eviction once compaction is finished and it may just waste space in the cache we could use for the blocks we are writing during compaction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4805:
URL: https://github.com/apache/hbase/pull/4805#issuecomment-1264320771
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 33s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 9s | master passed |
| +1 :green_heart: | compile | 0m 55s | master passed |
| +1 :green_heart: | shadedjars | 4m 31s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 34s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 59s | the patch passed |
| +1 :green_heart: | compile | 0m 58s | the patch passed |
| +1 :green_heart: | javac | 0m 58s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 34s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 34s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 351m 21s | hbase-server in the patch failed. |
| +1 :green_heart: | unit | 5m 45s | hbase-external-blockcache in the patch passed. |
| | | 381m 55s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4805 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 98eb9e15cb8d 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 23a56331db |
| Default Java | Temurin-1.8.0_345-b01 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/5/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/5/testReport/ |
| Max. process+thread count | 2496 (vs. ulimit of 30000) |
| modules | C: hbase-server hbase-external-blockcache U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4805/5/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] chrajeshbabu commented on a diff in pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
Posted by GitBox <gi...@apache.org>.
chrajeshbabu commented on code in PR #4805:
URL: https://github.com/apache/hbase/pull/4805#discussion_r984748399
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java:
##########
@@ -1341,8 +1341,10 @@ public HFileBlock readBlock(long dataBlockOffset, long onDiskBlockSize, final bo
// Cache the block if necessary
cacheConf.getBlockCache().ifPresent(cache -> {
if (cacheBlock && cacheConf.shouldCacheBlockOnRead(category)) {
+ // Using the wait on cache during compaction and prefetching.
cache.cacheBlock(cacheKey, cacheCompressed ? hfileBlock : unpacked,
- cacheConf.isInMemory());
+ cacheConf.isInMemory(), isCompaction ||
+ (pread && expectedBlockType == null && expectedDataBlockEncoding == null));
Review Comment:
cacheOnly option is enough for detecting prefetch and isCompaction for compaction operations changed that in new commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org