You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "wchevreuil (via GitHub)" <gi...@apache.org> on 2025/08/05 19:39:02 UTC
[PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
wchevreuil opened a new pull request, #7192:
URL: https://github.com/apache/hbase/pull/7192
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7192:
URL: https://github.com/apache/hbase/pull/7192#issuecomment-3165770555
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
|||| _ Prechecks _ |
|||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 10s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 17s | | master passed |
| +1 :green_heart: | compile | 1m 17s | | master passed |
| +1 :green_heart: | javadoc | 0m 44s | | master passed |
| +1 :green_heart: | shadedjars | 6m 4s | | branch has no errors when building our shaded downstream artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 5s | | the patch passed |
| +1 :green_heart: | compile | 1m 17s | | the patch passed |
| +1 :green_heart: | javac | 1m 17s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 43s | | the patch passed |
| +1 :green_heart: | shadedjars | 6m 3s | | patch has no errors when building our shaded downstream artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 14s | | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 258m 14s | | hbase-server in the patch passed. |
| | | 290m 4s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7192 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux e90e594a2553 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 3f758842dd1d6a37543217d6f9ad9fed5428afbd |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/2/testReport/ |
| Max. process+thread count | 5099 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/2/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "wchevreuil (via GitHub)" <gi...@apache.org>.
wchevreuil commented on PR #7192:
URL: https://github.com/apache/hbase/pull/7192#issuecomment-3229448315
Any further comments, @taklwu ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7192:
URL: https://github.com/apache/hbase/pull/7192#issuecomment-3166311500
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
|||| _ Prechecks _ |
|||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 26s | | master passed |
| +1 :green_heart: | compile | 1m 21s | | master passed |
| +1 :green_heart: | javadoc | 0m 46s | | master passed |
| +1 :green_heart: | shadedjars | 6m 18s | | branch has no errors when building our shaded downstream artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 13s | | the patch passed |
| +1 :green_heart: | compile | 1m 21s | | the patch passed |
| +1 :green_heart: | javac | 1m 21s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 45s | | the patch passed |
| +1 :green_heart: | shadedjars | 6m 9s | | patch has no errors when building our shaded downstream artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 16s | | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 222m 34s | | hbase-server in the patch passed. |
| | | 254m 0s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7192 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux 746c2270a385 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 7b7728d50e6e6a0e4b9ff26d8e02167ffa59cb28 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/3/testReport/ |
| Max. process+thread count | 5328 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/3/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "wchevreuil (via GitHub)" <gi...@apache.org>.
wchevreuil commented on PR #7192:
URL: https://github.com/apache/hbase/pull/7192#issuecomment-3223414864
> few questions below and inlines, otherwise LGTM
>
> 1. according from #5829, there is a file TestPrefetch.java in the diff, where did we remove it from this PR?
That changed from #5829 was removed when resolving conflicts from HBASE-28804.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "taklwu (via GitHub)" <gi...@apache.org>.
taklwu commented on code in PR #7192:
URL: https://github.com/apache/hbase/pull/7192#discussion_r2301812207
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java:
##########
@@ -1378,7 +1379,8 @@ public HFileBlock readBlock(long dataBlockOffset, long onDiskBlockSize, final bo
}
BlockType.BlockCategory category = hfileBlock.getBlockType().getCategory();
final boolean cacheCompressed = cacheConf.shouldCacheCompressed(category);
- final boolean cacheOnRead = cacheConf.shouldCacheBlockOnRead(category);
+ final boolean cacheOnRead =
Review Comment:
yeah, sorry I didn't explain my comment with links,
about the `cacheOnRead` , in the old PR #5905 , `cacheOnRead` has been used within the if block `if (cacheOnly && cacheCompressed && cacheOnRead) {` https://github.com/apache/hbase/pull/5905/files#diff-c910a49e7d962e49b199d22f38e1cd78d867351208b86bffdd56ffb9aa1aa596R1358 , but in this PR, we don't see those usage of `cacheOnRead`, did I miss anything?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7192:
URL: https://github.com/apache/hbase/pull/7192#issuecomment-3225669363
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 32s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any anti-patterns. |
|||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 38s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 36s | | master passed |
| +1 :green_heart: | compile | 4m 6s | | master passed |
| +1 :green_heart: | checkstyle | 0m 55s | | master passed |
| +1 :green_heart: | spotbugs | 2m 19s | | master passed |
| +1 :green_heart: | spotless | 0m 52s | | branch has no errors when running spotless:check. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 5s | | the patch passed |
| +1 :green_heart: | compile | 4m 3s | | the patch passed |
| +1 :green_heart: | javac | 4m 3s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 40s | [/results-checkstyle-hbase-server.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/4/artifact/yetus-general-check/output/results-checkstyle-hbase-server.txt) | hbase-server: The patch generated 16 new + 30 unchanged - 0 fixed = 46 total (was 30) |
| +1 :green_heart: | spotbugs | 2m 30s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 11m 48s | | Patch does not cause any errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 48s | | patch has no errors when running spotless:check. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 20s | | The patch does not generate ASF License warnings. |
| | | 44m 30s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/4/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7192 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 9f3ff94c352c 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 6fdffaff3d80ee306ab022fd01a7bdcf7c6d7cfb |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 85 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/4/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7192:
URL: https://github.com/apache/hbase/pull/7192#issuecomment-3165100211
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any anti-patterns. |
|||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 41s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 27s | | master passed |
| +1 :green_heart: | compile | 4m 3s | | master passed |
| +1 :green_heart: | checkstyle | 0m 52s | | master passed |
| +1 :green_heart: | spotbugs | 2m 6s | | master passed |
| +1 :green_heart: | spotless | 0m 47s | | branch has no errors when running spotless:check. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 56s | | the patch passed |
| +1 :green_heart: | compile | 3m 59s | | the patch passed |
| +1 :green_heart: | javac | 3m 59s | | the patch passed |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 38s | [/results-checkstyle-hbase-server.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/2/artifact/yetus-general-check/output/results-checkstyle-hbase-server.txt) | hbase-server: The patch generated 16 new + 30 unchanged - 0 fixed = 46 total (was 30) |
| +1 :green_heart: | spotbugs | 2m 16s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 11m 38s | | Patch does not cause any errors with Hadoop 3.3.6 3.4.0. |
| -1 :x: | spotless | 0m 37s | | patch has 22 errors when running spotless:check, run spotless:apply to fix. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 18s | | The patch does not generate ASF License warnings. |
| | | 42m 54s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7192 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 82e74d8ba000 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 3f758842dd1d6a37543217d6f9ad9fed5428afbd |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/2/artifact/yetus-general-check/output/patch-spotless.txt |
| Max. process+thread count | 84 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/2/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "wchevreuil (via GitHub)" <gi...@apache.org>.
wchevreuil commented on code in PR #7192:
URL: https://github.com/apache/hbase/pull/7192#discussion_r2300492200
##########
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestBytesReadFromFs.java:
##########
@@ -268,6 +268,10 @@ private void readLoadOnOpenDataSection(Path path, boolean hasBloomFilters) throw
CacheConfig cacheConfig = new CacheConfig(conf);
HFile.Reader reader = new HFilePreadReader(readerContext, hfile, cacheConfig, conf);
+ // Since HBASE-28466, we call fileInfo.initMetaAndIndex inside HFilePreadReader,
+ // which reads some blocks and increment the counters, so we need to reset it here.
+ ThreadLocalServerSideScanMetrics.getBytesReadFromFsAndReset();
+ ThreadLocalServerSideScanMetrics.getBlockReadOpsCountAndReset();
Review Comment:
Yeah, these are changes introduced in this PR last commit to fix the tests for HBASE-28466 changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "taklwu (via GitHub)" <gi...@apache.org>.
taklwu commented on code in PR #7192:
URL: https://github.com/apache/hbase/pull/7192#discussion_r2308351392
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java:
##########
@@ -1378,7 +1379,8 @@ public HFileBlock readBlock(long dataBlockOffset, long onDiskBlockSize, final bo
}
BlockType.BlockCategory category = hfileBlock.getBlockType().getCategory();
final boolean cacheCompressed = cacheConf.shouldCacheCompressed(category);
- final boolean cacheOnRead = cacheConf.shouldCacheBlockOnRead(category);
+ final boolean cacheOnRead =
Review Comment:
thanks for fixing it. LGTM now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7192:
URL: https://github.com/apache/hbase/pull/7192#issuecomment-3165994219
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any anti-patterns. |
|||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 10s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 9s | | master passed |
| +1 :green_heart: | compile | 4m 0s | | master passed |
| +1 :green_heart: | checkstyle | 0m 52s | | master passed |
| +1 :green_heart: | spotbugs | 2m 3s | | master passed |
| +1 :green_heart: | spotless | 0m 49s | | branch has no errors when running spotless:check. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 1s | | the patch passed |
| +1 :green_heart: | compile | 3m 57s | | the patch passed |
| +1 :green_heart: | javac | 3m 57s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 38s | [/results-checkstyle-hbase-server.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/3/artifact/yetus-general-check/output/results-checkstyle-hbase-server.txt) | hbase-server: The patch generated 16 new + 30 unchanged - 0 fixed = 46 total (was 30) |
| +1 :green_heart: | spotbugs | 2m 18s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 11m 33s | | Patch does not cause any errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 43s | | patch has no errors when running spotless:check. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 18s | | The patch does not generate ASF License warnings. |
| | | 42m 10s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7192 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 6e1e63b673f7 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 7b7728d50e6e6a0e4b9ff26d8e02167ffa59cb28 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 85 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/3/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "wchevreuil (via GitHub)" <gi...@apache.org>.
wchevreuil commented on code in PR #7192:
URL: https://github.com/apache/hbase/pull/7192#discussion_r2300457733
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java:
##########
@@ -1378,7 +1379,8 @@ public HFileBlock readBlock(long dataBlockOffset, long onDiskBlockSize, final bo
}
BlockType.BlockCategory category = hfileBlock.getBlockType().getCategory();
final boolean cacheCompressed = cacheConf.shouldCacheCompressed(category);
- final boolean cacheOnRead = cacheConf.shouldCacheBlockOnRead(category);
+ final boolean cacheOnRead =
Review Comment:
>this part is different from #5905 , where cacheOnRead has been defined but it does not use in line#1391 and line#1405, is it expected?
Sorry, ain't following you. This seems same code we added in [PR 5905, line #1351 of HFileReaderImpl.]([HFileReaderImpl.java](https://github.com/apache/hbase/pull/5905/files#diff-c910a49e7d962e49b199d22f38e1cd78d867351208b86bffdd56ffb9aa1aa596))
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "wchevreuil (via GitHub)" <gi...@apache.org>.
wchevreuil merged PR #7192:
URL: https://github.com/apache/hbase/pull/7192
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7192:
URL: https://github.com/apache/hbase/pull/7192#issuecomment-3156553793
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 30s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
|||| _ Prechecks _ |
|||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 9s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 16s | | master passed |
| +1 :green_heart: | compile | 1m 16s | | master passed |
| +1 :green_heart: | javadoc | 0m 43s | | master passed |
| +1 :green_heart: | shadedjars | 6m 3s | | branch has no errors when building our shaded downstream artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 16s | | the patch passed |
| +1 :green_heart: | compile | 1m 21s | | the patch passed |
| +1 :green_heart: | javac | 1m 21s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 45s | | the patch passed |
| +1 :green_heart: | shadedjars | 6m 13s | | patch has no errors when building our shaded downstream artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 19s | | hbase-common in the patch passed. |
| -1 :x: | unit | 15m 56s | [/patch-unit-hbase-server.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/1/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-server.txt) | hbase-server in the patch failed. |
| | | 44m 14s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7192 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux b00ee70f9fcc 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 98a0296a4e172906a8dfd5341b04e3dec74980e2 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/1/testReport/ |
| Max. process+thread count | 1864 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/1/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "wchevreuil (via GitHub)" <gi...@apache.org>.
wchevreuil commented on code in PR #7192:
URL: https://github.com/apache/hbase/pull/7192#discussion_r2300472107
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java:
##########
@@ -1378,7 +1379,8 @@ public HFileBlock readBlock(long dataBlockOffset, long onDiskBlockSize, final bo
}
BlockType.BlockCategory category = hfileBlock.getBlockType().getCategory();
final boolean cacheCompressed = cacheConf.shouldCacheCompressed(category);
- final boolean cacheOnRead = cacheConf.shouldCacheBlockOnRead(category);
+ final boolean cacheOnRead =
Review Comment:
>in other words, is these two APIs of cacheConf.shouldCacheBlockOnRead(category) and cacheConf.shouldCacheBlockOnRead(category, getHFileInfo(), conf) the same?
Those are not the same. The former just checks for the block type, whilst the latter performs additional checks to figure if the data block is already cached, in order to avoid redundant caching from file system.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "wchevreuil (via GitHub)" <gi...@apache.org>.
wchevreuil commented on code in PR #7192:
URL: https://github.com/apache/hbase/pull/7192#discussion_r2301980456
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java:
##########
@@ -1378,7 +1379,8 @@ public HFileBlock readBlock(long dataBlockOffset, long onDiskBlockSize, final bo
}
BlockType.BlockCategory category = hfileBlock.getBlockType().getCategory();
final boolean cacheCompressed = cacheConf.shouldCacheCompressed(category);
- final boolean cacheOnRead = cacheConf.shouldCacheBlockOnRead(category);
+ final boolean cacheOnRead =
Review Comment:
Ok, I think I got it now. This is probably due to the rebase on top of master branch. Master branch had this HBASE-28596, which was missing on the feature branch when PR #5905 was merged. So when resolving this conflict in the rebase, I simply picked the master version.
Reading this now, seems wrong to me, we should do as PR #595 was doing. I'm going to update it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7192:
URL: https://github.com/apache/hbase/pull/7192#issuecomment-3156556008
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any anti-patterns. |
|||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 10s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 22s | | master passed |
| +1 :green_heart: | compile | 4m 18s | | master passed |
| +1 :green_heart: | checkstyle | 0m 52s | | master passed |
| +1 :green_heart: | spotbugs | 2m 12s | | master passed |
| +1 :green_heart: | spotless | 0m 50s | | branch has no errors when running spotless:check. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 7s | | the patch passed |
| +1 :green_heart: | compile | 4m 3s | | the patch passed |
| +1 :green_heart: | javac | 4m 3s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 39s | [/results-checkstyle-hbase-server.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/1/artifact/yetus-general-check/output/results-checkstyle-hbase-server.txt) | hbase-server: The patch generated 16 new + 30 unchanged - 0 fixed = 46 total (was 30) |
| +1 :green_heart: | spotbugs | 2m 16s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 11m 30s | | Patch does not cause any errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 44s | | patch has no errors when running spotless:check. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 19s | | The patch does not generate ASF License warnings. |
| | | 43m 27s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7192 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 6fb30c2e3b3c 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 98a0296a4e172906a8dfd5341b04e3dec74980e2 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 84 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/1/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "taklwu (via GitHub)" <gi...@apache.org>.
taklwu commented on code in PR #7192:
URL: https://github.com/apache/hbase/pull/7192#discussion_r2274073504
##########
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestBytesReadFromFs.java:
##########
@@ -268,6 +268,10 @@ private void readLoadOnOpenDataSection(Path path, boolean hasBloomFilters) throw
CacheConfig cacheConfig = new CacheConfig(conf);
HFile.Reader reader = new HFilePreadReader(readerContext, hfile, cacheConfig, conf);
+ // Since HBASE-28466, we call fileInfo.initMetaAndIndex inside HFilePreadReader,
+ // which reads some blocks and increment the counters, so we need to reset it here.
+ ThreadLocalServerSideScanMetrics.getBytesReadFromFsAndReset();
+ ThreadLocalServerSideScanMetrics.getBlockReadOpsCountAndReset();
Review Comment:
it's not part of #7124 , can you point out where does this come from ? new changes to fix tests when HBASE-28466 is included?
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java:
##########
@@ -1378,7 +1379,8 @@ public HFileBlock readBlock(long dataBlockOffset, long onDiskBlockSize, final bo
}
BlockType.BlockCategory category = hfileBlock.getBlockType().getCategory();
final boolean cacheCompressed = cacheConf.shouldCacheCompressed(category);
- final boolean cacheOnRead = cacheConf.shouldCacheBlockOnRead(category);
+ final boolean cacheOnRead =
Review Comment:
this part is different from #5905 , where `cacheOnRead` has been defined but it does not use in line#1391 and line#1405, is it expected?
in other words, is these two APIs of `cacheConf.shouldCacheBlockOnRead(category)` and `cacheConf.shouldCacheBlockOnRead(category, getHFileInfo(), conf)` the same?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28463 Time Based Priority for BucketCache [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7192:
URL: https://github.com/apache/hbase/pull/7192#issuecomment-3226244153
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 29s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
|||| _ Prechecks _ |
|||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 10s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 19s | | master passed |
| +1 :green_heart: | compile | 1m 17s | | master passed |
| +1 :green_heart: | javadoc | 0m 45s | | master passed |
| +1 :green_heart: | shadedjars | 6m 9s | | branch has no errors when building our shaded downstream artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 7s | | the patch passed |
| +1 :green_heart: | compile | 1m 17s | | the patch passed |
| +1 :green_heart: | javac | 1m 17s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 42s | | the patch passed |
| +1 :green_heart: | shadedjars | 6m 6s | | patch has no errors when building our shaded downstream artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 15s | | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 234m 39s | | hbase-server in the patch passed. |
| | | 265m 51s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7192 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux 6892ebfbdca4 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 6fdffaff3d80ee306ab022fd01a7bdcf7c6d7cfb |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/4/testReport/ |
| Max. process+thread count | 4450 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7192/4/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org