You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "apurtell (via GitHub)" <gi...@apache.org> on 2023/05/15 21:34:53 UTC
[GitHub] [hbase] apurtell opened a new pull request, #5239: HBASE-27867 Close the L1 victim handler race
apurtell opened a new pull request, #5239:
URL: https://github.com/apache/hbase/pull/5239
When we evict a block from L1 and move it to L2 there is a brief window of time where we have removed the block from the L1 map and yet the victim handler has not completed execution. Some read-your-write use cases can be significantly impacted even though the window is small. Victim handling can be made atomic with respect to the unmapping operation. The upside is there will be no L1+L2 misses during the transition. The downside is if the victim handler takes a long time to execute – currently they are all very fast, so only a theoretical risk – then other removals or insertions in L1 can block until it completes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5239: HBASE-27867 Close the L1 victim handler race
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5239:
URL: https://github.com/apache/hbase/pull/5239#issuecomment-1548841271
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 15s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 38s | master passed |
| +1 :green_heart: | compile | 0m 48s | master passed |
| +1 :green_heart: | shadedjars | 4m 25s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 28s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 30s | the patch passed |
| +1 :green_heart: | compile | 0m 46s | the patch passed |
| +1 :green_heart: | javac | 0m 47s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 28s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 25s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 215m 33s | hbase-server in the patch passed. |
| | | 238m 45s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5239 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux cb057cd2526a 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 93ddd7060d |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/1/testReport/ |
| Max. process+thread count | 4355 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/1/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5239: HBASE-27867 Close the L1 victim handler race
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5239:
URL: https://github.com/apache/hbase/pull/5239#issuecomment-1553819047
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 8s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 8s | master passed |
| +1 :green_heart: | compile | 2m 51s | master passed |
| +1 :green_heart: | checkstyle | 0m 40s | master passed |
| +1 :green_heart: | spotless | 0m 45s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 1m 48s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 32s | the patch passed |
| +1 :green_heart: | compile | 3m 14s | the patch passed |
| +1 :green_heart: | javac | 3m 14s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 42s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 14m 29s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. |
| +1 :green_heart: | spotless | 0m 47s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 1m 54s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 8s | The patch does not generate ASF License warnings. |
| | | 45m 35s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5239 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux 90d1ee3c30a8 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 71d7996047 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Max. process+thread count | 86 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/2/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #5239: HBASE-27867 Close the L1 victim handler race
Posted by "Apache9 (via GitHub)" <gi...@apache.org>.
Apache9 commented on code in PR #5239:
URL: https://github.com/apache/hbase/pull/5239#discussion_r1195857656
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java:
##########
@@ -592,10 +593,27 @@ public int evictBlocksByHfileName(String hfileName) {
* @return the heap size of evicted block
*/
protected long evictBlock(LruCachedBlock block, boolean evictedByEvictionProcess) {
- LruCachedBlock previous = map.remove(block.getCacheKey());
- if (previous == null) {
+ // We need something effectively final for the lambda.
+ final AtomicBoolean evicted = new AtomicBoolean(false);
Review Comment:
MutableBoolean is enough? Anyway, not a big problem.
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java:
##########
@@ -592,10 +593,27 @@ public int evictBlocksByHfileName(String hfileName) {
* @return the heap size of evicted block
*/
protected long evictBlock(LruCachedBlock block, boolean evictedByEvictionProcess) {
- LruCachedBlock previous = map.remove(block.getCacheKey());
- if (previous == null) {
+ // We need something effectively final for the lambda.
+ final AtomicBoolean evicted = new AtomicBoolean(false);
+ map.computeIfPresent(block.getCacheKey(), (k, v) -> {
Review Comment:
Check the declaration, the map is a ConcurrentHashMap so it is fine. Better add some comments to say that we must use ConcurrentHashMap here, as for CSLM, the computeIfPresent method is not atomic, which means the supplier could be called multiple times.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a diff in pull request #5239: HBASE-27867 Close the L1 victim handler race
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #5239:
URL: https://github.com/apache/hbase/pull/5239#discussion_r1195879024
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java:
##########
@@ -592,10 +593,27 @@ public int evictBlocksByHfileName(String hfileName) {
* @return the heap size of evicted block
*/
protected long evictBlock(LruCachedBlock block, boolean evictedByEvictionProcess) {
- LruCachedBlock previous = map.remove(block.getCacheKey());
- if (previous == null) {
+ // We need something effectively final for the lambda.
+ final AtomicBoolean evicted = new AtomicBoolean(false);
+ map.computeIfPresent(block.getCacheKey(), (k, v) -> {
+ // Run the victim handler before we remove the mapping in the L1 map. It must complete
+ // quickly because other removal or insertion operations can be blocked in the meantime.
+ if (evictedByEvictionProcess && victimHandler != null) {
+ victimHandler.cacheBlock(k, v.getBuffer());
+ }
+ // Decrease the block's reference count, and if refCount is 0, then it'll auto-deallocate. DO
+ // NOT move this up because if do that then the victimHandler may access the buffer with
Review Comment:
nit: `because if do that` -> `because if we do that`
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java:
##########
@@ -592,10 +593,27 @@ public int evictBlocksByHfileName(String hfileName) {
* @return the heap size of evicted block
*/
protected long evictBlock(LruCachedBlock block, boolean evictedByEvictionProcess) {
- LruCachedBlock previous = map.remove(block.getCacheKey());
- if (previous == null) {
+ // We need something effectively final for the lambda.
+ final AtomicBoolean evicted = new AtomicBoolean(false);
+ map.computeIfPresent(block.getCacheKey(), (k, v) -> {
+ // Run the victim handler before we remove the mapping in the L1 map. It must complete
+ // quickly because other removal or insertion operations can be blocked in the meantime.
+ if (evictedByEvictionProcess && victimHandler != null) {
+ victimHandler.cacheBlock(k, v.getBuffer());
+ }
Review Comment:
with computeIfPresent, i think updates that fall into some bucket of keys get blocked and not all updates IIRC, but yes some updates do get blocked so comment seems good
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #5239: HBASE-27867 Close the L1 victim handler race
Posted by "apurtell (via GitHub)" <gi...@apache.org>.
apurtell commented on PR #5239:
URL: https://github.com/apache/hbase/pull/5239#issuecomment-1553771670
Rebased. Addressed review feedback.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5239: HBASE-27867 Close the L1 victim handler race
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5239:
URL: https://github.com/apache/hbase/pull/5239#issuecomment-1548698471
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 4s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 0s | master passed |
| +1 :green_heart: | compile | 2m 45s | master passed |
| +1 :green_heart: | checkstyle | 0m 41s | master passed |
| -1 :x: | spotless | 0m 52s | branch has 1 errors when running spotless:check, run spotless:apply to fix. |
| +1 :green_heart: | spotbugs | 1m 58s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 15s | the patch passed |
| +1 :green_heart: | compile | 2m 59s | the patch passed |
| +1 :green_heart: | javac | 2m 59s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 45s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 15m 4s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. |
| +1 :green_heart: | spotless | 0m 48s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 2m 11s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. |
| | | 46m 40s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5239 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux 9467bcfc65f6 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 93ddd7060d |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/1/artifact/yetus-general-check/output/branch-spotless.txt |
| Max. process+thread count | 86 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/1/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] apurtell commented on a diff in pull request #5239: HBASE-27867 Close the L1 victim handler race
Posted by "apurtell (via GitHub)" <gi...@apache.org>.
apurtell commented on code in PR #5239:
URL: https://github.com/apache/hbase/pull/5239#discussion_r1198367749
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java:
##########
@@ -592,10 +593,27 @@ public int evictBlocksByHfileName(String hfileName) {
* @return the heap size of evicted block
*/
protected long evictBlock(LruCachedBlock block, boolean evictedByEvictionProcess) {
- LruCachedBlock previous = map.remove(block.getCacheKey());
- if (previous == null) {
+ // We need something effectively final for the lambda.
+ final AtomicBoolean evicted = new AtomicBoolean(false);
Review Comment:
MutableBoolean would be fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] apurtell commented on a diff in pull request #5239: HBASE-27867 Close the L1 victim handler race
Posted by "apurtell (via GitHub)" <gi...@apache.org>.
apurtell commented on code in PR #5239:
URL: https://github.com/apache/hbase/pull/5239#discussion_r1198368591
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java:
##########
@@ -592,10 +593,27 @@ public int evictBlocksByHfileName(String hfileName) {
* @return the heap size of evicted block
*/
protected long evictBlock(LruCachedBlock block, boolean evictedByEvictionProcess) {
- LruCachedBlock previous = map.remove(block.getCacheKey());
- if (previous == null) {
+ // We need something effectively final for the lambda.
+ final AtomicBoolean evicted = new AtomicBoolean(false);
+ map.computeIfPresent(block.getCacheKey(), (k, v) -> {
+ // Run the victim handler before we remove the mapping in the L1 map. It must complete
+ // quickly because other removal or insertion operations can be blocked in the meantime.
+ if (evictedByEvictionProcess && victimHandler != null) {
+ victimHandler.cacheBlock(k, v.getBuffer());
+ }
Review Comment:
The language I chose should be ambiguous enough to cover the distinction. I didn't want to really get into it. Let me know if it needs improving.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #5239: HBASE-27867 Close the L1 victim handler race
Posted by "apurtell (via GitHub)" <gi...@apache.org>.
apurtell commented on PR #5239:
URL: https://github.com/apache/hbase/pull/5239#issuecomment-1555275724
@Apache9 please let me know if you have any more concerns or nits. I plan to merge this tomorrow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5239: HBASE-27867 Close the L1 victim handler race
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5239:
URL: https://github.com/apache/hbase/pull/5239#issuecomment-1548877623
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 35s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 40s | master passed |
| +1 :green_heart: | compile | 0m 51s | master passed |
| +1 :green_heart: | shadedjars | 5m 17s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 29s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 21s | the patch passed |
| +1 :green_heart: | compile | 0m 46s | the patch passed |
| +1 :green_heart: | javac | 0m 46s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 4s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 26s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 262m 12s | hbase-server in the patch failed. |
| | | 288m 41s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5239 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 7a2c55fabd40 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 93ddd7060d |
| Default Java | Temurin-1.8.0_352-b08 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/1/testReport/ |
| Max. process+thread count | 4460 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/1/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #5239: HBASE-27867 Close the L1 victim handler race
Posted by "apurtell (via GitHub)" <gi...@apache.org>.
apurtell commented on PR #5239:
URL: https://github.com/apache/hbase/pull/5239#issuecomment-1550382491
I checked out the branch I pushed and ran `mvn spotless:apply` and there were no differences.
$ mvn spotless:apply
....
$ git status
(no changes)
The test issue is not a unit test failure and does not appear to be related to this change. Looks like we may need to do a zombie cleanup.
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.1.0:test
(secondPartTestsExecution) on project hbase-server:
There was a timeout in the fork
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5239: HBASE-27867 Close the L1 victim handler race
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5239:
URL: https://github.com/apache/hbase/pull/5239#issuecomment-1553944028
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 19s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 25s | master passed |
| +1 :green_heart: | compile | 0m 47s | master passed |
| +1 :green_heart: | shadedjars | 4m 29s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 25s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 18s | the patch passed |
| +1 :green_heart: | compile | 0m 48s | the patch passed |
| +1 :green_heart: | javac | 0m 48s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 29s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 25s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 217m 28s | hbase-server in the patch failed. |
| | | 240m 20s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5239 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 54de897e5754 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 71d7996047 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/2/testReport/ |
| Max. process+thread count | 4250 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/2/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] apurtell commented on a diff in pull request #5239: HBASE-27867 Close the L1 victim handler race
Posted by "apurtell (via GitHub)" <gi...@apache.org>.
apurtell commented on code in PR #5239:
URL: https://github.com/apache/hbase/pull/5239#discussion_r1198367987
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java:
##########
@@ -592,10 +593,27 @@ public int evictBlocksByHfileName(String hfileName) {
* @return the heap size of evicted block
*/
protected long evictBlock(LruCachedBlock block, boolean evictedByEvictionProcess) {
- LruCachedBlock previous = map.remove(block.getCacheKey());
- if (previous == null) {
+ // We need something effectively final for the lambda.
+ final AtomicBoolean evicted = new AtomicBoolean(false);
+ map.computeIfPresent(block.getCacheKey(), (k, v) -> {
Review Comment:
Let me also bullet proof the lambda against multiple invocation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5239: HBASE-27867 Close the L1 victim handler race
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5239:
URL: https://github.com/apache/hbase/pull/5239#issuecomment-1553801819
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 6s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 49s | master passed |
| +1 :green_heart: | compile | 0m 48s | master passed |
| +1 :green_heart: | shadedjars | 5m 1s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 30s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 31s | the patch passed |
| +1 :green_heart: | compile | 0m 42s | the patch passed |
| +1 :green_heart: | javac | 0m 42s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 7s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 26s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 12m 37s | hbase-server in the patch failed. |
| | | 35m 37s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5239 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux de60e3bd8767 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 71d7996047 |
| Default Java | Temurin-1.8.0_352-b08 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/2/testReport/ |
| Max. process+thread count | 2013 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5239/2/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] apurtell merged pull request #5239: HBASE-27867 Close the L1 victim handler race
Posted by "apurtell (via GitHub)" <gi...@apache.org>.
apurtell merged PR #5239:
URL: https://github.com/apache/hbase/pull/5239
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org