You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2021/09/25 09:36:56 UTC
[GitHub] [hbase] comnetwork opened a new pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
comnetwork opened a new pull request #3699:
URL: https://github.com/apache/hbase/pull/3699
…m persistence file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3699:
URL: https://github.com/apache/hbase/pull/3699#issuecomment-927243191
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 26s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 39s | master passed |
| +1 :green_heart: | compile | 1m 3s | master passed |
| +1 :green_heart: | shadedjars | 8m 16s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 48s | the patch passed |
| +1 :green_heart: | compile | 1m 0s | the patch passed |
| +1 :green_heart: | javac | 1m 0s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 11s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 38s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 148m 32s | hbase-server in the patch failed. |
| | | 178m 12s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3699 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 697999d30388 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 2b26dfbaf4 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/3/testReport/ |
| Max. process+thread count | 4740 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/3/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] comnetwork commented on a change in pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
comnetwork commented on a change in pull request #3699:
URL: https://github.com/apache/hbase/pull/3699#discussion_r718130209
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketProtoUtils.java
##########
@@ -127,19 +130,23 @@ private BucketProtoUtils() {
}
static ConcurrentHashMap<BlockCacheKey, BucketEntry> fromPB(
- Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap)
+ Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap,
+ Function<BucketEntry, Recycler> createRecycler)
throws IOException {
ConcurrentHashMap<BlockCacheKey, BucketEntry> result = new ConcurrentHashMap<>();
for (BucketCacheProtos.BackingMapEntry entry : backingMap.getEntryList()) {
BucketCacheProtos.BlockCacheKey protoKey = entry.getKey();
BlockCacheKey key = new BlockCacheKey(protoKey.getHfilename(), protoKey.getOffset(),
protoKey.getPrimaryReplicaBlock(), fromPb(protoKey.getBlockType()));
BucketCacheProtos.BucketEntry protoValue = entry.getValue();
+ // TODO:We use ByteBuffAllocator.HEAP here, because we could not get the ByteBuffAllocator
Review comment:
@Apache9 ， thank you for review.
The original implementation here before my patch is `ByteBufferAllocator.HEAP` ，but I think `RpcServer#bbAllocator` is better(when `BucketEntry `is not restored from file, `BucketEntry#allocator` is `RpcServer#bbAllocator`).
The problem here is we could not get `RpcServer#bbAllocator ` in `BucketCache `ctor.There are two way to introduce `RpcServer#bbAllocator `, but both are not very elegantly and the modification are somewhat entensive:
1. Changing the `BucketCache `ctor to add a `ByteBufferAllocator `parameter and pass `RpcServer#bbAllocator `to `BlockCacheFactory.createBlockCache(Configuration) `in `HRegionServer(Configuration)`, which would make `BucketCache `and `BlockCacheFactory `depends on `ByteBufferAllocator `, but `BucketCache `indeed with no need for `ByteBufferAllocator ` unless it is restored from file, so it seems not very elegantly.
2. Another way is set `BucketEntry#allocator` to null here and adding `ByteBufferAllocator `parameter to `BlockCache#getBlock` method, passing `CacheConfig.byteBuffAllocator` (which is also from`RpcServer#bbAllocator ` ) to it when invoking `BlockCache#getBlock` in `HFileReaderImpl.getCachedBlock`, the influence for modifying `BlockCache#getBlock` is somwhat extensive and is also a little strange, so it is also not very elegantly.
So I maintain `ByteBufferAllocator.HEAP` here as same as the implementation before my patch. I think the first way is somewhat better if we need to use `RpcServer#bbAllocator ` here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 merged pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
Apache9 merged pull request #3699:
URL: https://github.com/apache/hbase/pull/3699
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
Apache9 commented on a change in pull request #3699:
URL: https://github.com/apache/hbase/pull/3699#discussion_r718087826
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketProtoUtils.java
##########
@@ -127,19 +130,23 @@ private BucketProtoUtils() {
}
static ConcurrentHashMap<BlockCacheKey, BucketEntry> fromPB(
- Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap)
+ Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap,
+ Function<BucketEntry, Recycler> createRecycler)
throws IOException {
ConcurrentHashMap<BlockCacheKey, BucketEntry> result = new ConcurrentHashMap<>();
for (BucketCacheProtos.BackingMapEntry entry : backingMap.getEntryList()) {
BucketCacheProtos.BlockCacheKey protoKey = entry.getKey();
BlockCacheKey key = new BlockCacheKey(protoKey.getHfilename(), protoKey.getOffset(),
protoKey.getPrimaryReplicaBlock(), fromPb(protoKey.getBlockType()));
BucketCacheProtos.BucketEntry protoValue = entry.getValue();
+ // TODO:We use ByteBuffAllocator.HEAP here, because we could not get the ByteBuffAllocator
Review comment:
Mind explaining that why using ByteBufferAllocator.HEAP is fine for now?
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketProtoUtils.java
##########
@@ -127,19 +130,23 @@ private BucketProtoUtils() {
}
static ConcurrentHashMap<BlockCacheKey, BucketEntry> fromPB(
- Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap)
+ Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap,
+ Function<BucketEntry, Recycler> createRecycler)
throws IOException {
ConcurrentHashMap<BlockCacheKey, BucketEntry> result = new ConcurrentHashMap<>();
for (BucketCacheProtos.BackingMapEntry entry : backingMap.getEntryList()) {
BucketCacheProtos.BlockCacheKey protoKey = entry.getKey();
BlockCacheKey key = new BlockCacheKey(protoKey.getHfilename(), protoKey.getOffset(),
protoKey.getPrimaryReplicaBlock(), fromPb(protoKey.getBlockType()));
BucketCacheProtos.BucketEntry protoValue = entry.getValue();
+ // TODO:We use ByteBuffAllocator.HEAP here, because we could not get the ByteBuffAllocator
Review comment:
OK, could be a follow on issue. Thanks for adding a TODO here to point this problem out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3699:
URL: https://github.com/apache/hbase/pull/3699#issuecomment-927156858
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 0s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 49s | master passed |
| +1 :green_heart: | compile | 1m 19s | master passed |
| +1 :green_heart: | shadedjars | 9m 0s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 41s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 46s | the patch passed |
| +1 :green_heart: | compile | 1m 20s | the patch passed |
| +1 :green_heart: | javac | 1m 20s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 2s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 42s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 203m 55s | hbase-server in the patch passed. |
| | | 238m 14s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3699 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux a2c091ec98ec 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 2b26dfbaf4 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/2/testReport/ |
| Max. process+thread count | 3327 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3699:
URL: https://github.com/apache/hbase/pull/3699#issuecomment-927122586
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 2s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 1s | master passed |
| +1 :green_heart: | compile | 1m 17s | master passed |
| +1 :green_heart: | shadedjars | 9m 4s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 44s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 49s | the patch passed |
| +1 :green_heart: | compile | 1m 18s | the patch passed |
| +1 :green_heart: | javac | 1m 18s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 5s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 42s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 204m 23s | hbase-server in the patch passed. |
| | | 239m 23s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3699 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux c093c487d490 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / d956828aac |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/1/testReport/ |
| Max. process+thread count | 3478 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3699:
URL: https://github.com/apache/hbase/pull/3699#issuecomment-927497730
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 3s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 1s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 8s | master passed |
| +1 :green_heart: | compile | 3m 21s | master passed |
| +1 :green_heart: | checkstyle | 1m 10s | master passed |
| +1 :green_heart: | spotbugs | 2m 13s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 7s | the patch passed |
| +1 :green_heart: | compile | 3m 17s | the patch passed |
| +1 :green_heart: | javac | 3m 17s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 10s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 20m 17s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. |
| +1 :green_heart: | spotbugs | 2m 20s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. |
| | | 51m 40s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/4/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3699 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux ed1510a04236 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / cbebf85b3c |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Max. process+thread count | 86 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/4/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
Apache9 commented on a change in pull request #3699:
URL: https://github.com/apache/hbase/pull/3699#discussion_r718087826
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketProtoUtils.java
##########
@@ -127,19 +130,23 @@ private BucketProtoUtils() {
}
static ConcurrentHashMap<BlockCacheKey, BucketEntry> fromPB(
- Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap)
+ Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap,
+ Function<BucketEntry, Recycler> createRecycler)
throws IOException {
ConcurrentHashMap<BlockCacheKey, BucketEntry> result = new ConcurrentHashMap<>();
for (BucketCacheProtos.BackingMapEntry entry : backingMap.getEntryList()) {
BucketCacheProtos.BlockCacheKey protoKey = entry.getKey();
BlockCacheKey key = new BlockCacheKey(protoKey.getHfilename(), protoKey.getOffset(),
protoKey.getPrimaryReplicaBlock(), fromPb(protoKey.getBlockType()));
BucketCacheProtos.BucketEntry protoValue = entry.getValue();
+ // TODO:We use ByteBuffAllocator.HEAP here, because we could not get the ByteBuffAllocator
Review comment:
Mind explaining that why using ByteBufferAllocator.HEAP is fine for now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3699:
URL: https://github.com/apache/hbase/pull/3699#issuecomment-927544218
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3699:
URL: https://github.com/apache/hbase/pull/3699#issuecomment-927148916
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 27s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 37s | master passed |
| +1 :green_heart: | compile | 1m 0s | master passed |
| +1 :green_heart: | shadedjars | 8m 19s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 46s | the patch passed |
| +1 :green_heart: | compile | 1m 0s | the patch passed |
| +1 :green_heart: | javac | 1m 0s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 17s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 148m 21s | hbase-server in the patch failed. |
| | | 178m 4s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3699 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux a1f6296f003b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 2b26dfbaf4 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/2/testReport/ |
| Max. process+thread count | 4718 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3699:
URL: https://github.com/apache/hbase/pull/3699#issuecomment-927129500
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 2s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 4s | master passed |
| +1 :green_heart: | compile | 3m 15s | master passed |
| +1 :green_heart: | checkstyle | 1m 11s | master passed |
| +1 :green_heart: | spotbugs | 2m 10s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 3s | the patch passed |
| +1 :green_heart: | compile | 3m 18s | the patch passed |
| +1 :green_heart: | javac | 3m 18s | the patch passed |
| -0 :warning: | checkstyle | 1m 9s | hbase-server: The patch generated 1 new + 39 unchanged - 0 fixed = 40 total (was 39) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 20m 24s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. |
| +1 :green_heart: | spotbugs | 2m 23s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. |
| | | 51m 22s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3699 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux ace8d7116cfa 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 2b26dfbaf4 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 85 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/2/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 merged pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
Apache9 merged pull request #3699:
URL: https://github.com/apache/hbase/pull/3699
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3699:
URL: https://github.com/apache/hbase/pull/3699#issuecomment-927101784
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 6m 21s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 21s | master passed |
| +1 :green_heart: | compile | 3m 17s | master passed |
| +1 :green_heart: | checkstyle | 1m 12s | master passed |
| +1 :green_heart: | spotbugs | 2m 12s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 3s | the patch passed |
| +1 :green_heart: | compile | 3m 20s | the patch passed |
| +1 :green_heart: | javac | 3m 20s | the patch passed |
| -0 :warning: | checkstyle | 1m 12s | hbase-server: The patch generated 1 new + 39 unchanged - 0 fixed = 40 total (was 39) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 20m 15s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. |
| +1 :green_heart: | spotbugs | 2m 27s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. |
| | | 57m 22s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3699 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 8bd4cd701eb5 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / d956828aac |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 85 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] comnetwork commented on a change in pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
comnetwork commented on a change in pull request #3699:
URL: https://github.com/apache/hbase/pull/3699#discussion_r718130209
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketProtoUtils.java
##########
@@ -127,19 +130,23 @@ private BucketProtoUtils() {
}
static ConcurrentHashMap<BlockCacheKey, BucketEntry> fromPB(
- Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap)
+ Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap,
+ Function<BucketEntry, Recycler> createRecycler)
throws IOException {
ConcurrentHashMap<BlockCacheKey, BucketEntry> result = new ConcurrentHashMap<>();
for (BucketCacheProtos.BackingMapEntry entry : backingMap.getEntryList()) {
BucketCacheProtos.BlockCacheKey protoKey = entry.getKey();
BlockCacheKey key = new BlockCacheKey(protoKey.getHfilename(), protoKey.getOffset(),
protoKey.getPrimaryReplicaBlock(), fromPb(protoKey.getBlockType()));
BucketCacheProtos.BucketEntry protoValue = entry.getValue();
+ // TODO:We use ByteBuffAllocator.HEAP here, because we could not get the ByteBuffAllocator
Review comment:
@Apache9 ， thank you for review.
The original implementation here before my patch is `ByteBufferAllocator.HEAP` ，but I think `RpcServer#bbAllocator` is better(when `BucketEntry `is not restored from file, `BucketEntry#allocator` is `RpcServer#bbAllocator`).
The problem here is we could not get `RpcServer#bbAllocator ` in `BucketCache `ctor.There are two way to introduce `RpcServer#bbAllocator `, but both are not very elegantly and the modification are somewhat entensive:
1. Changing the `BucketCache `ctor to add a `ByteBufferAllocator `parameter and pass `RpcServer#bbAllocator `to `BlockCacheFactory.createBlockCache(Configuration) `in `HRegionServer(Configuration)`, which would make `BucketCache `and `BlockCacheFactory `depends on `ByteBufferAllocator `, but `BucketCache `indeed with no need for `ByteBufferAllocator ` unless it is restored from file, so it seems not very elegantly.
2. Another way is set `BucketEntry#allocator` to null here and adding `ByteBufferAllocator `parameter to `BlockCache#getBlock` method, passing `CacheConfig.byteBuffAllocator` (which is also from`RpcServer#bbAllocator ` ) to it when invoking `BlockCache#getBlock` in `HFileReaderImpl.getCachedBlock`, the influence for modifying `BlockCache#getBlock` is somwhat extensive and is also a little strange, so it is also not very elegantly.
So I maintain `ByteBufferAllocator.HEAP` here as same as the implementation before my patch. I think the first way is somewhat better if we need to use `RpcServer#bbAllocator ` here.
Using `ByteBufferAllocator.HEAP` or `RpcServer#bbAllocator` do not affect correctness， but `RpcServer#bbAllocator` using DBB pool to alleviate Java GC.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] comnetwork commented on a change in pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
comnetwork commented on a change in pull request #3699:
URL: https://github.com/apache/hbase/pull/3699#discussion_r718130209
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketProtoUtils.java
##########
@@ -127,19 +130,23 @@ private BucketProtoUtils() {
}
static ConcurrentHashMap<BlockCacheKey, BucketEntry> fromPB(
- Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap)
+ Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap,
+ Function<BucketEntry, Recycler> createRecycler)
throws IOException {
ConcurrentHashMap<BlockCacheKey, BucketEntry> result = new ConcurrentHashMap<>();
for (BucketCacheProtos.BackingMapEntry entry : backingMap.getEntryList()) {
BucketCacheProtos.BlockCacheKey protoKey = entry.getKey();
BlockCacheKey key = new BlockCacheKey(protoKey.getHfilename(), protoKey.getOffset(),
protoKey.getPrimaryReplicaBlock(), fromPb(protoKey.getBlockType()));
BucketCacheProtos.BucketEntry protoValue = entry.getValue();
+ // TODO:We use ByteBuffAllocator.HEAP here, because we could not get the ByteBuffAllocator
Review comment:
@Apache9 ， thank you for review.
The original implementation here before my patch is `ByteBufferAllocator.HEAP` ，but I think `RpcServer#bbAllocator` is better(when `BucketEntry `is not restored from file, `BucketEntry#allocator` is `RpcServer#bbAllocator`).
The problem here is we could not get `RpcServer#bbAllocator ` in `BucketCache `ctor.There are two way to introduce `RpcServer#bbAllocator `, but both are not very elegantly and the modification are somewhat entensive:
1. Changing the `BucketCache `ctor to add a `ByteBufferAllocator `parameter and pass `RpcServer#bbAllocator `to `BlockCacheFactory.createBlockCache(Configuration) `in `HRegionServer(Configuration)`, which would make `BucketCache `and `BlockCacheFactory `depends on `ByteBufferAllocator `, but `BucketCache `indeed with no need for `ByteBufferAllocator ` unless it is restored from file, so it seems not very elegantly.
2. Another way is set `BucketEntry#allocator` to null here and adding `ByteBufferAllocator `parameter to `BlockCache#getBlock` method, passing `CacheConfig.byteBuffAllocator` (which is also from`RpcServer#bbAllocator ` ) to it when invoking `BlockCache#getBlock` in `HFileReaderImpl.getCachedBlock`, the influence for modifying `BlockCache#getBlock` is somwhat extensive and is also a little strange, so it is also not very elegantly.
So I maintain `ByteBufferAllocator.HEAP` here as same as the implementation before my patch. I think the first way is somewhat better if we need to use `RpcServer#bbAllocator ` here.
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketProtoUtils.java
##########
@@ -127,19 +130,23 @@ private BucketProtoUtils() {
}
static ConcurrentHashMap<BlockCacheKey, BucketEntry> fromPB(
- Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap)
+ Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap,
+ Function<BucketEntry, Recycler> createRecycler)
throws IOException {
ConcurrentHashMap<BlockCacheKey, BucketEntry> result = new ConcurrentHashMap<>();
for (BucketCacheProtos.BackingMapEntry entry : backingMap.getEntryList()) {
BucketCacheProtos.BlockCacheKey protoKey = entry.getKey();
BlockCacheKey key = new BlockCacheKey(protoKey.getHfilename(), protoKey.getOffset(),
protoKey.getPrimaryReplicaBlock(), fromPb(protoKey.getBlockType()));
BucketCacheProtos.BucketEntry protoValue = entry.getValue();
+ // TODO:We use ByteBuffAllocator.HEAP here, because we could not get the ByteBuffAllocator
Review comment:
@Apache9 ， thank you for review.
The original implementation here before my patch is `ByteBufferAllocator.HEAP` ，but I think `RpcServer#bbAllocator` is better(when `BucketEntry `is not restored from file, `BucketEntry#allocator` is `RpcServer#bbAllocator`).
The problem here is we could not get `RpcServer#bbAllocator ` in `BucketCache `ctor.There are two way to introduce `RpcServer#bbAllocator `, but both are not very elegantly and the modification are somewhat entensive:
1. Changing the `BucketCache `ctor to add a `ByteBufferAllocator `parameter and pass `RpcServer#bbAllocator `to `BlockCacheFactory.createBlockCache(Configuration) `in `HRegionServer(Configuration)`, which would make `BucketCache `and `BlockCacheFactory `depends on `ByteBufferAllocator `, but `BucketCache `indeed with no need for `ByteBufferAllocator ` unless it is restored from file, so it seems not very elegantly.
2. Another way is set `BucketEntry#allocator` to null here and adding `ByteBufferAllocator `parameter to `BlockCache#getBlock` method, passing `CacheConfig.byteBuffAllocator` (which is also from`RpcServer#bbAllocator ` ) to it when invoking `BlockCache#getBlock` in `HFileReaderImpl.getCachedBlock`, the influence for modifying `BlockCache#getBlock` is somwhat extensive and is also a little strange, so it is also not very elegantly.
So I maintain `ByteBufferAllocator.HEAP` here as same as the implementation before my patch. I think the first way is somewhat better if we need to use `RpcServer#bbAllocator ` here.
Using `ByteBufferAllocator.HEAP` or `RpcServer#bbAllocator` do not affect correctness， but `RpcServer#bbAllocator` using DBB pool to alleviate Java GC.
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketProtoUtils.java
##########
@@ -127,19 +130,23 @@ private BucketProtoUtils() {
}
static ConcurrentHashMap<BlockCacheKey, BucketEntry> fromPB(
- Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap)
+ Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap,
+ Function<BucketEntry, Recycler> createRecycler)
throws IOException {
ConcurrentHashMap<BlockCacheKey, BucketEntry> result = new ConcurrentHashMap<>();
for (BucketCacheProtos.BackingMapEntry entry : backingMap.getEntryList()) {
BucketCacheProtos.BlockCacheKey protoKey = entry.getKey();
BlockCacheKey key = new BlockCacheKey(protoKey.getHfilename(), protoKey.getOffset(),
protoKey.getPrimaryReplicaBlock(), fromPb(protoKey.getBlockType()));
BucketCacheProtos.BucketEntry protoValue = entry.getValue();
+ // TODO:We use ByteBuffAllocator.HEAP here, because we could not get the ByteBuffAllocator
Review comment:
@Apache9 ， thank you for review.
The original implementation here before my patch is `ByteBufferAllocator.HEAP` ，but I think `RpcServer#bbAllocator` is better(when `BucketEntry `is not restored from file, `BucketEntry#allocator` is `RpcServer#bbAllocator`).
The problem here is we could not get `RpcServer#bbAllocator ` in `BucketCache `ctor.There are two way to introduce `RpcServer#bbAllocator `, but both are not very elegantly and the modification are somewhat entensive:
1. Changing the `BucketCache `ctor to add a `ByteBufferAllocator `parameter and pass `RpcServer#bbAllocator `to `BlockCacheFactory.createBlockCache(Configuration) `in `HRegionServer(Configuration)`, which would make `BucketCache `and `BlockCacheFactory `depends on `ByteBufferAllocator `, but `BucketCache `indeed with no need for `ByteBufferAllocator ` unless it is restored from file, so it seems not very elegantly.
2. Another way is set `BucketEntry#allocator` to null here and adding `ByteBufferAllocator `parameter to `BlockCache#getBlock` method, passing `CacheConfig.byteBuffAllocator` (which is also from`RpcServer#bbAllocator ` ) to it when invoking `BlockCache#getBlock` in `HFileReaderImpl.getCachedBlock`, the influence for modifying `BlockCache#getBlock` is somwhat extensive and is also a little strange, so it is also not very elegantly.
So I maintain `ByteBufferAllocator.HEAP` here as same as the implementation before my patch. I think the first way is somewhat better if we need to use `RpcServer#bbAllocator ` here.
Using `ByteBufferAllocator.HEAP` or `RpcServer#bbAllocator` do not affect correctness here， but `RpcServer#bbAllocator` using DBB pool to alleviate Java GC.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3699:
URL: https://github.com/apache/hbase/pull/3699#issuecomment-927114941
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 26s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 0s | master passed |
| +1 :green_heart: | compile | 0m 59s | master passed |
| +1 :green_heart: | shadedjars | 8m 15s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 43s | the patch passed |
| +1 :green_heart: | compile | 1m 1s | the patch passed |
| +1 :green_heart: | javac | 1m 1s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 13s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 146m 54s | hbase-server in the patch passed. |
| | | 177m 6s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3699 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 2a7ef32bc2e2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / d956828aac |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/1/testReport/ |
| Max. process+thread count | 4608 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3699:
URL: https://github.com/apache/hbase/pull/3699#issuecomment-927250807
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 57s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 7s | master passed |
| +1 :green_heart: | compile | 1m 18s | master passed |
| +1 :green_heart: | shadedjars | 9m 7s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 43s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 46s | the patch passed |
| +1 :green_heart: | compile | 1m 18s | the patch passed |
| +1 :green_heart: | javac | 1m 18s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 8s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 42s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 207m 3s | hbase-server in the patch failed. |
| | | 241m 54s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3699 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux cf8e6557efc0 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 2b26dfbaf4 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/3/testReport/ |
| Max. process+thread count | 3453 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/3/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3699:
URL: https://github.com/apache/hbase/pull/3699#issuecomment-927228678
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 6s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 5s | master passed |
| +1 :green_heart: | compile | 3m 21s | master passed |
| +1 :green_heart: | checkstyle | 1m 11s | master passed |
| +1 :green_heart: | spotbugs | 2m 12s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 4s | the patch passed |
| +1 :green_heart: | compile | 3m 21s | the patch passed |
| +1 :green_heart: | javac | 3m 21s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 10s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 20m 17s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. |
| +1 :green_heart: | spotbugs | 2m 21s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. |
| | | 51m 33s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3699 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 6a1868cc498c 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 2b26dfbaf4 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Max. process+thread count | 86 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3699/3/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #3699: HBASE-26295 BucketCache could not free BucketEntry which restored fro…
Posted by GitBox <gi...@apache.org>.
Apache9 commented on a change in pull request #3699:
URL: https://github.com/apache/hbase/pull/3699#discussion_r718483937
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketProtoUtils.java
##########
@@ -127,19 +130,23 @@ private BucketProtoUtils() {
}
static ConcurrentHashMap<BlockCacheKey, BucketEntry> fromPB(
- Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap)
+ Map<Integer, String> deserializers, BucketCacheProtos.BackingMap backingMap,
+ Function<BucketEntry, Recycler> createRecycler)
throws IOException {
ConcurrentHashMap<BlockCacheKey, BucketEntry> result = new ConcurrentHashMap<>();
for (BucketCacheProtos.BackingMapEntry entry : backingMap.getEntryList()) {
BucketCacheProtos.BlockCacheKey protoKey = entry.getKey();
BlockCacheKey key = new BlockCacheKey(protoKey.getHfilename(), protoKey.getOffset(),
protoKey.getPrimaryReplicaBlock(), fromPb(protoKey.getBlockType()));
BucketCacheProtos.BucketEntry protoValue = entry.getValue();
+ // TODO:We use ByteBuffAllocator.HEAP here, because we could not get the ByteBuffAllocator
Review comment:
OK, could be a follow on issue. Thanks for adding a TODO here to point this problem out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org