You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2021/07/14 18:48:34 UTC
[GitHub] [hbase] gkanade opened a new pull request #3490: HBASE-24984 WAL corruption due to early DBBs re-use when Durability.A…
gkanade opened a new pull request #3490:
URL: https://github.com/apache/hbase/pull/3490
…SYNC_WAL is used with multi operation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #3490: HBASE-24984 WAL corruption due to early DBBs re-use when Durability.A…
Posted by GitBox <gi...@apache.org>.
huaxiangsun commented on a change in pull request #3490:
URL: https://github.com/apache/hbase/pull/3490#discussion_r670887993
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/ServerCall.java
##########
@@ -175,23 +178,17 @@ private void release(int mask) {
}
}
- @Override
- public void cleanup() {
- release(0b01);
+ public void retainByWAL() {
+ reference.incrementAndGet();
}
- public void retainByWAL() {
- for (;;) {
- int ref = reference.get();
- int nextRef = ref | 0b10;
- if (reference.compareAndSet(ref, nextRef)) {
- return;
+ public void releaseByWAL() {
Review comment:
As explained by Anoop, this is not a concern as FSWEntry() is called within handler thread, so bit for Call still holds, no cleanup will be called.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #3490: HBASE-24984 WAL corruption due to early DBBs re-use when Durability.A…
Posted by GitBox <gi...@apache.org>.
huaxiangsun commented on a change in pull request #3490:
URL: https://github.com/apache/hbase/pull/3490#discussion_r669999412
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/ServerCall.java
##########
@@ -175,23 +178,17 @@ private void release(int mask) {
}
}
- @Override
- public void cleanup() {
- release(0b01);
+ public void retainByWAL() {
+ reference.incrementAndGet();
}
- public void retainByWAL() {
- for (;;) {
- int ref = reference.get();
- int nextRef = ref | 0b10;
- if (reference.compareAndSet(ref, nextRef)) {
- return;
+ public void releaseByWAL() {
Review comment:
I saw the review request is closed. Let me post one comment here. I think there is one corner case here.
batch() -> FSWEntry (increase refCount)
consume() -> sync done, decrease refCount -> cleanup()
next FSWEntry(increase refCount) --> referring to the buffer which has been released.
In this case, it is better to pre-calculate how many refCount is there and set it at the very begin.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #3490: HBASE-24984 WAL corruption due to early DBBs re-use when Durability.A…
Posted by GitBox <gi...@apache.org>.
huaxiangsun commented on a change in pull request #3490:
URL: https://github.com/apache/hbase/pull/3490#discussion_r669999412
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/ServerCall.java
##########
@@ -175,23 +178,17 @@ private void release(int mask) {
}
}
- @Override
- public void cleanup() {
- release(0b01);
+ public void retainByWAL() {
+ reference.incrementAndGet();
}
- public void retainByWAL() {
- for (;;) {
- int ref = reference.get();
- int nextRef = ref | 0b10;
- if (reference.compareAndSet(ref, nextRef)) {
- return;
+ public void releaseByWAL() {
Review comment:
I saw the review request is closed. Let me post one comment here. I think there is one corner case.
batch() -> FSWEntry (increase refCount)
consume() -> sync done, decrease refCount -> cleanup()
next FSWEntry(increase refCount) --> referring to the buffer which has been released.
In this case, it is better to pre-calculate how many refCount is there and set it at the very begin.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3490: HBASE-24984 WAL corruption due to early DBBs re-use when Durability.A…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3490:
URL: https://github.com/apache/hbase/pull/3490#issuecomment-880144102
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 6s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 1s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 58s | master passed |
| +1 :green_heart: | compile | 3m 7s | master passed |
| +1 :green_heart: | checkstyle | 1m 3s | master passed |
| +1 :green_heart: | spotbugs | 2m 4s | master passed |
||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 1m 53s | root in the patch failed. |
| -1 :x: | compile | 1m 54s | hbase-server in the patch failed. |
| -0 :warning: | javac | 1m 54s | hbase-server in the patch failed. |
| +1 :green_heart: | checkstyle | 1m 0s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| -1 :x: | hadoopcheck | 2m 9s | The patch causes 16 errors with Hadoop v3.1.2. |
| -1 :x: | hadoopcheck | 4m 18s | The patch causes 16 errors with Hadoop v3.2.1. |
| -1 :x: | hadoopcheck | 6m 32s | The patch causes 16 errors with Hadoop v3.3.0. |
| -1 :x: | spotbugs | 0m 32s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. |
| | | 25m 6s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3490 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux de9a924234f8 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 0836695459 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-general-check/output/patch-mvninstall-root.txt |
| compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt |
| javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt |
| hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-general-check/output/patch-javac-3.1.2.txt |
| hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-general-check/output/patch-javac-3.2.1.txt |
| hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-general-check/output/patch-javac-3.3.0.txt |
| spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-general-check/output/patch-spotbugs-hbase-server.txt |
| Max. process+thread count | 96 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3490: HBASE-24984 WAL corruption due to early DBBs re-use when Durability.A…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3490:
URL: https://github.com/apache/hbase/pull/3490#issuecomment-880147921
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 27s | Docker mode activated. |
| -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 47s | master passed |
| +1 :green_heart: | compile | 1m 11s | master passed |
| +1 :green_heart: | shadedjars | 9m 39s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 43s | master passed |
||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 2m 16s | root in the patch failed. |
| -1 :x: | compile | 0m 54s | hbase-server in the patch failed. |
| -0 :warning: | javac | 0m 54s | hbase-server in the patch failed. |
| -1 :x: | shadedjars | 7m 20s | patch has 16 errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 38s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 0m 50s | hbase-server in the patch failed. |
| | | 31m 18s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3490 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux efe66223e3e5 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 0836695459 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-jdk8-hadoop3-check/output/patch-mvninstall-root.txt |
| compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt |
| javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt |
| shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/testReport/ |
| Max. process+thread count | 78 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3490: HBASE-24984 WAL corruption due to early DBBs re-use when Durability.A…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3490:
URL: https://github.com/apache/hbase/pull/3490#issuecomment-880148807
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 39s | master passed |
| +1 :green_heart: | compile | 1m 12s | master passed |
| +1 :green_heart: | shadedjars | 9m 41s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 54s | master passed |
||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 3m 7s | root in the patch failed. |
| -1 :x: | compile | 1m 13s | hbase-server in the patch failed. |
| -0 :warning: | javac | 1m 13s | hbase-server in the patch failed. |
| -1 :x: | shadedjars | 8m 34s | patch has 16 errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 50s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 1m 11s | hbase-server in the patch failed. |
| | | 33m 9s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3490 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux ee1533453ca6 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 0836695459 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-jdk11-hadoop3-check/output/patch-mvninstall-root.txt |
| compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt |
| javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt |
| shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/testReport/ |
| Max. process+thread count | 93 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3490/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] gkanade closed pull request #3490: HBASE-24984 WAL corruption due to early DBBs re-use when Durability.A…
Posted by GitBox <gi...@apache.org>.
gkanade closed pull request #3490:
URL: https://github.com/apache/hbase/pull/3490
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org