You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "hfutatzhanghb (via GitHub)" <gi...@apache.org> on 2024/01/10 15:16:01 UTC
[PR] HDFS-17334. FSEditLogAsync#enqueueEdit does not synchronized this before invoke wait method. [hadoop]
hfutatzhanghb opened a new pull request, #6434:
URL: https://github.com/apache/hadoop/pull/6434
### Description of PR
In method FSEditLogAsync#enqueueEdit , there exist the below codes:
```java
if (Thread.holdsLock(this)) {
// if queue is full, synchronized caller must immediately relinquish
// the monitor before re-offering to avoid deadlock with sync thread
// which needs the monitor to write transactions.
int permits = overflowMutex.drainPermits();
try {
do {
this.wait(1000); // will be notified by next logSync.
} while (!editPendingQ.offer(edit));
} finally {
overflowMutex.release(permits);
}
}
```
It maybe invoke this.wait(1000) without having object this's monitor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-17334. FSEditLogAsync#enqueueEdit does not synchronized this before invoke wait method. [hadoop]
Posted by "zhangshuyan0 (via GitHub)" <gi...@apache.org>.
zhangshuyan0 commented on PR #6434:
URL: https://github.com/apache/hadoop/pull/6434#issuecomment-1897739440
Line211 has already ensured that we have a monitor for this object:
https://github.com/apache/hadoop/blob/ba6ada73acc2bce560878272c543534c21c76f22/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogAsync.java#L211-L223
So, I think the description in this PR is not a problem. What's your opinion? @hfutatzhanghb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-17334. FSEditLogAsync#enqueueEdit does not synchronized this before invoke wait method. [hadoop]
Posted by "hfutatzhanghb (via GitHub)" <gi...@apache.org>.
hfutatzhanghb commented on PR #6434:
URL: https://github.com/apache/hadoop/pull/6434#issuecomment-1898012100
> > > Line211 has already ensured that we have a monitor for this object:
> > > https://github.com/apache/hadoop/blob/ba6ada73acc2bce560878272c543534c21c76f22/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogAsync.java#L211-L223
> > >
> > > So, I think the description in this PR is not a problem. What's your opinion? @hfutatzhanghb
> >
> >
> > @zhangshuyan0 Sir, `this.wait(1000);` is in do-while loop, when we invoke `this.wait(1000)` at first time, it will release object monitor. But in extreme situation, it will throw Exception when invoke `this.wait(1000)` at the second time, because current thread does not hold the object monitor. Waiting for your response~
>
> Let's see [java doc](https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html#wait-long-) :
>
> > Thus, on return from the wait method, the synchronization state of the object and of thread T is exactly as it was when the wait method was invoked.
>
> Therefore, after `this.wait(1000)` returns at first time, it obtains the monitor again. I think no exception will be thrown here. By the way, in this [java doc](https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html#wait-long-) , `synchronized -> while loop` is showed as a recommended usage. Looking forward to your response.
Sir, Thanks a lot for your explanations here. I will close this PR laterly. Thanks again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-17334. FSEditLogAsync#enqueueEdit does not synchronized this before invoke wait method. [hadoop]
Posted by "hfutatzhanghb (via GitHub)" <gi...@apache.org>.
hfutatzhanghb closed pull request #6434: HDFS-17334. FSEditLogAsync#enqueueEdit does not synchronized this before invoke wait method.
URL: https://github.com/apache/hadoop/pull/6434
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-17334. FSEditLogAsync#enqueueEdit does not synchronized this before invoke wait method. [hadoop]
Posted by "hfutatzhanghb (via GitHub)" <gi...@apache.org>.
hfutatzhanghb commented on PR #6434:
URL: https://github.com/apache/hadoop/pull/6434#issuecomment-1886067772
@Hexiaoqiao @zhangshuyan0 @tomscut Sir, could you please take a look at this PR when you have free time? Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-17334. FSEditLogAsync#enqueueEdit does not synchronized this before invoke wait method. [hadoop]
Posted by "zhangshuyan0 (via GitHub)" <gi...@apache.org>.
zhangshuyan0 commented on PR #6434:
URL: https://github.com/apache/hadoop/pull/6434#issuecomment-1897981697
> > Line211 has already ensured that we have a monitor for this object:
> > https://github.com/apache/hadoop/blob/ba6ada73acc2bce560878272c543534c21c76f22/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogAsync.java#L211-L223
> >
> > So, I think the description in this PR is not a problem. What's your opinion? @hfutatzhanghb
>
> @zhangshuyan0 Sir, `this.wait(1000);` is in do-while loop, when we invoke `this.wait(1000)` at first time, it will release object monitor. But in extreme situation, it will throw Exception when invoke `this.wait(1000)` at the second time, because current thread does not hold the object monitor. Waiting for your response~
Let's see [JAVA doc](https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html#wait-long-) :
> Thus, on return from the wait method, the synchronization state of the object and of thread T is exactly as it was when the wait method was invoked.
Therefore, after `this.wait(1000)` returns at first time, it obtains the monitor again. I think no exception will be thrown here. By the way, in this [JAVA doc](https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html#wait-long-) , `synchronize -> while loop` is showed as a recommended usage. Looking forward to your response.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-17334. FSEditLogAsync#enqueueEdit does not synchronized this before invoke wait method. [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #6434:
URL: https://github.com/apache/hadoop/pull/6434#issuecomment-1885348791
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 11m 25s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 18s | | trunk passed |
| +1 :green_heart: | compile | 1m 19s | | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 16s | | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 9s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 26s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 6s | | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 33s | | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 20s | | trunk passed |
| +1 :green_heart: | shadedclient | 38m 32s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 20s | | the patch passed |
| +1 :green_heart: | compile | 1m 22s | | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 22s | | the patch passed |
| +1 :green_heart: | compile | 1m 18s | | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 1m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 8s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 21s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 0s | | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 40s | | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 4m 7s | | the patch passed |
| -1 :x: | shadedclient | 44m 54s | | patch has errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 0m 26s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6434/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch failed. |
| +0 :ok: | asflicense | 0m 26s | | ASF License check generated no output? |
| | | 160m 5s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6434/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/6434 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 71e56375345d 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 33a14bba95491c9282718b5053ff75b63140b472 |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6434/1/testReport/ |
| Max. process+thread count | 566 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6434/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-17334. FSEditLogAsync#enqueueEdit does not synchronized this before invoke wait method. [hadoop]
Posted by "hfutatzhanghb (via GitHub)" <gi...@apache.org>.
hfutatzhanghb commented on PR #6434:
URL: https://github.com/apache/hadoop/pull/6434#issuecomment-1897837901
> Line211 has already ensured that we have a monitor for this object:
>
> https://github.com/apache/hadoop/blob/ba6ada73acc2bce560878272c543534c21c76f22/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogAsync.java#L211-L223
>
>
> So, I think the description in this PR is not a problem. What's your opinion? @hfutatzhanghb
@zhangshuyan0 Sir, `this.wait(1000);` is in do-while loop, when we invoke `this.wait(1000)` at first time, it will release object monitor. But in extreme situation, it will throw Exception when invoke `this.wait(1000)` at the second time, because current thread does not hold the object monitor. Waiting for your response~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org