You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "steveloughran (via GitHub)" <gi...@apache.org> on 2023/02/08 14:35:28 UTC
[GitHub] [hadoop] steveloughran opened a new pull request, #5369: Revert "HADOOP-18324. Interrupting RPC Client calls can lead to thread exhaustion. (#4527)
steveloughran opened a new pull request, #5369:
URL: https://github.com/apache/hadoop/pull/5369
Pulling HADOOP-18324 out of this release because of the problems of HDFS-16853
This reverts commit bc4d7b47747f58d8b0973c18a148be74df3e6c9a.
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g., 'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
### How was this patch tested?
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] omalley commented on pull request #5369: Revert "HADOOP-18324. Interrupting RPC Client calls can lead to thread exhaustion. (#4527)
Posted by "omalley (via GitHub)" <gi...@apache.org>.
omalley commented on PR #5369:
URL: https://github.com/apache/hadoop/pull/5369#issuecomment-1424579281
Strong -1
You are reintroducing a really bad bug that brought down our NN's in production for a race condition that we've only seen in unit tests. My PR for the race condition should fix the issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #5369: Revert "HADOOP-18324. Interrupting RPC Client calls can lead to thread exhaustion. (#4527)
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on PR #5369:
URL: https://github.com/apache/hadoop/pull/5369#issuecomment-1424005574
don't know what went wrong with the build; EOF during the npm build
```
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.11.2:install-node-and-yarn (install node and yarn) on project hadoop-yarn-applications-catalog-webapp: Could not extract the Node archive: Could not extract archive: '/home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5369/yetus-m2/hadoop-branch-3.3.5-patch-0/com/github/eirslett/node/12.22.1/node-12.22.1-linux-x64.tar.gz': EOFException -> [Help 1]
[ERROR]
```
This PR is a quick revert only of the IPC stuff. This is one of the two blockers for 3.3.5 and IMO it's dangerous to try and rush through a fix for this with time pressure. yes, the thread leakage issue will remain, but at least we know the shutdown works -and as that is a common activity, I'd prefer that to be working.
If people with private forks who have the original patch can help work on the ultimate fix, we can get it into the followup release
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran closed pull request #5369: Revert "HADOOP-18324. Interrupting RPC Client calls can lead to thread exhaustion. (#4527)
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran closed pull request #5369: Revert "HADOOP-18324. Interrupting RPC Client calls can lead to thread exhaustion. (#4527)
URL: https://github.com/apache/hadoop/pull/5369
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5369: Revert "HADOOP-18324. Interrupting RPC Client calls can lead to thread exhaustion. (#4527)
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5369:
URL: https://github.com/apache/hadoop/pull/5369#issuecomment-1422987851
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 7m 32s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 2 new or modified test files. |
|||| _ branch-3.3.5 Compile Tests _ |
| -1 :x: | mvninstall | 41m 11s | [/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5369/1/artifact/out/branch-mvninstall-root.txt) | root in branch-3.3.5 failed. |
| +1 :green_heart: | compile | 18m 3s | | branch-3.3.5 passed |
| +1 :green_heart: | checkstyle | 1m 1s | | branch-3.3.5 passed |
| +1 :green_heart: | mvnsite | 1m 37s | | branch-3.3.5 passed |
| +1 :green_heart: | javadoc | 0m 55s | | branch-3.3.5 passed |
| +1 :green_heart: | spotbugs | 2m 35s | | branch-3.3.5 passed |
| +1 :green_heart: | shadedclient | 28m 38s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 1s | | the patch passed |
| +1 :green_heart: | compile | 17m 13s | | the patch passed |
| +1 :green_heart: | javac | 17m 13s | | the patch passed |
| -1 :x: | blanks | 0m 0s | [/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5369/1/artifact/out/blanks-eol.txt) | The patch has 12 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 0m 55s | [/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5369/1/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt) | hadoop-common-project/hadoop-common: The patch generated 3 new + 145 unchanged - 0 fixed = 148 total (was 145) |
| +1 :green_heart: | mvnsite | 1m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 47s | | the patch passed |
| +1 :green_heart: | spotbugs | 2m 32s | | the patch passed |
| +1 :green_heart: | shadedclient | 28m 47s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 17m 40s | | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 1m 1s | | The patch does not generate ASF License warnings. |
| | | 172m 46s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5369/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5369 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 48153b4c4a76 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3.5 / 96e76d27e84a3ba4bb85380f32c18a6810e7e7bb |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~18.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5369/1/testReport/ |
| Max. process+thread count | 2464 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5369/1/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #5369: Revert "HADOOP-18324. Interrupting RPC Client calls can lead to thread exhaustion. (#4527)
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on PR #5369:
URL: https://github.com/apache/hadoop/pull/5369#issuecomment-1424673206
@omalley ok; just felt that as this was a regression and nobody else had seen it over many years, the risk was low. will abandon this PR and look forward to a fix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org