You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2022/12/12 13:14:14 UTC
[GitHub] [hadoop] adoroszlai opened a new pull request, #5207: HADOOP-18569. NFS Gateway may release buffer too early
adoroszlai opened a new pull request, #5207:
URL: https://github.com/apache/hadoop/pull/5207
## What changes were proposed in this pull request?
After upgrading Netty from 4.1.68 to 4.1.77 (HADOOP-18079), NFS Gateway started randomly crashing when writing data (can be easily reproduced by a few 10MB+ files). The problem was triggered by [reduced default chunk size in PooledByteBufAllocator](https://github.com/netty/netty/commit/f650303911) (in 4.1.75), but it turned out to be caused by a buffer released too early in NFS Gateway (HADOOP-11245).
https://issues.apache.org/jira/browse/HADOOP-18569
## How was this patch tested?
Deployed cluster with the change, tested write/read via NFS mount.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5207: HADOOP-18569. NFS Gateway may release buffer too early
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5207:
URL: https://github.com/apache/hadoop/pull/5207#issuecomment-1346729416
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 10m 19s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ branch-3.3.5 Compile Tests _ |
| +1 :green_heart: | mvninstall | 36m 38s | | branch-3.3.5 passed |
| +1 :green_heart: | compile | 17m 43s | | branch-3.3.5 passed |
| +1 :green_heart: | checkstyle | 0m 42s | | branch-3.3.5 passed |
| +1 :green_heart: | mvnsite | 0m 44s | | branch-3.3.5 passed |
| +1 :green_heart: | javadoc | 0m 45s | | branch-3.3.5 passed |
| +1 :green_heart: | spotbugs | 1m 10s | | branch-3.3.5 passed |
| +1 :green_heart: | shadedclient | 24m 19s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 25s | | the patch passed |
| +1 :green_heart: | compile | 17m 4s | | the patch passed |
| +1 :green_heart: | javac | 17m 4s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 36s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 45s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 41s | | the patch passed |
| +1 :green_heart: | spotbugs | 1m 8s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 22s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 0s | | hadoop-nfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 0s | | The patch does not generate ASF License warnings. |
| | | 141m 31s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5207/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5207 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux d379b18343c5 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3.5 / 3be989682dd86e87905796d63af654b126e8d863 |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~18.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5207/1/testReport/ |
| Max. process+thread count | 550 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-nfs U: hadoop-common-project/hadoop-nfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5207/1/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] adoroszlai merged pull request #5207: HADOOP-18569. NFS Gateway may release buffer too early
Posted by GitBox <gi...@apache.org>.
adoroszlai merged PR #5207:
URL: https://github.com/apache/hadoop/pull/5207
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] adoroszlai commented on pull request #5207: HADOOP-18569. NFS Gateway may release buffer too early
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on PR #5207:
URL: https://github.com/apache/hadoop/pull/5207#issuecomment-1346967676
Thanks @steveloughran, @szetszwo for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org