You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Feng Yuan (JIRA)" <ji...@apache.org> on 2018/04/18 11:30:00 UTC
[jira] [Created] (MAPREDUCE-7083) Map native output collector hang
on NativeBatchProcess.nativeFinsh
Feng Yuan created MAPREDUCE-7083:
------------------------------------
Summary: Map native output collector hang on NativeBatchProcess.nativeFinsh
Key: MAPREDUCE-7083
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7083
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: nativetask
Affects Versions: 3.0.0-alpha1
Reporter: Feng Yuan
Attachments: 4910BB94-7966-4BFB-B2EC-66FE42768F3A.png, 734822C1-F4CB-4B37-8F43-C598CB18F52D.png, js_1, js_native_1
I run the benchmark test demo terasort app.
Input size 100G and block size is 1g, so all is 100 files. One file have only one split.
terasort have 100 map, this mean every map process 1g data
everyone's jvm setting is:
{code:java}
-Xmx2048m -Xms2048m -Xmn256m -XX:MaxDirectMemorySize=128m -XX:SurvivorRatio=6 -XX:MaxPermSize=128m -XX:ParallelGCThreads=10
io.sort.mb is 512mb
{code}
This is heap snapshot
!4910BB94-7966-4BFB-B2EC-66FE42768F3A.png!
This is working output files
!734822C1-F4CB-4B37-8F43-C598CB18F52D.png!
This is Java stack
[^js_1]
This is native stack
[^js_native_1]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org