You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Jason Lowe (JIRA)" <ji...@apache.org> on 2016/02/08 16:19:39 UTC
[jira] [Commented] (TEZ-1944) OOM when using
tez.runtime.shuffle.memory-to-memory.enable=true
[ https://issues.apache.org/jira/browse/TEZ-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137059#comment-15137059 ]
Jason Lowe commented on TEZ-1944:
---------------------------------
This seems likely caused by the same problem reported in TEZ-1911.
> OOM when using tez.runtime.shuffle.memory-to-memory.enable=true
> ---------------------------------------------------------------
>
> Key: TEZ-1944
> URL: https://issues.apache.org/jira/browse/TEZ-1944
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rajesh Balamohan
>
> ETL workload at 10 TB scale
> {code}
> $HIVE_HOME/bin/hive -hiveconf tez.runtime.io.sort.factor=200 --hiveconf hive.tez.auto.reducer.parallelism=true --hiveconf tez.am.heartbeat.interval-ms.max=20 --hiveconf tez.runtime.io.sort.mb=1200 --hiveconf tez.runtime.sort.threads=2 --hiveconf hive.tez.container.size=4096 --hiveconf tez.runtime.shuffle.memory-to-memory.enable=true --hiveconf tez.runtime.shuffle.memory-to-memory.segments=4
> create table testData as select ss_sold_date_sk,ss_sold_time_sk,ss_item_sk,ss_customer_sk,ss_quantity,ss_sold_date from store_sales distribute by ss_sold_date;
> {code}
> Tasks starts throwing OOM with the following exception.
> {code}
> aused by: java.lang.OutOfMemoryError: Java heap space
> at org.apache.hadoop.io.BoundedByteArrayOutputStream.<init>(BoundedByteArrayOutputStream.java:56)
> at org.apache.hadoop.io.BoundedByteArrayOutputStream.<init>(BoundedByteArrayOutputStream.java:46)
> at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MapOutput.<init>(MapOutput.java:81)
> at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MapOutput.createMemoryMapOutput(MapOutput.java:133)
> at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeManager.unconditionalReserve(MergeManager.java:397)
> at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeManager.access$300(MergeManager.java:74)
> at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeManager$IntermediateMemoryToMemoryMerger.merge(MergeManager.java:530)
> at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeThread.run(MergeThread.java:89)
> , errorMessage=Shuffle Runner Failed:org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle$ShuffleError: error in shuffle in MemToMemMerger [Map_1]
> at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle$RunShuffleCallable.call(Shuffle.java:349)
> at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle$RunShuffleCallable.call(Shuffle.java:328)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Created this to track issues related to memory to memory merging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)