You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Hitesh Shah (JIRA)" <ji...@apache.org> on 2015/05/28 06:05:17 UTC
[jira] [Comment Edited] (TEZ-2440) Sorter should check for
indexCacheList.size() in flush()
[ https://issues.apache.org/jira/browse/TEZ-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14562261#comment-14562261 ]
Hitesh Shah edited comment on TEZ-2440 at 5/28/15 4:05 AM:
-----------------------------------------------------------
[~rajesh.balamohan] Is there a reason for this not to go into 0.7 and earlier branches?
was (Author: hitesh):
[~rajesh.balamohan] Is there a reason for this not to go into 0.7?
> Sorter should check for indexCacheList.size() in flush()
> --------------------------------------------------------
>
> Key: TEZ-2440
> URL: https://issues.apache.org/jira/browse/TEZ-2440
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rajesh Balamohan
> Assignee: Mit Desai
> Fix For: 0.8.0
>
> Attachments: TEZ-2440-1.patch, TEZ-2440-2.patch
>
>
> {noformat}
> 015-05-11 20:28:20,225 INFO [main] task.TezTaskRunner: Shutdown requested... returning
> 2015-05-11 20:28:20,225 INFO [main] task.TezChild: Got a shouldDie notification via hearbeats. Shutting down
> 2015-05-11 20:28:20,231 INFO [TezChild] impl.PipelinedSorter: Thread interrupted, cleaned up stale data, sorter threads shutdown=true, terminated=false
> 2015-05-11 20:28:20,231 INFO [TezChild] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter
> 2015-05-11 20:28:20,231 INFO [TezChild] runtime.LogicalIOProcessorRuntimeTask: Ignoring interrupt while waiting for the router thread to die
> 2015-05-11 20:28:20,232 INFO [TezChild] task.TezTaskRunner: Encounted an error while executing task: attempt_1429683757595_0875_1_07_000000_0
> java.lang.ArrayIndexOutOfBoundsException: -1
> at java.util.ArrayList.elementData(ArrayList.java:418)
> at java.util.ArrayList.get(ArrayList.java:431)
> at org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter.flush(PipelinedSorter.java:462)
> at org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput.close(OrderedPartitionedKVOutput.java:183)
> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.close(LogicalIOProcessorRuntimeTask.java:360)
> at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
> at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> {noformat}
> When a DAG is killed in the middle, sometimes these exceptions are thrown (e.g q_17 in TPC-DS). Even though it is completely harmless, it would be better to fix it to avoid distraction when debugging
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)