You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "László Bodor (Jira)" <ji...@apache.org> on 2021/11/29 14:23:00 UTC
[jira] [Updated] (TEZ-4356) Ignore some exceptions when the task is already in CLOSED state
[ https://issues.apache.org/jira/browse/TEZ-4356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
László Bodor updated TEZ-4356:
------------------------------
Description:
with HIVE-24207, we can be in a situation where there are input related events to process when the task is already CLOSED
in this case, we simply don't care about certain kinds of exceptions, and should ignore them in order to avoid task failure
{code}
java.lang.RuntimeException: java.io.IOException: java.io.IOException: java.io.IOException: Failed on local exception: java.nio.channels.ClosedByInterruptException; Host Details : local host is: "lbodor-MBP16.local/192.168.0.53"; destination host is: "localhost":54688;
at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:200) ~[tez-mapreduce-0.10.1.jar:0.10.1]
at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.<init>(TezGroupedSplitsInputFormat.java:139) ~[tez-mapreduce-0.10.1.jar:0.10.1]
at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat.getRecordReader(TezGroupedSplitsInputFormat.java:105) ~[tez-mapreduce-0.10.1.jar:0.10.1]
at org.apache.tez.mapreduce.lib.MRReaderMapred.setupOldRecordReader(MRReaderMapred.java:164) ~[tez-mapreduce-0.10.1.jar:0.10.1]
at org.apache.tez.mapreduce.lib.MRReaderMapred.<init>(MRReaderMapred.java:76) ~[tez-mapreduce-0.10.1.jar:0.10.1]
at org.apache.tez.mapreduce.input.MultiMRInput.initFromEvent(MultiMRInput.java:196) ~[tez-mapreduce-0.10.1.jar:0.10.1]
at org.apache.tez.mapreduce.input.MultiMRInput.handleEvents(MultiMRInput.java:154) ~[tez-mapreduce-0.10.1.jar:0.10.1]
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.handleEvent(LogicalIOProcessorRuntimeTask.java:729) [tez-runtime-internals-0.10.1.jar:0.10.1]
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.access$600(LogicalIOProcessorRuntimeTask.java:110) [tez-runtime-internals-0.10.1.jar:0.10.1]
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$1.runInternal(LogicalIOProcessorRuntimeTask.java:817) [tez-runtime-internals-0.10.1.jar:0.10.1]
at org.apache.tez.common.RunnableWithNdc.run(RunnableWithNdc.java:35) [tez-common-0.10.1.jar:0.10.1]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292]
Caused by: java.io.IOException: java.io.IOException: java.io.IOException: Failed on local exception: java.nio.channels.ClosedByInterruptException; Host Details : local host is: "lbodor-MBP16.local/192.168.0.53"; destination host is: "localhost":54688;
at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97) ~[hive-shims-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57) ~[hive-shims-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:449) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:197) ~[tez-mapreduce-0.10.1.jar:0.10.1]
... 11 more
Caused by: java.io.IOException: java.io.IOException: Failed on local exception: java.nio.channels.ClosedByInterruptException; Host Details : local host is: "lbodor-MBP16.local/192.168.0.53"; destination host is: "localhost":54688;
at org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat.getRecordReader(LlapInputFormat.java:141) ~[hive-llap-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.io.RecordReaderWrapper.create(RecordReaderWrapper.java:72) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:446) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:197) ~[tez-mapreduce-0.10.1.jar:0.10.1]
... 11 more
{code}
> Ignore some exceptions when the task is already in CLOSED state
> ---------------------------------------------------------------
>
> Key: TEZ-4356
> URL: https://issues.apache.org/jira/browse/TEZ-4356
> Project: Apache Tez
> Issue Type: Bug
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
>
> with HIVE-24207, we can be in a situation where there are input related events to process when the task is already CLOSED
> in this case, we simply don't care about certain kinds of exceptions, and should ignore them in order to avoid task failure
> {code}
> java.lang.RuntimeException: java.io.IOException: java.io.IOException: java.io.IOException: Failed on local exception: java.nio.channels.ClosedByInterruptException; Host Details : local host is: "lbodor-MBP16.local/192.168.0.53"; destination host is: "localhost":54688;
> at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:200) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.<init>(TezGroupedSplitsInputFormat.java:139) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat.getRecordReader(TezGroupedSplitsInputFormat.java:105) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> at org.apache.tez.mapreduce.lib.MRReaderMapred.setupOldRecordReader(MRReaderMapred.java:164) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> at org.apache.tez.mapreduce.lib.MRReaderMapred.<init>(MRReaderMapred.java:76) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> at org.apache.tez.mapreduce.input.MultiMRInput.initFromEvent(MultiMRInput.java:196) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> at org.apache.tez.mapreduce.input.MultiMRInput.handleEvents(MultiMRInput.java:154) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.handleEvent(LogicalIOProcessorRuntimeTask.java:729) [tez-runtime-internals-0.10.1.jar:0.10.1]
> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.access$600(LogicalIOProcessorRuntimeTask.java:110) [tez-runtime-internals-0.10.1.jar:0.10.1]
> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$1.runInternal(LogicalIOProcessorRuntimeTask.java:817) [tez-runtime-internals-0.10.1.jar:0.10.1]
> at org.apache.tez.common.RunnableWithNdc.run(RunnableWithNdc.java:35) [tez-common-0.10.1.jar:0.10.1]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292]
> Caused by: java.io.IOException: java.io.IOException: java.io.IOException: Failed on local exception: java.nio.channels.ClosedByInterruptException; Host Details : local host is: "lbodor-MBP16.local/192.168.0.53"; destination host is: "localhost":54688;
> at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97) ~[hive-shims-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57) ~[hive-shims-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:449) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:197) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> ... 11 more
> Caused by: java.io.IOException: java.io.IOException: Failed on local exception: java.nio.channels.ClosedByInterruptException; Host Details : local host is: "lbodor-MBP16.local/192.168.0.53"; destination host is: "localhost":54688;
> at org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat.getRecordReader(LlapInputFormat.java:141) ~[hive-llap-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.io.RecordReaderWrapper.create(RecordReaderWrapper.java:72) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:446) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:197) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> ... 11 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)