You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tez.apache.org by "László Bodor (Jira)" <ji...@apache.org> on 2021/11/30 09:16:00 UTC
[jira] [Resolved] (TEZ-4356) Ignore some exceptions when the task is already in CLOSED state
[ https://issues.apache.org/jira/browse/TEZ-4356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
László Bodor resolved TEZ-4356.
-------------------------------
Resolution: Won't Fix
> Ignore some exceptions when the task is already in CLOSED state
> ---------------------------------------------------------------
>
> Key: TEZ-4356
> URL: https://issues.apache.org/jira/browse/TEZ-4356
> Project: Apache Tez
> Issue Type: Bug
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> with HIVE-24207, we can be in a situation where there are input related events to process when the task is already CLOSED
> in this case, we simply don't care about certain kinds of exceptions and should ignore them in order to avoid task failure
> thinking about whether it is unsafe to ignore these exceptions...I'm assuming that processing events for a closed task is just a best-effort behavior and we never rely on that
> {code}
> java.lang.RuntimeException: java.io.IOException: java.io.IOException: java.io.IOException: Failed on local exception: java.nio.channels.ClosedByInterruptException; Host Details : local host is: "lbodor-MBP16.local/192.168.0.53"; destination host is: "localhost":54688;
> at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:200) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.<init>(TezGroupedSplitsInputFormat.java:139) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat.getRecordReader(TezGroupedSplitsInputFormat.java:105) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> at org.apache.tez.mapreduce.lib.MRReaderMapred.setupOldRecordReader(MRReaderMapred.java:164) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> at org.apache.tez.mapreduce.lib.MRReaderMapred.<init>(MRReaderMapred.java:76) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> at org.apache.tez.mapreduce.input.MultiMRInput.initFromEvent(MultiMRInput.java:196) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> at org.apache.tez.mapreduce.input.MultiMRInput.handleEvents(MultiMRInput.java:154) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.handleEvent(LogicalIOProcessorRuntimeTask.java:729) [tez-runtime-internals-0.10.1.jar:0.10.1]
> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.access$600(LogicalIOProcessorRuntimeTask.java:110) [tez-runtime-internals-0.10.1.jar:0.10.1]
> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$1.runInternal(LogicalIOProcessorRuntimeTask.java:817) [tez-runtime-internals-0.10.1.jar:0.10.1]
> at org.apache.tez.common.RunnableWithNdc.run(RunnableWithNdc.java:35) [tez-common-0.10.1.jar:0.10.1]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292]
> Caused by: java.io.IOException: java.io.IOException: java.io.IOException: Failed on local exception: java.nio.channels.ClosedByInterruptException; Host Details : local host is: "lbodor-MBP16.local/192.168.0.53"; destination host is: "localhost":54688;
> at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97) ~[hive-shims-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57) ~[hive-shims-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:449) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:197) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> ... 11 more
> Caused by: java.io.IOException: java.io.IOException: Failed on local exception: java.nio.channels.ClosedByInterruptException; Host Details : local host is: "lbodor-MBP16.local/192.168.0.53"; destination host is: "localhost":54688;
> at org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat.getRecordReader(LlapInputFormat.java:141) ~[hive-llap-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.io.RecordReaderWrapper.create(RecordReaderWrapper.java:72) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:446) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:197) ~[tez-mapreduce-0.10.1.jar:0.10.1]
> ... 11 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)