You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Kuhu Shukla (JIRA)" <ji...@apache.org> on 2017/01/19 22:54:27 UTC
[jira] [Updated] (TEZ-3587) Fetcher fetchInputs() can NPE on
srcAttempt due to missing entry in pathToAttemptMap
[ https://issues.apache.org/jira/browse/TEZ-3587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kuhu Shukla updated TEZ-3587:
-----------------------------
Description:
With Tez Shuffle Handler (or composite fetch on), unordered fetcher can fail to populate the pathToAttemptsMap with the correct value of "partition" member of the PathPartition object. This can cause an NPE.
{code}
[main] examples.HashJoinExample (TezExampleBase.java:runDag(190)) - DAG diagnostics: [Vertex failed, vertexName=joiner, vertexId=vertex_123_0001_2_02, diagnostics=[Task failed, taskId=task_123_0001_2_02_000001, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : java.lang.NullPointerException
at org.apache.tez.runtime.library.common.shuffle.Fetcher.fetchInputs(Fetcher.java:893)
at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:597)
at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:486)
at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:284)
at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:76)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
, errorMessage=Fetch failed:java.lang.NullPointerException
at org.apache.tez.runtime.library.common.shuffle.Fetcher.fetchInputs(Fetcher.java:893)
at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:597)
at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:486)
at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:284)
at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:76)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
was:
With Tez Shuffle Handler (or composite fetch on), unordered fetcher can fail to populate the pathToAttemptsMap with the coorect value of "partition" member of the PathPartition object. This can cause an NPE.
{code}
[main] examples.HashJoinExample (TezExampleBase.java:runDag(190)) - DAG diagnostics: [Vertex failed, vertexName=joiner, vertexId=vertex_123_0001_2_02, diagnostics=[Task failed, taskId=task_123_0001_2_02_000001, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : java.lang.NullPointerException
at org.apache.tez.runtime.library.common.shuffle.Fetcher.fetchInputs(Fetcher.java:893)
at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:597)
at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:486)
at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:284)
at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:76)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
, errorMessage=Fetch failed:java.lang.NullPointerException
at org.apache.tez.runtime.library.common.shuffle.Fetcher.fetchInputs(Fetcher.java:893)
at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:597)
at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:486)
at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:284)
at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:76)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
> Fetcher fetchInputs() can NPE on srcAttempt due to missing entry in pathToAttemptMap
> ------------------------------------------------------------------------------------
>
> Key: TEZ-3587
> URL: https://issues.apache.org/jira/browse/TEZ-3587
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Kuhu Shukla
> Assignee: Kuhu Shukla
>
> With Tez Shuffle Handler (or composite fetch on), unordered fetcher can fail to populate the pathToAttemptsMap with the correct value of "partition" member of the PathPartition object. This can cause an NPE.
> {code}
> [main] examples.HashJoinExample (TezExampleBase.java:runDag(190)) - DAG diagnostics: [Vertex failed, vertexName=joiner, vertexId=vertex_123_0001_2_02, diagnostics=[Task failed, taskId=task_123_0001_2_02_000001, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : java.lang.NullPointerException
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.fetchInputs(Fetcher.java:893)
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:597)
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:486)
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:284)
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:76)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> , errorMessage=Fetch failed:java.lang.NullPointerException
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.fetchInputs(Fetcher.java:893)
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:597)
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:486)
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:284)
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:76)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)