You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Sergey Shelukhin (JIRA)" <ji...@apache.org> on 2015/06/30 22:09:04 UTC
[jira] [Commented] (HIVE-11015) LLAP: MiniTez tez_smb_main,
tez_bmj_schema_evolution fail with NPE
[ https://issues.apache.org/jira/browse/HIVE-11015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14608983#comment-14608983 ]
Sergey Shelukhin commented on HIVE-11015:
-----------------------------------------
Root cause are threadlocals added to IOContext. The context is actually accessed from different threads for the same task. So... (numbers are object identities for execmappercontext and iocontext).
Note that most of the accesses come from TezChild, but the path is set from TezTaskEventRouter, so it is null in TezChild copy of the context
{noformat}
2015-06-30 12:25:36,305 ERROR [TezChild] io.IOContext: TODO# created {default => 1468048601}
2015-06-30 12:25:36,306 ERROR [TezChild] io.IOContext: TODO# created {Map 2 => 245048063}
2015-06-30 12:25:36,310 INFO [TezChild] mr.ExecMapperContext: TODO# Setting up 2088825942 with input Map 2 with 245048063
2015-06-30 12:25:37,422 ERROR [TezTaskEventRouter[attempt_1435692240627_0002_5_01_000002_0]] io.IOContext: TODO# created {default => 2096000139}
2015-06-30 12:25:37,422 ERROR [TezTaskEventRouter[attempt_1435692240627_0002_5_01_000002_0]] io.IOContext: TODO# created {Map 2 => 2106605654}
2015-06-30 12:25:37,423 ERROR [TezTaskEventRouter[attempt_1435692240627_0002_5_01_000002_0]] io.HiveContextAwareRecordReader: TODO# setting 2106605654 with hdfs://localhost:51461/build/ql/test/data/warehouse/tab_part/ds=2008-04-08/000002_0
2015-06-30 12:25:37,573 ERROR [TezChild] mr.ExecMapperContext: TODO# from 2088825942 and 245048063, got null
{noformat}
> LLAP: MiniTez tez_smb_main, tez_bmj_schema_evolution fail with NPE
> ------------------------------------------------------------------
>
> Key: HIVE-11015
> URL: https://issues.apache.org/jira/browse/HIVE-11015
> Project: Hive
> Issue Type: Sub-task
> Reporter: Sergey Shelukhin
>
> Didn't spend a lot of time investigating
> {noformat}
> 2015-06-15 17:00:49,334 ERROR [main]: SessionState (SessionState.java:printError(984)) - Vertex failed, vertexName=Map 2, vertexId=vertex_1434412732572_0002_5_01, diagnostics=[Task failed, taskId=task_1434412732572_0002_5_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1434412732572_0002_5_01_000000_0:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:181)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:146)
> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:349)
> at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
> at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:60)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:60)
> at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:35)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:255)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:157)
> ... 14 more
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.exec.MapOperator.getNominalPath(MapOperator.java:399)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeContexts(MapOperator.java:644)
> at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.getKeyValueReader(MapRecordProcessor.java:293)
> at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.initializeMapRecordSources(MapRecordProcessor.java:269)
> at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:223)
> ... 15 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)