[jira] Reopened: (HIVE-1422) skip counter update when RunningJob.getCounters() returns null

["Joydeep Sen Sarma (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joydeep Sen Sarma reopened HIVE-1422:
-------------------------------------


reopening. i think the bug still exists:

jc.getJob(rj.getJobID()) can return null if the jobtracker has retired the job by the time we poll for it. this happens very rarely. we are setting the returned value in the execdriver handle. and then trying to dereference it later on.

if we have a valid runningjob reference - i don't think we have seen the getCounters call return null (in the past).

> skip counter update when RunningJob.getCounters() returns null
> --------------------------------------------------------------
>
>                 Key: HIVE-1422
>                 URL: https://issues.apache.org/jira/browse/HIVE-1422
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.6.0
>            Reporter: John Sichi
>            Assignee: John Sichi
>             Fix For: 0.7.0
>
>         Attachments: HIVE-1422.1.patch
>
>
> Under heavy load circumstances on some Hadoop versions, we may get a NPE from trying to dereference a null Counters object.  I don't have a unit test which can reproduce it, but here's an example stack from a production cluster we saw today:
> 10/06/21 13:01:10 ERROR exec.ExecDriver: Ended Job = job_201005200457_701060 with exception 'java.lang.NullPointerException(null)'
> java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.exec.Operator.updateCounters(Operator.java:999)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.updateCounters(ExecDriver.java:503)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.progress(ExecDriver.java:390)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:697)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.