You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by George Liaw <ge...@gmail.com> on 2016/11/10 23:18:38 UTC
Hive Runtime Error processing row
I'm running into the below error occasionally and I'm not 100% certain
what's going on. Does anyone have a hunch what might be happening here or
where we can dig for more ideas? Removed row contents but there are
multiple columns.
Vertex failed, vertexName=Map 5, vertexId=vertex_1478744516303_0282_1_07,
> diagnostics=
> Task failed, taskId=task_1478744516303_0282_1_07_000071, diagnostics=
> TaskAttempt 0 failed, info=
> Error: Error while running task ( failure ) :
> attempt_1478744516303_0282_1_07_000071_0:java.lang.RuntimeException:
> java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row {"some_col":"some_val"...}
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:195)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:160)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row {"some_col":"some_val"...}
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:95)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:70)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:356)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:172)
> ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row {"some_col":"some_val"...}
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:574)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
> ... 17 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unexpected
> exception from MapJoinOperator : null
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:454)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:97)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> at
> org.apache.hadoop.hive.ql.exec.FilterOperator.process(FilterOperator.java:128)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:128)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:170)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:564)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableContainer$ReusableRowContainer.setFromOutput(HybridHashTableContainer.java:844)
> at
> org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableContainer$GetAdaptor.setFromRow(HybridHashTableContainer.java:725)
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.setMapJoinKey(MapJoinOperator.java:339)
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:390)
> ... 26 more
--
George A. Liaw
(408) 318-7920
george.a.liaw@gmail.com
LinkedIn <http://www.linkedin.com/in/georgeliaw/>
Re: Hive Runtime Error processing row
Posted by George Liaw <ge...@gmail.com>.
Sure thing.
https://issues.apache.org/jira/browse/HIVE-15188
On Fri, Nov 11, 2016 at 11:09 AM, Sergey Shelukhin <se...@hortonworks.com>
wrote:
> Hi. Can you file a JIRA with exception callstack? Seems to be a bug.
> Thanks!
>
> From: George Liaw <ge...@gmail.com>
> Reply-To: "user@hive.apache.org" <us...@hive.apache.org>
> Date: Thursday, November 10, 2016 at 17:27
> To: "user@hive.apache.org" <us...@hive.apache.org>
> Subject: Re: Hive Runtime Error processing row
>
> Gopal,
>
> Tested it out and that seemed to resolve the issue. Guess it'll have to be
> kept disabled for the time being.
>
> Thanks!
>
> On Thu, Nov 10, 2016 at 3:58 PM, George Liaw <ge...@gmail.com>
> wrote:
>
>> I'll give it a try. This is Hive 2.0.1
>>
>> On Thu, Nov 10, 2016 at 3:26 PM, Gopal Vijayaraghavan <go...@apache.org>
>> wrote:
>>
>>>
>>> > I'm running into the below error occasionally and I'm not 100% certain
>>> what's going on. Does anyone have a hunch what might be happening here or
>>> where we can dig for more ideas? Removed row contents but there are
>>> multiple columns.
>>>
>>> You can try a repro run by doing
>>>
>>> set hive.mapjoin.hybridgrace.hashtable=false;
>>>
>>> to dig into the issue.
>>>
>>> > at org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableCo
>>> ntainer$ReusableRowContainer.setFromOutput(HybridHashTableCo
>>> ntainer.java:844)
>>> > at org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableCo
>>> ntainer$GetAdaptor.setFromRow(HybridHashTableContainer.java:725)
>>>
>>> Which version of Hive is this?
>>>
>>> Cheers,
>>> Gopal
>>>
>>>
>>>
>>
>>
>> --
>> George A. Liaw
>>
>> (408) 318-7920
>> george.a.liaw@gmail.com
>> LinkedIn <http://www.linkedin.com/in/georgeliaw/>
>>
>
>
>
> --
> George A. Liaw
>
> (408) 318-7920
> george.a.liaw@gmail.com
> LinkedIn <http://www.linkedin.com/in/georgeliaw/>
>
--
George A. Liaw
(408) 318-7920
george.a.liaw@gmail.com
LinkedIn <http://www.linkedin.com/in/georgeliaw/>
Re: Hive Runtime Error processing row
Posted by Sergey Shelukhin <se...@hortonworks.com>.
Hi. Can you file a JIRA with exception callstack? Seems to be a bug. Thanks!
From: George Liaw <ge...@gmail.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Thursday, November 10, 2016 at 17:27
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Subject: Re: Hive Runtime Error processing row
Gopal,
Tested it out and that seemed to resolve the issue. Guess it'll have to be kept disabled for the time being.
Thanks!
On Thu, Nov 10, 2016 at 3:58 PM, George Liaw <ge...@gmail.com>> wrote:
I'll give it a try. This is Hive 2.0.1
On Thu, Nov 10, 2016 at 3:26 PM, Gopal Vijayaraghavan <go...@apache.org>> wrote:
> I'm running into the below error occasionally and I'm not 100% certain what's going on. Does anyone have a hunch what might be happening here or where we can dig for more ideas? Removed row contents but there are multiple columns.
You can try a repro run by doing
set hive.mapjoin.hybridgrace.hashtable=false;
to dig into the issue.
> at org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableContainer$ReusableRowContainer.setFromOutput(HybridHashTableContainer.java:844)
> at org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableContainer$GetAdaptor.setFromRow(HybridHashTableContainer.java:725)
Which version of Hive is this?
Cheers,
Gopal
--
George A. Liaw
(408) 318-7920<tel:%28408%29%20318-7920>
george.a.liaw@gmail.com<ma...@gmail.com>
LinkedIn<http://www.linkedin.com/in/georgeliaw/>
--
George A. Liaw
(408) 318-7920
george.a.liaw@gmail.com<ma...@gmail.com>
LinkedIn<http://www.linkedin.com/in/georgeliaw/>
Re: Hive Runtime Error processing row
Posted by George Liaw <ge...@gmail.com>.
Gopal,
Tested it out and that seemed to resolve the issue. Guess it'll have to be
kept disabled for the time being.
Thanks!
On Thu, Nov 10, 2016 at 3:58 PM, George Liaw <ge...@gmail.com>
wrote:
> I'll give it a try. This is Hive 2.0.1
>
> On Thu, Nov 10, 2016 at 3:26 PM, Gopal Vijayaraghavan <go...@apache.org>
> wrote:
>
>>
>> > I'm running into the below error occasionally and I'm not 100% certain
>> what's going on. Does anyone have a hunch what might be happening here or
>> where we can dig for more ideas? Removed row contents but there are
>> multiple columns.
>>
>> You can try a repro run by doing
>>
>> set hive.mapjoin.hybridgrace.hashtable=false;
>>
>> to dig into the issue.
>>
>> > at org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableCo
>> ntainer$ReusableRowContainer.setFromOutput(HybridHashTableCo
>> ntainer.java:844)
>> > at org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableCo
>> ntainer$GetAdaptor.setFromRow(HybridHashTableContainer.java:725)
>>
>> Which version of Hive is this?
>>
>> Cheers,
>> Gopal
>>
>>
>>
>
>
> --
> George A. Liaw
>
> (408) 318-7920
> george.a.liaw@gmail.com
> LinkedIn <http://www.linkedin.com/in/georgeliaw/>
>
--
George A. Liaw
(408) 318-7920
george.a.liaw@gmail.com
LinkedIn <http://www.linkedin.com/in/georgeliaw/>
Re: Hive Runtime Error processing row
Posted by George Liaw <ge...@gmail.com>.
I'll give it a try. This is Hive 2.0.1
On Thu, Nov 10, 2016 at 3:26 PM, Gopal Vijayaraghavan <go...@apache.org>
wrote:
>
> > I'm running into the below error occasionally and I'm not 100% certain
> what's going on. Does anyone have a hunch what might be happening here or
> where we can dig for more ideas? Removed row contents but there are
> multiple columns.
>
> You can try a repro run by doing
>
> set hive.mapjoin.hybridgrace.hashtable=false;
>
> to dig into the issue.
>
> > at org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableContainer$
> ReusableRowContainer.setFromOutput(HybridHashTableContainer.java:844)
> > at org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableContainer$
> GetAdaptor.setFromRow(HybridHashTableContainer.java:725)
>
> Which version of Hive is this?
>
> Cheers,
> Gopal
>
>
>
--
George A. Liaw
(408) 318-7920
george.a.liaw@gmail.com
LinkedIn <http://www.linkedin.com/in/georgeliaw/>
Re: Hive Runtime Error processing row
Posted by Gopal Vijayaraghavan <go...@apache.org>.
> I'm running into the below error occasionally and I'm not 100% certain what's going on. Does anyone have a hunch what might be happening here or where we can dig for more ideas? Removed row contents but there are multiple columns.
You can try a repro run by doing
set hive.mapjoin.hybridgrace.hashtable=false;
to dig into the issue.
> at org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableContainer$ReusableRowContainer.setFromOutput(HybridHashTableContainer.java:844)
> at org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableContainer$GetAdaptor.setFromRow(HybridHashTableContainer.java:725)
Which version of Hive is this?
Cheers,
Gopal