You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Daniel Haviv <da...@veracity-group.com> on 2015/10/13 20:14:29 UTC

Generated ORC files cause NPE in Hive

Hi,
We are inserting streaming data into a hive orc table via a simple insert
statement passed to HiveContext.
When trying to read the files generated using Hive 1.2.1 we are getting NPE:
        at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:91)
        at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
        at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:290)
        at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:148)
        ... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row
        at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:52)
        at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83)
        ... 17 more
Caused by: java.lang.NullPointerException
        at java.lang.System.arraycopy(Native Method)
        at org.apache.hadoop.io.Text.set(Text.java:225)
        at
org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow$StringExtractorByValue.extract(VectorExtractRow.java:472)
        at
org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.extractRow(VectorExtractRow.java:732)
        at
org.apache.hadoop.hive.ql.exec.vector.VectorReduceSinkOperator.process(VectorReduceSinkOperator.java:102)
        at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
        at
org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:138)
        at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
        at
org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:97)
        at
org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:162)
        at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:45)
        ... 18 more

Is this a known issue ?

Re: Generated ORC files cause NPE in Hive

Posted by Alexander Pivovarov <ap...@gmail.com>.
Daniel,

Looks like we already have Jira for that error
https://issues.apache.org/jira/browse/HIVE-11431

Could you put details on how to reproduce the issue to the ticket?

Thank you
Alex

On Tue, Oct 13, 2015 at 11:14 AM, Daniel Haviv <
daniel.haviv@veracity-group.com> wrote:

> Hi,
> We are inserting streaming data into a hive orc table via a simple insert
> statement passed to HiveContext.
> When trying to read the files generated using Hive 1.2.1 we are getting
> NPE:
>         at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:91)
>         at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
>         at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:290)
>         at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:148)
>         ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row
>         at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:52)
>         at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83)
>         ... 17 more
> Caused by: java.lang.NullPointerException
>         at java.lang.System.arraycopy(Native Method)
>         at org.apache.hadoop.io.Text.set(Text.java:225)
>         at
> org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow$StringExtractorByValue.extract(VectorExtractRow.java:472)
>         at
> org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.extractRow(VectorExtractRow.java:732)
>         at
> org.apache.hadoop.hive.ql.exec.vector.VectorReduceSinkOperator.process(VectorReduceSinkOperator.java:102)
>         at
> org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
>         at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:138)
>         at
> org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
>         at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:97)
>         at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:162)
>         at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:45)
>         ... 18 more
>
> Is this a known issue ?
>
>