You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hivemall.apache.org by "Makoto Yui (JIRA)" <ji...@apache.org> on 2019/02/05 08:18:00 UTC
[jira] [Updated] (HIVEMALL-236) to_json cause
KryoException/NullPointerException with ArrayList due to Kryo bug
[ https://issues.apache.org/jira/browse/HIVEMALL-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Makoto Yui updated HIVEMALL-236:
--------------------------------
Affects Version/s: 0.5.2
> to_json cause KryoException/NullPointerException with ArrayList due to Kryo bug
> -------------------------------------------------------------------------------
>
> Key: HIVEMALL-236
> URL: https://issues.apache.org/jira/browse/HIVEMALL-236
> Project: Hivemall
> Issue Type: Bug
> Affects Versions: 0.5.2
> Reporter: Makoto Yui
> Assignee: Makoto Yui
> Priority: Major
>
> {code:java}
> SELECT
> from_json(
> '{ "person" : { "name" : "makoto" , "age" : 37 } }',
> 'struct<name:string,age:int>',
> array('person')
> ){code}
> to_json cause the following Kryo exception depending on underlying Kryo version.
> {code:java}
> Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> columnNames (hivemall.tools.json.ToJsonUDF)
> genericUDF (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)
> colExprMap (org.apache.hadoop.hive.ql.exec.SelectOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
> aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:139)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:17)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:139)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:17)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:672)
> at org.apache.hadoop.hive.ql.exec.Utilities.deserializeObjectByKryo(Utilities.java:915)
> at org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan(Utilities.java:823)
> at org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan(Utilities.java:837)
> at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:330)
> ... 17 more
> Caused by: java.lang.NullPointerException
> at java.util.Arrays$ArrayList.size(Arrays.java:3818)
> at java.util.AbstractList.add(AbstractList.java:108)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:105)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> ... 44 more{code}
> BTW, the following query will run without error:
> {code:java}
> select
> from_json(
> '[0.1,1.1,2.2]',
> 'array<double>'
> ),{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)