You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "aloyszhang (JIRA)" <ji...@apache.org> on 2019/07/08 03:41:00 UTC
[jira] [Created] (HIVE-21964) jdbc handler class cast exception
aloyszhang created HIVE-21964:
---------------------------------
Summary: jdbc handler class cast exception
Key: HIVE-21964
URL: https://issues.apache.org/jira/browse/HIVE-21964
Project: Hive
Issue Type: Improvement
Components: JDBC
Affects Versions: 3.1.1
Reporter: aloyszhang
Using hive jdbc handler to query external mysql data source with type decimal type, it throws class cast Exception :
{code:java}
2019-07-08T11:11:50,424 ERROR [7787918f-3111-4706-a3b3-0097fa1bc117 main] CliDriver: Failed with exception java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException: java.math.BigDecimal cannot be cast to org.apache.hadoop.hive.common.type.HiveDecimal
java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException: java.math.BigDecimal cannot be cast to org.apache.hadoop.hive.common.type.HiveDecimal
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:162)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:2691)
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.getResults(ReExecDriver.java:229)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:226)
at org.apache.hadoop.util.RunJar.main(RunJar.java:141)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException: java.math.BigDecimal cannot be cast to org.apache.hadoop.hive.common.type.HiveDecimal
at org.apache.hadoop.hive.ql.exec.ListSinkOperator.process(ListSinkOperator.java:98)
at org.apache.hadoop.hive.ql.exec.Operator.baseForward(Operator.java:995)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:941)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:928)
at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
at org.apache.hadoop.hive.ql.exec.Operator.baseForward(Operator.java:995)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:941)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:125)
at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:519)
at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:511)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:146)
... 14 more
Caused by: java.lang.ClassCastException: java.math.BigDecimal cannot be cast to org.apache.hadoop.hive.common.type.HiveDecimal
at org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaHiveDecimalObjectInspector.getPrimitiveJavaObject(JavaHiveDecimalObjectInspector.java:55)
at org.apache.hadoop.hive.serde2.lazy.LazyUtils.writePrimitiveUTF8(LazyUtils.java:329)
at org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:292)
at org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serializeField(LazySimpleSerDe.java:247)
at org.apache.hadoop.hive.serde2.DelimitedJSONSerDe.serializeField(DelimitedJSONSerDe.java:72)
at org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.doSerialize(LazySimpleSerDe.java:231)
at org.apache.hadoop.hive.serde2.AbstractEncodingAwareSerDe.serialize(AbstractEncodingAwareSerDe.java:55)
at org.apache.hadoop.hive.serde2.DefaultFetchFormatter.convert(DefaultFetchFormatter.java:67)
at org.apache.hadoop.hive.serde2.DefaultFetchFormatter.convert(DefaultFetchFormatter.java:36)
at org.apache.hadoop.hive.ql.exec.ListSinkOperator.process(ListSinkOperator.java:94)
... 24 more
{code}
Some problem with type date and timestamp.
I debug the code and find that was caused by hive-jdbc-handler return result wity type
java.math.Decimal , but when serialize the result row by calling JavaHiveDecimalObjectInspector#getPrimitiveJavaObject , java.math.Decimal cann't cast to org.apache.hadoop.hive.common.type.HiveDecimal.
So, IMO, this is a bug in jdbc-handler code when processing decimal, date and timestamp teyps. I think jdbc-handler should return result with type cast like this :
{code:java}
@Override
public Map<String, Object> next() {
try {
ResultSetMetaData metadata = rs.getMetaData();
int numColumns = metadata.getColumnCount();
Map<String, Object> record = new HashMap<String, Object>(numColumns);
for (int i = 0; i < numColumns; i++) {
String key = metadata.getColumnName(i + 1);
Object value;
if (columnTypes!=null && columnTypes.get(i) instanceof PrimitiveTypeInfo) {
// This is not a complete list, barely make information schema work
switch (((PrimitiveTypeInfo)columnTypes.get(i)).getTypeName()) {
case "int":
case "smallint":
case "tinyint":
value = rs.getInt(i + 1);
break;
case "bigint":
value = rs.getLong(i + 1);
break;
case "float":
value = rs.getFloat(i + 1);
break;
case "double":
value = rs.getDouble(i + 1);
break;
case "decimal":
case "bigdecimal":
value = HiveDecimal.create(rs.getBigDecimal(i + 1));
break;
case "boolean":
value = rs.getBoolean(i + 1);
break;
case "string":
case "char":
case "varchar":
value = rs.getString(i + 1);
break;
case "date":
case "datetime":
value = new Date(rs.getDate(i + 1).toLocalDate());
break;
case "timestamp":
value = new Timestamp((rs.getTimestamp(i + 1)).toLocalDateTime());
break;
default:
value = rs.getObject(i + 1);
break;
}
} else {
value = rs.getObject(i + 1);
}
record.put(key, value);
}
return record;
}
catch (Exception e) {
LOGGER.warn("next() threw exception", e);
return null;
}
}
{code}
As I am quite a novice, So I want to know whether this change has other effects or not .
Any suggestions are appreciated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)