ClassCastException: org.apache.hadoop.io.LongWritable cannot be cast to org.apache.hadoop.hive.serde2.io.TimestampWritable

[Dmitry Goldenberg <dg...@gmail.com>]

Hi,

I apologize for the wide distribution and if this is not the right mailing
list for this.

We write Avro files to Parquet and load them to HDFS so they can be
accessed via an EXTERNAL Hive table.  These records have two timestamp
fields which are expressed in the Avro schema as type = long and
logicalType=timestamp-millis.

When trying to do a SELECT * FROM <the table> we get the error as included
below. Basically, the long values cannot be converted to timestamps.  This
appears similar to https://issues.apache.org/jira/browse/HIVE-13534.

Could someone suggest a workaround? Would we have to make the timestamp
fields strings? Was hoping to avoid that... Thanks

-Dmitry

Bad status for request TFetchResultsReq(

fetchType=0, operationHandle=TOperationHandle(hasResultSet=True,
modifiedRowCount=None, operationType=0,

operationId=THandleIdentifier(secret='\x94yB\xb2\xf47K\x98\xaa\xce\\\xab\xdc_\xcdH',
guid='~\xf9\xd5x\x1e\xe1I*\x91\xddu\x92\xa7\xec\xc6\xda')),

orientation=4, maxRows=100):



TFetchResultsResp(status=TStatus(errorCode=0, errorMessage='java.io.
IOException:

org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException: org.apache.hadoop.io.LongWritable cannot be
cast to org.apache.hadoop.hive.serde2.io.TimestampWritable',

sqlState=None, infoMessages=['*org.apache.hive.service.cli.HiveSQLException:
java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException:

java.lang.ClassCastException: org.apache.hadoop.io.LongWritable cannot be
cast to org.apache.hadoop.hive.serde2.io.TimestampWritable:14:13',

'org.apache.hive.service.cli.operation.SQLOperation:
getNextRowSet:SQLOperation.java:463',

'org.apache.hive.service.cli.operation.OperationManager:
getOperationNextRowSet:OperationManager.java:294',

'org.apache.hive.service.cli.session.HiveSessionImpl:
fetchResults:HiveSessionImpl.java:769',

'org.apache.hive.service.cli.CLIService:fetchResults:CLIService.java:462',

'org.apache.hive.service.cli.thrift.ThriftCLIService:
FetchResults:ThriftCLIService.java:694',

'org.apache.hive.service.cli.thrift.TCLIService$Processor$
FetchResults:getResult:TCLIService.java:1553',

'org.apache.hive.service.cli.thrift.TCLIService$Processor$
FetchResults:getResult:TCLIService.java:1538',

'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39',

'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39',

'org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$
Server$TUGIAssumingProcessor:process:HadoopThriftAuthBridge.java:747',

'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:
TThreadPoolServer.java:286',

'java.util.concurrent.ThreadPoolExecutor:runWorker:
ThreadPoolExecutor.java:1149',

'java.util.concurrent.ThreadPoolExecutor$Worker:run:
ThreadPoolExecutor.java:624',

'java.lang.Thread:run:Thread.java:748',



'*java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException:

java.lang.ClassCastException:

rg.apache.hadoop.io.LongWritable cannot be cast to
org.apache.hadoop.hive.serde2.io.TimestampWritable:16:2',

'org.apache.hadoop.hive.ql.exec.FetchTask:fetch:FetchTask.java:154',

'org.apache.hadoop.hive.ql.Driver:getResults:Driver.java:2069',

'org.apache.hive.service.cli.operation.SQLOperation:
getNextRowSet:SQLOperation.java:458',

'*org.apache.hadoop.hive.ql.metadata.HiveException:java.lang.ClassCastException:
org.apache.hadoop.io.LongWritable cannot be cast to
org.apache.hadoop.hive.serde2.io.TimestampWritable:23:7',

'org.apache.hadoop.hive.ql.exec.ListSinkOperator:processOp:ListSinkOperator.
java:90',

'org.apache.hadoop.hive.ql.exec.Operator:forward:Operator.java:815',

'org.apache.hadoop.hive.ql.exec.SelectOperator:processOp:
SelectOperator.java:84',

'org.apache.hadoop.hive.ql.exec.Operator:forward:Operator.java:815',

'org.apache.hadoop.hive.ql.exec.TableScanOperator:
processOp:TableScanOperator.java:98',

'org.apache.hadoop.hive.ql.exec.FetchOperator:pushRow:
FetchOperator.java:425',

'org.apache.hadoop.hive.ql.exec.FetchOperator:pushRow:
FetchOperator.java:417',

'org.apache.hadoop.hive.ql.exec.FetchTask:fetch:FetchTask.java:140',

'*java.lang.ClassCastException:org.apache.hadoop.io.LongWritable cannot be
cast to org.apache.hadoop.hive.serde2.io.TimestampWritable:28:5',

'org.apache.hadoop.hive.serde2.objectinspector.primitive.
WritableTimestampObjectInspector:getPrimitiveJavaObject:
WritableTimestampObjectInspector.java:39',

'org.apache.hadoop.hive.serde2.objectinspector.primitive.
WritableTimestampObjectInspector:getPrimitiveJavaObject:
WritableTimestampObjectInspector.java:25',

'org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils:
copyToStandardObject:ObjectInspectorUtils.java:336',

'org.apache.hadoop.hive.serde2.SerDeUtils:toThriftPayload:SerDeUtils.
java:167',

'org.apache.hadoop.hive.ql.exec.FetchFormatter$ThriftFormatter:convert:
FetchFormatter.java:61',

'org.apache.hadoop.hive.ql.exec.ListSinkOperator:processOp:ListSinkOperator.java:87'],
statusCode=3), results=None, hasMoreRows=None)