You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/07/13 18:31:03 UTC

[GitHub] [hudi] zhedoubushishi commented on issue #1790: [SUPPORT] Querying MoR tables with DecimalType columns via Spark SQL fails

zhedoubushishi commented on issue #1790:
URL: https://github.com/apache/hudi/issues/1790#issuecomment-657720726


   I think it's this pr that fix the bug for your exception: https://github.com/apache/hudi/pull/1677/files.
   
   For the ```hive-exec-1.2.1-spark2-amzn-1.jar``` issue, I don't think it's specifically related to EMR because all the  Spark 2.x depends on ```1.2.1.spark2``` hive version: https://github.com/apache/spark/blob/v2.4.6/pom.xml#L129.
   
   To solve this issue, I would suggest using ```HiveDecimalUtils.enforcePrecisionScale(HiveDecimalWritable writable, DecimalTypeInfo typeInfo)``` instead of https://github.com/apache/hudi/blob/2603cfb33e272632d7f36a53e1b13fe86dbb8627/hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieRealtimeRecordReaderUtils.java#L213. Because ```HiveDecimalUtils.enforcePrecisionScale(HiveDecimalWritable writable, DecimalTypeInfo typeInfo)``` is compatible with both Hive 2.3.1 and Hive 1.2.0.
   
   It this makes sense, I can create a pr for this.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org