You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/07/23 00:19:27 UTC
[GitHub] [hudi] qingyuan18 commented on issue #1854: query MOR table using spark sql error
qingyuan18 commented on issue #1854:
URL: https://github.com/apache/hudi/issues/1854#issuecomment-662761051
> Is the table (xxxx.xxxx_acidtest2) registered as Hive table. If so, can you provide the complete table description of the table (desc formatted
>
> ) in Hive metastore.
hi Balaji
Thanks for your support!
yes, it's hive managed table , please see below table details:
hive -e "desc formatted xxxx.xxxx_acidtest2"
# col_name data_type comment
case_no string
case_id string
evnt_nm string
evnt_crt_loc_dt string
evnt_crt_loc_ts string
ordr_ver_no string
evnt_stat_desc string
prcs_tp_nm string
lanid string
team string
cntry_nm string
# Detailed Table Information
Database: xxxx
OwnerType: USER
CreateTime: Mon Jul 20 17:29:11 AEST 2020
LastAccessTime: UNKNOWN
Retention: 0
Location: hdfs://nameservice1/data/operations/hive/raw/xxxx_acidtest2
Table Type: MANAGED_TABLE
Table Parameters:
transient_lastDdlTime 1595230151
# Storage Information
SerDe Library: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe
InputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat
OutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat
Compressed: No
Num Buckets: -1
Bucket Columns: []
Sort Columns: []
Storage Desc Params:
serialization.format 1
I have added the spark.sql.hive.convertMetastoreParquet=false to try to avoid spark sql use it native parquet parser
Thanks again
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org