[jira] [Updated] (HUDI-907) Test Presto mor query support changes in HDFS Env

["Udit Mehrotra (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Udit Mehrotra updated HUDI-907:
-------------------------------
    Fix Version/s:     (was: 0.9.0)
                   0.10.0

> Test Presto mor query support changes in HDFS Env
> -------------------------------------------------
>
>                 Key: HUDI-907
>                 URL: https://issues.apache.org/jira/browse/HUDI-907
>             Project: Apache Hudi
>          Issue Type: Sub-task
>          Components: Presto Integration
>    Affects Versions: 0.9.0
>            Reporter: Bhavani Sudha
>            Assignee: Bhavani Sudha
>            Priority: Major
>             Fix For: 0.10.0
>
>
> Test presto integration for HDFS environment as well in addition to S3.
>  
> Blockers faced so far
> [~bdscheller] I tried to apply your presto patch to test mor queries on Presto. The way I set it up was create a docker image from your presto patch and use that image in hudi local docker environment. I observed couple of issues there:
>  * I got NoClassDefFoundError for these classes:
>  ** org/apache/parquet/avro/AvroSchemaConverter
>  ** org/apache/parquet/hadoop/ParquetFileReader
>  ** org/apache/parquet/io/InputFile
>  ** org/apache/parquet/format/TypeDefinedOrder
> I was able to get around the first three errors by shading org.apache.parquet inside hudi-presto-bundle and changing presto-hive to depend on the hudi-presto-bundle. However, for the last one shading dint help because its already a Thrift generated class. I am wondering you  also ran into similar issues while testing S3.  
> Could you please elaborate your test set up so we can do similar thing for HDFS as well. If we need to add more changes to hudi-presto-bundle, we would need to prioritize that for 0.5.3 release asap.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)