[GitHub] [hudi] RexXiong commented on a diff in pull request #5250: [HUDI-3817] shade parquet dependency for hudi-hadoop-mr-bundle

[GitBox <gi...@apache.org>]

RexXiong commented on code in PR #5250:
URL: https://github.com/apache/hudi/pull/5250#discussion_r929470525


##########
packaging/hudi-hadoop-mr-bundle/pom.xml:
##########
@@ -67,8 +67,9 @@
                 <includes>
                   <include>org.apache.hudi:hudi-common</include>
                   <include>org.apache.hudi:hudi-hadoop-mr</include>
-
+                  <!-- Parquet -->
                   <include>org.apache.parquet:parquet-avro</include>
+                  <include>org.apache.parquet:parquet-hadoop-bundle</include>

Review Comment:
   There are two proposals：
   1）The read and write engines (according to spark parquet-avro version)use the same version which is the meaning of this patch
   2）The read engines such as hive use their own parquet-* version
   
   for hive2 parquet-hadoop version is 1.8.1,  hive3 parquet-hadoop version is 1.10.0, which is not compatible with the version of parquet-avro. 
   
   So  the second solution may take 1.8.1 for hive2, 1.10.0 for hive3 ,but has also a bit inconsistent with the write engines.
   
   
   
   
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org