You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Pratyaksh Sharma (Jira)" <ji...@apache.org> on 2019/10/15 07:42:00 UTC

[jira] [Commented] (HUDI-259) Hadoop 3 support for Hudi writing

    [ https://issues.apache.org/jira/browse/HUDI-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16951692#comment-16951692 ] 

Pratyaksh Sharma commented on HUDI-259:
---------------------------------------

Hi [~vinoth], here are the pom changes in hoodie-0.4.7 -> 
 # pom.xml - hadoop version updated to 3.1.0, hive version updated to 3.1.0, spark version updated to 2.3.2 and hbase version updated to 2.0.2
 # Also since our production kafka cluster is by default ssl enabled, I had to update spark-streaming-kafka artifact to spark-streaming-kafka-0-10_2.11. Also one supporting dependency of {{spark-sql-kafka-0-10_2.11}} had to be included so as to be able to rewrite KafkaOffsetGen.java class. 

 

After a long time, now I can focus on fixing test cases again, so thought of discussing the changes with you here as suggested by you. :)

> Hadoop 3 support for Hudi writing
> ---------------------------------
>
>                 Key: HUDI-259
>                 URL: https://issues.apache.org/jira/browse/HUDI-259
>             Project: Apache Hudi (incubating)
>          Issue Type: Improvement
>          Components: Usability
>            Reporter: Vinoth Chandar
>            Priority: Major
>
> Sample issues
>  
> [https://github.com/apache/incubator-hudi/issues/735]
> [https://github.com/apache/incubator-hudi/issues/877#issuecomment-528433568] 
> [https://github.com/apache/incubator-hudi/issues/898]
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)