[jira] [Commented] (HUDI-60) [UMBRELLA] Support Apache Beam / Hudi IO

["Vinoth Chandar (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HUDI-60?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17823691#comment-17823691 ] 

Vinoth Chandar commented on HUDI-60:
------------------------------------

Thanks for taking this up [~xuzifu] ! 

> [UMBRELLA] Support Apache Beam / Hudi IO
> ----------------------------------------
>
>                 Key: HUDI-60
>                 URL: https://issues.apache.org/jira/browse/HUDI-60
>             Project: Apache Hudi
>          Issue Type: Epic
>          Components: spark, Utilities
>            Reporter: Vinoth Chandar
>            Assignee: xy
>            Priority: Major
>              Labels: gsoc, gsoc2021, hudi-umbrellas, mentor
>             Fix For: 0.15.0
>
>
> We would like to add a HudiIO for Beam, along the lines of [https://github.com/apache/beam/blob/master/sdks/java/io/parquet/src/main/java/org/apache/beam/sdk/io/parquet/ParquetIO.java] 
> For the initial cut : we can leave the table services turned off on the writer and advise users to run them independently?
> During this work - we can also look into anything need to be fixed on the java-client module, which works with GenericRecords as well (used by the Kafka Connect Sink). So if thats in shape, this can be much easier.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)