[idea] A new IO connector named DataLakeIO, which support to connect Beam and data lake, such as Delta Lake, Apache Hudi, Apache iceberg.

[张涛 <zh...@nanhulab.ac.cn>]

Hi, we developed a new IO connector named DataLakeIO, to connect Beam and data lake, such as Delta Lake, Apache Hudi, Apache iceberg. Beam can use DataLakeIO to read data from data lake, and write data to data lake. We did not find data lake IO on https://beam.apache.org/documentation/io/built-in/, we want to contribute this new IO connector to Beam, what should we do next? Thank you very much!

Re: [idea] A new IO connector named DataLakeIO, which support to connect Beam and data lake, such as Delta Lake, Apache Hudi, Apache iceberg.

[Sachin Agarwal via dev <de...@beam.apache.org>]

This is wonderful to hear -
https://beam.apache.org/contribute/get-started-contributing/#contribute-code
has the process to contribute; we're very much looking forward to seeing
your DataLakeIO!

On Fri, Aug 5, 2022 at 9:02 AM 张涛 <zh...@nanhulab.ac.cn> wrote:

>
> Hi, we developed a new IO connector named DataLakeIO, to connect Beam and
> data lake, such as Delta Lake, Apache Hudi, Apache iceberg. Beam can use
> DataLakeIO to read data from data lake, and write data to data lake. We did
> not find data lake IO on
> https://beam.apache.org/documentation/io/built-in/, we want to contribute
> this new IO connector to Beam, what should we do next? Thank you very
> much!
>