You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@hudi.apache.org by leesf <le...@gmail.com> on 2020/08/09 15:24:00 UTC
[ANNOUNCE] Hudi Community Weekly Update(2020-08-02 ~ 2020-08-09)
Dear community,
Nice to share Hudi community weekly update for 2020-08-02 ~ 2020-08-09 with
updates on features, bugfixs.
=======================================
Features
[Writer Core] Support for RFC-12/Bootstrapping of external datasets to hudi
[1]
[Writer Core] Spark Streaming with async compaction support [2]
[Spark Integration] Speedup spark read queries by caching metaclient in
HoodieROPathFilter [3]
[Metrics] Added a console metrics reporter and associated unit tests. [4]
[Hive Integration] Abstract hudi-sync-common, and support hudi-hive-sync,
hudi-dla-sync [5]
[Writer Core] Parallelize fetching of source data files/partitions [6]
[Spark Integration] Support Spark Datasource for MOR table - RDD approach
[7]
[Writer Core] Implement CLI support for performing bootstrap [8]
[Metrics] Hudi Supports Prometheus Pushgateway [9]
=======================================
Bugs
[Writer Core] lack of insert info in delta_commit inflight [10]
[DeltaStreamer] Fix Jcommander issue for --hoodie-conf in DeltaStreamer [11]
[DeltaStreamer] Fix NPE when no new data in kafka using HoodieDeltaStreamer
[12]
[1] https://issues.apache.org/jira/browse/HUDI-242
[2] https://issues.apache.org/jira/browse/HUDI-575
[3] https://issues.apache.org/jira/browse/HUDI-1144
[4] https://issues.apache.org/jira/browse/HUDI-1149
[5] https://issues.apache.org/jira/browse/HUDI-875
[6] https://issues.apache.org/jira/browse/HUDI-999
[7] https://issues.apache.org/jira/browse/HUDI-69
[8] https://issues.apache.org/jira/browse/HUDI-971
[9] https://issues.apache.org/jira/browse/HUDI-210
[10] https://issues.apache.org/jira/browse/HUDI-525
[11] https://issues.apache.org/jira/browse/HUDI-1140
[12] https://issues.apache.org/jira/browse/HUDI-1151
Best,
Leesf
Re: [ANNOUNCE] Hudi Community Weekly Update(2020-08-02 ~ 2020-08-09)
Posted by Vinoth Chandar <vi...@apache.org>.
+1 thanks leesf. I actually find these very useful when composing the
reports also.:)
On Sun, Aug 9, 2020 at 5:32 PM vino yang <ya...@gmail.com> wrote:
> Thanks to leesf for continuously updating Hudi weekly.
>
> It is great to see that more and more improvements are being proposed in
> the community.
>
> Best,
> Vino
>
> leesf <le...@gmail.com> 于2020年8月9日周日 下午11:24写道：
>
> > Dear community,
> >
> > Nice to share Hudi community weekly update for 2020-08-02 ~ 2020-08-09
> > with updates on features, bugfixs.
> >
> > =======================================
> > Features
> >
> > [Writer Core] Support for RFC-12/Bootstrapping of external datasets to
> > hudi [1]
> > [Writer Core] Spark Streaming with async compaction support [2]
> > [Spark Integration] Speedup spark read queries by caching metaclient in
> > HoodieROPathFilter [3]
> > [Metrics] Added a console metrics reporter and associated unit tests. [4]
> > [Hive Integration] Abstract hudi-sync-common, and support hudi-hive-sync,
> > hudi-dla-sync [5]
> > [Writer Core] Parallelize fetching of source data files/partitions [6]
> > [Spark Integration] Support Spark Datasource for MOR table - RDD approach
> > [7]
> > [Writer Core] Implement CLI support for performing bootstrap [8]
> > [Metrics] Hudi Supports Prometheus Pushgateway [9]
> >
> > =======================================
> > Bugs
> >
> > [Writer Core] lack of insert info in delta_commit inflight [10]
> > [DeltaStreamer] Fix Jcommander issue for --hoodie-conf in DeltaStreamer
> > [11]
> > [DeltaStreamer] Fix NPE when no new data in kafka using
> > HoodieDeltaStreamer [12]
> >
> > [1] https://issues.apache.org/jira/browse/HUDI-242
> > [2] https://issues.apache.org/jira/browse/HUDI-575
> > [3] https://issues.apache.org/jira/browse/HUDI-1144
> > [4] https://issues.apache.org/jira/browse/HUDI-1149
> > [5] https://issues.apache.org/jira/browse/HUDI-875
> > [6] https://issues.apache.org/jira/browse/HUDI-999
> > [7] https://issues.apache.org/jira/browse/HUDI-69
> > [8] https://issues.apache.org/jira/browse/HUDI-971
> > [9] https://issues.apache.org/jira/browse/HUDI-210
> > [10] https://issues.apache.org/jira/browse/HUDI-525
> > [11] https://issues.apache.org/jira/browse/HUDI-1140
> > [12] https://issues.apache.org/jira/browse/HUDI-1151
> >
> >
> >
> > Best,
> > Leesf
> >
>
Re: [ANNOUNCE] Hudi Community Weekly Update(2020-08-02 ~ 2020-08-09)
Posted by Vinoth Chandar <vi...@apache.org>.
+1 thanks leesf. I actually find these very useful when composing the
reports also.:)
On Sun, Aug 9, 2020 at 5:32 PM vino yang <ya...@gmail.com> wrote:
> Thanks to leesf for continuously updating Hudi weekly.
>
> It is great to see that more and more improvements are being proposed in
> the community.
>
> Best,
> Vino
>
> leesf <le...@gmail.com> 于2020年8月9日周日 下午11:24写道：
>
> > Dear community,
> >
> > Nice to share Hudi community weekly update for 2020-08-02 ~ 2020-08-09
> > with updates on features, bugfixs.
> >
> > =======================================
> > Features
> >
> > [Writer Core] Support for RFC-12/Bootstrapping of external datasets to
> > hudi [1]
> > [Writer Core] Spark Streaming with async compaction support [2]
> > [Spark Integration] Speedup spark read queries by caching metaclient in
> > HoodieROPathFilter [3]
> > [Metrics] Added a console metrics reporter and associated unit tests. [4]
> > [Hive Integration] Abstract hudi-sync-common, and support hudi-hive-sync,
> > hudi-dla-sync [5]
> > [Writer Core] Parallelize fetching of source data files/partitions [6]
> > [Spark Integration] Support Spark Datasource for MOR table - RDD approach
> > [7]
> > [Writer Core] Implement CLI support for performing bootstrap [8]
> > [Metrics] Hudi Supports Prometheus Pushgateway [9]
> >
> > =======================================
> > Bugs
> >
> > [Writer Core] lack of insert info in delta_commit inflight [10]
> > [DeltaStreamer] Fix Jcommander issue for --hoodie-conf in DeltaStreamer
> > [11]
> > [DeltaStreamer] Fix NPE when no new data in kafka using
> > HoodieDeltaStreamer [12]
> >
> > [1] https://issues.apache.org/jira/browse/HUDI-242
> > [2] https://issues.apache.org/jira/browse/HUDI-575
> > [3] https://issues.apache.org/jira/browse/HUDI-1144
> > [4] https://issues.apache.org/jira/browse/HUDI-1149
> > [5] https://issues.apache.org/jira/browse/HUDI-875
> > [6] https://issues.apache.org/jira/browse/HUDI-999
> > [7] https://issues.apache.org/jira/browse/HUDI-69
> > [8] https://issues.apache.org/jira/browse/HUDI-971
> > [9] https://issues.apache.org/jira/browse/HUDI-210
> > [10] https://issues.apache.org/jira/browse/HUDI-525
> > [11] https://issues.apache.org/jira/browse/HUDI-1140
> > [12] https://issues.apache.org/jira/browse/HUDI-1151
> >
> >
> >
> > Best,
> > Leesf
> >
>
Re: [ANNOUNCE] Hudi Community Weekly Update(2020-08-02 ~ 2020-08-09)
Posted by vino yang <ya...@gmail.com>.
Thanks to leesf for continuously updating Hudi weekly.
It is great to see that more and more improvements are being proposed in
the community.
Best,
Vino
leesf <le...@gmail.com> 于2020年8月9日周日 下午11:24写道：
> Dear community,
>
> Nice to share Hudi community weekly update for 2020-08-02 ~ 2020-08-09
> with updates on features, bugfixs.
>
> =======================================
> Features
>
> [Writer Core] Support for RFC-12/Bootstrapping of external datasets to
> hudi [1]
> [Writer Core] Spark Streaming with async compaction support [2]
> [Spark Integration] Speedup spark read queries by caching metaclient in
> HoodieROPathFilter [3]
> [Metrics] Added a console metrics reporter and associated unit tests. [4]
> [Hive Integration] Abstract hudi-sync-common, and support hudi-hive-sync,
> hudi-dla-sync [5]
> [Writer Core] Parallelize fetching of source data files/partitions [6]
> [Spark Integration] Support Spark Datasource for MOR table - RDD approach
> [7]
> [Writer Core] Implement CLI support for performing bootstrap [8]
> [Metrics] Hudi Supports Prometheus Pushgateway [9]
>
> =======================================
> Bugs
>
> [Writer Core] lack of insert info in delta_commit inflight [10]
> [DeltaStreamer] Fix Jcommander issue for --hoodie-conf in DeltaStreamer
> [11]
> [DeltaStreamer] Fix NPE when no new data in kafka using
> HoodieDeltaStreamer [12]
>
> [1] https://issues.apache.org/jira/browse/HUDI-242
> [2] https://issues.apache.org/jira/browse/HUDI-575
> [3] https://issues.apache.org/jira/browse/HUDI-1144
> [4] https://issues.apache.org/jira/browse/HUDI-1149
> [5] https://issues.apache.org/jira/browse/HUDI-875
> [6] https://issues.apache.org/jira/browse/HUDI-999
> [7] https://issues.apache.org/jira/browse/HUDI-69
> [8] https://issues.apache.org/jira/browse/HUDI-971
> [9] https://issues.apache.org/jira/browse/HUDI-210
> [10] https://issues.apache.org/jira/browse/HUDI-525
> [11] https://issues.apache.org/jira/browse/HUDI-1140
> [12] https://issues.apache.org/jira/browse/HUDI-1151
>
>
>
> Best,
> Leesf
>
Re: [ANNOUNCE] Hudi Community Weekly Update(2020-08-02 ~ 2020-08-09)
Posted by vino yang <ya...@gmail.com>.
Thanks to leesf for continuously updating Hudi weekly.
It is great to see that more and more improvements are being proposed in
the community.
Best,
Vino
leesf <le...@gmail.com> 于2020年8月9日周日 下午11:24写道：
> Dear community,
>
> Nice to share Hudi community weekly update for 2020-08-02 ~ 2020-08-09
> with updates on features, bugfixs.
>
> =======================================
> Features
>
> [Writer Core] Support for RFC-12/Bootstrapping of external datasets to
> hudi [1]
> [Writer Core] Spark Streaming with async compaction support [2]
> [Spark Integration] Speedup spark read queries by caching metaclient in
> HoodieROPathFilter [3]
> [Metrics] Added a console metrics reporter and associated unit tests. [4]
> [Hive Integration] Abstract hudi-sync-common, and support hudi-hive-sync,
> hudi-dla-sync [5]
> [Writer Core] Parallelize fetching of source data files/partitions [6]
> [Spark Integration] Support Spark Datasource for MOR table - RDD approach
> [7]
> [Writer Core] Implement CLI support for performing bootstrap [8]
> [Metrics] Hudi Supports Prometheus Pushgateway [9]
>
> =======================================
> Bugs
>
> [Writer Core] lack of insert info in delta_commit inflight [10]
> [DeltaStreamer] Fix Jcommander issue for --hoodie-conf in DeltaStreamer
> [11]
> [DeltaStreamer] Fix NPE when no new data in kafka using
> HoodieDeltaStreamer [12]
>
> [1] https://issues.apache.org/jira/browse/HUDI-242
> [2] https://issues.apache.org/jira/browse/HUDI-575
> [3] https://issues.apache.org/jira/browse/HUDI-1144
> [4] https://issues.apache.org/jira/browse/HUDI-1149
> [5] https://issues.apache.org/jira/browse/HUDI-875
> [6] https://issues.apache.org/jira/browse/HUDI-999
> [7] https://issues.apache.org/jira/browse/HUDI-69
> [8] https://issues.apache.org/jira/browse/HUDI-971
> [9] https://issues.apache.org/jira/browse/HUDI-210
> [10] https://issues.apache.org/jira/browse/HUDI-525
> [11] https://issues.apache.org/jira/browse/HUDI-1140
> [12] https://issues.apache.org/jira/browse/HUDI-1151
>
>
>
> Best,
> Leesf
>