You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Raymond Xu (Jira)" <ji...@apache.org> on 2022/03/12 13:52:00 UTC
[jira] [Updated] (HUDI-2919) Migrate Hudi to proper Incremental Build System (Bazel)
[ https://issues.apache.org/jira/browse/HUDI-2919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raymond Xu updated HUDI-2919:
-----------------------------
Component/s: dev-experience
> Migrate Hudi to proper Incremental Build System (Bazel)
> -------------------------------------------------------
>
> Key: HUDI-2919
> URL: https://issues.apache.org/jira/browse/HUDI-2919
> Project: Apache Hudi
> Issue Type: Improvement
> Components: dev-experience
> Reporter: Alexey Kudinkin
> Priority: Major
> Labels: developer-experience
>
> Even though Maven claims that it supports incremental builds, that is not what i personally observed in a day to day operations: even single file change trickles down into *whole project* being rebuilt which is currently taking about {*}6.5 mins{*}.
> This is a substantial productivity drain, so migration to more modern Build System (like Bazel), while not easy, should bring the fruits quite fast, paying itself off well over by just how much time we would be saving on not waiting for these builds to complete.
>
> {code:java}
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary for Hudi 0.11.0-SNAPSHOT:
> [INFO]
> [INFO] Hudi ............................................... SUCCESS [ 5.150 s]
> [INFO] hudi-common ........................................ SUCCESS [ 18.957 s]
> [INFO] hudi-aws ........................................... SUCCESS [ 4.052 s]
> [INFO] hudi-timeline-service .............................. SUCCESS [ 5.437 s]
> [INFO] hudi-client ........................................ SUCCESS [ 0.291 s]
> [INFO] hudi-client-common ................................. SUCCESS [ 13.902 s]
> [INFO] hudi-hadoop-mr ..................................... SUCCESS [ 5.227 s]
> [INFO] hudi-spark-client .................................. SUCCESS [ 23.578 s]
> [INFO] hudi-sync-common ................................... SUCCESS [ 1.630 s]
> [INFO] hudi-hive-sync ..................................... SUCCESS [ 10.545 s]
> [INFO] hudi-spark-datasource .............................. SUCCESS [ 0.090 s]
> [INFO] hudi-spark-common_2.12 ............................. SUCCESS [ 17.497 s]
> [INFO] hudi-spark3_2.12 ................................... SUCCESS [ 10.558 s]
> [INFO] hudi-spark_2.12 .................................... SUCCESS [ 46.709 s]
> [INFO] hudi-utilities_2.12 ................................ SUCCESS [ 9.061 s]
> [INFO] hudi-utilities-bundle_2.12 ......................... SUCCESS [ 18.046 s]
> [INFO] hudi-cli ........................................... SUCCESS [ 16.890 s]
> [INFO] hudi-java-client ................................... SUCCESS [ 3.887 s]
> [INFO] hudi-flink-client .................................. SUCCESS [ 10.512 s]
> [INFO] hudi-spark2_2.12 ................................... SUCCESS [ 15.522 s]
> [INFO] hudi-dla-sync ...................................... SUCCESS [ 2.017 s]
> [INFO] hudi-sync .......................................... SUCCESS [ 0.075 s]
> [INFO] hudi-hadoop-mr-bundle .............................. SUCCESS [ 6.188 s]
> [INFO] hudi-hive-sync-bundle .............................. SUCCESS [ 1.783 s]
> [INFO] hudi-spark3-bundle_2.12 ............................ SUCCESS [ 17.597 s]
> [INFO] hudi-presto-bundle ................................. SUCCESS [ 4.982 s]
> [INFO] hudi-timeline-server-bundle ........................ SUCCESS [ 6.391 s]
> [INFO] hudi-hadoop-docker ................................. SUCCESS [ 2.110 s]
> [INFO] hudi-hadoop-base-docker ............................ SUCCESS [ 1.404 s]
> [INFO] hudi-hadoop-namenode-docker ........................ SUCCESS [ 1.124 s]
> [INFO] hudi-hadoop-datanode-docker ........................ SUCCESS [ 1.090 s]
> [INFO] hudi-hadoop-history-docker ......................... SUCCESS [ 1.098 s]
> [INFO] hudi-hadoop-hive-docker ............................ SUCCESS [ 2.019 s]
> [INFO] hudi-hadoop-sparkbase-docker ....................... SUCCESS [ 0.993 s]
> [INFO] hudi-hadoop-sparkmaster-docker ..................... SUCCESS [ 1.106 s]
> [INFO] hudi-hadoop-sparkworker-docker ..................... SUCCESS [ 1.063 s]
> [INFO] hudi-hadoop-sparkadhoc-docker ...................... SUCCESS [ 1.031 s]
> [INFO] hudi-hadoop-presto-docker .......................... SUCCESS [ 1.103 s]
> [INFO] hudi-integ-test .................................... SUCCESS [ 14.574 s]
> [INFO] hudi-integ-test-bundle ............................. SUCCESS [ 33.876 s]
> [INFO] hudi-examples ...................................... SUCCESS [ 9.117 s]
> [INFO] hudi-flink_2.12 .................................... SUCCESS [ 5.476 s]
> [INFO] hudi-kafka-connect ................................. SUCCESS [ 3.303 s]
> [INFO] hudi-flink-bundle_2.12 ............................. SUCCESS [ 21.331 s]
> [INFO] hudi-kafka-connect-bundle .......................... SUCCESS [ 15.754 s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 06:34 min
> [INFO] Finished at: 2021-12-02T12:06:21-08:00
> [INFO] ------------------------------------------------------------------------ {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)