[ANNOUNCE] Hudi Community Update(2021-10-10 ~ 2021-10-24)

[leesf <le...@gmail.com>]

Dear community,

Nice to share Hudi community bi-weekly updates for 2021-10-10 ~ 2021-10-24
with updates on features, bug fixes and tests.


=======================================
Features

[Spark SQL] support 'drop partition' sql [1]
[Flink Integration] Support merging small files for flink insert operation
[2]
[Core] Add HoodieData abstraction and refactor compaction actions in
hudi-client module [3]




[1] https://issues.apache.org/jira/browse/HUDI-2482
[2] https://issues.apache.org/jira/browse/HUDI-2578
[3] https://issues.apache.org/jira/browse/HUDI-2501


=======================================
Bugs

[Flink] Fix metadata table for flink [1]
[Core] Insert duplicate records when precombined is deactivated for
"insert" operation [2]
[Flink] AppendWriteFunction throws NPE when checkpointing without written
data [3]
[Core] Fixed wrong validation for metadataTableEnabled in Hoodie Table [4]
[Core] Metadata table compaction trigger max delta commits [5]
[Core] Fixing glob pattern to skip all hoodie meta paths [6]
[Core] Fix clustering handle errors [7]
[Flink Integration] Flink streaming reader misses the rolling over file
handles [8]
[Flink Integration] Support DefaultHoodieRecordPayload for flink [9]
[Flink Integration] Tweak some default config options for flink [10]
[Flink Integration] Embedded timeline server on JobManager [11]
[Flink Integration] Shade javax.servlet for flink bundle jar [12]
[Flink Integration] Simplify the view storage config properties [13]
[Flink Integration] Shaded hive for flink bundle jar [14]
[Flink Integration] Remove include-flink-sql-connector-hive profile from
flink bundle [15]
[Core] BitCaskDiskMap - avoiding hostname resolution when logging messages
[16]
[Flink Integration] Strength flink compaction rollback strategy [17]
[Core] Replace json based payload with protobuf for Transaction protocol
[18]
[CI] Generate more dependency list file for other bundles [19]
[Core] Metadata table compaction trigger max delta commits [20]
[Core] Fix write empty array when write.precombine.field is decimal type
[21]
[Core] Tuning HoodieROTablePathFilter by caching hoodieTableFileSystemView,
aiming to reduce unnecessary list/get requests [22]
[Core] Metadata table support for rolling back the first commit [23]



[1] https://issues.apache.org/jira/browse/HUDI-2537
[2] https://issues.apache.org/jira/browse/HUDI-2496
[3] https://issues.apache.org/jira/browse/HUDI-2542
[4] https://issues.apache.org/jira/browse/HUDI-2540
[5] https://issues.apache.org/jira/browse/HUDI-2532
[6] https://issues.apache.org/jira/browse/HUDI-2494
[7] https://issues.apache.org/jira/browse/HUDI-2435
[8] https://issues.apache.org/jira/browse/HUDI-2548
[9] https://issues.apache.org/jira/browse/HUDI-2551
[10] https://issues.apache.org/jira/browse/HUDI-2556
[11] https://issues.apache.org/jira/browse/HUDI-2562
[12] https://issues.apache.org/jira/browse/HUDI-2557
[13] https://issues.apache.org/jira/browse/HUDI-2568
[14] https://issues.apache.org/jira/browse/HUDI-2569
[15] https://issues.apache.org/jira/browse/HUDI-2571
[16] https://issues.apache.org/jira/browse/HUDI-2561
[17] https://issues.apache.org/jira/browse/HUDI-2572
[18] https://issues.apache.org/jira/browse/HUDI-2469
[19] https://issues.apache.org/jira/browse/HUDI-2507
[20] https://issues.apache.org/jira/browse/HUDI-2553
[21] https://issues.apache.org/jira/browse/HUDI-2592
[22] https://issues.apache.org/jira/browse/HUDI-2489
[23] https://issues.apache.org/jira/browse/HUDI-2468

======================================
Tests

[Tests] Fixing some test failures to unblock broken CI master [1]
[Tests] Fix few Cleaner tests with metadata table enabled [2]
[Tests] Fix flakiness in TestHoodieDeltaStreamer [3]
[Tests] Refactor TestWriteCopyOnWrite test cases [4]



[1] https://issues.apache.org/jira/browse/HUDI-2552
[2] https://issues.apache.org/jira/browse/HUDI-2472
[3] https://issues.apache.org/jira/browse/HUDI-2077
[4] https://issues.apache.org/jira/browse/HUDI-2583



Best,
Leesf