[ANNOUNCE] Hudi Community Update(2021-08-15 ~ 2021-08-29)

[leesf <le...@gmail.com>]

Dear community,

Nice to share Hudi community bi-weekly updates for 2021-08-15 ~ 2021-08-29
with updates on features, bug fixes and tests.


=======================================
Features

[Flink Integration] Support Flink batch upsert [1]
[Core] AHoodie columns sort partitioner for bulk insert [2]



[1] https://issues.apache.org/jira/browse/HUDI-2316
[2] https://issues.apache.org/jira/browse/HUDI-2345


=======================================
Bugs

[Core] fix FileSliceMetrics utils bug [1]
[Core] HoodieCompactionConfig get HoodieCleaningPolicy NullPointerException
[2]
[Core] Include _hoodie_operation meta column in removeMetadataFields [3]
[Core] Use correct meta columns while preparing dataset for bulk insert [4]
[Spark Integraion] Create Table If Not Exists Failed After Alter Table [5]
[Flink Integration] Merge the data set for flink bounded source when
changelog mode turns off [6]
[Flink Integration] Optimize Bootstrap operator [7]
[Spark Integration] Support referencing subquery with column aliases by
table alias [8]
[Flink Integration] The upgrade downgrade action of flink writer should be
singleton [9]
[Spark Integration] MERGE INTO doesn't work for tables created using CTAS
[10]
[Core] Catch Throwable in BoundedInMemoryExecutor  [11]
[Core] Use the caller classloader for ReflectionUtils [12]
[Core] Refact HoodieFlinkStreamer to reuse the pipeline of HoodieTableSink
[13]
[Core] Optimizing overwriteField method with Objects.equals [14]
[Flink Integration] Improvement flink streaming reader [15]


[1] https://issues.apache.org/jira/browse/HUDI-2301
[2] https://issues.apache.org/jira/browse/HUDI-2167
[3] https://issues.apache.org/jira/browse/HUDI-1363
[4] https://issues.apache.org/jira/browse/HUDI-2322
[5] https://issues.apache.org/jira/browse/HUDI-2339
[6] https://issues.apache.org/jira/browse/HUDI-2340
[7] https://issues.apache.org/jira/browse/HUDI-2342
[8] https://issues.apache.org/jira/browse/HUDI-2259
[9] https://issues.apache.org/jira/browse/HUDI-2352
[10] https://issues.apache.org/jira/browse/HUDI-2357
[11] https://issues.apache.org/jira/browse/HUDI-2368
[12] https://issues.apache.org/jira/browse/HUDI-2321
[13] https://issues.apache.org/jira/browse/HUDI-2229
[14] https://issues.apache.org/jira/browse/HUDI-2365
[15] https://issues.apache.org/jira/browse/HUDI-2371


======================================
Tests

[Tests] Adding spark delete node to integ test suite [1]
[Tests] Add basic "hoodie_is_deleted" unit tests to TestDataSource classes
[2]
[Tests] Refactor HoodieSparkSqlWriterSuite to add setup and teardown [3]

[1] https://issues.apache.org/jira/browse/HUDI-2349
[2] https://issues.apache.org/jira/browse/HUDI-2359
[3] https://issues.apache.org/jira/browse/HUDI-2264

Best,
Leesf