[ANNOUNCE] Hudi Weekly Community Update(2020-02-23~20200301)

[leesf <le...@gmail.com>]

Dear community,

Happy to share Hudi community weekly update for 2020-02-23 ~ 2020-03-01
with updates on development, features, bugs and tests.

Development

[KeyGenerator] A discussion to support for complex record keys with
TimestampBasedKeyGenerator, and may combine TimestampBasedKeyGenerator with
ComplexKeyGenerator [1]
[Improve Performance] A discussion about improving the merge performance
for cow is still under discussion, it aims to use spark operators(e.g.
mapToPair) to merge records instead of ExternalSpillableMap. [2]
[Test coverage] Test coverage is now integrated to codecov.io, it helps to
improve code quality. [3]
[Release] Code is frozen for next release(0.5.2), and rc1 will be sent in
the next few days. [4]

[1]
https://lists.apache.org/thread.html/r177c2e0d499cb2c7ec1a4f62646b23d545683e2036df1a36e57e0fa3%40%3Cdev.hudi.apache.org%3E
[2]
https://lists.apache.org/thread.html/rcfd072e3ae90f401eb005e282a4a80fba10438e62ebb32f46f983d95%40%3Cdev.hudi.apache.org%3E
[3]
https://lists.apache.org/thread.html/ra753a535860a6f46f92f9c3e5a5e09c983f558989171c5de745818d9%40%3Cdev.hudi.apache.org%3E
[4]
https://lists.apache.org/thread.html/reeb63ccddee3324a60a59253f234672170434642049a65c97208c75a%40%3Cdev.hudi.apache.org%3


Features

[Spark Integration] Enable incremental pulling from defined partitions. [5]
[Code Cleanup] Cleanup package structure in hudi-client. [6]
[CI] Aggregate code coverage and publish to codecov.io during CI [7]


[5] https://jira.apache.org/jira/browse/HUDI-597
[6] https://jira.apache.org/jira/browse/HUDI-554
[7] https://jira.apache.org/jira/browse/HUDI-627


Bugs

[Writer] Fixing performance issues around DiskBasedMap & kryo. [8]
[Release] Fix could not get sources warnings while compiling [9]

[8] https://jira.apache.org/jira/browse/HUDI-625
[9] https://jira.apache.org/jira/browse/HUDI-636


Tests

[Tests] Adding unit tests for PriorityBasedFileSystemView [10]

[10] https://jira.apache.org/jira/browse/HUDI-618

Best,
Leesf