[ANNOUNCE] Weekly Community Update 2020/06

[Konstantin Knauf <ko...@ververica.com>]

Dear community,

happy to share this week's community update. Activity on the dev@ mailing
list has picked up quite a bit this week and more and more concrete design
proposals for Flink 1.11 are brought up for discussion. Besides that, Flink
1.10 and flink-shaded 10.0 are both close to being released.

Flink Development
==============

* [releases] Since Friday the community is voting on release candidate #3
for *Flink 1.10*. No blockers found so far. [1]

* [releases] Chesnay has kicked off the release of *flink-shaded 10.0*.
This release will allow Apache Flink to support Zookeeper 3.5 alongside
Zookeeper 3.4 (currently supported version). [2,3]

* [connectors] As part of his work on a JDBC exactly-once sink, Roman
proposes to redesign the *TwoPhaseCommitSinkFunction* abstraction in
FLIP-94 . The new abstraction would work for both the current exactly-once
KafkaProducer, the new JDBC sink and could potentially also subsume
existing WAL sinks like the Cassandra sink. No feedback so far. [4,5]

* [ecosystem] Gyula has started an interesting discussion on *Apache Atlas
integration* for Flink. The integration itself would live in Apache Atlas,
but Flink would need to provide the required hooks and expose job metadata
(particularly of sources & sinks) [6]

* [web ui] FLIP-75, a collection of *improvements to Flink's web user
interface*, will be split up into multiple "sub-FLIPs" for voting. [7]

* [python, sql] In Flink 1.10, the Python Table API will support Python
UDFs for the first time. Xingbo has prepared a proposal for *User Defined
Table Function *support in the* Python* Table API. [8]

* [python, sql] Dian Fu proposes to support scalar, *vectorized* *UDFs* in
the *Python* Table API. Instead of exchanging serialized rows between Java
and Python process, the Flink operator would send batches of rows in a
columnar format to the Python process. Thereby, the proposal aims to
improve performance and increase interoperability with libraries such as
Pandas. [9]

* [python] Hequn proposes to support the *Flink's machine learning API* on
top of the *Python* Table API. No feedback so far. [10]

* [distribution] Hequn has started a discussion about moving the binaries
of Flink's new *machine learning *modules into *opt/ *directory of the
Flink distribution. While everyone seems to agree, that a) we need a long
term plan of how to distribute components of the ever growing Flink project
and b) the Flink distribution should be rather lean, it is not yet clear
how to deal with the machine learning modules at this point in time. [11]

* [development process] Removing a deprecated interface does not require a
FLIP. [12]

[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Release-1-10-0-release-candidate-3-tp37405.html
[2]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Release-flink-shaded-10-0-release-candidate-1-tp37417.html
[3]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Release-flink-shaded-10-0-tp37306.html
[4]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-94-Rework-2-phase-commit-abstractions-tp37164.html
[5]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-94%3A+Rework+2-phase+commit+abstractions
[6]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Discussion-Job-generation-submission-hooks-Atlas-integration-tp37298.html
[7]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-tp33540.html
[8]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-User-Defined-Table-Function-in-PyFlink-tp37149.html
[9]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-scalar-vectorized-Python-UDF-in-PyFlink-tp37264.html
[10]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-Python-ML-Pipeline-API-tp37291.html
[11]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Include-flink-ml-api-and-flink-ml-lib-in-opt-tp36476.html
[12]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Does-removing-deprecated-interfaces-needs-another-FLIP-tp37365p37370.html

Notable Bugs
==========

* [FLINK-15918] [1.9.2] The "uptime" metrics is not reset during restarts,
when jobmanager.execution.failover-strategy: region is configured (which is
the default). [13]

[13] https://issues.apache.org/jira/browse/FLINK-15918

Events, Blog Posts, Misc
===================

* *Kartik Khare* has published a guide on unit testing Apache Flink on the
Apache Flink blog. [14]

* The first set of speakers for Flink Forward San Francisco has been
announced [15]. For a 50% discount check out this thread [16].

* Upcoming Meetups
   * On February 19th, Apache Flink Meetup London, "Monitoring and
Analysing Communication & Trade Events as Graphs", hosted by *Christos
Hadjinikolis* [17]

[14]
https://flink.apache.org/news/2020/02/07/a-guide-for-unit-testing-in-apache-flink.html
[15] https://www.flink-forward.org/sf-2020/speakers
[16]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Community-Discounts-for-Flink-Forward-SF-2020-Registrations-td37055.html
[17] https://www.meetup.com/Apache-Flink-London-Meetup/events/268400545/

Cheers,

Konstantin (@snntrable)

-- 

Konstantin Knauf | Solutions Architect

+49 160 91394525


Follow us @VervericaData Ververica <https://www.ververica.com/>


--

Join Flink Forward <https://flink-forward.org/> - The Apache Flink
Conference

Stream Processing | Event Driven | Real Time

--

Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany

--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
(Tony) Cheng