You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by mb...@apache.org on 2024/10/28 21:55:04 UTC
(flink-web) branch asf-site updated (c99fc80e5 -> 28de19d0b)
This is an automated email from the ASF dual-hosted git repository.
mbalassi pushed a change to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git
from c99fc80e5 Rebuild website
new 294a79bf4 Flink Kubernetes Operator 1.10.0
new 28de19d0b Rebuild website
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../08/26/apache-flink-0.6-available/index.html | 15 +-
.../09/26/apache-flink-0.6.1-available/index.html | 11 +-
content/2014/10/03/upcoming-events/index.html | 11 +-
.../11/04/apache-flink-0.7.0-available/index.html | 15 +-
.../11/18/hadoop-compatibility-in-flink/index.html | 91 +-
.../index.html | 15 +-
.../01/21/apache-flink-0.8.0-available/index.html | 15 +-
.../january-2015-in-the-flink-community/index.html | 23 +-
.../02/09/introducing-flink-streaming/index.html | 585 ++-
.../index.html | 29 +-
.../index.html | 15 +-
.../march-2015-in-the-flink-community/index.html | 19 +-
.../index.html | 11 +-
.../05/11/juggling-with-bits-and-bytes/index.html | 25 +-
.../april-2015-in-the-flink-community/index.html | 17 +-
.../06/24/announcing-apache-flink-0.9.0/index.html | 15 +-
.../index.html | 195 +-
.../09/01/apache-flink-0.9.1-available/index.html | 11 +-
.../09/03/announcing-flink-forward-2015/index.html | 15 +-
.../index.html | 61 +-
.../16/announcing-apache-flink-0.10.0/index.html | 11 +-
.../2015/11/27/flink-0.10.1-released/index.html | 15 +-
.../index.html | 11 +-
.../index.html | 129 +-
.../index.html | 11 +-
.../2016/02/11/flink-0.10.2-released/index.html | 15 +-
.../03/08/announcing-apache-flink-1.0.0/index.html | 15 +-
content/2016/04/06/flink-1.0.1-released/index.html | 15 +-
.../index.html | 157 +-
.../index.html | 11 +-
content/2016/04/22/flink-1.0.2-released/index.html | 15 +-
content/2016/05/11/flink-1.0.3-released/index.html | 15 +-
.../index.html | 11 +-
.../08/04/announcing-apache-flink-1.1.0/index.html | 73 +-
content/2016/08/04/flink-1.1.1-released/index.html | 11 +-
.../index.html | 11 +-
.../09/05/apache-flink-1.1.2-released/index.html | 15 +-
.../10/12/apache-flink-1.1.3-released/index.html | 21 +-
.../apache-flink-in-2016-year-in-review/index.html | 11 +-
.../12/21/apache-flink-1.1.4-released/index.html | 25 +-
.../02/06/announcing-apache-flink-1.2.0/index.html | 11 +-
.../03/23/apache-flink-1.1.5-released/index.html | 11 +-
.../index.html | 11 +-
.../index.html | 19 +-
.../04/26/apache-flink-1.2.1-released/index.html | 15 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../06/23/apache-flink-1.3.1-released/index.html | 11 +-
.../index.html | 85 +-
.../08/05/apache-flink-1.3.2-released/index.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../apache-flink-in-2017-year-in-review/index.html | 11 +-
.../index.html | 21 +-
.../02/15/apache-flink-1.4.1-released/index.html | 11 +-
.../index.html | 11 +-
.../03/08/apache-flink-1.4.2-released/index.html | 11 +-
.../03/15/apache-flink-1.3.3-released/index.html | 11 +-
.../index.html | 11 +-
.../07/12/apache-flink-1.5.1-released/index.html | 11 +-
.../07/31/apache-flink-1.5.2-released/index.html | 11 +-
.../index.html | 11 +-
.../08/21/apache-flink-1.5.3-released/index.html | 11 +-
.../09/20/apache-flink-1.5.4-released/index.html | 11 +-
.../09/20/apache-flink-1.6.1-released/index.html | 11 +-
.../10/29/apache-flink-1.5.5-released/index.html | 11 +-
.../10/29/apache-flink-1.6.2-released/index.html | 11 +-
.../index.html | 11 +-
.../12/21/apache-flink-1.7.1-released/index.html | 11 +-
.../12/22/apache-flink-1.6.3-released/index.html | 11 +-
.../12/26/apache-flink-1.5.6-released/index.html | 11 +-
.../index.html | 15 +-
.../02/15/apache-flink-1.7.2-released/index.html | 11 +-
.../index.html | 15 +-
.../02/25/apache-flink-1.6.4-released/index.html | 11 +-
.../index.html | 11 +-
.../index.html | 41 +-
.../index.html | 11 +-
.../index.html | 11 +-
content/2019/05/03/pulsar-flink.html | 11 +-
.../03/when-flink-pulsar-come-together/index.html | 151 +-
.../index.html | 75 +-
content/2019/05/14/temporal-tables.html | 11 +-
.../index.html | 73 +-
content/2019/05/19/state-ttl.html | 11 +-
.../index.html | 15 +-
content/2019/06/05/flink-network-stack.html | 11 +-
.../index.html | 159 +-
content/2019/06/26/broadcast-state.html | 11 +-
.../07/02/apache-flink-1.8.1-released/index.html | 11 +-
content/2019/07/23/flink-network-stack-2.html | 11 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../flink-community-update-september19/index.html | 11 +-
.../09/11/apache-flink-1.8.2-released/index.html | 11 +-
.../index.html | 11 +-
.../10/18/apache-flink-1.9.1-released/index.html | 11 +-
.../index.html | 11 +-
.../index.html | 235 +-
.../12/11/apache-flink-1.8.3-released/index.html | 11 +-
.../index.html | 119 +-
.../index.html | 203 +-
.../01/30/apache-flink-1.9.2-released/index.html | 11 +-
.../index.html | 291 +-
.../index.html | 11 +-
.../index.html | 21 +-
.../index.html | 11 +-
.../index.html | 153 +-
.../index.html | 15 +-
.../30/flink-community-update-april20/index.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
content/2020/04/09/pyflink-udf-support-flink.html | 11 +-
.../index.html | 131 +-
.../index.html | 11 +-
.../04/24/apache-flink-1.9.3-released/index.html | 11 +-
.../index.html | 11 +-
.../05/06/flink-community-update-may20/index.html | 11 +-
.../05/12/apache-flink-1.10.1-released/index.html | 15 +-
.../index.html | 23 +-
.../06/10/flink-community-update-june20/index.html | 11 +-
.../index.html | 11 +-
.../index.html | 15 +-
.../index.html | 15 +-
.../index.html | 19 +-
.../07/21/apache-flink-1.11.1-released/index.html | 11 +-
content/2020/07/23/catalogs.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
...ql-demo-building-e2e-streaming-application.html | 11 +-
.../07/29/flink-community-update-july20/index.html | 15 +-
.../index.html | 273 +-
.../08/04/pyflink-pandas-udf-support-flink.html | 11 +-
.../index.html | 15 +-
.../index.html | 67 +-
.../index.html | 11 +-
content/2020/08/19/statefun.html | 11 +-
.../08/20/the-state-of-flink-on-docker/index.html | 15 +-
.../08/25/apache-flink-1.10.2-released/index.html | 15 +-
.../flink-1.11-memory-management-improvements.html | 11 +-
.../index.html | 11 +-
.../04/flink-community-update-august20/index.html | 15 +-
.../09/17/apache-flink-1.11.2-released/index.html | 11 +-
.../index.html | 73 +-
.../index.html | 11 +-
.../index.html | 11 +-
...om-aligned-to-unaligned-checkpoints-part-1.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 23 +-
content/2020/12/15/pipelined-region-sheduling.html | 11 +-
.../12/18/apache-flink-1.11.3-released/index.html | 11 +-
.../index.html | 11 +-
content/2021/01/07/pulsar-flink-connector-270.html | 11 +-
.../index.html | 15 +-
.../index.html | 11 +-
content/2021/01/18/rocksdb.html | 11 +-
.../index.html | 11 +-
.../01/19/apache-flink-1.12.1-released/index.html | 11 +-
.../01/29/apache-flink-1.10.3-released/index.html | 11 +-
.../index.html | 11 +-
content/2021/02/10/native-k8s-with-ha.html | 11 +-
.../03/03/apache-flink-1.12.2-released/index.html | 11 +-
.../index.html | 157 +-
content/2021/03/11/batch-execution-mode.html | 11 +-
.../index.html | 65 +-
.../04/29/apache-flink-1.12.3-released/index.html | 11 +-
.../index.html | 33 +-
content/2021/05/06/reactive-mode.html | 11 +-
.../index.html | 15 +-
.../05/21/apache-flink-1.12.4-released/index.html | 11 +-
.../05/28/apache-flink-1.13.1-released/index.html | 11 +-
content/2021/07/07/backpressure.html | 11 +-
.../index.html | 15 +-
.../08/06/apache-flink-1.12.5-released/index.html | 11 +-
.../08/06/apache-flink-1.13.2-released/index.html | 11 +-
.../08/09/apache-flink-1.11.4-released/index.html | 11 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../2021/09/07/connector-table-sql-api-part1.html | 11 +-
.../2021/09/07/connector-table-sql-api-part2.html | 11 +-
.../index.html | 201 +-
.../index.html | 601 ++-
.../index.html | 11 +-
.../10/19/apache-flink-1.13.3-released/index.html | 11 +-
.../index.html | 15 +-
.../index.html | 81 +-
content/2021/10/26/sort-shuffle-part1.html | 11 +-
content/2021/10/26/sort-shuffle-part2.html | 11 +-
.../index.html | 15 +-
content/2021/11/03/flink-backward.html | 11 +-
.../index.html | 19 +-
content/2021/12/10/log4j-cve.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 15 +-
.../index.html | 15 +-
.../2022/01/04/scheduler-performance-part-one.html | 11 +-
.../2022/01/04/scheduler-performance-part-two.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
content/2022/01/20/pravega-connector-101.html | 11 +-
.../01/20/pravega-flink-connector-101/index.html | 143 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../02/22/scala-free-in-one-fifteen/index.html | 31 +-
content/2022/02/22/scala-free.html | 11 +-
.../index.html | 15 +-
.../the-generic-asynchronous-base-sink/index.html | 135 +-
.../index.html | 11 +-
.../index.html | 11 +-
content/2022/05/06/async-sink-base.html | 11 +-
.../index.html | 89 +-
.../index.html | 11 +-
content/2022/05/06/pyflink-1.15-thread-mode.html | 11 +-
content/2022/05/06/restore-modes.html | 11 +-
.../index.html | 15 +-
.../index.html | 11 +-
content/2022/05/18/latency-part1.html | 11 +-
.../index.html | 11 +-
content/2022/05/23/latency-part2.html | 11 +-
content/2022/05/30/changelog-state-backend.html | 11 +-
.../index.html | 15 +-
.../index.html | 19 +-
.../index.html | 15 +-
content/2022/06/17/adaptive-batch-scheduler.html | 11 +-
.../index.html | 15 +-
.../index.html | 11 +-
content/2022/07/11/final-checkpoint-part1.html | 11 +-
content/2022/07/11/final-checkpoint-part2.html | 11 +-
.../index.html | 15 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../index.html | 21 +-
.../08/regarding-akkas-licensing-change/index.html | 11 +-
.../index.html | 15 +-
.../index.html | 15 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../11/25/async-sink-rate-limiting-strategy.html | 11 +-
.../index.html | 199 +-
.../index.html | 11 +-
.../index.html | 15 +-
.../index.html | 15 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 19 +-
.../index.html | 15 +-
.../index.html | 79 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 19 +-
.../index.html | 19 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../index.html | 15 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 43 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 45 +-
.../index.html | 11 +-
.../preview-release-of-apache-flink-2.0/index.html | 11 +-
.../index.html | 143 +-
content/404.html | 5 +-
content/categories/index.html | 11 +-
content/categories/index.xml | 3 +-
content/categories/page/1/index.html | 11 +-
content/community.html | 11 +-
content/community/index.html | 11 +-
content/documentation/flink-cdc-master/index.html | 15 +-
content/documentation/flink-cdc-stable/index.html | 15 +-
.../flink-kubernetes-operator-master/index.html | 15 +-
.../flink-kubernetes-operator-stable/index.html | 19 +-
content/documentation/flink-master/index.html | 15 +-
content/documentation/flink-preview/index.html | 15 +-
content/documentation/flink-stable/index.html | 15 +-
.../flink-stateful-functions-master/index.html | 15 +-
.../flink-stateful-functions-stable/index.html | 15 +-
content/documentation/flinkml-master/index.html | 15 +-
content/documentation/flinkml-stable/index.html | 15 +-
content/documentation/index.html | 13 +-
content/documentation/index.xml | 47 +-
content/downloads/index.html | 51 +-
.../apache-beam-how-beam-runs-on-top-of-flink.html | 11 +-
.../2020/06/23/flink-on-zeppelin-part2.html | 11 +-
...c78caad1f76107eb0e3a2cbda31c2e72b28582940ebc.js | 5481 ++++++++++++++++++++
...071a5f86ebd9c58d0d17fedfc9fb56db3a4db935ec5c.js | 1 -
...87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js} | 2 +-
content/en/index.html | 11 +-
content/en/sitemap.xml | 7 +-
.../feature/2019/09/13/state-processor-api.html | 11 +-
.../2017/07/04/flink-rescalable-state.html | 11 +-
.../2018/01/30/incremental-checkpointing.html | 11 +-
.../01/end-to-end-exactly-once-apache-flink.html | 11 +-
.../features/2019/03/11/prometheus-monitoring.html | 11 +-
.../2020/03/27/flink-for-data-warehouse.html | 11 +-
content/flink-packages/index.html | 15 +-
content/getting-help.html | 11 +-
content/getting-help/index.html | 11 +-
content/getting-started/index.html | 13 +-
content/getting-started/index.xml | 27 +-
content/getting-started/training-course/index.html | 15 +-
content/getting-started/with-flink-cdc/index.html | 15 +-
.../with-flink-kubernetes-operator/index.html | 15 +-
content/getting-started/with-flink-ml/index.html | 15 +-
.../with-flink-stateful-functions/index.html | 15 +-
content/getting-started/with-flink/index.html | 15 +-
.../code-style-and-quality-common/index.html | 71 +-
.../code-style-and-quality-components/index.html | 51 +-
.../code-style-and-quality-formatting/index.html | 45 +-
.../code-style-and-quality-java/index.html | 43 +-
.../code-style-and-quality-preamble/index.html | 49 +-
.../index.html | 43 +-
.../code-style-and-quality-scala/index.html | 47 +-
.../how-to-contribute/contribute-code/index.html | 15 +-
.../contribute-documentation/index.html | 19 +-
.../documentation-style-guide/index.html | 21 +-
content/how-to-contribute/getting-help/index.html | 21 +-
.../how-to-contribute/improve-website/index.html | 23 +-
content/how-to-contribute/index.html | 13 +-
content/how-to-contribute/index.xml | 72 +-
content/how-to-contribute/overview/index.html | 15 +-
content/how-to-contribute/reviewing-prs/index.html | 19 +-
content/index.html | 50 +-
content/index.xml | 1506 +-----
content/material/index.html | 23 +-
content/news/2014/08/26/release-0.6.html | 11 +-
content/news/2014/09/26/release-0.6.1.html | 11 +-
content/news/2014/10/03/upcoming_events.html | 11 +-
content/news/2014/11/04/release-0.7.0.html | 11 +-
content/news/2014/11/18/hadoop-compatibility.html | 11 +-
content/news/2015/01/06/december-in-flink.html | 11 +-
content/news/2015/01/21/release-0.8.html | 11 +-
content/news/2015/02/04/january-in-flink.html | 11 +-
content/news/2015/02/09/streaming-example.html | 11 +-
.../news/2015/03/02/february-2015-in-flink.html | 11 +-
.../13/peeking-into-Apache-Flinks-Engine-Room.html | 11 +-
content/news/2015/04/07/march-in-flink.html | 11 +-
.../news/2015/04/13/release-0.9.0-milestone1.html | 11 +-
.../2015/05/11/Juggling-with-Bits-and-Bytes.html | 11 +-
.../news/2015/05/14/Community-update-April.html | 11 +-
.../24/announcing-apache-flink-0.9.0-release.html | 11 +-
.../news/2015/08/24/introducing-flink-gelly.html | 11 +-
content/news/2015/09/01/release-0.9.1.html | 11 +-
content/news/2015/09/03/flink-forward.html | 11 +-
content/news/2015/09/16/off-heap-memory.html | 11 +-
content/news/2015/11/16/release-0.10.0.html | 11 +-
content/news/2015/11/27/release-0.10.1.html | 11 +-
content/news/2015/12/04/Introducing-windows.html | 11 +-
content/news/2015/12/11/storm-compatibility.html | 11 +-
content/news/2015/12/18/a-year-in-review.html | 11 +-
content/news/2016/02/11/release-0.10.2.html | 11 +-
content/news/2016/03/08/release-1.0.0.html | 11 +-
content/news/2016/04/06/cep-monitoring.html | 11 +-
content/news/2016/04/06/release-1.0.1.html | 11 +-
.../news/2016/04/14/flink-forward-announce.html | 11 +-
content/news/2016/04/22/release-1.0.2.html | 11 +-
content/news/2016/05/11/release-1.0.3.html | 11 +-
content/news/2016/05/24/stream-sql.html | 11 +-
content/news/2016/08/08/release-1.1.0.html | 11 +-
content/news/2016/08/11/release-1.1.1.html | 11 +-
content/news/2016/08/24/ff16-keynotes-panels.html | 11 +-
content/news/2016/09/05/release-1.1.2.html | 11 +-
content/news/2016/10/12/release-1.1.3.html | 11 +-
content/news/2016/12/19/2016-year-in-review.html | 11 +-
content/news/2016/12/21/release-1.1.4.html | 11 +-
content/news/2017/02/06/release-1.2.0.html | 11 +-
content/news/2017/03/23/release-1.1.5.html | 11 +-
content/news/2017/03/29/table-sql-api-update.html | 11 +-
content/news/2017/04/04/dynamic-tables.html | 11 +-
content/news/2017/04/26/release-1.2.1.html | 11 +-
content/news/2017/05/16/official-docker-image.html | 11 +-
content/news/2017/06/01/release-1.3.0.html | 11 +-
content/news/2017/06/23/release-1.3.1.html | 11 +-
content/news/2017/08/05/release-1.3.2.html | 11 +-
.../2017/11/22/release-1.4-and-1.5-timeline.html | 11 +-
content/news/2017/12/12/release-1.4.0.html | 11 +-
content/news/2017/12/21/2017-year-in-review.html | 11 +-
content/news/2018/02/15/release-1.4.1.html | 11 +-
content/news/2018/03/08/release-1.4.2.html | 11 +-
content/news/2018/03/15/release-1.3.3.html | 11 +-
content/news/2018/05/25/release-1.5.0.html | 11 +-
content/news/2018/07/12/release-1.5.1.html | 11 +-
content/news/2018/07/31/release-1.5.2.html | 11 +-
content/news/2018/08/09/release-1.6.0.html | 11 +-
content/news/2018/08/21/release-1.5.3.html | 11 +-
content/news/2018/09/20/release-1.5.4.html | 11 +-
content/news/2018/09/20/release-1.6.1.html | 11 +-
content/news/2018/10/29/release-1.5.5.html | 11 +-
content/news/2018/10/29/release-1.6.2.html | 11 +-
content/news/2018/11/30/release-1.7.0.html | 11 +-
content/news/2018/12/21/release-1.7.1.html | 11 +-
content/news/2018/12/22/release-1.6.3.html | 11 +-
content/news/2018/12/26/release-1.5.6.html | 11 +-
.../2019/02/13/unified-batch-streaming-blink.html | 11 +-
content/news/2019/02/15/release-1.7.2.html | 11 +-
.../news/2019/02/25/monitoring-best-practices.html | 11 +-
content/news/2019/02/25/release-1.6.4.html | 11 +-
content/news/2019/03/06/ffsf-preview.html | 11 +-
content/news/2019/04/09/release-1.8.0.html | 11 +-
content/news/2019/04/17/sod.html | 11 +-
content/news/2019/07/02/release-1.8.1.html | 11 +-
content/news/2019/08/22/release-1.9.0.html | 11 +-
content/news/2019/09/10/community-update.html | 11 +-
content/news/2019/09/11/release-1.8.2.html | 11 +-
content/news/2019/10/18/release-1.9.1.html | 11 +-
.../query-pulsar-streams-using-apache-flink.html | 11 +-
content/news/2019/12/09/flink-kubernetes-kudo.html | 11 +-
content/news/2019/12/11/release-1.8.3.html | 11 +-
content/news/2020/01/15/demo-fraud-detection.html | 11 +-
...ked-interacting-with-state-in-apache-flink.html | 11 +-
content/news/2020/01/30/release-1.9.2.html | 11 +-
.../a-guide-for-unit-testing-in-apache-flink.html | 11 +-
content/news/2020/02/11/release-1.10.0.html | 11 +-
content/news/2020/02/20/ddl.html | 11 +-
.../news/2020/03/24/demo-fraud-detection-2.html | 11 +-
content/news/2020/04/01/community-update.html | 11 +-
.../news/2020/04/07/release-statefun-2.0.0.html | 11 +-
.../04/15/flink-serialization-tuning-vol-1.html | 11 +-
.../memory-management-improvements-flink-1.10.html | 11 +-
content/news/2020/04/24/release-1.9.3.html | 11 +-
content/news/2020/05/04/season-of-docs.html | 11 +-
content/news/2020/05/07/community-update.html | 11 +-
content/news/2020/05/12/release-1.10.1.html | 11 +-
.../news/2020/06/09/release-statefun-2.1.0.html | 11 +-
content/news/2020/06/11/community-update.html | 11 +-
.../news/2020/06/15/flink-on-zeppelin-part1.html | 11 +-
content/news/2020/07/06/release-1.11.0.html | 11 +-
content/news/2020/07/14/application-mode.html | 11 +-
content/news/2020/07/21/release-1.11.1.html | 11 +-
content/news/2020/07/27/community-update.html | 11 +-
.../news/2020/07/30/demo-fraud-detection-3.html | 11 +-
content/news/2020/08/06/external-resource.html | 11 +-
content/news/2020/08/20/flink-docker.html | 11 +-
content/news/2020/08/25/release-1.10.2.html | 11 +-
content/news/2020/09/04/community-update.html | 11 +-
content/news/2020/09/17/release-1.11.2.html | 11 +-
.../news/2020/09/28/release-statefun-2.2.0.html | 11 +-
.../2020/10/13/stateful-serverless-internals.html | 11 +-
.../news/2020/11/11/release-statefun-2.2.1.html | 11 +-
content/news/2020/12/10/release-1.12.0.html | 11 +-
content/news/2020/12/18/release-1.11.3.html | 11 +-
.../news/2021/01/02/release-statefun-2.2.2.html | 11 +-
.../01/11/batch-fine-grained-fault-tolerance.html | 11 +-
content/news/2021/01/19/release-1.12.1.html | 11 +-
content/news/2021/01/29/release-1.10.3.html | 11 +-
content/news/2021/03/03/release-1.12.2.html | 11 +-
.../news/2021/04/15/release-statefun-3.0.0.html | 11 +-
content/news/2021/04/29/release-1.12.3.html | 11 +-
content/news/2021/05/03/release-1.13.0.html | 11 +-
content/news/2021/05/21/release-1.12.4.html | 11 +-
content/news/2021/05/28/release-1.13.1.html | 11 +-
content/news/2021/08/06/release-1.12.5.html | 11 +-
content/news/2021/08/06/release-1.13.2.html | 11 +-
content/news/2021/08/09/release-1.11.4.html | 11 +-
content/news/2021/08/31/release-1.14.0-rc0.html | 11 +-
.../news/2021/08/31/release-statefun-3.1.0.html | 11 +-
content/news/2021/09/29/release-1.14.0.html | 11 +-
content/news/2021/10/19/release-1.13.3.html | 11 +-
content/news/2021/12/16/log4j-patch-releases.html | 11 +-
.../news/2021/12/22/log4j-statefun-release.html | 11 +-
content/news/2022/01/07/release-ml-2.0.0.html | 11 +-
content/news/2022/01/17/release-1.14.3.html | 11 +-
.../news/2022/01/31/release-statefun-3.2.0.html | 11 +-
content/news/2022/02/18/release-1.13.6.html | 11 +-
content/news/2022/03/11/release-1.14.4.html | 11 +-
.../04/03/release-kubernetes-operator-0.1.0.html | 11 +-
content/news/2022/05/05/1.15-announcement.html | 11 +-
.../news/2022/05/11/release-table-store-0.1.0.html | 11 +-
.../06/05/release-kubernetes-operator-1.0.0.html | 11 +-
content/news/2022/06/22/release-1.14.5.html | 11 +-
content/news/2022/07/06/release-1.15.1.html | 11 +-
content/news/2022/07/12/release-ml-2.1.0.html | 11 +-
.../07/25/release-kubernetes-operator-1.1.0.html | 11 +-
content/news/2022/08/25/release-1.15.2.html | 11 +-
.../news/2022/08/29/release-table-store-0.2.0.html | 11 +-
content/news/2022/09/08/akka-license-change.html | 11 +-
content/news/2022/09/28/release-1.14.6.html | 11 +-
.../10/07/release-kubernetes-operator-1.2.0.html | 11 +-
.../news/2022/10/13/release-table-store-0.2.1.html | 11 +-
content/news/2022/10/28/1.16-announcement.html | 11 +-
content/news/2022/11/10/release-1.15.3.html | 11 +-
.../12/14/release-kubernetes-operator-1.3.0.html | 11 +-
.../01/10/release-kubernetes-operator-1.3.1.html | 11 +-
.../news/2023/01/13/release-table-store-0.3.0.html | 11 +-
.../2023/01/20/delegation-token-framework.html | 11 +-
content/news/2023/01/30/release-1.16.1.html | 11 +-
.../02/27/release-kubernetes-operator-1.4.0.html | 11 +-
content/news/2023/03/23/release-1.17.0.html | 11 +-
content/news/2023/04/19/release-ml-2.2.0.html | 11 +-
.../05/17/release-kubernetes-operator-1.5.0.html | 11 +-
content/news/2023/05/25/release-1.16.2.html | 11 +-
content/news/2023/05/25/release-1.17.1.html | 11 +-
content/news/2023/07/03/sigmod-system-award.html | 11 +-
.../news/2023/08/04/externalized-connectors.html | 11 +-
.../08/15/release-kubernetes-operator-1.6.0.html | 11 +-
content/news/2023/10/10/release-1.18.0.html | 11 +-
.../10/27/release-kubernetes-operator-1.6.1.html | 11 +-
.../11/22/release-kubernetes-operator-1.7.0.html | 11 +-
content/news/2023/11/29/release-1.16.3.html | 11 +-
content/news/2023/11/29/release-1.17.2.html | 11 +-
content/news/2024/01/19/release-1.18.1.html | 11 +-
content/news/2024/03/18/release-1.19.0.html | 11 +-
.../03/21/release-kubernetes-operator-1.8.0.html | 11 +-
content/news/2024/05/17/release-cdc-3.1.0.html | 11 +-
content/news/2024/06/14/release-1.19.1.html | 11 +-
.../news/2024/06/18/release-release-cdc-3.1.1.html | 11 +-
.../07/02/release-kubernetes-operator-1.9.0.html | 11 +-
content/news/2024/08/02/release-1.20.0.html | 11 +-
.../news/2024/09/05/release-release-cdc-3.2.0.html | 11 +-
content/news/2024/10/23/release-2.0-preview.html | 11 +-
.../10/30/release-kubernetes-operator-1.10.0.html | 10 +
content/posts/index.html | 78 +-
content/posts/index.xml | 1286 +----
content/posts/page/1/index.html | 11 +-
content/posts/page/10/index.html | 70 +-
content/posts/page/11/index.html | 69 +-
content/posts/page/12/index.html | 78 +-
content/posts/page/13/index.html | 79 +-
content/posts/page/14/index.html | 74 +-
content/posts/page/15/index.html | 76 +-
content/posts/page/16/index.html | 75 +-
content/posts/page/17/index.html | 68 +-
content/posts/page/18/index.html | 66 +-
content/posts/page/19/index.html | 63 +-
content/posts/page/2/index.html | 83 +-
content/posts/page/20/index.html | 65 +-
content/posts/page/21/index.html | 64 +-
content/posts/page/22/index.html | 64 +-
content/posts/page/23/index.html | 74 +-
content/posts/page/24/index.html | 50 +-
content/posts/page/3/index.html | 81 +-
content/posts/page/4/index.html | 80 +-
content/posts/page/5/index.html | 81 +-
content/posts/page/6/index.html | 75 +-
content/posts/page/7/index.html | 75 +-
content/posts/page/8/index.html | 92 +-
content/posts/page/9/index.html | 84 +-
content/powered-by.html | 11 +-
content/powered-by/index.html | 11 +-
content/roadmap.html | 11 +-
content/roadmap/index.html | 11 +-
content/security.html | 11 +-
content/security/index.html | 11 +-
content/sitemap.xml | 2 +-
content/tags/index.html | 11 +-
content/tags/index.xml | 3 +-
content/tags/page/1/index.html | 11 +-
content/use-cases.html | 11 +-
content/use-cases/index.html | 11 +-
content/what-is-flink-ml/index.html | 15 +-
content/what-is-flink-table-store/index.html | 15 +-
content/what-is-flink/community/index.html | 15 +-
.../what-is-flink/flink-applications/index.html | 189 +-
.../what-is-flink/flink-architecture/index.html | 45 +-
content/what-is-flink/flink-operations/index.html | 15 +-
content/what-is-flink/index.html | 11 +-
content/what-is-flink/index.xml | 51 +-
content/what-is-flink/powered-by/index.html | 19 +-
content/what-is-flink/roadmap/index.html | 27 +-
content/what-is-flink/security/index.html | 19 +-
content/what-is-flink/special-thanks/index.html | 21 +-
content/what-is-flink/use-cases/index.html | 51 +-
content/what-is-stateful-functions/index.html | 15 +-
.../index.html | 15 +-
...2eb071355a29b36d64578b7cdee6175344a9829c42b3.js | 586 +++
...f0b6a621002c25984bc235f8a3e44cd643474a6ed955.js | 1 -
...5c2f35150dcdb7bfab03cf9337bc3eeb8cd11fd4fb4.js} | 2 +-
content/zh/404.html | 5 +-
content/zh/categories/index.html | 11 +-
content/zh/categories/index.xml | 3 +-
content/zh/categories/page/1/index.html | 11 +-
content/zh/community.html | 11 +-
content/zh/community/index.html | 11 +-
.../zh/documentation/flink-cdc-master/index.html | 15 +-
.../zh/documentation/flink-cdc-stable/index.html | 15 +-
.../flink-kubernetes-operator-master/index.html | 15 +-
.../flink-kubernetes-operator-stable/index.html | 19 +-
content/zh/documentation/flink-master/index.html | 15 +-
content/zh/documentation/flink-preview/index.html | 15 +-
content/zh/documentation/flink-stable/index.html | 15 +-
.../flink-stateful-functions-master/index.html | 15 +-
.../flink-stateful-functions-stable/index.html | 15 +-
content/zh/documentation/flinkml-master/index.html | 15 +-
content/zh/documentation/flinkml-stable/index.html | 15 +-
content/zh/documentation/index.html | 13 +-
content/zh/documentation/index.xml | 47 +-
content/zh/downloads/index.html | 51 +-
content/zh/flink-packages/index.html | 15 +-
content/zh/getting-help.html | 11 +-
content/zh/getting-help/index.html | 11 +-
content/zh/getting-started/index.html | 13 +-
content/zh/getting-started/index.xml | 27 +-
.../zh/getting-started/training-course/index.html | 15 +-
.../zh/getting-started/with-flink-cdc/index.html | 15 +-
.../with-flink-kubernetes-operator/index.html | 15 +-
.../zh/getting-started/with-flink-ml/index.html | 15 +-
.../with-flink-stateful-functions/index.html | 15 +-
content/zh/getting-started/with-flink/index.html | 15 +-
.../code-style-and-quality-common/index.html | 67 +-
.../code-style-and-quality-components/index.html | 59 +-
.../code-style-and-quality-formatting/index.html | 45 +-
.../code-style-and-quality-java/index.html | 43 +-
.../code-style-and-quality-preamble/index.html | 55 +-
.../index.html | 45 +-
.../code-style-and-quality-scala/index.html | 47 +-
.../how-to-contribute/contribute-code/index.html | 23 +-
.../contribute-documentation/index.html | 33 +-
.../documentation-style-guide/index.html | 37 +-
.../zh/how-to-contribute/getting-help/index.html | 35 +-
.../how-to-contribute/improve-website/index.html | 29 +-
content/zh/how-to-contribute/index.html | 13 +-
content/zh/how-to-contribute/index.xml | 114 +-
content/zh/how-to-contribute/overview/index.html | 23 +-
.../zh/how-to-contribute/reviewing-prs/index.html | 33 +-
content/zh/index.html | 50 +-
content/zh/index.xml | 311 +-
content/zh/material/index.html | 27 +-
content/zh/powered-by.html | 11 +-
content/zh/powered-by/index.html | 11 +-
content/zh/roadmap.html | 11 +-
content/zh/roadmap/index.html | 11 +-
content/zh/security.html | 11 +-
content/zh/security/index.html | 11 +-
content/zh/tags/index.html | 11 +-
content/zh/tags/index.xml | 3 +-
content/zh/tags/page/1/index.html | 11 +-
content/zh/use-cases.html | 11 +-
content/zh/use-cases/index.html | 11 +-
content/zh/what-is-flink-ml/index.html | 15 +-
content/zh/what-is-flink-table-store/index.html | 15 +-
content/zh/what-is-flink/community/index.html | 35 +-
.../zh/what-is-flink/flink-applications/index.html | 205 +-
.../zh/what-is-flink/flink-architecture/index.html | 79 +-
.../zh/what-is-flink/flink-operations/index.html | 27 +-
content/zh/what-is-flink/index.html | 11 +-
content/zh/what-is-flink/index.xml | 90 +-
content/zh/what-is-flink/powered-by/index.html | 23 +-
content/zh/what-is-flink/roadmap/index.html | 35 +-
content/zh/what-is-flink/security/index.html | 19 +-
content/zh/what-is-flink/special-thanks/index.html | 23 +-
content/zh/what-is-flink/use-cases/index.html | 73 +-
content/zh/what-is-stateful-functions/index.html | 15 +-
.../index.html | 15 +-
docs/config.toml | 4 +-
...024-10-30-release-kubernetes-operator-1.10.0.md | 79 +
docs/data/flink_kubernetes_operator.yml | 14 +-
docs/data/release_archive.yml | 3 +
664 files changed, 15784 insertions(+), 9808 deletions(-)
copy content/2024/{07/02/apache-flink-kubernetes-operator-1.9.0-release-announcement => 10/25/apache-flink-kubernetes-operator-1.10.0-release-announcement}/index.html (73%)
create mode 100644 content/en.search-data.min.09ed85a135e7d7713367c78caad1f76107eb0e3a2cbda31c2e72b28582940ebc.js
delete mode 100644 content/en.search-data.min.46c32874242b2142916a071a5f86ebd9c58d0d17fedfc9fb56db3a4db935ec5c.js
rename content/{en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js => en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js} (58%)
create mode 100644 content/news/2024/10/30/release-kubernetes-operator-1.10.0.html
create mode 100644 content/zh.search-data.min.5b267a075983b27a84862eb071355a29b36d64578b7cdee6175344a9829c42b3.js
delete mode 100644 content/zh.search-data.min.5b6680b104c66c787a86f0b6a621002c25984bc235f8a3e44cd643474a6ed955.js
rename content/{zh.search.min.157539026f48a69c87cc0132304d2f362cbffe8bf1e21c392594f8392bed607e.js => zh.search.min.be67f4d8a763cede2d4455c2f35150dcdb7bfab03cf9337bc3eeb8cd11fd4fb4.js} (58%)
create mode 100644 docs/content/posts/2024-10-30-release-kubernetes-operator-1.10.0.md
(flink-web) 01/02: Flink Kubernetes Operator 1.10.0
Posted by mb...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
mbalassi pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git
commit 294a79bf4b31a54a9d637aa2dc2ba6f286a34a66
Author: mateczagany <cz...@gmail.com>
AuthorDate: Mon Oct 14 09:59:33 2024 +0200
Flink Kubernetes Operator 1.10.0
---
docs/config.toml | 4 +-
...024-10-30-release-kubernetes-operator-1.10.0.md | 79 ++++++++++++++++++++++
docs/data/flink_kubernetes_operator.yml | 14 ++--
docs/data/release_archive.yml | 3 +
4 files changed, 91 insertions(+), 9 deletions(-)
diff --git a/docs/config.toml b/docs/config.toml
index c00b79c41..940684bd1 100644
--- a/docs/config.toml
+++ b/docs/config.toml
@@ -51,8 +51,8 @@ posts = "/:year/:month/:day/:title/"
FlinkMLStableShortVersion = "2.3"
FlinkCDCStableVersion = "3.2.0"
FlinkCDCStableShortVersion = "3.2"
- FlinkKubernetesOperatorStableVersion = "1.9.0"
- FlinkKubernetesOperatorStableShortVersion = "1.9"
+ FlinkKubernetesOperatorStableVersion = "1.10.0"
+ FlinkKubernetesOperatorStableShortVersion = "1.10"
ScalaVersion = "2.12.7"
ScalaShortVersion = "2.12"
JavaVersion = "1.8"
diff --git a/docs/content/posts/2024-10-30-release-kubernetes-operator-1.10.0.md b/docs/content/posts/2024-10-30-release-kubernetes-operator-1.10.0.md
new file mode 100644
index 000000000..4ef59f3c8
--- /dev/null
+++ b/docs/content/posts/2024-10-30-release-kubernetes-operator-1.10.0.md
@@ -0,0 +1,79 @@
+---
+title: "Apache Flink Kubernetes Operator 1.10.0 Release Announcement"
+date: "2024-10-25T18:00:00.000Z"
+authors:
+- mateczagany:
+ name: "Mate Czagany"
+- 1996fanrui:
+ name: "Rui Fan"
+ twitter: "1996fanrui"
+aliases:
+- /news/2024/10/30/release-kubernetes-operator-1.10.0.html
+---
+
+The Apache Flink community is excited to announce the release of Flink Kubernetes Operator 1.10.0!
+
+The release includes several improvements to the autoscaler, and introduces a new Kubernetes custom resource called FlinkStateSnapshot to manage job snapshots.
+The process of job upgrades has also been enhanced which makes it possible to now use the last-state upgrade mode with session jobs.
+
+We encourage you to [download the release](https://flink.apache.org/downloads.html) and share your experience with the
+community through the Flink [mailing lists](https://flink.apache.org/community.html#mailing-lists) or
+[JIRA](https://issues.apache.org/jira/browse/flink)! We're looking forward to your feedback!
+
+## Highlights
+
+### FlinkStateSnapshot
+
+With this version comes also a new custom resource called FlinkStateSnapshot.
+This is used to describe savepoint or checkpoint for a Flink job.
+The savepoint/checkpoint fields found in FlinkDeployment and FlinkSessionJob status are therefore deprecated, and the operator will create new FlinkStateSnapshot resources for periodic, update and manual savepoints/checkpoints.
+
+Users can also create new FlinkStateSnapshot resources, which will instruct the operator to trigger new checkpoint/savepoint.
+
+This new feature is enabled by default, unless disabled by setting `kubernetes.operator.snapshot.resource.enabled` to false or if the FlinkStateSnapshot CRD was not found on the Kubernetes cluster.
+
+You can read more about this feature [here](https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.10/docs/custom-resource/snapshots/).
+
+
+### Last-State Upgrade Mode
+
+For deployments using last-state upgrade mode where HA metadata is not available, the operator will fallback to cancel the job via REST API and extract the last checkpoint info after cancellation if the job is healthy, making upgrades more robust.
+
+This change makes it possible to finally use the last-state upgrade mode for session jobs as well.
+
+
+### Autoscaler Delayed Scale Down
+
+With the introduction of the configuration option `job.autoscaler.scale-down.interval`, the operator can now optimize multiple scale-down operations to a single one to prevent too many unnecessary downscales, thus improving job availability.
+Please note that `job.autoscaler.scale-up.grace-period` has been removed with this change.
+
+
+### Other Autoscaler Improvements
+- Optimized cases where partitions or key groups cannot be evenly distributed to subtasks
+- Introduced `autoscaler.standalone.jdbc.event-handler.ttl` to support cleaning up historical event handler records in JDBC event handler
+- Autoscaler is now compatible with Flink 1.20
+
+
+## Release Notes
+
+The release notes can be found [here](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12354833).
+
+## Release Resources
+
+The source artifacts and helm chart are available on the Downloads page of the Flink website. You can easily try out the new features shipped in the official 1.8.0 release by adding the Helm chart to your own local registry:
+
+```
+$ helm repo add flink-kubernetes-operator-1.10.0 https://archive.apache.org/dist/flink/flink-kubernetes-operator-1.10.0/
+$ helm install flink-kubernetes-operator flink-kubernetes-operator-1.10.0/flink-kubernetes-operator --set webhook.create=false
+```
+
+You can also find official Kubernetes Operator Docker images of the new version on [Dockerhub](https://hub.docker.com/r/apache/flink-kubernetes-operator).
+
+For more details, check the [updated documentation](https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.10/) and the release notes. We encourage you to download the release and share your feedback with the community through the Flink mailing lists or JIRA.
+
+## List of Contributors
+
+Angela Chen, Ferenc Csaky, Gyula Fora, Mate Czagany, Matyas Orhidi, Naresh Kumar Reddy Gaddam,
+Roc Marshal, Rui Fan, Sam Barker, Yuepeng Pan, big face cat, chenyuzhi459, kartik-3513,
+r-sidd, 阿洋
+
diff --git a/docs/data/flink_kubernetes_operator.yml b/docs/data/flink_kubernetes_operator.yml
index 5ed27e35a..15d610478 100644
--- a/docs/data/flink_kubernetes_operator.yml
+++ b/docs/data/flink_kubernetes_operator.yml
@@ -15,6 +15,13 @@
# specific language governing permissions and limitations
# under the License
+1.10:
+ name: "Apache Flink Kubernetes Operator 1.10.0"
+ source_release_url: "https://www.apache.org/dyn/closer.lua/flink/flink-kubernetes-operator-1.10.0/flink-kubernetes-operator-1.10.0-src.tgz"
+ source_release_asc_url: "https://downloads.apache.org/flink/flink-kubernetes-operator-1.10.0/flink-kubernetes-operator-1.10.0-src.tgz.asc"
+ source_release_sha512_url: "https://downloads.apache.org/flink/flink-kubernetes-operator-1.10.0/flink-kubernetes-operator-1.10.0-src.tgz.sha512"
+ compatibility: ["1.20.x", "1.19.x", "1.18.x", "1.17.x", "1.16.x"]
+
1.9:
name: "Apache Flink Kubernetes Operator 1.9.0"
source_release_url: "https://www.apache.org/dyn/closer.lua/flink/flink-kubernetes-operator-1.9.0/flink-kubernetes-operator-1.9.0-src.tgz"
@@ -22,10 +29,3 @@
source_release_sha512_url: "https://downloads.apache.org/flink/flink-kubernetes-operator-1.9.0/flink-kubernetes-operator-1.9.0-src.tgz.sha512"
compatibility: ["1.19.x", "1.18.x", "1.17.x", "1.16.x", "1.15.x"]
-1.8:
- name: "Apache Flink Kubernetes Operator 1.8.0"
- source_release_url: "https://www.apache.org/dyn/closer.lua/flink/flink-kubernetes-operator-1.8.0/flink-kubernetes-operator-1.8.0-src.tgz"
- source_release_asc_url: "https://downloads.apache.org/flink/flink-kubernetes-operator-1.8.0/flink-kubernetes-operator-1.8.0-src.tgz.asc"
- source_release_sha512_url: "https://downloads.apache.org/flink/flink-kubernetes-operator-1.8.0/flink-kubernetes-operator-1.8.0-src.tgz.sha512"
- compatibility: ["1.18.x", "1.17.x", "1.16.x", "1.15.x"]
-
diff --git a/docs/data/release_archive.yml b/docs/data/release_archive.yml
index 40b0de796..ef3aa5e4f 100644
--- a/docs/data/release_archive.yml
+++ b/docs/data/release_archive.yml
@@ -733,6 +733,9 @@ release_archive:
release_date: 2021-01-07
flink_kubernetes_operator:
+ - version_short: 1.10
+ version_long: 1.10.0
+ release_date: 2024-10-25
- version_short: 1.9
version_long: 1.9.0
release_date: 2024-07-02
(flink-web) 02/02: Rebuild website
Posted by mb...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
mbalassi pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git
commit 28de19d0bf53815201efc1cefecbb1b1c441f6d0
Author: mateczagany <cz...@gmail.com>
AuthorDate: Sun Oct 27 11:19:07 2024 +0100
Rebuild website
---
.../08/26/apache-flink-0.6-available/index.html | 15 +-
.../09/26/apache-flink-0.6.1-available/index.html | 11 +-
content/2014/10/03/upcoming-events/index.html | 11 +-
.../11/04/apache-flink-0.7.0-available/index.html | 15 +-
.../11/18/hadoop-compatibility-in-flink/index.html | 91 +-
.../index.html | 15 +-
.../01/21/apache-flink-0.8.0-available/index.html | 15 +-
.../january-2015-in-the-flink-community/index.html | 23 +-
.../02/09/introducing-flink-streaming/index.html | 585 ++-
.../index.html | 29 +-
.../index.html | 15 +-
.../march-2015-in-the-flink-community/index.html | 19 +-
.../index.html | 11 +-
.../05/11/juggling-with-bits-and-bytes/index.html | 25 +-
.../april-2015-in-the-flink-community/index.html | 17 +-
.../06/24/announcing-apache-flink-0.9.0/index.html | 15 +-
.../index.html | 195 +-
.../09/01/apache-flink-0.9.1-available/index.html | 11 +-
.../09/03/announcing-flink-forward-2015/index.html | 15 +-
.../index.html | 61 +-
.../16/announcing-apache-flink-0.10.0/index.html | 11 +-
.../2015/11/27/flink-0.10.1-released/index.html | 15 +-
.../index.html | 11 +-
.../index.html | 129 +-
.../index.html | 11 +-
.../2016/02/11/flink-0.10.2-released/index.html | 15 +-
.../03/08/announcing-apache-flink-1.0.0/index.html | 15 +-
content/2016/04/06/flink-1.0.1-released/index.html | 15 +-
.../index.html | 157 +-
.../index.html | 11 +-
content/2016/04/22/flink-1.0.2-released/index.html | 15 +-
content/2016/05/11/flink-1.0.3-released/index.html | 15 +-
.../index.html | 11 +-
.../08/04/announcing-apache-flink-1.1.0/index.html | 73 +-
content/2016/08/04/flink-1.1.1-released/index.html | 11 +-
.../index.html | 11 +-
.../09/05/apache-flink-1.1.2-released/index.html | 15 +-
.../10/12/apache-flink-1.1.3-released/index.html | 21 +-
.../apache-flink-in-2016-year-in-review/index.html | 11 +-
.../12/21/apache-flink-1.1.4-released/index.html | 25 +-
.../02/06/announcing-apache-flink-1.2.0/index.html | 11 +-
.../03/23/apache-flink-1.1.5-released/index.html | 11 +-
.../index.html | 11 +-
.../index.html | 19 +-
.../04/26/apache-flink-1.2.1-released/index.html | 15 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../06/23/apache-flink-1.3.1-released/index.html | 11 +-
.../index.html | 85 +-
.../08/05/apache-flink-1.3.2-released/index.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../apache-flink-in-2017-year-in-review/index.html | 11 +-
.../index.html | 21 +-
.../02/15/apache-flink-1.4.1-released/index.html | 11 +-
.../index.html | 11 +-
.../03/08/apache-flink-1.4.2-released/index.html | 11 +-
.../03/15/apache-flink-1.3.3-released/index.html | 11 +-
.../index.html | 11 +-
.../07/12/apache-flink-1.5.1-released/index.html | 11 +-
.../07/31/apache-flink-1.5.2-released/index.html | 11 +-
.../index.html | 11 +-
.../08/21/apache-flink-1.5.3-released/index.html | 11 +-
.../09/20/apache-flink-1.5.4-released/index.html | 11 +-
.../09/20/apache-flink-1.6.1-released/index.html | 11 +-
.../10/29/apache-flink-1.5.5-released/index.html | 11 +-
.../10/29/apache-flink-1.6.2-released/index.html | 11 +-
.../index.html | 11 +-
.../12/21/apache-flink-1.7.1-released/index.html | 11 +-
.../12/22/apache-flink-1.6.3-released/index.html | 11 +-
.../12/26/apache-flink-1.5.6-released/index.html | 11 +-
.../index.html | 15 +-
.../02/15/apache-flink-1.7.2-released/index.html | 11 +-
.../index.html | 15 +-
.../02/25/apache-flink-1.6.4-released/index.html | 11 +-
.../index.html | 11 +-
.../index.html | 41 +-
.../index.html | 11 +-
.../index.html | 11 +-
content/2019/05/03/pulsar-flink.html | 11 +-
.../03/when-flink-pulsar-come-together/index.html | 151 +-
.../index.html | 75 +-
content/2019/05/14/temporal-tables.html | 11 +-
.../index.html | 73 +-
content/2019/05/19/state-ttl.html | 11 +-
.../index.html | 15 +-
content/2019/06/05/flink-network-stack.html | 11 +-
.../index.html | 159 +-
content/2019/06/26/broadcast-state.html | 11 +-
.../07/02/apache-flink-1.8.1-released/index.html | 11 +-
content/2019/07/23/flink-network-stack-2.html | 11 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../flink-community-update-september19/index.html | 11 +-
.../09/11/apache-flink-1.8.2-released/index.html | 11 +-
.../index.html | 11 +-
.../10/18/apache-flink-1.9.1-released/index.html | 11 +-
.../index.html | 11 +-
.../index.html | 235 +-
.../12/11/apache-flink-1.8.3-released/index.html | 11 +-
.../index.html | 119 +-
.../index.html | 203 +-
.../01/30/apache-flink-1.9.2-released/index.html | 11 +-
.../index.html | 291 +-
.../index.html | 11 +-
.../index.html | 21 +-
.../index.html | 11 +-
.../index.html | 153 +-
.../index.html | 15 +-
.../30/flink-community-update-april20/index.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
content/2020/04/09/pyflink-udf-support-flink.html | 11 +-
.../index.html | 131 +-
.../index.html | 11 +-
.../04/24/apache-flink-1.9.3-released/index.html | 11 +-
.../index.html | 11 +-
.../05/06/flink-community-update-may20/index.html | 11 +-
.../05/12/apache-flink-1.10.1-released/index.html | 15 +-
.../index.html | 23 +-
.../06/10/flink-community-update-june20/index.html | 11 +-
.../index.html | 11 +-
.../index.html | 15 +-
.../index.html | 15 +-
.../index.html | 19 +-
.../07/21/apache-flink-1.11.1-released/index.html | 11 +-
content/2020/07/23/catalogs.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
...ql-demo-building-e2e-streaming-application.html | 11 +-
.../07/29/flink-community-update-july20/index.html | 15 +-
.../index.html | 273 +-
.../08/04/pyflink-pandas-udf-support-flink.html | 11 +-
.../index.html | 15 +-
.../index.html | 67 +-
.../index.html | 11 +-
content/2020/08/19/statefun.html | 11 +-
.../08/20/the-state-of-flink-on-docker/index.html | 15 +-
.../08/25/apache-flink-1.10.2-released/index.html | 15 +-
.../flink-1.11-memory-management-improvements.html | 11 +-
.../index.html | 11 +-
.../04/flink-community-update-august20/index.html | 15 +-
.../09/17/apache-flink-1.11.2-released/index.html | 11 +-
.../index.html | 73 +-
.../index.html | 11 +-
.../index.html | 11 +-
...om-aligned-to-unaligned-checkpoints-part-1.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 23 +-
content/2020/12/15/pipelined-region-sheduling.html | 11 +-
.../12/18/apache-flink-1.11.3-released/index.html | 11 +-
.../index.html | 11 +-
content/2021/01/07/pulsar-flink-connector-270.html | 11 +-
.../index.html | 15 +-
.../index.html | 11 +-
content/2021/01/18/rocksdb.html | 11 +-
.../index.html | 11 +-
.../01/19/apache-flink-1.12.1-released/index.html | 11 +-
.../01/29/apache-flink-1.10.3-released/index.html | 11 +-
.../index.html | 11 +-
content/2021/02/10/native-k8s-with-ha.html | 11 +-
.../03/03/apache-flink-1.12.2-released/index.html | 11 +-
.../index.html | 157 +-
content/2021/03/11/batch-execution-mode.html | 11 +-
.../index.html | 65 +-
.../04/29/apache-flink-1.12.3-released/index.html | 11 +-
.../index.html | 33 +-
content/2021/05/06/reactive-mode.html | 11 +-
.../index.html | 15 +-
.../05/21/apache-flink-1.12.4-released/index.html | 11 +-
.../05/28/apache-flink-1.13.1-released/index.html | 11 +-
content/2021/07/07/backpressure.html | 11 +-
.../index.html | 15 +-
.../08/06/apache-flink-1.12.5-released/index.html | 11 +-
.../08/06/apache-flink-1.13.2-released/index.html | 11 +-
.../08/09/apache-flink-1.11.4-released/index.html | 11 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../2021/09/07/connector-table-sql-api-part1.html | 11 +-
.../2021/09/07/connector-table-sql-api-part2.html | 11 +-
.../index.html | 201 +-
.../index.html | 601 ++-
.../index.html | 11 +-
.../10/19/apache-flink-1.13.3-released/index.html | 11 +-
.../index.html | 15 +-
.../index.html | 81 +-
content/2021/10/26/sort-shuffle-part1.html | 11 +-
content/2021/10/26/sort-shuffle-part2.html | 11 +-
.../index.html | 15 +-
content/2021/11/03/flink-backward.html | 11 +-
.../index.html | 19 +-
content/2021/12/10/log4j-cve.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 15 +-
.../index.html | 15 +-
.../2022/01/04/scheduler-performance-part-one.html | 11 +-
.../2022/01/04/scheduler-performance-part-two.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
content/2022/01/20/pravega-connector-101.html | 11 +-
.../01/20/pravega-flink-connector-101/index.html | 143 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../02/22/scala-free-in-one-fifteen/index.html | 31 +-
content/2022/02/22/scala-free.html | 11 +-
.../index.html | 15 +-
.../the-generic-asynchronous-base-sink/index.html | 135 +-
.../index.html | 11 +-
.../index.html | 11 +-
content/2022/05/06/async-sink-base.html | 11 +-
.../index.html | 89 +-
.../index.html | 11 +-
content/2022/05/06/pyflink-1.15-thread-mode.html | 11 +-
content/2022/05/06/restore-modes.html | 11 +-
.../index.html | 15 +-
.../index.html | 11 +-
content/2022/05/18/latency-part1.html | 11 +-
.../index.html | 11 +-
content/2022/05/23/latency-part2.html | 11 +-
content/2022/05/30/changelog-state-backend.html | 11 +-
.../index.html | 15 +-
.../index.html | 19 +-
.../index.html | 15 +-
content/2022/06/17/adaptive-batch-scheduler.html | 11 +-
.../index.html | 15 +-
.../index.html | 11 +-
content/2022/07/11/final-checkpoint-part1.html | 11 +-
content/2022/07/11/final-checkpoint-part2.html | 11 +-
.../index.html | 15 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../index.html | 21 +-
.../08/regarding-akkas-licensing-change/index.html | 11 +-
.../index.html | 15 +-
.../index.html | 15 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../11/25/async-sink-rate-limiting-strategy.html | 11 +-
.../index.html | 199 +-
.../index.html | 11 +-
.../index.html | 15 +-
.../index.html | 15 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 19 +-
.../index.html | 15 +-
.../index.html | 79 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 19 +-
.../index.html | 19 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../index.html | 15 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 43 +-
.../index.html | 15 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 11 +-
.../index.html | 45 +-
.../index.html | 11 +-
.../preview-release-of-apache-flink-2.0/index.html | 11 +-
.../index.html | 143 +-
content/404.html | 5 +-
content/categories/index.html | 11 +-
content/categories/index.xml | 3 +-
content/categories/page/1/index.html | 11 +-
content/community.html | 11 +-
content/community/index.html | 11 +-
content/documentation/flink-cdc-master/index.html | 15 +-
content/documentation/flink-cdc-stable/index.html | 15 +-
.../flink-kubernetes-operator-master/index.html | 15 +-
.../flink-kubernetes-operator-stable/index.html | 19 +-
content/documentation/flink-master/index.html | 15 +-
content/documentation/flink-preview/index.html | 15 +-
content/documentation/flink-stable/index.html | 15 +-
.../flink-stateful-functions-master/index.html | 15 +-
.../flink-stateful-functions-stable/index.html | 15 +-
content/documentation/flinkml-master/index.html | 15 +-
content/documentation/flinkml-stable/index.html | 15 +-
content/documentation/index.html | 13 +-
content/documentation/index.xml | 47 +-
content/downloads/index.html | 51 +-
.../apache-beam-how-beam-runs-on-top-of-flink.html | 11 +-
.../2020/06/23/flink-on-zeppelin-part2.html | 11 +-
...c78caad1f76107eb0e3a2cbda31c2e72b28582940ebc.js | 5481 ++++++++++++++++++++
...071a5f86ebd9c58d0d17fedfc9fb56db3a4db935ec5c.js | 1 -
...87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js} | 2 +-
content/en/index.html | 11 +-
content/en/sitemap.xml | 7 +-
.../feature/2019/09/13/state-processor-api.html | 11 +-
.../2017/07/04/flink-rescalable-state.html | 11 +-
.../2018/01/30/incremental-checkpointing.html | 11 +-
.../01/end-to-end-exactly-once-apache-flink.html | 11 +-
.../features/2019/03/11/prometheus-monitoring.html | 11 +-
.../2020/03/27/flink-for-data-warehouse.html | 11 +-
content/flink-packages/index.html | 15 +-
content/getting-help.html | 11 +-
content/getting-help/index.html | 11 +-
content/getting-started/index.html | 13 +-
content/getting-started/index.xml | 27 +-
content/getting-started/training-course/index.html | 15 +-
content/getting-started/with-flink-cdc/index.html | 15 +-
.../with-flink-kubernetes-operator/index.html | 15 +-
content/getting-started/with-flink-ml/index.html | 15 +-
.../with-flink-stateful-functions/index.html | 15 +-
content/getting-started/with-flink/index.html | 15 +-
.../code-style-and-quality-common/index.html | 71 +-
.../code-style-and-quality-components/index.html | 51 +-
.../code-style-and-quality-formatting/index.html | 45 +-
.../code-style-and-quality-java/index.html | 43 +-
.../code-style-and-quality-preamble/index.html | 49 +-
.../index.html | 43 +-
.../code-style-and-quality-scala/index.html | 47 +-
.../how-to-contribute/contribute-code/index.html | 15 +-
.../contribute-documentation/index.html | 19 +-
.../documentation-style-guide/index.html | 21 +-
content/how-to-contribute/getting-help/index.html | 21 +-
.../how-to-contribute/improve-website/index.html | 23 +-
content/how-to-contribute/index.html | 13 +-
content/how-to-contribute/index.xml | 72 +-
content/how-to-contribute/overview/index.html | 15 +-
content/how-to-contribute/reviewing-prs/index.html | 19 +-
content/index.html | 50 +-
content/index.xml | 1506 +-----
content/material/index.html | 23 +-
content/news/2014/08/26/release-0.6.html | 11 +-
content/news/2014/09/26/release-0.6.1.html | 11 +-
content/news/2014/10/03/upcoming_events.html | 11 +-
content/news/2014/11/04/release-0.7.0.html | 11 +-
content/news/2014/11/18/hadoop-compatibility.html | 11 +-
content/news/2015/01/06/december-in-flink.html | 11 +-
content/news/2015/01/21/release-0.8.html | 11 +-
content/news/2015/02/04/january-in-flink.html | 11 +-
content/news/2015/02/09/streaming-example.html | 11 +-
.../news/2015/03/02/february-2015-in-flink.html | 11 +-
.../13/peeking-into-Apache-Flinks-Engine-Room.html | 11 +-
content/news/2015/04/07/march-in-flink.html | 11 +-
.../news/2015/04/13/release-0.9.0-milestone1.html | 11 +-
.../2015/05/11/Juggling-with-Bits-and-Bytes.html | 11 +-
.../news/2015/05/14/Community-update-April.html | 11 +-
.../24/announcing-apache-flink-0.9.0-release.html | 11 +-
.../news/2015/08/24/introducing-flink-gelly.html | 11 +-
content/news/2015/09/01/release-0.9.1.html | 11 +-
content/news/2015/09/03/flink-forward.html | 11 +-
content/news/2015/09/16/off-heap-memory.html | 11 +-
content/news/2015/11/16/release-0.10.0.html | 11 +-
content/news/2015/11/27/release-0.10.1.html | 11 +-
content/news/2015/12/04/Introducing-windows.html | 11 +-
content/news/2015/12/11/storm-compatibility.html | 11 +-
content/news/2015/12/18/a-year-in-review.html | 11 +-
content/news/2016/02/11/release-0.10.2.html | 11 +-
content/news/2016/03/08/release-1.0.0.html | 11 +-
content/news/2016/04/06/cep-monitoring.html | 11 +-
content/news/2016/04/06/release-1.0.1.html | 11 +-
.../news/2016/04/14/flink-forward-announce.html | 11 +-
content/news/2016/04/22/release-1.0.2.html | 11 +-
content/news/2016/05/11/release-1.0.3.html | 11 +-
content/news/2016/05/24/stream-sql.html | 11 +-
content/news/2016/08/08/release-1.1.0.html | 11 +-
content/news/2016/08/11/release-1.1.1.html | 11 +-
content/news/2016/08/24/ff16-keynotes-panels.html | 11 +-
content/news/2016/09/05/release-1.1.2.html | 11 +-
content/news/2016/10/12/release-1.1.3.html | 11 +-
content/news/2016/12/19/2016-year-in-review.html | 11 +-
content/news/2016/12/21/release-1.1.4.html | 11 +-
content/news/2017/02/06/release-1.2.0.html | 11 +-
content/news/2017/03/23/release-1.1.5.html | 11 +-
content/news/2017/03/29/table-sql-api-update.html | 11 +-
content/news/2017/04/04/dynamic-tables.html | 11 +-
content/news/2017/04/26/release-1.2.1.html | 11 +-
content/news/2017/05/16/official-docker-image.html | 11 +-
content/news/2017/06/01/release-1.3.0.html | 11 +-
content/news/2017/06/23/release-1.3.1.html | 11 +-
content/news/2017/08/05/release-1.3.2.html | 11 +-
.../2017/11/22/release-1.4-and-1.5-timeline.html | 11 +-
content/news/2017/12/12/release-1.4.0.html | 11 +-
content/news/2017/12/21/2017-year-in-review.html | 11 +-
content/news/2018/02/15/release-1.4.1.html | 11 +-
content/news/2018/03/08/release-1.4.2.html | 11 +-
content/news/2018/03/15/release-1.3.3.html | 11 +-
content/news/2018/05/25/release-1.5.0.html | 11 +-
content/news/2018/07/12/release-1.5.1.html | 11 +-
content/news/2018/07/31/release-1.5.2.html | 11 +-
content/news/2018/08/09/release-1.6.0.html | 11 +-
content/news/2018/08/21/release-1.5.3.html | 11 +-
content/news/2018/09/20/release-1.5.4.html | 11 +-
content/news/2018/09/20/release-1.6.1.html | 11 +-
content/news/2018/10/29/release-1.5.5.html | 11 +-
content/news/2018/10/29/release-1.6.2.html | 11 +-
content/news/2018/11/30/release-1.7.0.html | 11 +-
content/news/2018/12/21/release-1.7.1.html | 11 +-
content/news/2018/12/22/release-1.6.3.html | 11 +-
content/news/2018/12/26/release-1.5.6.html | 11 +-
.../2019/02/13/unified-batch-streaming-blink.html | 11 +-
content/news/2019/02/15/release-1.7.2.html | 11 +-
.../news/2019/02/25/monitoring-best-practices.html | 11 +-
content/news/2019/02/25/release-1.6.4.html | 11 +-
content/news/2019/03/06/ffsf-preview.html | 11 +-
content/news/2019/04/09/release-1.8.0.html | 11 +-
content/news/2019/04/17/sod.html | 11 +-
content/news/2019/07/02/release-1.8.1.html | 11 +-
content/news/2019/08/22/release-1.9.0.html | 11 +-
content/news/2019/09/10/community-update.html | 11 +-
content/news/2019/09/11/release-1.8.2.html | 11 +-
content/news/2019/10/18/release-1.9.1.html | 11 +-
.../query-pulsar-streams-using-apache-flink.html | 11 +-
content/news/2019/12/09/flink-kubernetes-kudo.html | 11 +-
content/news/2019/12/11/release-1.8.3.html | 11 +-
content/news/2020/01/15/demo-fraud-detection.html | 11 +-
...ked-interacting-with-state-in-apache-flink.html | 11 +-
content/news/2020/01/30/release-1.9.2.html | 11 +-
.../a-guide-for-unit-testing-in-apache-flink.html | 11 +-
content/news/2020/02/11/release-1.10.0.html | 11 +-
content/news/2020/02/20/ddl.html | 11 +-
.../news/2020/03/24/demo-fraud-detection-2.html | 11 +-
content/news/2020/04/01/community-update.html | 11 +-
.../news/2020/04/07/release-statefun-2.0.0.html | 11 +-
.../04/15/flink-serialization-tuning-vol-1.html | 11 +-
.../memory-management-improvements-flink-1.10.html | 11 +-
content/news/2020/04/24/release-1.9.3.html | 11 +-
content/news/2020/05/04/season-of-docs.html | 11 +-
content/news/2020/05/07/community-update.html | 11 +-
content/news/2020/05/12/release-1.10.1.html | 11 +-
.../news/2020/06/09/release-statefun-2.1.0.html | 11 +-
content/news/2020/06/11/community-update.html | 11 +-
.../news/2020/06/15/flink-on-zeppelin-part1.html | 11 +-
content/news/2020/07/06/release-1.11.0.html | 11 +-
content/news/2020/07/14/application-mode.html | 11 +-
content/news/2020/07/21/release-1.11.1.html | 11 +-
content/news/2020/07/27/community-update.html | 11 +-
.../news/2020/07/30/demo-fraud-detection-3.html | 11 +-
content/news/2020/08/06/external-resource.html | 11 +-
content/news/2020/08/20/flink-docker.html | 11 +-
content/news/2020/08/25/release-1.10.2.html | 11 +-
content/news/2020/09/04/community-update.html | 11 +-
content/news/2020/09/17/release-1.11.2.html | 11 +-
.../news/2020/09/28/release-statefun-2.2.0.html | 11 +-
.../2020/10/13/stateful-serverless-internals.html | 11 +-
.../news/2020/11/11/release-statefun-2.2.1.html | 11 +-
content/news/2020/12/10/release-1.12.0.html | 11 +-
content/news/2020/12/18/release-1.11.3.html | 11 +-
.../news/2021/01/02/release-statefun-2.2.2.html | 11 +-
.../01/11/batch-fine-grained-fault-tolerance.html | 11 +-
content/news/2021/01/19/release-1.12.1.html | 11 +-
content/news/2021/01/29/release-1.10.3.html | 11 +-
content/news/2021/03/03/release-1.12.2.html | 11 +-
.../news/2021/04/15/release-statefun-3.0.0.html | 11 +-
content/news/2021/04/29/release-1.12.3.html | 11 +-
content/news/2021/05/03/release-1.13.0.html | 11 +-
content/news/2021/05/21/release-1.12.4.html | 11 +-
content/news/2021/05/28/release-1.13.1.html | 11 +-
content/news/2021/08/06/release-1.12.5.html | 11 +-
content/news/2021/08/06/release-1.13.2.html | 11 +-
content/news/2021/08/09/release-1.11.4.html | 11 +-
content/news/2021/08/31/release-1.14.0-rc0.html | 11 +-
.../news/2021/08/31/release-statefun-3.1.0.html | 11 +-
content/news/2021/09/29/release-1.14.0.html | 11 +-
content/news/2021/10/19/release-1.13.3.html | 11 +-
content/news/2021/12/16/log4j-patch-releases.html | 11 +-
.../news/2021/12/22/log4j-statefun-release.html | 11 +-
content/news/2022/01/07/release-ml-2.0.0.html | 11 +-
content/news/2022/01/17/release-1.14.3.html | 11 +-
.../news/2022/01/31/release-statefun-3.2.0.html | 11 +-
content/news/2022/02/18/release-1.13.6.html | 11 +-
content/news/2022/03/11/release-1.14.4.html | 11 +-
.../04/03/release-kubernetes-operator-0.1.0.html | 11 +-
content/news/2022/05/05/1.15-announcement.html | 11 +-
.../news/2022/05/11/release-table-store-0.1.0.html | 11 +-
.../06/05/release-kubernetes-operator-1.0.0.html | 11 +-
content/news/2022/06/22/release-1.14.5.html | 11 +-
content/news/2022/07/06/release-1.15.1.html | 11 +-
content/news/2022/07/12/release-ml-2.1.0.html | 11 +-
.../07/25/release-kubernetes-operator-1.1.0.html | 11 +-
content/news/2022/08/25/release-1.15.2.html | 11 +-
.../news/2022/08/29/release-table-store-0.2.0.html | 11 +-
content/news/2022/09/08/akka-license-change.html | 11 +-
content/news/2022/09/28/release-1.14.6.html | 11 +-
.../10/07/release-kubernetes-operator-1.2.0.html | 11 +-
.../news/2022/10/13/release-table-store-0.2.1.html | 11 +-
content/news/2022/10/28/1.16-announcement.html | 11 +-
content/news/2022/11/10/release-1.15.3.html | 11 +-
.../12/14/release-kubernetes-operator-1.3.0.html | 11 +-
.../01/10/release-kubernetes-operator-1.3.1.html | 11 +-
.../news/2023/01/13/release-table-store-0.3.0.html | 11 +-
.../2023/01/20/delegation-token-framework.html | 11 +-
content/news/2023/01/30/release-1.16.1.html | 11 +-
.../02/27/release-kubernetes-operator-1.4.0.html | 11 +-
content/news/2023/03/23/release-1.17.0.html | 11 +-
content/news/2023/04/19/release-ml-2.2.0.html | 11 +-
.../05/17/release-kubernetes-operator-1.5.0.html | 11 +-
content/news/2023/05/25/release-1.16.2.html | 11 +-
content/news/2023/05/25/release-1.17.1.html | 11 +-
content/news/2023/07/03/sigmod-system-award.html | 11 +-
.../news/2023/08/04/externalized-connectors.html | 11 +-
.../08/15/release-kubernetes-operator-1.6.0.html | 11 +-
content/news/2023/10/10/release-1.18.0.html | 11 +-
.../10/27/release-kubernetes-operator-1.6.1.html | 11 +-
.../11/22/release-kubernetes-operator-1.7.0.html | 11 +-
content/news/2023/11/29/release-1.16.3.html | 11 +-
content/news/2023/11/29/release-1.17.2.html | 11 +-
content/news/2024/01/19/release-1.18.1.html | 11 +-
content/news/2024/03/18/release-1.19.0.html | 11 +-
.../03/21/release-kubernetes-operator-1.8.0.html | 11 +-
content/news/2024/05/17/release-cdc-3.1.0.html | 11 +-
content/news/2024/06/14/release-1.19.1.html | 11 +-
.../news/2024/06/18/release-release-cdc-3.1.1.html | 11 +-
.../07/02/release-kubernetes-operator-1.9.0.html | 11 +-
content/news/2024/08/02/release-1.20.0.html | 11 +-
.../news/2024/09/05/release-release-cdc-3.2.0.html | 11 +-
content/news/2024/10/23/release-2.0-preview.html | 11 +-
.../10/30/release-kubernetes-operator-1.10.0.html | 10 +
content/posts/index.html | 78 +-
content/posts/index.xml | 1286 +----
content/posts/page/1/index.html | 11 +-
content/posts/page/10/index.html | 70 +-
content/posts/page/11/index.html | 69 +-
content/posts/page/12/index.html | 78 +-
content/posts/page/13/index.html | 79 +-
content/posts/page/14/index.html | 74 +-
content/posts/page/15/index.html | 76 +-
content/posts/page/16/index.html | 75 +-
content/posts/page/17/index.html | 68 +-
content/posts/page/18/index.html | 66 +-
content/posts/page/19/index.html | 63 +-
content/posts/page/2/index.html | 83 +-
content/posts/page/20/index.html | 65 +-
content/posts/page/21/index.html | 64 +-
content/posts/page/22/index.html | 64 +-
content/posts/page/23/index.html | 74 +-
content/posts/page/24/index.html | 50 +-
content/posts/page/3/index.html | 81 +-
content/posts/page/4/index.html | 80 +-
content/posts/page/5/index.html | 81 +-
content/posts/page/6/index.html | 75 +-
content/posts/page/7/index.html | 75 +-
content/posts/page/8/index.html | 92 +-
content/posts/page/9/index.html | 84 +-
content/powered-by.html | 11 +-
content/powered-by/index.html | 11 +-
content/roadmap.html | 11 +-
content/roadmap/index.html | 11 +-
content/security.html | 11 +-
content/security/index.html | 11 +-
content/sitemap.xml | 2 +-
content/tags/index.html | 11 +-
content/tags/index.xml | 3 +-
content/tags/page/1/index.html | 11 +-
content/use-cases.html | 11 +-
content/use-cases/index.html | 11 +-
content/what-is-flink-ml/index.html | 15 +-
content/what-is-flink-table-store/index.html | 15 +-
content/what-is-flink/community/index.html | 15 +-
.../what-is-flink/flink-applications/index.html | 189 +-
.../what-is-flink/flink-architecture/index.html | 45 +-
content/what-is-flink/flink-operations/index.html | 15 +-
content/what-is-flink/index.html | 11 +-
content/what-is-flink/index.xml | 51 +-
content/what-is-flink/powered-by/index.html | 19 +-
content/what-is-flink/roadmap/index.html | 27 +-
content/what-is-flink/security/index.html | 19 +-
content/what-is-flink/special-thanks/index.html | 21 +-
content/what-is-flink/use-cases/index.html | 51 +-
content/what-is-stateful-functions/index.html | 15 +-
.../index.html | 15 +-
...2eb071355a29b36d64578b7cdee6175344a9829c42b3.js | 586 +++
...f0b6a621002c25984bc235f8a3e44cd643474a6ed955.js | 1 -
...5c2f35150dcdb7bfab03cf9337bc3eeb8cd11fd4fb4.js} | 2 +-
content/zh/404.html | 5 +-
content/zh/categories/index.html | 11 +-
content/zh/categories/index.xml | 3 +-
content/zh/categories/page/1/index.html | 11 +-
content/zh/community.html | 11 +-
content/zh/community/index.html | 11 +-
.../zh/documentation/flink-cdc-master/index.html | 15 +-
.../zh/documentation/flink-cdc-stable/index.html | 15 +-
.../flink-kubernetes-operator-master/index.html | 15 +-
.../flink-kubernetes-operator-stable/index.html | 19 +-
content/zh/documentation/flink-master/index.html | 15 +-
content/zh/documentation/flink-preview/index.html | 15 +-
content/zh/documentation/flink-stable/index.html | 15 +-
.../flink-stateful-functions-master/index.html | 15 +-
.../flink-stateful-functions-stable/index.html | 15 +-
content/zh/documentation/flinkml-master/index.html | 15 +-
content/zh/documentation/flinkml-stable/index.html | 15 +-
content/zh/documentation/index.html | 13 +-
content/zh/documentation/index.xml | 47 +-
content/zh/downloads/index.html | 51 +-
content/zh/flink-packages/index.html | 15 +-
content/zh/getting-help.html | 11 +-
content/zh/getting-help/index.html | 11 +-
content/zh/getting-started/index.html | 13 +-
content/zh/getting-started/index.xml | 27 +-
.../zh/getting-started/training-course/index.html | 15 +-
.../zh/getting-started/with-flink-cdc/index.html | 15 +-
.../with-flink-kubernetes-operator/index.html | 15 +-
.../zh/getting-started/with-flink-ml/index.html | 15 +-
.../with-flink-stateful-functions/index.html | 15 +-
content/zh/getting-started/with-flink/index.html | 15 +-
.../code-style-and-quality-common/index.html | 67 +-
.../code-style-and-quality-components/index.html | 59 +-
.../code-style-and-quality-formatting/index.html | 45 +-
.../code-style-and-quality-java/index.html | 43 +-
.../code-style-and-quality-preamble/index.html | 55 +-
.../index.html | 45 +-
.../code-style-and-quality-scala/index.html | 47 +-
.../how-to-contribute/contribute-code/index.html | 23 +-
.../contribute-documentation/index.html | 33 +-
.../documentation-style-guide/index.html | 37 +-
.../zh/how-to-contribute/getting-help/index.html | 35 +-
.../how-to-contribute/improve-website/index.html | 29 +-
content/zh/how-to-contribute/index.html | 13 +-
content/zh/how-to-contribute/index.xml | 114 +-
content/zh/how-to-contribute/overview/index.html | 23 +-
.../zh/how-to-contribute/reviewing-prs/index.html | 33 +-
content/zh/index.html | 50 +-
content/zh/index.xml | 311 +-
content/zh/material/index.html | 27 +-
content/zh/powered-by.html | 11 +-
content/zh/powered-by/index.html | 11 +-
content/zh/roadmap.html | 11 +-
content/zh/roadmap/index.html | 11 +-
content/zh/security.html | 11 +-
content/zh/security/index.html | 11 +-
content/zh/tags/index.html | 11 +-
content/zh/tags/index.xml | 3 +-
content/zh/tags/page/1/index.html | 11 +-
content/zh/use-cases.html | 11 +-
content/zh/use-cases/index.html | 11 +-
content/zh/what-is-flink-ml/index.html | 15 +-
content/zh/what-is-flink-table-store/index.html | 15 +-
content/zh/what-is-flink/community/index.html | 35 +-
.../zh/what-is-flink/flink-applications/index.html | 205 +-
.../zh/what-is-flink/flink-architecture/index.html | 79 +-
.../zh/what-is-flink/flink-operations/index.html | 27 +-
content/zh/what-is-flink/index.html | 11 +-
content/zh/what-is-flink/index.xml | 90 +-
content/zh/what-is-flink/powered-by/index.html | 23 +-
content/zh/what-is-flink/roadmap/index.html | 35 +-
content/zh/what-is-flink/security/index.html | 19 +-
content/zh/what-is-flink/special-thanks/index.html | 23 +-
content/zh/what-is-flink/use-cases/index.html | 73 +-
content/zh/what-is-stateful-functions/index.html | 15 +-
.../index.html | 15 +-
660 files changed, 15693 insertions(+), 9799 deletions(-)
diff --git a/content/2014/08/26/apache-flink-0.6-available/index.html b/content/2014/08/26/apache-flink-0.6-available/index.html
index 8ccb2c6cc..9a6be9f2f 100644
--- a/content/2014/08/26/apache-flink-0.6-available/index.html
+++ b/content/2014/08/26/apache-flink-0.6-available/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -16,26 +16,25 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="We are happy to announce the availability of Flink 0.6. This is the first release of the system inside the Apache Incubator and under the name Flink. Releases up to 0.5 were under the name Stratosphere, the academic and open source project that Flink originates from.
-What is Flink? # Apache Flink is a general-purpose data processing engine for clusters. It runs on YARN clusters on top of data stored in Hadoop, as well as stand-alone.">
+What is Flink? # Apache Flink is a general-purpose data processing engine for clusters. It runs on YARN clusters on top of data stored in Hadoop, as well as stand-alone.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Apache Flink 0.6 available" />
<meta property="og:description" content="We are happy to announce the availability of Flink 0.6. This is the first release of the system inside the Apache Incubator and under the name Flink. Releases up to 0.5 were under the name Stratosphere, the academic and open source project that Flink originates from.
-What is Flink? # Apache Flink is a general-purpose data processing engine for clusters. It runs on YARN clusters on top of data stored in Hadoop, as well as stand-alone." />
+What is Flink? # Apache Flink is a general-purpose data processing engine for clusters. It runs on YARN clusters on top of data stored in Hadoop, as well as stand-alone." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2014/08/26/apache-flink-0.6-available/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2014-08-26T10:00:00+00:00" />
<meta property="article:modified_time" content="2014-08-26T10:00:00+00:00" />
-
<title>Apache Flink 0.6 available | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2014/09/26/apache-flink-0.6.1-available/index.html b/content/2014/09/26/apache-flink-0.6.1-available/index.html
index 47c19bb64..289c35dcc 100644
--- a/content/2014/09/26/apache-flink-0.6.1-available/index.html
+++ b/content/2014/09/26/apache-flink-0.6.1-available/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -26,18 +26,17 @@ Download the release today." />
<meta property="og:url" content="https://flink.apache.org/2014/09/26/apache-flink-0.6.1-available/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2014-09-26T10:00:00+00:00" />
<meta property="article:modified_time" content="2014-09-26T10:00:00+00:00" />
-
<title>Apache Flink 0.6.1 available | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -60,7 +59,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -267,7 +266,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2014/10/03/upcoming-events/index.html b/content/2014/10/03/upcoming-events/index.html
index f87143627..56483a512 100644
--- a/content/2014/10/03/upcoming-events/index.html
+++ b/content/2014/10/03/upcoming-events/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -22,18 +22,17 @@
<meta property="og:url" content="https://flink.apache.org/2014/10/03/upcoming-events/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2014-10-03T10:00:00+00:00" />
<meta property="article:modified_time" content="2014-10-03T10:00:00+00:00" />
-
<title>Upcoming Events | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2014/11/04/apache-flink-0.7.0-available/index.html b/content/2014/11/04/apache-flink-0.7.0-available/index.html
index 953e6a236..92f46fd17 100644
--- a/content/2014/11/04/apache-flink-0.7.0-available/index.html
+++ b/content/2014/11/04/apache-flink-0.7.0-available/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -18,28 +18,27 @@
<meta name="description" content="We are pleased to announce the availability of Flink 0.7.0. This release includes new user-facing features as well as performance and bug fixes, brings the Scala and Java APIs in sync, and introduces Flink Streaming. A total of 34 people have contributed to this release, a big thanks to all of them!
Download Flink 0.7.0 here
See the release changelog here
-Overview of major new features # Flink Streaming: The gem of the 0.">
+Overview of major new features # Flink Streaming: The gem of the 0.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Apache Flink 0.7.0 available" />
<meta property="og:description" content="We are pleased to announce the availability of Flink 0.7.0. This release includes new user-facing features as well as performance and bug fixes, brings the Scala and Java APIs in sync, and introduces Flink Streaming. A total of 34 people have contributed to this release, a big thanks to all of them!
Download Flink 0.7.0 here
See the release changelog here
-Overview of major new features # Flink Streaming: The gem of the 0." />
+Overview of major new features # Flink Streaming: The gem of the 0." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2014/11/04/apache-flink-0.7.0-available/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2014-11-04T10:00:00+00:00" />
<meta property="article:modified_time" content="2014-11-04T10:00:00+00:00" />
-
<title>Apache Flink 0.7.0 available | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -62,7 +61,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -269,7 +268,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2014/11/18/hadoop-compatibility-in-flink/index.html b/content/2014/11/18/hadoop-compatibility-in-flink/index.html
index 17b9b9c92..66a113a45 100644
--- a/content/2014/11/18/hadoop-compatibility-in-flink/index.html
+++ b/content/2014/11/18/hadoop-compatibility-in-flink/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -22,18 +22,17 @@
<meta property="og:url" content="https://flink.apache.org/2014/11/18/hadoop-compatibility-in-flink/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2014-11-18T10:00:00+00:00" />
<meta property="article:modified_time" content="2014-11-18T10:00:00+00:00" />
-
<title>Hadoop Compatibility in Flink | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -505,46 +504,46 @@ https://github.com/alex-shpak/hugo-book
</ul>
<p>in Flink programs without changing a line of code. Moreover, Flink also natively supports all Hadoop data types (<code>Writables</code> and <code>WritableComparable</code>).</p>
<p>The following code snippet shows a simple Flink WordCount program that solely uses Hadoop data types, InputFormat, OutputFormat, Mapper, and Reducer functions.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// Definition of Hadoop Mapper function
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">public</span> <span class="kd">class</span> <span class="nc">Tokenizer</span> <span class="kd">implements</span> <span class="n">Mapper</span><span class="o"><</span><span class="n">LongWritable</span><span class="o">,</span> <span class="n">Text</span><span class="o">,</span> <span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">></span [...]
-</span></span><span class="line"><span class="cl"><span class="c1">// Definition of Hadoop Reducer function
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">public</span> <span class="kd">class</span> <span class="nc">Counter</span> <span class="kd">implements</span> <span class="n">Reducer</span><span class="o"><</span><span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">,</span> <span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">></span> [...]
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">static</span> <span class="kt">void</span> <span class="nf">main</span><span class="o">(</span><span class="n">String</span><span class="o">[]</span> <span class="n">args</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">final</span> <span class="n">String</span> <span class="n">inputPath</span> <span class="o">=</span> <span class="n">args</span><span class="o">[</span><span class="n">0</span><span class="o">];</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">final</span> <span class="n">String</span> <span class="n">outputPath</span> <span class="o">=</span> <span class="n">args</span><span class="o">[</span><span class="n">1</span><span class="o">];</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="kd">final</span> <span class="n">ExecutionEnvironment</span> <span class="n">env</span> <span class="o">=</span> <span class="n">ExecutionEnvironment</span><span class="o">.</span><span class="na">getExecutionEnvironment</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">// Setup Hadoop’s TextInputFormat
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">HadoopInputFormat</span><span class="o"><</span><span class="n">LongWritable</span><span class="o">,</span> <span class="n">Text</span><span class="o">></span> <span class="n">hadoopInputFormat</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">HadoopInputFormat</span><span class="o"><</span><span class="n">LongWritable</span><span class="o">,</span> <span class="n">Text</span><span class="o">>(</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">TextInputFormat</span><span class="o">(),</span> <span class="n">LongWritable</span><span class="o">.</span><span class="na">class</span><span class="o">,</span> <span class="n">Text</span><span class="o">.</span><span class="na">class</span><span class="o">,</span> <span class="k">new</span> <span class="n">JobConf</span><span class="o">());</span>
-</span></span><span class="line"><span class="cl"> <span class="n">TextInputFormat</span><span class="o">.</span><span class="na">addInputPath</span><span class="o">(</span><span class="n">hadoopInputFormat</span><span class="o">.</span><span class="na">getJobConf</span><span class="o">(),</span> <span class="k">new</span> <span class="n">Path</span><span class="o">(</span><span class="n">inputPath</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">// Read a DataSet with the Hadoop InputFormat
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">DataSet</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">LongWritable</span><span class="o">,</span> <span class="n">Text</span><span class="o">>></span> <span class="n">text</span> <span class="o">=</span> <span class="n">env</span><span class="o">.</span><span class="na">createInput</span><span class="o">(</span><span class="n" [...]
-</span></span><span class="line"><span class="cl"> <span class="n">DataSet</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">>></span> <span class="n">words</span> <span class="o">=</span> <span class="n">text</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// Wrap Tokenizer Mapper function
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="o">.</span><span class="na">flatMap</span><span class="o">(</span><span class="k">new</span> <span class="n">HadoopMapFunction</span><span class="o"><</span><span class="n">LongWritable</span><span class="o">,</span> <span class="n">Text</span><span class="o">,</span> <span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">>(</span><span c [...]
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">groupBy</span><span class="o">(</span><span class="n">0</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// Wrap Counter Reducer function (used as Reducer and Combiner)
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="o">.</span><span class="na">reduceGroup</span><span class="o">(</span><span class="k">new</span> <span class="n">HadoopReduceCombineFunction</span><span class="o"><</span><span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">,</span> <span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">>(</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">Counter</span><span class="o">(),</span> <span class="k">new</span> <span class="n">Counter</span><span class="o">()));</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">// Setup Hadoop’s TextOutputFormat
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">HadoopOutputFormat</span><span class="o"><</span><span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">></span> <span class="n">hadoopOutputFormat</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">HadoopOutputFormat</span><span class="o"><</span><span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">>(</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">TextOutputFormat</span><span class="o"><</span><span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">>(),</span> <span class="k">new</span> <span class="n">JobConf</span><span class="o">());</span>
-</span></span><span class="line"><span class="cl"> <span class="n">hadoopOutputFormat</span><span class="o">.</span><span class="na">getJobConf</span><span class="o">().</span><span class="na">set</span><span class="o">(</span><span class="s">"mapred.textoutputformat.separator"</span><span class="o">,</span> <span class="s">" "</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="n">TextOutputFormat</span><span class="o">.</span><span class="na">setOutputPath</span><span class="o">(</span><span class="n">hadoopOutputFormat</span><span class="o">.</span><span class="na">getJobConf</span><span class="o">(),</span> <span class="k">new</span> <span class="n">Path</span><span class="o">(</span><span class="n">outputPath</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">// Output & Execute
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">words</span><span class="o">.</span><span class="na">output</span><span class="o">(</span><span class="n">hadoopOutputFormat</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="n">env</span><span class="o">.</span><span class="na">execute</span><span class="o">(</span><span class="s">"Hadoop Compat WordCount"</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>As you can see, Flink represents Hadoop key-value pairs as <code>Tuple2<key, value></code> tuples. Note, that the program uses Flink’s <code>groupBy()</code> transformation to group data on the key field (field 0 of the <code>Tuple2<key, value></code>) before it is given to the Reducer function. At the moment, the compatibility package does not evaluate custom Hadoop partitioners, sorting comparators, or grouping comparators.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// Definition of Hadoop Mapper function</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">Tokenizer</span><span class="w"> </span><span class="kd">implements</span><span class="w"> </span><span class="n">Mapper</span><span class="o"><</span><span class="n">LongWritable</span><span class="p">,</span><span class="w"> </span><span class="n">Text</span><span class="p">,</span><span class="w"> </span [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// Definition of Hadoop Reducer function</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">Counter</span><span class="w"> </span><span class="kd">implements</span><span class="w"> </span><span class="n">Reducer</span><span class="o"><</span><span class="n">Text</span><span class="p">,</span><span class="w"> </span><span class="n">LongWritable</span><span class="p">,</span><span class="w"> </span> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kd">static</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">main</span><span class="p">(</span><span class="n">String</span><span class="o">[]</span><span class="w"> </span><span class="n">args</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">inputPath</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">args</span><span class="o">[</span><span class="n">0</span><span class="o">]</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">outputPath</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">args</span><span class="o">[</span><span class="n">1</span><span class="o">]</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="n">ExecutionEnvironment</span><span class="w"> </span><span class="n">env</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">ExecutionEnvironment</span><span class="p">.</span><span class="na">getExecutionEnvironment</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// Setup Hadoop’s TextInputFormat</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">HadoopInputFormat</span><span class="o"><</span><span class="n">LongWritable</span><span class="p">,</span><span class="w"> </span><span class="n">Text</span><span class="o">></span><span class="w"> </span><span class="n">hadoopInputFormat</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">HadoopInputFormat</span><span class="o"><</span><span class="n">LongWritable</span><span class="p">,</span><span class="w"> </span><span class="n">Text</span><span class="o">></span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">TextInputFormat</span><span class="p">(),</span><span class="w"> </span><span class="n">LongWritable</span><span class="p">.</span><span class="na">class</span><span class="p">,</span><span class="w"> </span><span class="n">Text</span><span class="p">.</span><span class="na">class</span><span class="p">,</span><span class="w"> </span><s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">TextInputFormat</span><span class="p">.</span><span class="na">addInputPath</span><span class="p">(</span><span class="n">hadoopInputFormat</span><span class="p">.</span><span class="na">getJobConf</span><span class="p">(),</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Path</span><span class="p">(</span><span class="n">inputPath</span><span c [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// Read a DataSet with the Hadoop InputFormat</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">DataSet</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">LongWritable</span><span class="p">,</span><span class="w"> </span><span class="n">Text</span><span class="o">>></span><span class="w"> </span><span class="n">text</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">env</span><span c [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">DataSet</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Text</span><span class="p">,</span><span class="w"> </span><span class="n">LongWritable</span><span class="o">>></span><span class="w"> </span><span class="n">words</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">text</span><span [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// Wrap Tokenizer Mapper function</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">flatMap</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">HadoopMapFunction</span><span class="o"><</span><span class="n">LongWritable</span><span class="p">,</span><span class="w"> </span><span class="n">Text</span><span class="p">,</span><span class="w"> </span><span class="n">Text</span><span class="p">,</span><sp [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">groupBy</span><span class="p">(</span><span class="n">0</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// Wrap Counter Reducer function (used as Reducer and Combiner)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">reduceGroup</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">HadoopReduceCombineFunction</span><span class="o"><</span><span class="n">Text</span><span class="p">,</span><span class="w"> </span><span class="n">LongWritable</span><span class="p">,</span><span class="w"> </span><span class="n">Text</span><span class=" [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Counter</span><span class="p">(),</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Counter</span><span class="p">()));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// Setup Hadoop’s TextOutputFormat</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">HadoopOutputFormat</span><span class="o"><</span><span class="n">Text</span><span class="p">,</span><span class="w"> </span><span class="n">LongWritable</span><span class="o">></span><span class="w"> </span><span class="n">hadoopOutputFormat</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">HadoopOutputFormat</span><span class="o"><</span><span class="n">Text</span><span class="p">,</span><span class="w"> </span><span class="n">LongWritable</span><span class="o">></span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">TextOutputFormat</span><span class="o"><</span><span class="n">Text</span><span class="p">,</span><span class="w"> </span><span class="n">LongWritable</span><span class="o">></span><span class="p">(),</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">JobConf</span><span class="p">());</s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">hadoopOutputFormat</span><span class="p">.</span><span class="na">getJobConf</span><span class="p">().</span><span class="na">set</span><span class="p">(</span><span class="s">"mapred.textoutputformat.separator"</span><span class="p">,</span><span class="w"> </span><span class="s">" "</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">TextOutputFormat</span><span class="p">.</span><span class="na">setOutputPath</span><span class="p">(</span><span class="n">hadoopOutputFormat</span><span class="p">.</span><span class="na">getJobConf</span><span class="p">(),</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Path</span><span class="p">(</span><span class="n">outputPath</span><sp [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// Output & Execute</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">words</span><span class="p">.</span><span class="na">output</span><span class="p">(</span><span class="n">hadoopOutputFormat</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">env</span><span class="p">.</span><span class="na">execute</span><span class="p">(</span><span class="s">"Hadoop Compat WordCount"</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>As you can see, Flink represents Hadoop key-value pairs as <code>Tuple2<key, value></code> tuples. Note, that the program uses Flink’s <code>groupBy()</code> transformation to group data on the key field (field 0 of the <code>Tuple2<key, value></code>) before it is given to the Reducer function. At the moment, the compatibility package does not evaluate custom Hadoop partitioners, sorting comparators, or grouping comparators.</p>
<p>Hadoop functions can be used at any position within a Flink program and of course also be mixed with native Flink functions. This means that instead of assembling a workflow of Hadoop jobs in an external driver method or using a workflow scheduler such as <a href="http://oozie.apache.org">Apache Oozie</a>, you can implement an arbitrary complex Flink program consisting of multiple Hadoop Input- and OutputFormats, Mapper and Reducer functions. When executing such a Flink program, data [...]
<center>
<img src="/img/blog/hcompat-flow.png" style="width:100%;margin:15px">
diff --git a/content/2015/01/06/december-2014-in-the-flink-community/index.html b/content/2015/01/06/december-2014-in-the-flink-community/index.html
index e15135bc0..80951c718 100644
--- a/content/2015/01/06/december-2014-in-the-flink-community/index.html
+++ b/content/2015/01/06/december-2014-in-the-flink-community/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -16,26 +16,25 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="This is the first blog post of a “newsletter” like series where we give a summary of the monthly activity in the Flink community. As the Flink project grows, this can serve as a “tl;dr” for people that are not following the Flink dev and user mailing lists, or those that are simply overwhelmed by the traffic.
-Flink graduation # The biggest news is that the Apache board approved Flink as a top-level Apache project!">
+Flink graduation # The biggest news is that the Apache board approved Flink as a top-level Apache project!">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="December 2014 in the Flink community" />
<meta property="og:description" content="This is the first blog post of a “newsletter” like series where we give a summary of the monthly activity in the Flink community. As the Flink project grows, this can serve as a “tl;dr” for people that are not following the Flink dev and user mailing lists, or those that are simply overwhelmed by the traffic.
-Flink graduation # The biggest news is that the Apache board approved Flink as a top-level Apache project!" />
+Flink graduation # The biggest news is that the Apache board approved Flink as a top-level Apache project!" />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2015/01/06/december-2014-in-the-flink-community/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-01-06T10:00:00+00:00" />
<meta property="article:modified_time" content="2015-01-06T10:00:00+00:00" />
-
<title>December 2014 in the Flink community | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2015/01/21/apache-flink-0.8.0-available/index.html b/content/2015/01/21/apache-flink-0.8.0-available/index.html
index 90e24063a..ad2053759 100644
--- a/content/2015/01/21/apache-flink-0.8.0-available/index.html
+++ b/content/2015/01/21/apache-flink-0.8.0-available/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -18,28 +18,27 @@
<meta name="description" content="We are pleased to announce the availability of Flink 0.8.0. This release includes new user-facing features as well as performance and bug fixes, extends the support for filesystems and introduces the Scala API and flexible windowing semantics for Flink Streaming. A total of 33 people have contributed to this release, a big thanks to all of them!
Download Flink 0.8.0
See the release changelog
-Overview of major new features # Extended filesystem support: The former DistributedFileSystem interface has been generalized to HadoopFileSystem now supporting all sub classes of org.">
+Overview of major new features # Extended filesystem support: The former DistributedFileSystem interface has been generalized to HadoopFileSystem now supporting all sub classes of org.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Apache Flink 0.8.0 available" />
<meta property="og:description" content="We are pleased to announce the availability of Flink 0.8.0. This release includes new user-facing features as well as performance and bug fixes, extends the support for filesystems and introduces the Scala API and flexible windowing semantics for Flink Streaming. A total of 33 people have contributed to this release, a big thanks to all of them!
Download Flink 0.8.0
See the release changelog
-Overview of major new features # Extended filesystem support: The former DistributedFileSystem interface has been generalized to HadoopFileSystem now supporting all sub classes of org." />
+Overview of major new features # Extended filesystem support: The former DistributedFileSystem interface has been generalized to HadoopFileSystem now supporting all sub classes of org." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2015/01/21/apache-flink-0.8.0-available/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-01-21T10:00:00+00:00" />
<meta property="article:modified_time" content="2015-01-21T10:00:00+00:00" />
-
<title>Apache Flink 0.8.0 available | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -62,7 +61,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -269,7 +268,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2015/02/04/january-2015-in-the-flink-community/index.html b/content/2015/02/04/january-2015-in-the-flink-community/index.html
index 210351c86..10b6daf1e 100644
--- a/content/2015/02/04/january-2015-in-the-flink-community/index.html
+++ b/content/2015/02/04/january-2015-in-the-flink-community/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -16,30 +16,29 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="Happy 2015! Here is a (hopefully digestible) summary of what happened last month in the Flink community.
-0.8.0 release # Flink 0.8.0 was released. See here for the release notes.
-Flink roadmap # The community has published a roadmap for 2015 on the Flink wiki. Check it out to see what is coming up in Flink, and pick up an issue to contribute!
-Articles in the press # The Apache Software Foundation announced Flink as a Top-Level Project.">
+0.8.0 release # Flink 0.8.0 was released. See here for the release notes.
+Flink roadmap # The community has published a roadmap for 2015 on the Flink wiki. Check it out to see what is coming up in Flink, and pick up an issue to contribute!
+Articles in the press # The Apache Software Foundation announced Flink as a Top-Level Project.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="January 2015 in the Flink community" />
<meta property="og:description" content="Happy 2015! Here is a (hopefully digestible) summary of what happened last month in the Flink community.
-0.8.0 release # Flink 0.8.0 was released. See here for the release notes.
-Flink roadmap # The community has published a roadmap for 2015 on the Flink wiki. Check it out to see what is coming up in Flink, and pick up an issue to contribute!
-Articles in the press # The Apache Software Foundation announced Flink as a Top-Level Project." />
+0.8.0 release # Flink 0.8.0 was released. See here for the release notes.
+Flink roadmap # The community has published a roadmap for 2015 on the Flink wiki. Check it out to see what is coming up in Flink, and pick up an issue to contribute!
+Articles in the press # The Apache Software Foundation announced Flink as a Top-Level Project." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2015/02/04/january-2015-in-the-flink-community/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-02-04T10:00:00+00:00" />
<meta property="article:modified_time" content="2015-02-04T10:00:00+00:00" />
-
<title>January 2015 in the Flink community | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -62,7 +61,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -269,7 +268,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2015/02/09/introducing-flink-streaming/index.html b/content/2015/02/09/introducing-flink-streaming/index.html
index f23802a3d..3c80747a0 100644
--- a/content/2015/02/09/introducing-flink-streaming/index.html
+++ b/content/2015/02/09/introducing-flink-streaming/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ In this post, we go through an example that uses the Flink Streaming API to comp
<meta property="og:url" content="https://flink.apache.org/2015/02/09/introducing-flink-streaming/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-02-09T12:00:00+00:00" />
<meta property="article:modified_time" content="2015-02-09T12:00:00+00:00" />
-
<title>Introducing Flink Streaming | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -551,39 +550,39 @@ found <a href="https://github.com/mbalassi/flink/blob/stockprices/flink-staging/
</span></span><span class="line"><span class="cl"><span class="o">}</span></span></span></code></pre></div>
</div>
<div data-lang="java7" markdown="1">
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">static</span> <span class="kt">void</span> <span class="nf">main</span><span class="o">(</span><span class="n">String</span><span class="o">[]</span> <span class="n">args</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="kd">final</span> <span class="n">StreamExecutionEnvironment</span> <span class="n">env</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="n">StreamExecutionEnvironment</span><span class="o">.</span><span class="na">getExecutionEnvironment</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">//Read from a socket stream at map it to StockPrice objects
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span> <span class="n">socketStockStream</span> <span class="o">=</span> <span class="n">env</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">socketTextStream</span><span class="o">(</span><span class="s">"localhost"</span><span class="o">,</span> <span class="n">9999</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">map</span><span class="o">(</span><span class="k">new</span> <span class="n">MapFunction</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">StockPrice</span><span class="o">>()</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">String</span><span class="o">[]</span> <span class="n">tokens</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">StockPrice</span> <span class="nf">map</span><span class="o">(</span><span class="n">String</span> <span class="n">value</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">tokens</span> <span class="o">=</span> <span class="n">value</span><span class="o">.</span><span class="na">split</span><span class="o">(</span><span class="s">","</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="k">new</span> <span class="n">StockPrice</span><span class="o">(</span><span class="n">tokens</span><span class="o">[</span><span class="n">0</span><span class="o">],</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Double</span><span class="o">.</span><span class="na">parseDouble</span><span class="o">(</span><span class="n">tokens</span><span class="o">[</span><span class="n">1</span><span class="o">]));</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">});</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">//Generate other stock streams
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span> <span class="n">SPX_stream</span> <span class="o">=</span> <span class="n">env</span><span class="o">.</span><span class="na">addSource</span><span class="o">(</span><span class="k">new</span> <span class="n">StockSource</span><span class="o">(</span><span class="s">"SPX"</span [...]
-</span></span><span class="line"><span class="cl"> <span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span> <span class="n">FTSE_stream</span> <span class="o">=</span> <span class="n">env</span><span class="o">.</span><span class="na">addSource</span><span class="o">(</span><span class="k">new</span> <span class="n">StockSource</span><span class="o">(</span><span class="s">"FTSE"</span><span class="o">,</span> <sp [...]
-</span></span><span class="line"><span class="cl"> <span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span> <span class="n">DJI_stream</span> <span class="o">=</span> <span class="n">env</span><span class="o">.</span><span class="na">addSource</span><span class="o">(</span><span class="k">new</span> <span class="n">StockSource</span><span class="o">(</span><span class="s">"DJI"</span><span class="o">,</span> <span [...]
-</span></span><span class="line"><span class="cl"> <span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span> <span class="n">BUX_stream</span> <span class="o">=</span> <span class="n">env</span><span class="o">.</span><span class="na">addSource</span><span class="o">(</span><span class="k">new</span> <span class="n">StockSource</span><span class="o">(</span><span class="s">"BUX"</span><span class="o">,</span> <span [...]
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">//Merge all stock streams together
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span> <span class="n">stockStream</span> <span class="o">=</span> <span class="n">socketStockStream</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">merge</span><span class="o">(</span><span class="n">SPX_stream</span><span class="o">,</span> <span class="n">FTSE_stream</span><span class="o">,</span> <span class="n">DJI_stream</span><span class="o">,</span> <span class="n">BUX_stream</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="n">stockStream</span><span class="o">.</span><span class="na">print</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="n">env</span><span class="o">.</span><span class="na">execute</span><span class="o">(</span><span class="s">"Stock stream"</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> </span></span></code></pre></div>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">static</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">main</span><span class="p">(</span><span class="n">String</span><span class="o">[]</span><span class="w"> </span><span class="n">args</span><span class="p">)</span><span class="w [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="n">StreamExecutionEnvironment</span><span class="w"> </span><span class="n">env</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">StreamExecutionEnvironment</span><span class="p">.</span><span class="na">getExecutionEnvironment</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">//Read from a socket stream at map it to StockPrice objects</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span><span class="w"> </span><span class="n">socketStockStream</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">env</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">socketTextStream</span><span class="p">(</span><span class="s">"localhost"</span><span class="p">,</span><span class="w"> </span><span class="n">9999</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">map</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">MapFunction</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">StockPrice</span><span class="o">></span><span class="p">()</span><span class="w"> </span><span class="p">{</span><spa [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">String</span><span class="o">[]</span><span class="w"> </span><span class="n">tokens</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">StockPrice</span><span class="w"> </span><span class="nf">map</span><span class="p">(</span><span class="n">String</span><span class="w"> </span><span class="n">value</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span><span class="w"> </ [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">tokens</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">value</span><span class="p">.</span><span class="na">split</span><span class="p">(</span><span class="s">","</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">StockPrice</span><span class="p">(</span><span class="n">tokens</span><span class="o">[</span><span class="n">0</span><span class="o">]</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Double</span><span class="p">.</span><span class="na">parseDouble</span><span class="p">(</span><span class="n">tokens</span><span class="o">[</span><span class="n">1</span><span class="o">]</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">});</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">//Generate other stock streams</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span><span class="w"> </span><span class="n">SPX_stream</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">env</span><span class="p">.</span><span class="na">addSource</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span><span class="w"> </span><span class="n">FTSE_stream</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">env</span><span class="p">.</span><span class="na">addSource</span><span class="p">(</span><span class="k">new</span><span class="w"> </span>< [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span><span class="w"> </span><span class="n">DJI_stream</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">env</span><span class="p">.</span><span class="na">addSource</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span><span class="w"> </span><span class="n">BUX_stream</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">env</span><span class="p">.</span><span class="na">addSource</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">//Merge all stock streams together</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span><span class="w"> </span><span class="n">stockStream</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">socketStockStream</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">merge</span><span class="p">(</span><span class="n">SPX_stream</span><span class="p">,</span><span class="w"> </span><span class="n">FTSE_stream</span><span class="p">,</span><span class="w"> </span><span class="n">DJI_stream</span><span class="p">,</span><span class="w"> </span><span class="n">BUX_stream</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">stockStream</span><span class="p">.</span><span class="na">print</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">env</span><span class="p">.</span><span class="na">execute</span><span class="p">(</span><span class="s">"Stock stream"</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span></span></span></code></pre></div>
</div>
</div>
<p>See
@@ -611,54 +610,54 @@ of this example, the data streams are simply generated using the
</span></span><span class="line"><span class="cl"><span class="o">}</span></span></span></code></pre></div>
</div>
<div data-lang="java7" markdown="1">
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">SYMBOLS</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ArrayList</span><span class="o"><</span><spa [...]
-</span></span><span class="line"><span class="cl"> <span class="n">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span><span class="s">"SPX"</span><span class="o">,</span> <span class="s">"FTSE"</span><span class="o">,</span> <span class="s">"DJI"</span><span class="o">,</span> <span class="s">"DJT"</span><span class="o">,</span> <span class="s">"BUX"</span><span class="o">,</span> <span class="s">"DAX& [...]
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">StockPrice</span> <span class="kd">implements</span> <span class="n">Serializable</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">String</span> <span class="n">symbol</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">Double</span> <span class="n">price</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="nf">StockPrice</span><span class="o">()</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="nf">StockPrice</span><span class="o">(</span><span class="n">String</span> <span class="n">symbol</span><span class="o">,</span> <span class="n">Double</span> <span class="n">price</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">this</span><span class="o">.</span><span class="na">symbol</span> <span class="o">=</span> <span class="n">symbol</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="k">this</span><span class="o">.</span><span class="na">price</span> <span class="o">=</span> <span class="n">price</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">String</span> <span class="nf">toString</span><span class="o">()</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="s">"StockPrice{"</span> <span class="o">+</span>
-</span></span><span class="line"><span class="cl"> <span class="s">"symbol='"</span> <span class="o">+</span> <span class="n">symbol</span> <span class="o">+</span> <span class="sc">'\''</span> <span class="o">+</span>
-</span></span><span class="line"><span class="cl"> <span class="s">", count="</span> <span class="o">+</span> <span class="n">price</span> <span class="o">+</span>
-</span></span><span class="line"><span class="cl"> <span class="sc">'}'</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">final</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">StockSource</span> <span class="kd">implements</span> <span class="n">SourceFunction</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">Double</span> <span class="n">price</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">String</span> <span class="n">symbol</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">Integer</span> <span class="n">sigma</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="nf">StockSource</span><span class="o">(</span><span class="n">String</span> <span class="n">symbol</span><span class="o">,</span> <span class="n">Integer</span> <span class="n">sigma</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">this</span><span class="o">.</span><span class="na">symbol</span> <span class="o">=</span> <span class="n">symbol</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="k">this</span><span class="o">.</span><span class="na">sigma</span> <span class="o">=</span> <span class="n">sigma</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">invoke</span><span class="o">(</span><span class="n">Collector</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span> <span class="n">collector</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">price</span> <span class="o">=</span> <span class="n">DEFAULT_PRICE</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Random</span> <span class="n">random</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Random</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="k">while</span> <span class="o">(</span><span class="kc">true</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">price</span> <span class="o">=</span> <span class="n">price</span> <span class="o">+</span> <span class="n">random</span><span class="o">.</span><span class="na">nextGaussian</span><span class="o">()</span> <span class="o">*</span> <span class="n">sigma</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="n">collector</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="k">new</span> <span class="n">StockPrice</span><span class="o">(</span><span class="n">symbol</span><span class="o">,</span> <span class="n">price</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Thread</span><span class="o">.</span><span class="na">sleep</span><span class="o">(</span><span class="n">random</span><span class="o">.</span><span class="na">nextInt</span><span class="o">(</span><span class="n">200</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span></span></span></code></pre></div>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">private</span><span class="w"> </span><span class="kd">static</span><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">SYMBOLS</span><span class="w"> </span> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Arrays</span><span class="p">.</span><span class="na">asList</span><span class="p">(</span><span class="s">"SPX"</span><span class="p">,</span><span class="w"> </span><span class="s">"FTSE"</span><span class="p">,</span><span class="w"> </span><span class="s">"DJI"</span><span class="p">,</span><span class="w"> </span><span class="s">"DJT"</span><span class= [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kd">static</span><span class="w"> </span><span class="kd">class</span> <span class="nc">StockPrice</span><span class="w"> </span><span class="kd">implements</span><span class="w"> </span><span class="n">Serializable</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">symbol</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">Double</span><span class="w"> </span><span class="n">price</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="nf">StockPrice</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="nf">StockPrice</span><span class="p">(</span><span class="n">String</span><span class="w"> </span><span class="n">symbol</span><span class="p">,</span><span class="w"> </span><span class="n">Double</span><span class="w"> </span><span class="n">price</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">symbol</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">symbol</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">price</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">price</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="nf">toString</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="s">"StockPrice{"</span><span class="w"> </span><span class="o">+</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="s">"symbol='"</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">symbol</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="sc">'\''</span><span class="w"> </span><span class="o">+</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="s">", count="</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">price</span><span class="w"> </span><span class="o">+</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="sc">'}'</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="kd">static</span><span class="w"> </span><span class="kd">class</span> <span class="nc">StockSource</span><span class="w"> </span><span class="kd">implements</span><span class="w"> </span><span class="n">SourceFunction</span><span class="o"><</span><span class="n">StockPrice</span><span c [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">Double</span><span class="w"> </span><span class="n">price</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">symbol</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">Integer</span><span class="w"> </span><span class="n">sigma</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="nf">StockSource</span><span class="p">(</span><span class="n">String</span><span class="w"> </span><span class="n">symbol</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="w"> </span><span class="n">sigma</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">symbol</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">symbol</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">sigma</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">sigma</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">invoke</span><span class="p">(</span><span class="n">Collector</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span><span class="w"> </span><span class="n">collector</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</ [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">price</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">DEFAULT_PRICE</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Random</span><span class="w"> </span><span class="n">random</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Random</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">while</span><span class="w"> </span><span class="p">(</span><span class="kc">true</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">price</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">price</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">random</span><span class="p">.</span><span class="na">nextGaussian</span><span class="p">()</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="n [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">collector</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">StockPrice</span><span class="p">(</span><span class="n">symbol</span><span class="p">,</span><span class="w"> </span><span class="n">price</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Thread</span><span class="p">.</span><span class="na">sleep</span><span class="p">(</span><span class="n">random</span><span class="p">.</span><span class="na">nextInt</span><span class="p">(</span><span class="n">200</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span></span></span></code></pre></div>
</div>
</div>
<p>To read from the text socket stream please make sure that you have a
@@ -716,40 +715,40 @@ performed on named fields of POJOs, making the code more readable.</p>
</span></span><span class="line"><span class="cl"><span class="o">}</span></span></span></code></pre></div>
</div>
<div data-lang="java7" markdown="1">
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">//Define the desired time window
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">WindowedDataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span> <span class="n">windowedStream</span> <span class="o">=</span> <span class="n">stockStream</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">window</span><span class="o">(</span><span class="n">Time</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">10</span><span class="o">,</span> <span class="n">TimeUnit</span><span class="o">.</span><span class="na">SECONDS</span><span class="o">))</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">every</span><span class="o">(</span><span class="n">Time</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">5</span><span class="o">,</span> <span class="n">TimeUnit</span><span class="o">.</span><span class="na">SECONDS</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">//Compute some simple statistics on a rolling window
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span> <span class="n">lowest</span> <span class="o">=</span> <span class="n">windowedStream</span><span class="o">.</span><span class="na">minBy</span><span class="o">(</span><span class="s">"price"</span><span class="o">).</span><span class="na">flatten</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span> <span class="n">maxByStock</span> <span class="o">=</span> <span class="n">windowedStream</span><span class="o">.</span><span class="na">groupBy</span><span class="o">(</span><span class="s">"symbol"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">maxBy</span><span class="o">(</span><span class="s">"price"</span><span class="o">).</span><span class="na">flatten</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span> <span class="n">rollingMean</span> <span class="o">=</span> <span class="n">windowedStream</span><span class="o">.</span><span class="na">groupBy</span><span class="o">(</span><span class="s">"symbol"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">mapWindow</span><span class="o">(</span><span class="k">new</span> <span class="n">WindowMean</span><span class="o">()).</span><span class="na">flatten</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">//Compute the mean of a window
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">public</span> <span class="kd">final</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">WindowMean</span> <span class="kd">implements</span>
-</span></span><span class="line"><span class="cl"> <span class="n">WindowMapFunction</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">,</span> <span class="n">StockPrice</span><span class="o">></span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">Double</span> <span class="n">sum</span> <span class="o">=</span> <span class="n">0</span><span class="o">.</span><span class="na">0</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">Integer</span> <span class="n">count</span> <span class="o">=</span> <span class="n">0</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">String</span> <span class="n">symbol</span> <span class="o">=</span> <span class="s">""</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">mapWindow</span><span class="o">(</span><span class="n">Iterable</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span> <span class="n">values</span><span class="o">,</span> <span class="n">Collector</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span> <span class="n">out</span><s [...]
-</span></span><span class="line"><span class="cl"> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">(</span><span class="n">values</span><span class="o">.</span><span class="na">iterator</span><span class="o">().</span><span class="na">hasNext</span><span class="o">())</span> <span class="o">{</span><span class="n">s</span>
-</span></span><span class="line"><span class="cl"> <span class="nf">for</span> <span class="o">(</span><span class="n">StockPrice</span> <span class="n">sp</span> <span class="o">:</span> <span class="n">values</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">sum</span> <span class="o">+=</span> <span class="n">sp</span><span class="o">.</span><span class="na">price</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="n">symbol</span> <span class="o">=</span> <span class="n">sp</span><span class="o">.</span><span class="na">symbol</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="n">count</span><span class="o">++;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="n">out</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="k">new</span> <span class="n">StockPrice</span><span class="o">(</span><span class="n">symbol</span><span class="o">,</span> <span class="n">sum</span> <span class="o">/</span> <span class="n">count</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span></span></span></code></pre></div>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">//Define the desired time window</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">WindowedDataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span><span class="w"> </span><span class="n">windowedStream</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">stockStream</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">window</span><span class="p">(</span><span class="n">Time</span><span class="p">.</span><span class="na">of</span><span class="p">(</span><span class="n">10</span><span class="p">,</span><span class="w"> </span><span class="n">TimeUnit</span><span class="p">.</span><span class="na">SECONDS</span><span class="p">))</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">every</span><span class="p">(</span><span class="n">Time</span><span class="p">.</span><span class="na">of</span><span class="p">(</span><span class="n">5</span><span class="p">,</span><span class="w"> </span><span class="n">TimeUnit</span><span class="p">.</span><span class="na">SECONDS</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">//Compute some simple statistics on a rolling window</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span><span class="w"> </span><span class="n">lowest</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">windowedStream</span><span class="p">.</span><span class="na">minBy</span><span class="p">(</span><span class="s">"price"</span><span class="p" [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span><span class="w"> </span><span class="n">maxByStock</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">windowedStream</span><span class="p">.</span><span class="na">groupBy</span><span class="p">(</span><span class="s">"symbol"</span><span cl [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">maxBy</span><span class="p">(</span><span class="s">"price"</span><span class="p">).</span><span class="na">flatten</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span><span class="w"> </span><span class="n">rollingMean</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">windowedStream</span><span class="p">.</span><span class="na">groupBy</span><span class="p">(</span><span class="s">"symbol"</span><span c [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">mapWindow</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">WindowMean</span><span class="p">()).</span><span class="na">flatten</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">//Compute the mean of a window</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="kd">static</span><span class="w"> </span><span class="kd">class</span> <span class="nc">WindowMean</span><span class="w"> </span><span class="kd">implements</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">WindowMapFunction</span><span class="o"><</span><span class="n">StockPrice</span><span class="p">,</span><span class="w"> </span><span class="n">StockPrice</span><span class="o">></span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">Double</span><span class="w"> </span><span class="n">sum</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">0</span><span class="p">.</span><span class="na">0</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">Integer</span><span class="w"> </span><span class="n">count</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">0</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">symbol</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">""</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">mapWindow</span><span class="p">(</span><span class="n">Iterable</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span><span class="w"> </span><span class="n">values</span><span class="p">,</span><span class="w"> </span><span class="n">Collector< [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">values</span><span class="p">.</span><span class="na">iterator</span><span class="p">().</span><span class="na">hasNext</span><span class="p">())</span><span class="w"> </span><span class="p">{</span><span class="n">s</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nf">for</span><span class="w"> </span><span class="p">(</span><span class="n">StockPrice</span><span class="w"> </span><span class="n">sp</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="n">values</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">sum</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="n">sp</span><span class="p">.</span><span class="na">price</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">symbol</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">sp</span><span class="p">.</span><span class="na">symbol</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">count</span><span class="o">++</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">out</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">StockPrice</span><span class="p">(</span><span class="n">symbol</span><span class="p">,</span><span class="w"> </span><span class="n">sum</span><span class="w"> </span><span class="o">/</span><span class="w"> </span><span class=" [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span></span></span></code></pre></div>
</div>
</div>
<p>Let us note that to print a windowed stream one has to flatten it first,
@@ -795,58 +794,58 @@ every 30 seconds.</p>
</span></span><span class="line"><span class="cl"><span class="o">}</span></span></span></code></pre></div>
</div>
<div data-lang="java7" markdown="1">
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">Double</span> <span class="n">DEFAULT_PRICE</span> <span class="o">=</span> <span class="n">1000</span><span class="o">.;</span>
-</span></span><span class="line"><span class="cl"><span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">StockPrice</span> <span class="n">DEFAULT_STOCK_PRICE</span> <span class="o">=</span> <span class="k">new</span> <span class="n">StockPrice</span><span class="o">(</span><span class="s">""</span><span class="o">,</span> <span class="n">DEFAULT_PRICE</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">//Use delta policy to create price change warnings
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">priceWarnings</span> <span class="o">=</span> <span class="n">stockStream</span><span class="o">.</span><span class="na">groupBy</span><span class="o">(</span><span class="s">"symbol"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">window</span><span class="o">(</span><span class="n">Delta</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">0</span><span class="o">.</span><span class="na">05</span><span class="o">,</span> <span class="k">new</span> <span class="n">DeltaFunction</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">>()</span> <span [...]
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">double</span> <span class="nf">getDelta</span><span class="o">(</span><span class="n">StockPrice</span> <span class="n">oldDataPoint</span><span class="o">,</span> <span class="n">StockPrice</span> <span class="n">newDataPoint</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">Math</span><span class="o">.</span><span class="na">abs</span><span class="o">(</span><span class="n">oldDataPoint</span><span class="o">.</span><span class="na">price</span> <span class="o">-</span> <span class="n">newDataPoint</span><span class="o">.</span><span class="na">price</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">},</span> <span class="n">DEFAULT_STOCK_PRICE</span><span class="o">))</span>
-</span></span><span class="line"><span class="cl"><span class="o">.</span><span class="na">mapWindow</span><span class="o">(</span><span class="k">new</span> <span class="n">SendWarning</span><span class="o">()).</span><span class="na">flatten</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">//Count the number of warnings every half a minute
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Count</span><span class="o">></span> <span class="n">warningsPerStock</span> <span class="o">=</span> <span class="n">priceWarnings</span><span class="o">.</span><span class="na">map</span><span class="o">(</span><span class="k">new</span> <span class="n">MapFunction</span><span class="o"><</span><span class="n">String</span><s [...]
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">Count</span> <span class="nf">map</span><span class="o">(</span><span class="n">String</span> <span class="n">value</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="k">new</span> <span class="n">Count</span><span class="o">(</span><span class="n">value</span><span class="o">,</span> <span class="n">1</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}).</span><span class="na">groupBy</span><span class="o">(</span><span class="s">"symbol"</span><span class="o">).</span><span class="na">window</span><span class="o">(</span><span class="n">Time</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">30</span><span class="o">,</span> <span class="n">TimeUnit</span><span class="o">.</span><span class="na">SECONDS</span><span [...]
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">Count</span> <span class="kd">implements</span> <span class="n">Serializable</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">String</span> <span class="n">symbol</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">Integer</span> <span class="n">count</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="nf">Count</span><span class="o">()</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="nf">Count</span><span class="o">(</span><span class="n">String</span> <span class="n">symbol</span><span class="o">,</span> <span class="n">Integer</span> <span class="n">count</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">this</span><span class="o">.</span><span class="na">symbol</span> <span class="o">=</span> <span class="n">symbol</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="k">this</span><span class="o">.</span><span class="na">count</span> <span class="o">=</span> <span class="n">count</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">String</span> <span class="nf">toString</span><span class="o">()</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="s">"Count{"</span> <span class="o">+</span>
-</span></span><span class="line"><span class="cl"> <span class="s">"symbol='"</span> <span class="o">+</span> <span class="n">symbol</span> <span class="o">+</span> <span class="sc">'\''</span> <span class="o">+</span>
-</span></span><span class="line"><span class="cl"> <span class="s">", count="</span> <span class="o">+</span> <span class="n">count</span> <span class="o">+</span>
-</span></span><span class="line"><span class="cl"> <span class="sc">'}'</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">static</span> <span class="kd">final</span> <span class="kd">class</span> <span class="nc">SendWarning</span> <span class="kd">implements</span> <span class="n">MapWindowFunction</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">,</span> <span class="n">String</span><span class="o">></span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">mapWindow</span><span class="o">(</span><span class="n">Iterable</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span> <span class="n">values</span><span class="o">,</span> <span class="n">Collector</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">out</span><span [...]
-</span></span><span class="line"><span class="cl"> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">(</span><span class="n">values</span><span class="o">.</span><span class="na">iterator</span><span class="o">().</span><span class="na">hasNext</span><span class="o">())</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">out</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="n">values</span><span class="o">.</span><span class="na">iterator</span><span class="o">().</span><span class="na">next</span><span class="o">().</span><span class="na">symbol</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span></span></span></code></pre></div>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">private</span><span class="w"> </span><span class="kd">static</span><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="n">Double</span><span class="w"> </span><span class="n">DEFAULT_PRICE</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">1000</span><span [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">private</span><span class="w"> </span><span class="kd">static</span><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="n">StockPrice</span><span class="w"> </span><span class="n">DEFAULT_STOCK_PRICE</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">StockPrice</s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">//Use delta policy to create price change warnings</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">priceWarnings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">stockStream</span><span class="p">.</span><span class="na">groupBy</span><span class="p">(</span><span class="s">"symbol"</span><span class= [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">window</span><span class="p">(</span><span class="n">Delta</span><span class="p">.</span><span class="na">of</span><span class="p">(</span><span class="n">0</span><span class="p">.</span><span class="na">05</span><span class="p">,</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">DeltaFunction</span><span class="o">< [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">double</span><span class="w"> </span><span class="nf">getDelta</span><span class="p">(</span><span class="n">StockPrice</span><span class="w"> </span><span class="n">oldDataPoint</span><span class="p">,</span><span class="w"> </span><span class="n">StockPrice</span><span class="w"> </span><span class="n">newDataPoint</span><span cl [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Math</span><span class="p">.</span><span class="na">abs</span><span class="p">(</span><span class="n">oldDataPoint</span><span class="p">.</span><span class="na">price</span><span class="w"> </span><span class="o">-</span><span class="w"> </span><span class="n">newDataPoint</span><span class="p">.</span><span class="na">price</sp [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">},</span><span class="w"> </span><span class="n">DEFAULT_STOCK_PRICE</span><span class="p">))</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">.</span><span class="na">mapWindow</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">SendWarning</span><span class="p">()).</span><span class="na">flatten</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">//Count the number of warnings every half a minute</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Count</span><span class="o">></span><span class="w"> </span><span class="n">warningsPerStock</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">priceWarnings</span><span class="p">.</span><span class="na">map</span><span class="p">(</span><span class="k">new</span><span class="w"> </span>< [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">Count</span><span class="w"> </span><span class="nf">map</span><span class="p">(</span><span class="n">String</span><span class="w"> </span><span class="n">value</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span><span class="w"> </span><span class= [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Count</span><span class="p">(</span><span class="n">value</span><span class="p">,</span><span class="w"> </span><span class="n">1</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}).</span><span class="na">groupBy</span><span class="p">(</span><span class="s">"symbol"</span><span class="p">).</span><span class="na">window</span><span class="p">(</span><span class="n">Time</span><span class="p">.</span><span class="na">of</span><span class="p">(</span><span class="n">30</span><span class="p">,</span><span class="w"> </span><span class="n">TimeUnit</span><span cl [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kd">static</span><span class="w"> </span><span class="kd">class</span> <span class="nc">Count</span><span class="w"> </span><span class="kd">implements</span><span class="w"> </span><span class="n">Serializable</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">symbol</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">Integer</span><span class="w"> </span><span class="n">count</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="nf">Count</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="nf">Count</span><span class="p">(</span><span class="n">String</span><span class="w"> </span><span class="n">symbol</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="w"> </span><span class="n">count</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">symbol</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">symbol</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">count</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">count</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="nf">toString</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="s">"Count{"</span><span class="w"> </span><span class="o">+</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="s">"symbol='"</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">symbol</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="sc">'\''</span><span class="w"> </span><span class="o">+</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="s">", count="</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">count</span><span class="w"> </span><span class="o">+</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="sc">'}'</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kd">static</span><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="kd">class</span> <span class="nc">SendWarning</span><span class="w"> </span><span class="kd">implements</span><span class="w"> </span><span class="n">MapWindowFunction</span><span class="o"><</span><span class="n">StockPrice</span><spa [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">mapWindow</span><span class="p">(</span><span class="n">Iterable</span><span class="o"><</span><span class="n">StockPrice</span><span class="o">></span><span class="w"> </span><span class="n">values</span><span class="p">,</span><span class="w"> </span><span class="n">Collector< [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">values</span><span class="p">.</span><span class="na">iterator</span><span class="p">().</span><span class="na">hasNext</span><span class="p">())</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">out</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="n">values</span><span class="p">.</span><span class="na">iterator</span><span class="p">().</span><span class="na">next</span><span class="p">().</span><span class="na">symbol</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span></span></span></code></pre></div>
</div>
</div>
<p><a href="#top">Back to top</a></p>
@@ -884,55 +883,55 @@ but for the sake of this example we generate dummy tweet data.</p>
</span></span><span class="line"><span class="cl"><span class="o">}</span></span></span></code></pre></div>
</div>
<div data-lang="java7" markdown="1">
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">//Read a stream of tweets
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">tweetStream</span> <span class="o">=</span> <span class="n">env</span><span class="o">.</span><span class="na">addSource</span><span class="o">(</span><span class="k">new</span> <span class="n">TweetSource</span><span class="o">());</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">//Extract the stock symbols
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">mentionedSymbols</span> <span class="o">=</span> <span class="n">tweetStream</span><span class="o">.</span><span class="na">flatMap</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">FlatMapFunction</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">String</span><span class="o">>()</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">flatMap</span><span class="o">(</span><span class="n">String</span> <span class="n">value</span><span class="o">,</span> <span class="n">Collector</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">out</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span cla [...]
-</span></span><span class="line"><span class="cl"> <span class="n">String</span><span class="o">[]</span> <span class="n">words</span> <span class="o">=</span> <span class="n">value</span><span class="o">.</span><span class="na">split</span><span class="o">(</span><span class="s">" "</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="o">(</span><span class="n">String</span> <span class="n">word</span> <span class="o">:</span> <span class="n">words</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">out</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="n">word</span><span class="o">.</span><span class="na">toUpperCase</span><span class="o">());</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}).</span><span class="na">filter</span><span class="o">(</span><span class="k">new</span> <span class="n">FilterFunction</span><span class="o"><</span><span class="n">String</span><span class="o">>()</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">boolean</span> <span class="nf">filter</span><span class="o">(</span><span class="n">String</span> <span class="n">value</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">SYMBOLS</span><span class="o">.</span><span class="na">contains</span><span class="o">(</span><span class="n">value</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">});</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">//Count the extracted symbols
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Count</span><span class="o">></span> <span class="n">tweetsPerStock</span> <span class="o">=</span> <span class="n">mentionedSymbols</span><span class="o">.</span><span class="na">map</span><span class="o">(</span><span class="k">new</span> <span class="n">MapFunction</span><span class="o"><</span><span class="n">String</span>< [...]
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">Count</span> <span class="nf">map</span><span class="o">(</span><span class="n">String</span> <span class="n">value</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="k">new</span> <span class="n">Count</span><span class="o">(</span><span class="n">value</span><span class="o">,</span> <span class="n">1</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}).</span><span class="na">groupBy</span><span class="o">(</span><span class="s">"symbol"</span><span class="o">).</span><span class="na">window</span><span class="o">(</span><span class="n">Time</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">30</span><span class="o">,</span> <span class="n">TimeUnit</span><span class="o">.</span><span class="na">SECONDS</span><span [...]
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">static</span> <span class="kd">final</span> <span class="kd">class</span> <span class="nc">TweetSource</span> <span class="kd">implements</span> <span class="n">SourceFunction</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Random</span> <span class="n">random</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="n">StringBuilder</span> <span class="n">stringBuilder</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">invoke</span><span class="o">(</span><span class="n">Collector</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">collector</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">random</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Random</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="n">stringBuilder</span> <span class="o">=</span> <span class="k">new</span> <span class="n">StringBuilder</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="k">while</span> <span class="o">(</span><span class="kc">true</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">stringBuilder</span><span class="o">.</span><span class="na">setLength</span><span class="o">(</span><span class="n">0</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="n">0</span><span class="o">;</span> <span class="n">i</span> <span class="o"><</span> <span class="n">3</span><span class="o">;</span> <span class="n">i</span><span class="o">++)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">stringBuilder</span><span class="o">.</span><span class="na">append</span><span class="o">(</span><span class="s">" "</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="n">stringBuilder</span><span class="o">.</span><span class="na">append</span><span class="o">(</span><span class="n">SYMBOLS</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">random</span><span class="o">.</span><span class="na">nextInt</span><span class="o">(</span><span class="n">SYMBOLS</span><span class="o">.</span><span class="na">size</span><span class="o">( [...]
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="n">collector</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="n">stringBuilder</span><span class="o">.</span><span class="na">toString</span><span class="o">());</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Thread</span><span class="o">.</span><span class="na">sleep</span><span class="o">(</span><span class="n">500</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span></span></span></code></pre></div>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">//Read a stream of tweets</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">tweetStream</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">env</span><span class="p">.</span><span class="na">addSource</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span cla [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">//Extract the stock symbols</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">mentionedSymbols</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tweetStream</span><span class="p">.</span><span class="na">flatMap</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">FlatMapFunction</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">String</span><span class="o">></span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">flatMap</span><span class="p">(</span><span class="n">String</span><span class="w"> </span><span class="n">value</span><span class="p">,</span><span class="w"> </span><span class="n">Collector</span><span class="o"><</span><span class="n">String</span><span class="o">></span><sp [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">String</span><span class="o">[]</span><span class="w"> </span><span class="n">words</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">value</span><span class="p">.</span><span class="na">split</span><span class="p">(</span><span class="s">" "</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="n">String</span><span class="w"> </span><span class="n">word</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="n">words</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">out</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="n">word</span><span class="p">.</span><span class="na">toUpperCase</span><span class="p">());</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}).</span><span class="na">filter</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">FilterFunction</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">boolean</span><span class="w"> </span><span class="nf">filter</span><span class="p">(</span><span class="n">String</span><span class="w"> </span><span class="n">value</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span><span class="w"> </span><span [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">SYMBOLS</span><span class="p">.</span><span class="na">contains</span><span class="p">(</span><span class="n">value</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">});</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">//Count the extracted symbols</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Count</span><span class="o">></span><span class="w"> </span><span class="n">tweetsPerStock</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">mentionedSymbols</span><span class="p">.</span><span class="na">map</span><span class="p">(</span><span class="k">new</span><span class="w"> </span> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">Count</span><span class="w"> </span><span class="nf">map</span><span class="p">(</span><span class="n">String</span><span class="w"> </span><span class="n">value</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span><span class="w"> </span><span class= [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Count</span><span class="p">(</span><span class="n">value</span><span class="p">,</span><span class="w"> </span><span class="n">1</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}).</span><span class="na">groupBy</span><span class="p">(</span><span class="s">"symbol"</span><span class="p">).</span><span class="na">window</span><span class="p">(</span><span class="n">Time</span><span class="p">.</span><span class="na">of</span><span class="p">(</span><span class="n">30</span><span class="p">,</span><span class="w"> </span><span class="n">TimeUnit</span><span cl [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kd">static</span><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="kd">class</span> <span class="nc">TweetSource</span><span class="w"> </span><span class="kd">implements</span><span class="w"> </span><span class="n">SourceFunction</span><span class="o"><</span><span class="n">String</span><span class [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Random</span><span class="w"> </span><span class="n">random</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">StringBuilder</span><span class="w"> </span><span class="n">stringBuilder</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">invoke</span><span class="p">(</span><span class="n">Collector</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">collector</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</span [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">random</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Random</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">stringBuilder</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">StringBuilder</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">while</span><span class="w"> </span><span class="p">(</span><span class="kc">true</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">stringBuilder</span><span class="p">.</span><span class="na">setLength</span><span class="p">(</span><span class="n">0</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="kt">int</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">0</span><span class="p">;</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o"><</span><span class="w"> </span><span cl [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">stringBuilder</span><span class="p">.</span><span class="na">append</span><span class="p">(</span><span class="s">" "</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">stringBuilder</span><span class="p">.</span><span class="na">append</span><span class="p">(</span><span class="n">SYMBOLS</span><span class="p">.</span><span class="na">get</span><span class="p">(</span><span class="n">random</span><span class="p">.</span><span class="na">nextInt</span><span class="p">(</span><span class="n">SYMBOLS</span><span class="p">.</span><span class="na [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">collector</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="n">stringBuilder</span><span class="p">.</span><span class="na">toString</span><span class="p">());</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Thread</span><span class="p">.</span><span class="na">sleep</span><span class="p">(</span><span class="n">500</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span></span></span></code></pre></div>
</div>
</div>
<p><a href="#top">Back to top</a></p>
@@ -976,77 +975,77 @@ these data streams are potentially infinite, we apply the join on a
</span></span><span class="line"><span class="cl"><span class="o">}</span></span></span></code></pre></div>
</div>
<div data-lang="java7" markdown="1">
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">//Join warnings and parsed tweets
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span> <span class="n">tweetsAndWarning</span> <span class="o">=</span> <span class="n">warningsPerStock</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">join</span><span class="o">(</span><span class="n">tweetsPerStock</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">onWindow</span><span class="o">(</span><span class="n">30</span><span class="o">,</span> <span class="n">TimeUnit</span><span class="o">.</span><span class="na">SECONDS</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">where</span><span class="o">(</span><span class="s">"symbol"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">equalTo</span><span class="o">(</span><span class="s">"symbol"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">with</span><span class="o">(</span><span class="k">new</span> <span class="n">JoinFunction</span><span class="o"><</span><span class="n">Count</span><span class="o">,</span> <span class="n">Count</span><span class="o">,</span> <span class="n">Tuple2</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>>() [...]
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">Tuple2</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Integer</span><span class="o">></span> <span class="nf">join</span><span class="o">(</span><span class="n">Count</span> <span class="n">first</span><span class="o">,</span> <span class="n">Count</span> <span class="n">second</span><span class="o">)</span> <span class="kd"> [...]
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="k">new</span> <span class="n">Tuple2</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>(</span><span class="n">first</span><span class="o">.</span><span class="na">count</span><span class="o">,</span> <span class="n">second</span><span class="o">.</span><span class="na">count</span><span class="o"> [...]
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">});</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">//Compute rolling correlation
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Double</span><span class="o">></span> <span class="n">rollingCorrelation</span> <span class="o">=</span> <span class="n">tweetsAndWarning</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">window</span><span class="o">(</span><span class="n">Time</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">30</span><span class="o">,</span> <span class="n">TimeUnit</span><span class="o">.</span><span class="na">SECONDS</span><span class="o">))</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">mapWindow</span><span class="o">(</span><span class="k">new</span> <span class="n">WindowCorrelation</span><span class="o">());</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">rollingCorrelation</span><span class="o">.</span><span class="na">print</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">static</span> <span class="kd">final</span> <span class="kd">class</span> <span class="nc">WindowCorrelation</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">implements</span> <span class="n">WindowMapFunction</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>,</span> <span class="n">Double</span><span class="o">></span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">Integer</span> <span class="n">leftSum</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">Integer</span> <span class="n">rightSum</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">Integer</span> <span class="n">count</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">Double</span> <span class="n">leftMean</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">Double</span> <span class="n">rightMean</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">Double</span> <span class="n">cov</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">Double</span> <span class="n">leftSd</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">Double</span> <span class="n">rightSd</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">mapWindow</span><span class="o">(</span><span class="n">Iterable</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span> <span class="n">values</span><span class="o">,</span> <span class="n">Collector</span><spa [...]
-</span></span><span class="line"><span class="cl"> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="n">leftSum</span> <span class="o">=</span> <span class="n">0</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="n">rightSum</span> <span class="o">=</span> <span class="n">0</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="n">count</span> <span class="o">=</span> <span class="n">0</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="n">cov</span> <span class="o">=</span> <span class="n">0</span><span class="o">.;</span>
-</span></span><span class="line"><span class="cl"> <span class="n">leftSd</span> <span class="o">=</span> <span class="n">0</span><span class="o">.;</span>
-</span></span><span class="line"><span class="cl"> <span class="n">rightSd</span> <span class="o">=</span> <span class="n">0</span><span class="o">.;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">//compute mean for both sides, save count
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">for</span> <span class="o">(</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Integer</span><span class="o">></span> <span class="n">pair</span> <span class="o">:</span> <span class="n">values</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">leftSum</span> <span class="o">+=</span> <span class="n">pair</span><span class="o">.</span><span class="na">f0</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="n">rightSum</span> <span class="o">+=</span> <span class="n">pair</span><span class="o">.</span><span class="na">f1</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="n">count</span><span class="o">++;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="n">leftMean</span> <span class="o">=</span> <span class="n">leftSum</span><span class="o">.</span><span class="na">doubleValue</span><span class="o">()</span> <span class="o">/</span> <span class="n">count</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="n">rightMean</span> <span class="o">=</span> <span class="n">rightSum</span><span class="o">.</span><span class="na">doubleValue</span><span class="o">()</span> <span class="o">/</span> <span class="n">count</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">//compute covariance & std. deviations
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">for</span> <span class="o">(</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Integer</span><span class="o">></span> <span class="n">pair</span> <span class="o">:</span> <span class="n">values</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">cov</span> <span class="o">+=</span> <span class="o">(</span><span class="n">pair</span><span class="o">.</span><span class="na">f0</span> <span class="o">-</span> <span class="n">leftMean</span><span class="o">)</span> <span class="o">*</span> <span class="o">(</span><span class="n">pair</span><span class="o">.</span><span class="na">f1</span> <span class="o">-</span> <span class="n">rightMean</span><span clas [...]
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="o">(</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Integer</span><span class="o">></span> <span class="n">pair</span> <span class="o">:</span> <span class="n">values</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">leftSd</span> <span class="o">+=</span> <span class="n">Math</span><span class="o">.</span><span class="na">pow</span><span class="o">(</span><span class="n">pair</span><span class="o">.</span><span class="na">f0</span> <span class="o">-</span> <span class="n">leftMean</span><span class="o">,</span> <span class="n">2</span><span class="o">)</span> <span class="o">/</span> <span class="n">count</span><span class [...]
-</span></span><span class="line"><span class="cl"> <span class="n">rightSd</span> <span class="o">+=</span> <span class="n">Math</span><span class="o">.</span><span class="na">pow</span><span class="o">(</span><span class="n">pair</span><span class="o">.</span><span class="na">f1</span> <span class="o">-</span> <span class="n">rightMean</span><span class="o">,</span> <span class="n">2</span><span class="o">)</span> <span class="o">/</span> <span class="n">count</span><span cla [...]
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="n">leftSd</span> <span class="o">=</span> <span class="n">Math</span><span class="o">.</span><span class="na">sqrt</span><span class="o">(</span><span class="n">leftSd</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="n">rightSd</span> <span class="o">=</span> <span class="n">Math</span><span class="o">.</span><span class="na">sqrt</span><span class="o">(</span><span class="n">rightSd</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="n">out</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="n">cov</span> <span class="o">/</span> <span class="o">(</span><span class="n">leftSd</span> <span class="o">*</span> <span class="n">rightSd</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span></span></span></code></pre></div>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">//Join warnings and parsed tweets</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Integer</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="o">>></span><span class="w"> </span><span class="n">tweetsAndWarning</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">warnin [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">join</span><span class="p">(</span><span class="n">tweetsPerStock</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">onWindow</span><span class="p">(</span><span class="n">30</span><span class="p">,</span><span class="w"> </span><span class="n">TimeUnit</span><span class="p">.</span><span class="na">SECONDS</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">where</span><span class="p">(</span><span class="s">"symbol"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">equalTo</span><span class="p">(</span><span class="s">"symbol"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">with</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">JoinFunction</span><span class="o"><</span><span class="n">Count</span><span class="p">,</span><span class="w"> </span><span class="n">Count</span><span class="p">,</span><span class="w"> </span><span class="n">Tuple2</span><span class="o"><</span><span class= [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Integer</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="o">></span><span class="w"> </span><span class="nf">join</span><span class="p">(</span><span class="n">Count</span><span class="w"> </span><span class="n">first</span><sp [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Integer</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="o">></span><span class="p">(</span><span class="n">first</span><span class="p">.</span><span class="na">count</span>< [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">});</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">//Compute rolling correlation</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Double</span><span class="o">></span><span class="w"> </span><span class="n">rollingCorrelation</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tweetsAndWarning</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">window</span><span class="p">(</span><span class="n">Time</span><span class="p">.</span><span class="na">of</span><span class="p">(</span><span class="n">30</span><span class="p">,</span><span class="w"> </span><span class="n">TimeUnit</span><span class="p">.</span><span class="na">SECONDS</span><span class="p">))</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">mapWindow</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">WindowCorrelation</span><span class="p">());</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">rollingCorrelation</span><span class="p">.</span><span class="na">print</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kd">static</span><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="kd">class</span> <span class="nc">WindowCorrelation</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">implements</span><span class="w"> </span><span class="n">WindowMapFunction</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Integer</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="o">></span><span class="p">,</span><span class="w"> </span><span class="n">Double</span><span class="o" [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">Integer</span><span class="w"> </span><span class="n">leftSum</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">Integer</span><span class="w"> </span><span class="n">rightSum</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">Integer</span><span class="w"> </span><span class="n">count</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">Double</span><span class="w"> </span><span class="n">leftMean</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">Double</span><span class="w"> </span><span class="n">rightMean</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">Double</span><span class="w"> </span><span class="n">cov</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">Double</span><span class="w"> </span><span class="n">leftSd</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">Double</span><span class="w"> </span><span class="n">rightSd</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">mapWindow</span><span class="p">(</span><span class="n">Iterable</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Integer</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="o">>> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">leftSum</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">0</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">rightSum</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">0</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">count</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">0</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">cov</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">0</span><span class="p">.;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">leftSd</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">0</span><span class="p">.;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">rightSd</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">0</span><span class="p">.;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">//compute mean for both sides, save count</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Integer</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="o">></span><span class="w"> </span><span class="n">pair</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="n"> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">leftSum</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="n">pair</span><span class="p">.</span><span class="na">f0</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">rightSum</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="n">pair</span><span class="p">.</span><span class="na">f1</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">count</span><span class="o">++</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">leftMean</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">leftSum</span><span class="p">.</span><span class="na">doubleValue</span><span class="p">()</span><span class="w"> </span><span class="o">/</span><span class="w"> </span><span class="n">count</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">rightMean</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">rightSum</span><span class="p">.</span><span class="na">doubleValue</span><span class="p">()</span><span class="w"> </span><span class="o">/</span><span class="w"> </span><span class="n">count</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">//compute covariance & std. deviations</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Integer</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="o">></span><span class="w"> </span><span class="n">pair</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="n"> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">cov</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="p">(</span><span class="n">pair</span><span class="p">.</span><span class="na">f0</span><span class="w"> </span><span class="o">-</span><span class="w"> </span><span class="n">leftMean</span><span class="p">)</span><span class="w"> </span><span class="o">*</span><span class="w"> </span>< [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Integer</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="o">></span><span class="w"> </span><span class="n">pair</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="n"> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">leftSd</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="n">Math</span><span class="p">.</span><span class="na">pow</span><span class="p">(</span><span class="n">pair</span><span class="p">.</span><span class="na">f0</span><span class="w"> </span><span class="o">-</span><span class="w"> </span><span class="n">leftMean</span><span class="p"> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">rightSd</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="n">Math</span><span class="p">.</span><span class="na">pow</span><span class="p">(</span><span class="n">pair</span><span class="p">.</span><span class="na">f1</span><span class="w"> </span><span class="o">-</span><span class="w"> </span><span class="n">rightMean</span><span class="p [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">leftSd</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Math</span><span class="p">.</span><span class="na">sqrt</span><span class="p">(</span><span class="n">leftSd</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">rightSd</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Math</span><span class="p">.</span><span class="na">sqrt</span><span class="p">(</span><span class="n">rightSd</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">out</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="n">cov</span><span class="w"> </span><span class="o">/</span><span class="w"> </span><span class="p">(</span><span class="n">leftSd</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="n">rightSd</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span></span></span></code></pre></div>
</div>
</div>
<p><a href="#top">Back to top</a></p>
diff --git a/content/2015/03/02/february-2015-in-the-flink-community/index.html b/content/2015/03/02/february-2015-in-the-flink-community/index.html
index d9c90fdab..ca99b28d6 100644
--- a/content/2015/03/02/february-2015-in-the-flink-community/index.html
+++ b/content/2015/03/02/february-2015-in-the-flink-community/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -16,28 +16,27 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="February might be the shortest month of the year, but this does not mean that the Flink community has not been busy adding features to the system and fixing bugs. Here’s a rundown of the activity in the Flink community last month.
-0.8.1 release # Flink 0.8.1 was released. This bugfixing release resolves a total of 22 issues.
-New committer # Max Michels has been voted a committer by the Flink PMC.">
+0.8.1 release # Flink 0.8.1 was released. This bugfixing release resolves a total of 22 issues.
+New committer # Max Michels has been voted a committer by the Flink PMC.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="February 2015 in the Flink community" />
<meta property="og:description" content="February might be the shortest month of the year, but this does not mean that the Flink community has not been busy adding features to the system and fixing bugs. Here’s a rundown of the activity in the Flink community last month.
-0.8.1 release # Flink 0.8.1 was released. This bugfixing release resolves a total of 22 issues.
-New committer # Max Michels has been voted a committer by the Flink PMC." />
+0.8.1 release # Flink 0.8.1 was released. This bugfixing release resolves a total of 22 issues.
+New committer # Max Michels has been voted a committer by the Flink PMC." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2015/03/02/february-2015-in-the-flink-community/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-03-02T10:00:00+00:00" />
<meta property="article:modified_time" content="2015-03-02T10:00:00+00:00" />
-
<title>February 2015 in the Flink community | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -60,7 +59,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -267,7 +266,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -559,11 +558,11 @@ the web. Here are some examples:</p>
<p>Gelly, Flink’s Graph API allows users to manipulate graph-shaped data
directly. Here’s for example a calculation of shortest paths in a
graph:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">Graph</span><span class="o"><</span><span class="n">Long</span><span class="o">,</span> <span class="n">Double</span><span class="o">,</span> <span class="n">Double</span><span class="o">></span> <span class="n">graph</span> <span class="o">=</span> <span class="n">Graph</span><span class="o">.</span><span class="na">fromDataSet</s [...]
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">DataSet</span><span class="o"><</span><span class="n">Vertex</span><span class="o"><</span><span class="n">Long</span><span class="o">,</span> <span class="n">Double</span><span class="o">>></span> <span class="n">singleSourceShortestPaths</span> <span class="o">=</span> <span class="n">graph</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">run</span><span class="o">(</span><span class="k">new</span> <span class="n">SingleSourceShortestPaths</span><span class="o"><</span><span class="n">Long</span><span class="o">>(</span><span class="n">srcVertexId</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">maxIterations</span><span class="o">)).</span><span class="na">getVertices</span><span class="o">();</span></span></span></code></pre></div>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">Graph</span><span class="o"><</span><span class="n">Long</span><span class="p">,</span><span class="w"> </span><span class="n">Double</span><span class="p">,</span><span class="w"> </span><span class="n">Double</span><span class="o">></span><span class="w"> </span><span class="n">graph</span><span class="w"> </span><span class="o"> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataSet</span><span class="o"><</span><span class="n">Vertex</span><span class="o"><</span><span class="n">Long</span><span class="p">,</span><span class="w"> </span><span class="n">Double</span><span class="o">>></span><span class="w"> </span><span class="n">singleSourceShortestPaths</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">grap [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">run</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">SingleSourceShortestPaths</span><span class="o"><</span><span class="n">Long</span><span class="o">></span><span class="p">(</span><span class="n">srcVertexId</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">maxIterations</span><span class="p">)).</span><span class="na">getVertices</span><span class="p">();</span></span></span></code></pre></div>
<p>See more Gelly examples
<a href="https://github.com/apache/flink/tree/master/flink-libraries/flink-gelly-examples">here</a>.</p>
<h3 id="flink-expressions">
diff --git a/content/2015/03/13/peeking-into-apache-flinks-engine-room/index.html b/content/2015/03/13/peeking-into-apache-flinks-engine-room/index.html
index 7a21258ef..be0e2203b 100644
--- a/content/2015/03/13/peeking-into-apache-flinks-engine-room/index.html
+++ b/content/2015/03/13/peeking-into-apache-flinks-engine-room/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -15,25 +15,24 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
-<meta name="description" content="Join Processing in Apache Flink # Joins are prevalent operations in many data processing applications. Most data processing systems feature APIs that make joining data sets very easy. However, the internal algorithms for join processing are much more involved – especially if large data sets need to be efficiently handled. Therefore, join processing serves as a good example to discuss the salient design points and implementation details of a data process [...]
+<meta name="description" content="Join Processing in Apache Flink # Joins are prevalent operations in many data processing applications. Most data processing systems feature APIs that make joining data sets very easy. However, the internal algorithms for join processing are much more involved – especially if large data sets need to be efficiently handled. Therefore, join processing serves as a good example to discuss the salient design points and implementation details of a data processi [...]
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Peeking into Apache Flink's Engine Room" />
-<meta property="og:description" content="Join Processing in Apache Flink # Joins are prevalent operations in many data processing applications. Most data processing systems feature APIs that make joining data sets very easy. However, the internal algorithms for join processing are much more involved – especially if large data sets need to be efficiently handled. Therefore, join processing serves as a good example to discuss the salient design points and implementation details of a data [...]
+<meta property="og:description" content="Join Processing in Apache Flink # Joins are prevalent operations in many data processing applications. Most data processing systems feature APIs that make joining data sets very easy. However, the internal algorithms for join processing are much more involved – especially if large data sets need to be efficiently handled. Therefore, join processing serves as a good example to discuss the salient design points and implementation details of a data p [...]
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2015/03/13/peeking-into-apache-flinks-engine-room/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-03-13T10:00:00+00:00" />
<meta property="article:modified_time" content="2015-03-13T10:00:00+00:00" />
-
<title>Peeking into Apache Flink's Engine Room | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2015/04/07/march-2015-in-the-flink-community/index.html b/content/2015/04/07/march-2015-in-the-flink-community/index.html
index ff7776e14..94b2a55ae 100644
--- a/content/2015/04/07/march-2015-in-the-flink-community/index.html
+++ b/content/2015/04/07/march-2015-in-the-flink-community/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -16,28 +16,27 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="March has been a busy month in the Flink community.
-Scaling ALS # Flink committers employed at data Artisans published a blog post on how they scaled matrix factorization with Flink and Google Compute Engine to matrices with 28 billion elements.
-Learn about the internals of Flink # The community has started an effort to better document the internals of Flink. Check out the first articles on the Flink wiki on how Flink manages memory, how tasks in Flink exchange data, type extraction and serialization in Flink, as well as how Flink builds on Akka for distributed coordination.">
+Scaling ALS # Flink committers employed at data Artisans published a blog post on how they scaled matrix factorization with Flink and Google Compute Engine to matrices with 28 billion elements.
+Learn about the internals of Flink # The community has started an effort to better document the internals of Flink. Check out the first articles on the Flink wiki on how Flink manages memory, how tasks in Flink exchange data, type extraction and serialization in Flink, as well as how Flink builds on Akka for distributed coordination.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="March 2015 in the Flink community" />
<meta property="og:description" content="March has been a busy month in the Flink community.
-Scaling ALS # Flink committers employed at data Artisans published a blog post on how they scaled matrix factorization with Flink and Google Compute Engine to matrices with 28 billion elements.
-Learn about the internals of Flink # The community has started an effort to better document the internals of Flink. Check out the first articles on the Flink wiki on how Flink manages memory, how tasks in Flink exchange data, type extraction and serialization in Flink, as well as how Flink builds on Akka for distributed coordination." />
+Scaling ALS # Flink committers employed at data Artisans published a blog post on how they scaled matrix factorization with Flink and Google Compute Engine to matrices with 28 billion elements.
+Learn about the internals of Flink # The community has started an effort to better document the internals of Flink. Check out the first articles on the Flink wiki on how Flink manages memory, how tasks in Flink exchange data, type extraction and serialization in Flink, as well as how Flink builds on Akka for distributed coordination." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2015/04/07/march-2015-in-the-flink-community/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-04-07T10:00:00+00:00" />
<meta property="article:modified_time" content="2015-04-07T10:00:00+00:00" />
-
<title>March 2015 in the Flink community | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -60,7 +59,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -267,7 +266,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2015/04/13/announcing-flink-0.9.0-milestone1-preview-release/index.html b/content/2015/04/13/announcing-flink-0.9.0-milestone1-preview-release/index.html
index 97ee0ec08..2085826da 100644
--- a/content/2015/04/13/announcing-flink-0.9.0-milestone1-preview-release/index.html
+++ b/content/2015/04/13/announcing-flink-0.9.0-milestone1-preview-release/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ You can download the release here and check out the latest documentation here."
<meta property="og:url" content="https://flink.apache.org/2015/04/13/announcing-flink-0.9.0-milestone1-preview-release/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-04-13T10:00:00+00:00" />
<meta property="article:modified_time" content="2015-04-13T10:00:00+00:00" />
-
<title>Announcing Flink 0.9.0-milestone1 preview release | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2015/05/11/juggling-with-bits-and-bytes/index.html b/content/2015/05/11/juggling-with-bits-and-bytes/index.html
index 7f536a4f6..256799dfe 100644
--- a/content/2015/05/11/juggling-with-bits-and-bytes/index.html
+++ b/content/2015/05/11/juggling-with-bits-and-bytes/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -15,25 +15,24 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
-<meta name="description" content="How Apache Flink operates on binary data # Nowadays, a lot of open-source systems for analyzing large data sets are implemented in Java or other JVM-based programming languages. The most well-known example is Apache Hadoop, but also newer frameworks such as Apache Spark, Apache Drill, and also Apache Flink run on JVMs. A common challenge that JVM-based data analysis engines face is to store large amounts of data in memory - both for caching and for effi [...]
+<meta name="description" content="How Apache Flink operates on binary data # Nowadays, a lot of open-source systems for analyzing large data sets are implemented in Java or other JVM-based programming languages. The most well-known example is Apache Hadoop, but also newer frameworks such as Apache Spark, Apache Drill, and also Apache Flink run on JVMs. A common challenge that JVM-based data analysis engines face is to store large amounts of data in memory - both for caching and for effic [...]
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Juggling with Bits and Bytes" />
-<meta property="og:description" content="How Apache Flink operates on binary data # Nowadays, a lot of open-source systems for analyzing large data sets are implemented in Java or other JVM-based programming languages. The most well-known example is Apache Hadoop, but also newer frameworks such as Apache Spark, Apache Drill, and also Apache Flink run on JVMs. A common challenge that JVM-based data analysis engines face is to store large amounts of data in memory - both for caching and f [...]
+<meta property="og:description" content="How Apache Flink operates on binary data # Nowadays, a lot of open-source systems for analyzing large data sets are implemented in Java or other JVM-based programming languages. The most well-known example is Apache Hadoop, but also newer frameworks such as Apache Spark, Apache Drill, and also Apache Flink run on JVMs. A common challenge that JVM-based data analysis engines face is to store large amounts of data in memory - both for caching and fo [...]
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2015/05/11/juggling-with-bits-and-bytes/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-05-11T10:00:00+00:00" />
<meta property="article:modified_time" content="2015-05-11T10:00:00+00:00" />
-
<title>Juggling with Bits and Bytes | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -543,11 +542,11 @@ However, this approach has a few notable drawbacks. First of all it is not trivi
</ul>
<p>Each TypeInformation provides a serializer for the data type it represents. For example, a BasicTypeInfo returns a serializer that writes the respective primitive type, the serializer of a WritableTypeInfo delegates de/serialization to the write() and readFields() methods of the object implementing Hadoop’s Writable interface, and a GenericTypeInfo returns a serializer that delegates serialization to Kryo. Object serialization to a DataOutput which is backed by Flink MemorySegments go [...]
<p>Tuple, Pojo, and CaseClass types are composite types, i.e., containers for one or more possibly nested data types. As such, their serializers and comparators are also composite and delegate the serialization and comparison of their member data types to the respective serializers and comparators. The following figure illustrates the serialization of a (nested) <code>Tuple3<Integer, Double, Person></code> object where <code>Person</code> is a POJO and defined as follows:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">Person</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">int</span> <span class="n">id</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">String</span> <span class="n">name</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><center>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">Person</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">id</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">name</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><center>
<img src="/img/blog/data-serialization.png" style="width:80%;margin:15px">
</center>
<p>Flink’s type system can be easily extended by providing custom TypeInformations, Serializers, and Comparators to improve the performance of serializing and comparing custom data types.</p>
diff --git a/content/2015/05/14/april-2015-in-the-flink-community/index.html b/content/2015/05/14/april-2015-in-the-flink-community/index.html
index 978e3a4a4..78c85368b 100644
--- a/content/2015/05/14/april-2015-in-the-flink-community/index.html
+++ b/content/2015/05/14/april-2015-in-the-flink-community/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -16,26 +16,27 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="April was an packed month for Apache Flink.
-Flink runner for Google Cloud Dataflow # A Flink runner for Google Cloud Dataflow was announced. See the blog posts by data Artisans and the Google Cloud Platform Blog. Google Cloud Dataflow programs can be written using and open-source SDK and run in multiple backends, either as a managed service inside Google’s infrastructure, or leveraging open source runners, including Apache Flink.">
+Flink runner for Google Cloud Dataflow # A Flink runner for Google Cloud Dataflow was announced. See the blog posts by data Artisans and the Google Cloud Platform Blog. Google Cloud Dataflow programs can be written using and open-source SDK and run in multiple backends, either as a managed service inside Google’s infrastructure, or leveraging open source runners, including Apache Flink.
+Flink 0.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="April 2015 in the Flink community" />
<meta property="og:description" content="April was an packed month for Apache Flink.
-Flink runner for Google Cloud Dataflow # A Flink runner for Google Cloud Dataflow was announced. See the blog posts by data Artisans and the Google Cloud Platform Blog. Google Cloud Dataflow programs can be written using and open-source SDK and run in multiple backends, either as a managed service inside Google’s infrastructure, or leveraging open source runners, including Apache Flink." />
+Flink runner for Google Cloud Dataflow # A Flink runner for Google Cloud Dataflow was announced. See the blog posts by data Artisans and the Google Cloud Platform Blog. Google Cloud Dataflow programs can be written using and open-source SDK and run in multiple backends, either as a managed service inside Google’s infrastructure, or leveraging open source runners, including Apache Flink.
+Flink 0." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2015/05/14/april-2015-in-the-flink-community/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-05-14T10:00:00+00:00" />
<meta property="article:modified_time" content="2015-05-14T10:00:00+00:00" />
-
<title>April 2015 in the Flink community | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +59,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +266,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2015/06/24/announcing-apache-flink-0.9.0/index.html b/content/2015/06/24/announcing-apache-flink-0.9.0/index.html
index 511185282..3e4b9279e 100644
--- a/content/2015/06/24/announcing-apache-flink-0.9.0/index.html
+++ b/content/2015/06/24/announcing-apache-flink-0.9.0/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -16,26 +16,25 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="The Apache Flink community is pleased to announce the availability of the 0.9.0 release. The release is the result of many months of hard work within the Flink community. It contains many new features and improvements which were previewed in the 0.9.0-milestone1 release and have been polished since then. This is the largest Flink release so far.
-Download the release and check out the documentation. Feedback through the Flinkmailing lists is, as always, very welcome!">
+Download the release and check out the documentation. Feedback through the Flink mailing lists is, as always, very welcome!">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Announcing Apache Flink 0.9.0" />
<meta property="og:description" content="The Apache Flink community is pleased to announce the availability of the 0.9.0 release. The release is the result of many months of hard work within the Flink community. It contains many new features and improvements which were previewed in the 0.9.0-milestone1 release and have been polished since then. This is the largest Flink release so far.
-Download the release and check out the documentation. Feedback through the Flinkmailing lists is, as always, very welcome!" />
+Download the release and check out the documentation. Feedback through the Flink mailing lists is, as always, very welcome!" />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2015/06/24/announcing-apache-flink-0.9.0/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-06-24T14:00:00+00:00" />
<meta property="article:modified_time" content="2015-06-24T14:00:00+00:00" />
-
<title>Announcing Apache Flink 0.9.0 | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2015/08/24/introducing-gelly-graph-processing-with-apache-flink/index.html b/content/2015/08/24/introducing-gelly-graph-processing-with-apache-flink/index.html
index 31064979f..6f7523ad8 100644
--- a/content/2015/08/24/introducing-gelly-graph-processing-with-apache-flink/index.html
+++ b/content/2015/08/24/introducing-gelly-graph-processing-with-apache-flink/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ Gelly allows Flink users to perform end-to-end data analysis in a single system.
<meta property="og:url" content="https://flink.apache.org/2015/08/24/introducing-gelly-graph-processing-with-apache-flink/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-08-24T00:00:00+00:00" />
<meta property="article:modified_time" content="2015-08-24T00:00:00+00:00" />
-
<title>Introducing Gelly: Graph Processing with Apache Flink | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -558,8 +557,8 @@ of vertices, edges and the node degrees.</p>
<p>The transformation methods enable several Graph operations, using high-level functions similar to
the ones provided by the batch processing API. These transformations can be applied one after the
other, yielding a new Graph after each step, in a fashion similar to operators on DataSets:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">inputGraph</span><span class="o">.</span><span class="na">getUndirected</span><span class="o">().</span><span class="na">mapEdges</span><span class="o">(</span><span class="k">new</span> <span class="n">CustomEdgeMapper</span><span class="o">());</span>
-</span></span></code></pre></div><p>Transformations can be applied on:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">inputGraph</span><span class="p">.</span><span class="na">getUndirected</span><span class="p">().</span><span class="na">mapEdges</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">CustomEdgeMapper</span><span class="p">());</span><span class="w">
+</span></span></span></code></pre></div><p>Transformations can be applied on:</p>
<ol>
<li><strong>Vertices</strong>: <code>mapVertices</code>, <code>joinWithVertices</code>, <code>filterOnVertices</code>, <code>addVertex</code>, …</li>
<li><strong>Edges</strong>: <code>mapEdges</code>, <code>filterOnEdges</code>, <code>removeEdge</code>, …</li>
@@ -587,8 +586,8 @@ one or more values per vertex, the more general <code>groupReduceOnEdges()</cod
</center>
<p>Assume you would want to compute the sum of the values of all incoming neighbors for each vertex.
We will call the <code>reduceOnNeighbors()</code> aggregation method since the sum is an associative and commutative operation and the neighbors’ values are needed:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">graph</span><span class="o">.</span><span class="na">reduceOnNeighbors</span><span class="o">(</span><span class="k">new</span> <span class="n">SumValues</span><span class="o">(),</span> <span class="n">EdgeDirection</span><span class="o">.</span><span class="na">IN</span><span class="o">);</span>
-</span></span></code></pre></div><p>The vertex with id 1 is the only node that has no incoming edges. The result is therefore:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">graph</span><span class="p">.</span><span class="na">reduceOnNeighbors</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">SumValues</span><span class="p">(),</span><span class="w"> </span><span class="n">EdgeDirection</span><span class="p">.</span><span class="na">IN</span><span class="p">);</ [...]
+</span></span></span></code></pre></div><p>The vertex with id 1 is the only node that has no incoming edges. The result is therefore:</p>
<center>
<img src="/img/blog/reduce-on-neighbors.png" style="width:90%;margin:15px">
</center>
@@ -740,23 +739,23 @@ play-count triplets.</p>
<p>After reading the <code>(user-id, song-id, play-count)</code> triplets from a CSV file and after parsing a
text file in order to retrieve the list of songs that a user would not want to include in a
playlist, we use a coGroup function to filter out the mismatches.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// read the user-song-play triplets.
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataSet</span><span class="o"><</span><span class="n">Tuple3</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span> <span class="n">triplets</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="n">getUserSongTripletsData</span><span class="o">(</span><span class="n">env</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// read the mismatches dataset and extract the songIDs
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataSet</span><span class="o"><</span><span class="n">Tuple3</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span> <span class="n">validTriplets</span> <span class="o">=</span> <span class="n">triplets</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">coGroup</span><span class="o">(</span><span class="n">mismatches</span><span class="o">).</span><span class="na">where</span><span class="o">(</span><span class="n">1</span><span class="o">).</span><span class="na">equalTo</span><span class="o">(</span><span class="n">0</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">with</span><span class="o">(</span><span class="k">new</span> <span class="n">CoGroupFunction</span><span class="o">()</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kt">void</span> <span class="nf">coGroup</span><span class="o">(</span><span class="n">Iterable</span> <span class="n">triplets</span><span class="o">,</span> <span class="n">Iterable</span> <span class="n">invalidSongs</span><span class="o">,</span> <span class="n">Collector</span> <span class="n">out</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">(!</span><span class="n">invalidSongs</span><span class="o">.</span><span class="na">iterator</span><span class="o">().</span><span class="na">hasNext</span><span class="o">())</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="o">(</span><span class="n">Tuple3</span> <span class="n">triplet</span> <span class="o">:</span> <span class="n">triplets</span><span class="o">)</span> <span class="o">{</span> <span class="c1">// valid triplet
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">out</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="n">triplet</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span></code></pre></div><p>The coGroup simply takes the triplets whose song-id (second field) matches the song-id from the
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// read the user-song-play triplets.</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataSet</span><span class="o"><</span><span class="n">Tuple3</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="o">>></span><span class="w"> </span><span class="n">triplets</span><span class="w"> </span><span [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">getUserSongTripletsData</span><span class="p">(</span><span class="n">env</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// read the mismatches dataset and extract the songIDs</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataSet</span><span class="o"><</span><span class="n">Tuple3</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="o">>></span><span class="w"> </span><span class="n">validTriplets</span><span class="w"> </span> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">coGroup</span><span class="p">(</span><span class="n">mismatches</span><span class="p">).</span><span class="na">where</span><span class="p">(</span><span class="n">1</span><span class="p">).</span><span class="na">equalTo</span><span class="p">(</span><span class="n">0</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">with</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">CoGroupFunction</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">coGroup</span><span class="p">(</span><span class="n">Iterable</span><span class="w"> </span><span class="n">triplets</span><span class="p">,</span><span class="w"> </span><span class="n">Iterable</span><span class="w"> </span><span class="n">invalidSongs</span><span class="p">,</span><span class="w"> </span><span class="n">C [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="o">!</span><span class="n">invalidSongs</span><span class="p">.</span><span class="na">iterator</span><span class="p">().</span><span class="na">hasNext</span><span class="p">())</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="n">Tuple3</span><span class="w"> </span><span class="n">triplet</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="n">triplets</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="c1">// valid tr [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">out</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="n">triplet</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>The coGroup simply takes the triplets whose song-id (second field) matches the song-id from the
mismatches list (first field) and if the iterator was empty for a certain triplet, meaning that
there were no mismatches found, the triplet associated with that song is collected.</p>
<h4 id="compute-the-top-songs-per-user">
@@ -770,33 +769,33 @@ certain song.</p>
<center>
<img src="/img/blog/user-song-graph.png" style="width:90%;margin:15px">
</center>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// create a user -> song weighted bipartite graph where the edge weights
-</span></span></span><span class="line"><span class="cl"><span class="c1">// correspond to play counts
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">Graph</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">NullValue</span><span class="o">,</span> <span class="n">Integer</span><span class="o">></span> <span class="n">userSongGraph</span> <span class="o">=</span> <span class="n">Graph</span><span class="o">.</span><span class="na">fromTupleDataSet</span><span class="o">(</span><span cla [...]
-</span></span></code></pre></div><p>Consult the <a href="//nightlies.apache.org/flink/flink-docs-master/dev/libs/gelly/">Gelly guide</a> for guidelines
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// create a user -> song weighted bipartite graph where the edge weights</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// correspond to play counts</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">Graph</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">NullValue</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="o">></span><span class="w"> </span><span class="n">userSongGraph</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class [...]
+</span></span></span></code></pre></div><p>Consult the <a href="//nightlies.apache.org/flink/flink-docs-master/dev/libs/gelly/">Gelly guide</a> for guidelines
on how to create a graph from a given DataSet of edges or from a collection.</p>
<p>To retrieve the top songs per user, we call the groupReduceOnEdges function as it perform an
aggregation over the first hop neighborhood taking just the edges into consideration. We will
basically iterate through the edge value and collect the target (song) of the maximum weight edge.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">//get the top track (most listened to) for each user
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataSet</span><span class="o"><</span><span class="n">Tuple2</span><span class="o">></span> <span class="n">usersWithTopTrack</span> <span class="o">=</span> <span class="n">userSongGraph</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">groupReduceOnEdges</span><span class="o">(</span><span class="k">new</span> <span class="n">GetTopSongPerUser</span><span class="o">(),</span> <span class="n">EdgeDirection</span><span class="o">.</span><span class="na">OUT</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="kd">class</span> <span class="nc">GetTopSongPerUser</span> <span class="kd">implements</span> <span class="n">EdgesFunctionWithVertexValue</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kt">void</span> <span class="nf">iterateEdges</span><span class="o">(</span><span class="n">Vertex</span> <span class="n">vertex</span><span class="o">,</span> <span class="n">Iterable</span><span class="o"><</span><span class="n">Edge</span><span class="o">></span> <span class="n">edges</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kt">int</span> <span class="n">maxPlaycount</span> <span class="o">=</span> <span class="n">0</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="n">String</span> <span class="n">topSong</span> <span class="o">=</span> <span class="s">""</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="o">(</span><span class="n">Edge</span> <span class="n">edge</span> <span class="o">:</span> <span class="n">edges</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">(</span><span class="n">edge</span><span class="o">.</span><span class="na">getValue</span><span class="o">()</span> <span class="o">></span> <span class="n">maxPlaycount</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">maxPlaycount</span> <span class="o">=</span> <span class="n">edge</span><span class="o">.</span><span class="na">getValue</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="n">topSong</span> <span class="o">=</span> <span class="n">edge</span><span class="o">.</span><span class="na">getTarget</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="k">new</span> <span class="n">Tuple2</span><span class="o">(</span><span class="n">vertex</span><span class="o">.</span><span class="na">getId</span><span class="o">(),</span> <span class="n">topSong</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><h4 id="creating-a-user-user-similarity-graph">
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">//get the top track (most listened to) for each user</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataSet</span><span class="o"><</span><span class="n">Tuple2</span><span class="o">></span><span class="w"> </span><span class="n">usersWithTopTrack</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">userSongGraph</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">groupReduceOnEdges</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">GetTopSongPerUser</span><span class="p">(),</span><span class="w"> </span><span class="n">EdgeDirection</span><span class="p">.</span><span class="na">OUT</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">class</span> <span class="nc">GetTopSongPerUser</span><span class="w"> </span><span class="kd">implements</span><span class="w"> </span><span class="n">EdgesFunctionWithVertexValue</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">iterateEdges</span><span class="p">(</span><span class="n">Vertex</span><span class="w"> </span><span class="n">vertex</span><span class="p">,</span><span class="w"> </span><span class="n">Iterable</span><span class="o"><</span><span class="n">Edge</span><span class="o">></span><span class="w"> </span><span class="n">edges</span><s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">maxPlaycount</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">0</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">topSong</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">""</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="n">Edge</span><span class="w"> </span><span class="n">edge</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="n">edges</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">edge</span><span class="p">.</span><span class="na">getValue</span><span class="p">()</span><span class="w"> </span><span class="o">></span><span class="w"> </span><span class="n">maxPlaycount</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">maxPlaycount</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">edge</span><span class="p">.</span><span class="na">getValue</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">topSong</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">edge</span><span class="p">.</span><span class="na">getTarget</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Tuple2</span><span class="p">(</span><span class="n">vertex</span><span class="p">.</span><span class="na">getId</span><span class="p">(),</span><span class="w"> </span><span class="n">topSong</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><h4 id="creating-a-user-user-similarity-graph">
Creating a User-User Similarity Graph
<a class="anchor" href="#creating-a-user-user-similarity-graph">#</a>
</h4>
@@ -815,60 +814,60 @@ common interest (right-hand side of the image).</p>
<p>Afterwards, we perform a <code>distinct()</code> operation to avoid creation of duplicate data.
Considering that we now have the DataSet of edges which present interest, creating a graph is as
straightforward as a call to the <code>Graph.fromDataSet()</code> method.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// create a user-user similarity graph:
-</span></span></span><span class="line"><span class="cl"><span class="c1">// two users that listen to the same song are connected
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataSet</span><span class="o"><</span><span class="n">Edge</span><span class="o">></span> <span class="n">similarUsers</span> <span class="o">=</span> <span class="n">userSongGraph</span><span class="o">.</span><span class="na">getEdges</span><span class="o">()</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// filter out user-song edges that are below the playcount threshold
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="o">.</span><span class="na">filter</span><span class="o">(</span><span class="k">new</span> <span class="n">FilterFunction</span><span class="o"><</span><span class="n">Edge</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>>()</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">boolean</span> <span class="nf">filter</span><span class="o">(</span><span class="n">Edge</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">></span> <span class="n">edge</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="o">(</span><span class="n">edge</span><span class="o">.</span><span class="na">getValue</span><span class="o">()</span> <span class="o">></span> <span class="n">playcountThreshold</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">})</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">groupBy</span><span class="o">(</span><span class="n">1</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">reduceGroup</span><span class="o">(</span><span class="k">new</span> <span class="n">GroupReduceFunction</span><span class="o">()</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kt">void</span> <span class="nf">reduce</span><span class="o">(</span><span class="n">Iterable</span><span class="o"><</span><span class="n">Edge</span><span class="o">></span> <span class="n">edges</span><span class="o">,</span> <span class="n">Collector</span><span class="o"><</span><span class="n">Edge</span><span class="o">></span> <span class="n">out</span><span class="o">)</span> <span class= [...]
-</span></span><span class="line"><span class="cl"> <span class="n">List</span> <span class="n">users</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ArrayList</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="o">(</span><span class="n">Edge</span> <span class="n">edge</span> <span class="o">:</span> <span class="n">edges</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">users</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="n">edge</span><span class="o">.</span><span class="na">getSource</span><span class="o">());</span>
-</span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="n">0</span><span class="o">;</span> <span class="n">i</span> <span class="o"><</span> <span class="n">users</span><span class="o">.</span><span class="na">size</span><span class="o">()</span> <span class="o">-</span> <span class="n">1</span><span class="o">;</span> <span c [...]
-</span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">j</span> <span class="o">=</span> <span class="n">i</span><span class="o">+</span><span class="n">1</span><span class="o">;</span> <span class="n">j</span> <span class="o"><</span> <span class="n">users</span><span class="o">.</span><span class="na">size</span><span class="o">()</span> <span class="o">-</span> <spa [...]
-</span></span><span class="line"><span class="cl"> <span class="n">out</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="k">new</span> <span class="n">Edge</span><span class="o">(</span><span class="n">users</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">i</span><span class="o">),</span> <span class="n">users</span><span class="o">.</span><span class="na">get</ [...]
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">})</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">distinct</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">Graph</span> <span class="n">similarUsersGraph</span> <span class="o">=</span> <span class="n">Graph</span><span class="o">.</span><span class="na">fromDataSet</span><span class="o">(</span><span class="n">similarUsers</span><span class="o">).</span><span class="na">getUndirected</span><span class="o">();</span>
-</span></span></code></pre></div><p>After having created a user-user graph, it would make sense to detect the various communities
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// create a user-user similarity graph:</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// two users that listen to the same song are connected</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataSet</span><span class="o"><</span><span class="n">Edge</span><span class="o">></span><span class="w"> </span><span class="n">similarUsers</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">userSongGraph</span><span class="p">.</span><span class="na">getEdges</span><span class="p">()</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// filter out user-song edges that are below the playcount threshold</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">filter</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">FilterFunction</span><span class="o"><</span><span class="n">Edge</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="o">>></span><span class="p">()</ [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">boolean</span><span class="w"> </span><span class="nf">filter</span><span class="p">(</span><span class="n">Edge</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="o">></span><span class="w"> </span><span class="n">edge</sp [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="p">(</span><span class="n">edge</span><span class="p">.</span><span class="na">getValue</span><span class="p">()</span><span class="w"> </span><span class="o">></span><span class="w"> </span><span class="n">playcountThreshold</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">})</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">groupBy</span><span class="p">(</span><span class="n">1</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">reduceGroup</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">GroupReduceFunction</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">reduce</span><span class="p">(</span><span class="n">Iterable</span><span class="o"><</span><span class="n">Edge</span><span class="o">></span><span class="w"> </span><span class="n">edges</span><span class="p">,</span><span class="w"> </span><span class="n">Collector</span><span class="o"><</span><span class="n">Edg [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">List</span><span class="w"> </span><span class="n">users</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">ArrayList</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="n">Edge</span><span class="w"> </span><span class="n">edge</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="n">edges</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">users</span><span class="p">.</span><span class="na">add</span><span class="p">(</span><span class="n">edge</span><span class="p">.</span><span class="na">getSource</span><span class="p">());</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="kt">int</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">0</span><span class="p">;</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o"><</span><span class="w"> </s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="kt">int</span><span class="w"> </span><span class="n">j</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">i</span><span class="o">+</span><span class="n">1</span><span class="p">;</span><span class="w"> </span><span class="n">j</span><span class="w"> </ [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">out</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">Edge</span><span class="p">(</span><span class="n">users</span><span class="p">.</span><span class="na">get</span><span class="p">(</span><span class="n">i</span><span class="p">),</span><span class="w"> </spa [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">})</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">distinct</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">Graph</span><span class="w"> </span><span class="n">similarUsersGraph</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Graph</span><span class="p">.</span><span class="na">fromDataSet</span><span class="p">(</span><span class="n">similarUsers</span><span class="p">).</span><span class="na">getUndirected</span><span class="p">();</span><span class="w">
+</span></span></span></code></pre></div><p>After having created a user-user graph, it would make sense to detect the various communities
formed. To do so, we first initialize each vertex with a numeric label using the
<code>joinWithVertices()</code> function that takes a data set of Tuple2 as a parameter and joins
the id of a vertex with the first element of the tuple, afterwards applying a map function.
Finally, we call the <code>run()</code> method with the LabelPropagation library method passed
as a parameter. In the end, the vertices will be updated to contain the most frequent label
among their neighbors.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// detect user communities using label propagation
-</span></span></span><span class="line"><span class="cl"><span class="c1">// initialize each vertex with a unique numeric label
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataSet</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Long</span><span class="o">>></span> <span class="n">idsWithInitialLabels</span> <span class="o">=</span> <span class="n">DataSetUtils</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">zipWithUniqueId</span><span class="o">(</span><span class="n">similarUsersGraph</span><span class="o">.</span><span class="na">getVertexIds</span><span class="o">())</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">map</span><span class="o">(</span><span class="k">new</span> <span class="n">MapFunction</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Long</span><span class="o">,</span> <span class="n">String</span><span class="o">>,</span> <span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="o">,</s [...]
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Long</span><span class="o">></span> <span class="nf">map</span><span class="o">(</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Long</span><span class="o">,</span> <span class="n">String</span><span class="o">></span> <span class="n [...]
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="k">new</span> <span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Long</span><span class="o">>(</span><span class="n">tuple2</span><span class="o">.</span><span class="na">f1</span><span class="o">,</span> <span class="n">tuple2</span><span class="o">.</span><span class="na">f0</span><span class="o">) [...]
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">});</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// update the vertex values and run the label propagation algorithm
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataSet</span><span class="o"><</span><span class="n">Vertex</span><span class="o">></span> <span class="n">verticesWithCommunity</span> <span class="o">=</span> <span class="n">similarUsersGraph</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">joinWithVertices</span><span class="o">(</span><span class="n">idsWithlLabels</span><span class="o">,</span> <span class="k">new</span> <span class="n">MapFunction</span><span class="o">()</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">Long</span> <span class="nf">map</span><span class="o">(</span><span class="n">Tuple2</span> <span class="n">idWithLabel</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">idWithLabel</span><span class="o">.</span><span class="na">f1</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">})</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">run</span><span class="o">(</span><span class="k">new</span> <span class="n">LabelPropagation</span><span class="o">(</span><span class="n">numIterations</span><span class="o">))</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">getVertices</span><span class="o">();</span>
-</span></span></code></pre></div><p><a href="#top">Back to top</a></p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// detect user communities using label propagation</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// initialize each vertex with a unique numeric label</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataSet</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">Long</span><span class="o">>></span><span class="w"> </span><span class="n">idsWithInitialLabels</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">DataSetUt [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">zipWithUniqueId</span><span class="p">(</span><span class="n">similarUsersGraph</span><span class="p">.</span><span class="na">getVertexIds</span><span class="p">())</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">map</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">MapFunction</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Long</span><span class="p">,</span><span class="w"> </span><span class="n">String</span><span class="o">></span><span class="p">,</span><span c [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">Long</span><span class="o">></span><span class="w"> </span><span class="nf">map</span><span class="p">(</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Long</sp [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">Long</span><span class="o">></span><span class="p">(</span><span class="n">tuple2</span><span class="p">.</span><span class="na">f1</span [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">});</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// update the vertex values and run the label propagation algorithm</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataSet</span><span class="o"><</span><span class="n">Vertex</span><span class="o">></span><span class="w"> </span><span class="n">verticesWithCommunity</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">similarUsersGraph</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">joinWithVertices</span><span class="p">(</span><span class="n">idsWithlLabels</span><span class="p">,</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">MapFunction</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">Long</span><span class="w"> </span><span class="nf">map</span><span class="p">(</span><span class="n">Tuple2</span><span class="w"> </span><span class="n">idWithLabel</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">idWithLabel</span><span class="p">.</span><span class="na">f1</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">})</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">run</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">LabelPropagation</span><span class="p">(</span><span class="n">numIterations</span><span class="p">))</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">getVertices</span><span class="p">();</span><span class="w">
+</span></span></span></code></pre></div><p><a href="#top">Back to top</a></p>
<h2 id="ongoing-and-future-work">
Ongoing and Future Work
<a class="anchor" href="#ongoing-and-future-work">#</a>
diff --git a/content/2015/09/01/apache-flink-0.9.1-available/index.html b/content/2015/09/01/apache-flink-0.9.1-available/index.html
index 77bc6c910..3a728c9f4 100644
--- a/content/2015/09/01/apache-flink-0.9.1-available/index.html
+++ b/content/2015/09/01/apache-flink-0.9.1-available/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -28,18 +28,17 @@ The following issues were fixed for this release:" />
<meta property="og:url" content="https://flink.apache.org/2015/09/01/apache-flink-0.9.1-available/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-09-01T08:00:00+00:00" />
<meta property="article:modified_time" content="2015-09-01T08:00:00+00:00" />
-
<title>Apache Flink 0.9.1 available | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -62,7 +61,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -269,7 +268,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2015/09/03/announcing-flink-forward-2015/index.html b/content/2015/09/03/announcing-flink-forward-2015/index.html
index 3d7ba214b..882852ae3 100644
--- a/content/2015/09/03/announcing-flink-forward-2015/index.html
+++ b/content/2015/09/03/announcing-flink-forward-2015/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -16,26 +16,25 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="Flink Forward 2015 is the first conference with Flink at its center that aims to bring together the Apache Flink community in a single place. The organizers are starting this conference in October 12 and 13 from Berlin, the place where Apache Flink started.
- The conference program has been announced by the organizers and a program committee consisting of Flink PMC members. The agenda contains talks from industry and academia as well as a dedicated session on hands-on Flink training.">
+The conference program has been announced by the organizers and a program committee consisting of Flink PMC members. The agenda contains talks from industry and academia as well as a dedicated session on hands-on Flink training.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Announcing Flink Forward 2015" />
<meta property="og:description" content="Flink Forward 2015 is the first conference with Flink at its center that aims to bring together the Apache Flink community in a single place. The organizers are starting this conference in October 12 and 13 from Berlin, the place where Apache Flink started.
- The conference program has been announced by the organizers and a program committee consisting of Flink PMC members. The agenda contains talks from industry and academia as well as a dedicated session on hands-on Flink training." />
+The conference program has been announced by the organizers and a program committee consisting of Flink PMC members. The agenda contains talks from industry and academia as well as a dedicated session on hands-on Flink training." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2015/09/03/announcing-flink-forward-2015/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-09-03T08:00:00+00:00" />
<meta property="article:modified_time" content="2015-09-03T08:00:00+00:00" />
-
<title>Announcing Flink Forward 2015 | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2015/09/16/off-heap-memory-in-apache-flink-and-the-curious-jit-compiler/index.html b/content/2015/09/16/off-heap-memory-in-apache-flink-and-the-curious-jit-compiler/index.html
index d40a36cb0..3c075e5da 100644
--- a/content/2015/09/16/off-heap-memory-in-apache-flink-and-the-curious-jit-compiler/index.html
+++ b/content/2015/09/16/off-heap-memory-in-apache-flink-and-the-curious-jit-compiler/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -22,18 +22,17 @@
<meta property="og:url" content="https://flink.apache.org/2015/09/16/off-heap-memory-in-apache-flink-and-the-curious-jit-compiler/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-09-16T08:00:00+00:00" />
<meta property="article:modified_time" content="2015-09-16T08:00:00+00:00" />
-
<title>Off-heap Memory in Apache Flink and the curious JIT compiler | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -558,31 +557,31 @@ OffHeapMemorySegment (subclass) : 3,847 msecs
<p>Fortunately, there is a nice trick to do this without introducing code branches and specialized handling of the two different memory types. The trick is based on the way that the <code>sun.misc.Unsafe</code> methods interpret object references. To illustrate this, we take the method that gets a long integer from a memory position:</p>
<pre tabindex="0"><code>sun.misc.Unsafe.getLong(Object reference, long offset)
</code></pre><p>The method accepts an object reference, takes its memory address, and add the offset to obtain a pointer. It then fetches the eight bytes at the address pointed to and interprets them as a long integer. Since the method accepts <em>null</em> as the reference (and interprets it a <em>zero</em>) one can write a method that fetches a long integer seamlessly from heap and off-heap memory as follows:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">HybridMemorySegment</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="kd">final</span> <span class="kt">byte</span><span class="o">[]</span> <span class="n">heapMemory</span><span class="o">;</span> <span class="c1">// non-null in heap case, null in off-heap case
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="kd">private</span> <span class="kd">final</span> <span class="kt">long</span> <span class="n">address</span><span class="o">;</span> <span class="c1">// may be absolute, or relative to byte[]
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">// method of interest
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="kd">public</span> <span class="kt">long</span> <span class="nf">getLong</span><span class="o">(</span><span class="kt">int</span> <span class="n">pos</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">UNSAFE</span><span class="o">.</span><span class="na">getLong</span><span class="o">(</span><span class="n">heapMemory</span><span class="o">,</span> <span class="n">address</span> <span class="o">+</span> <span class="n">pos</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">// initialize for heap memory
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="kd">public</span> <span class="nf">HybridMemorySegment</span><span class="o">(</span><span class="kt">byte</span><span class="o">[]</span> <span class="n">heapMemory</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">this</span><span class="o">.</span><span class="na">heapMemory</span> <span class="o">=</span> <span class="n">heapMemory</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="k">this</span><span class="o">.</span><span class="na">address</span> <span class="o">=</span> <span class="n">UNSAFE</span><span class="o">.</span><span class="na">arrayBaseOffset</span><span class="o">(</span><span class="kt">byte</span><span class="o">[].</span><span class="na">class</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">// initialize for off-heap memory
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="kd">public</span> <span class="nf">HybridMemorySegment</span><span class="o">(</span><span class="kt">long</span> <span class="n">offheapPointer</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">this</span><span class="o">.</span><span class="na">heapMemory</span> <span class="o">=</span> <span class="kc">null</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="k">this</span><span class="o">.</span><span class="na">address</span> <span class="o">=</span> <span class="n">offheapPointer</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>To check whether both cases (heap and off-heap) really result in the same code paths (no hidden branches inside the <code>Unsafe.getLong(Object, long)</code> method) one can check out the C++ source code of <code>sun.misc.Unsafe</code>, available here: <a href="http://hg.openjdk.java.net/jdk8/jdk8/hotspot/file/tip/src/share/vm/prims/unsafe.cpp">http://hg.openjdk.java.net/jdk8/jdk8/hotspot/file/tip/src/share/vm/prims/unsafe.cpp</a></p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">HybridMemorySegment</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="kt">byte</span><span class="o">[]</span><span class="w"> </span><span class="n">heapMemory</span><span class="p">;</span><span class="w"> </span><span class="c1">// non-null in heap case, null in off-heap case</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="kt">long</span><span class="w"> </span><span class="n">address</span><span class="p">;</span><span class="w"> </span><span class="c1">// may be absolute, or relative to byte[]</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// method of interest</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">long</span><span class="w"> </span><span class="nf">getLong</span><span class="p">(</span><span class="kt">int</span><span class="w"> </span><span class="n">pos</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">UNSAFE</span><span class="p">.</span><span class="na">getLong</span><span class="p">(</span><span class="n">heapMemory</span><span class="p">,</span><span class="w"> </span><span class="n">address</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">pos</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// initialize for heap memory</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="nf">HybridMemorySegment</span><span class="p">(</span><span class="kt">byte</span><span class="o">[]</span><span class="w"> </span><span class="n">heapMemory</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">heapMemory</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">heapMemory</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">address</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">UNSAFE</span><span class="p">.</span><span class="na">arrayBaseOffset</span><span class="p">(</span><span class="kt">byte</span><span class="o">[]</span><span class="p">.</span><span class="na">class</span><span class="p">)</span><span cl [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// initialize for off-heap memory</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="nf">HybridMemorySegment</span><span class="p">(</span><span class="kt">long</span><span class="w"> </span><span class="n">offheapPointer</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">heapMemory</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">null</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">address</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">offheapPointer</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>To check whether both cases (heap and off-heap) really result in the same code paths (no hidden branches inside the <code>Unsafe.getLong(Object, long)</code> method) one can check out the C++ source code of <code>sun.misc.Unsafe</code>, available here: <a href="http://hg.openjdk.java.net/jdk8/jdk8/hotspot/file/tip/src/share/vm/prims/unsafe.cpp">http://hg.openjdk.java.net/jdk8/jdk8/hotspot/file/tip/src/share/vm/prims/unsafe.cpp</a></p>
<p>Of particular interest is the macro in line 155, which is the base of all GET methods. Tracing the function calls (many are no-ops), one can see that both variants of Unsafe’s <code>getLong()</code> result in the same code:
Either <code>0 + absolutePointer</code> or <code>objectRefAddress + offset</code>.</p>
<h2 id="summary">
diff --git a/content/2015/11/16/announcing-apache-flink-0.10.0/index.html b/content/2015/11/16/announcing-apache-flink-0.10.0/index.html
index d1ac53bc8..ba3610f38 100644
--- a/content/2015/11/16/announcing-apache-flink-0.10.0/index.html
+++ b/content/2015/11/16/announcing-apache-flink-0.10.0/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ For Flink 0.10.0, the focus of the community was to graduate the DataStream API
<meta property="og:url" content="https://flink.apache.org/2015/11/16/announcing-apache-flink-0.10.0/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-11-16T08:00:00+00:00" />
<meta property="article:modified_time" content="2015-11-16T08:00:00+00:00" />
-
<title>Announcing Apache Flink 0.10.0 | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2015/11/27/flink-0.10.1-released/index.html b/content/2015/11/27/flink-0.10.1-released/index.html
index c14ec5591..14ca1f2de 100644
--- a/content/2015/11/27/flink-0.10.1-released/index.html
+++ b/content/2015/11/27/flink-0.10.1-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -17,27 +17,26 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="Today, the Flink community released the first bugfix release of the 0.10 series of Flink.
We recommend all users updating to this release, by bumping the version of your Flink dependencies and updating the binaries on the server.
-Issues fixed # [FLINK-2879] - Links in documentation are broken [FLINK-2938] - Streaming docs not in sync with latest state changes [FLINK-2942] - Dangling operators in web UI's program visualization (non-deterministic) [FLINK-2967] - TM address detection might not always detect the right interface on slow networks / overloaded JMs [FLINK-2977] - Cannot access HBase in a Kerberos secured Yarn cluster [FLINK-2987] - Flink 0.">
+Issues fixed # [FLINK-2879] - Links in documentation are broken [FLINK-2938] - Streaming docs not in sync with latest state changes [FLINK-2942] - Dangling operators in web UI's program visualization (non-deterministic) [FLINK-2967] - TM address detection might not always detect the right interface on slow networks / overloaded JMs [FLINK-2977] - Cannot access HBase in a Kerberos secured Yarn cluster [FLINK-2987] - Flink 0.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Flink 0.10.1 released" />
<meta property="og:description" content="Today, the Flink community released the first bugfix release of the 0.10 series of Flink.
We recommend all users updating to this release, by bumping the version of your Flink dependencies and updating the binaries on the server.
-Issues fixed # [FLINK-2879] - Links in documentation are broken [FLINK-2938] - Streaming docs not in sync with latest state changes [FLINK-2942] - Dangling operators in web UI's program visualization (non-deterministic) [FLINK-2967] - TM address detection might not always detect the right interface on slow networks / overloaded JMs [FLINK-2977] - Cannot access HBase in a Kerberos secured Yarn cluster [FLINK-2987] - Flink 0." />
+Issues fixed # [FLINK-2879] - Links in documentation are broken [FLINK-2938] - Streaming docs not in sync with latest state changes [FLINK-2942] - Dangling operators in web UI's program visualization (non-deterministic) [FLINK-2967] - TM address detection might not always detect the right interface on slow networks / overloaded JMs [FLINK-2977] - Cannot access HBase in a Kerberos secured Yarn cluster [FLINK-2987] - Flink 0." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2015/11/27/flink-0.10.1-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-11-27T08:00:00+00:00" />
<meta property="article:modified_time" content="2015-11-27T08:00:00+00:00" />
-
<title>Flink 0.10.1 released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -60,7 +59,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -267,7 +266,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2015/12/04/introducing-stream-windows-in-apache-flink/index.html b/content/2015/12/04/introducing-stream-windows-in-apache-flink/index.html
index 327dc1532..0b5cda00b 100644
--- a/content/2015/12/04/introducing-stream-windows-in-apache-flink/index.html
+++ b/content/2015/12/04/introducing-stream-windows-in-apache-flink/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -22,18 +22,17 @@
<meta property="og:url" content="https://flink.apache.org/2015/12/04/introducing-stream-windows-in-apache-flink/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-12-04T10:00:00+00:00" />
<meta property="article:modified_time" content="2015-12-04T10:00:00+00:00" />
-
<title>Introducing Stream Windows in Apache Flink | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2015/12/11/storm-compatibility-in-apache-flink-how-to-run-existing-storm-topologies-on-flink/index.html b/content/2015/12/11/storm-compatibility-in-apache-flink-how-to-run-existing-storm-topologies-on-flink/index.html
index 2fd417b9a..3d330fce5 100644
--- a/content/2015/12/11/storm-compatibility-in-apache-flink-how-to-run-existing-storm-topologies-on-flink/index.html
+++ b/content/2015/12/11/storm-compatibility-in-apache-flink-how-to-run-existing-storm-topologies-on-flink/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -22,18 +22,17 @@
<meta property="og:url" content="https://flink.apache.org/2015/12/11/storm-compatibility-in-apache-flink-how-to-run-existing-storm-topologies-on-flink/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-12-11T10:00:00+00:00" />
<meta property="article:modified_time" content="2015-12-11T10:00:00+00:00" />
-
<title>Storm Compatibility in Apache Flink: How to run existing Storm topologies on Flink | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -518,36 +517,36 @@ However, it is powerful enough to cover many use cases.</p>
For this, you only need to replace the dependency <code>storm-core</code> by <code>flink-storm</code> in your Storm project and <strong>change two lines of code</strong> in your original Storm program.</p>
<p>The following example shows a simple Storm-Word-Count-Program that can be executed in Flink.
First, the program is assembled the Storm way without any code change to Spouts, Bolts, or the topology itself.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// assemble topology, the Storm way
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">TopologyBuilder</span> <span class="n">builder</span> <span class="o">=</span> <span class="k">new</span> <span class="n">TopologyBuilder</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"><span class="n">builder</span><span class="o">.</span><span class="na">setSpout</span><span class="o">(</span><span class="s">"source"</span><span class="o">,</span> <span class="k">new</span> <span class="n">StormFileSpout</span><span class="o">(</span><span class="n">inputFilePath</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"><span class="n">builder</span><span class="o">.</span><span class="na">setBolt</span><span class="o">(</span><span class="s">"tokenizer"</span><span class="o">,</span> <span class="k">new</span> <span class="n">StormBoltTokenizer</span><span class="o">())</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">shuffleGrouping</span><span class="o">(</span><span class="s">"source"</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"><span class="n">builder</span><span class="o">.</span><span class="na">setBolt</span><span class="o">(</span><span class="s">"counter"</span><span class="o">,</span> <span class="k">new</span> <span class="n">StormBoltCounter</span><span class="o">())</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">fieldsGrouping</span><span class="o">(</span><span class="s">"tokenizer"</span><span class="o">,</span> <span class="k">new</span> <span class="n">Fields</span><span class="o">(</span><span class="s">"word"</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"><span class="n">builder</span><span class="o">.</span><span class="na">setBolt</span><span class="o">(</span><span class="s">"sink"</span><span class="o">,</span> <span class="k">new</span> <span class="n">StormBoltFileSink</span><span class="o">(</span><span class="n">outputFilePath</span><span class="o">))</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">shuffleGrouping</span><span class="o">(</span><span class="s">"counter"</span><span class="o">);</span>
-</span></span></code></pre></div><p>In order to execute the topology, we need to translate it to a <code>FlinkTopology</code> and submit it to a local or remote Flink cluster, very similar to submitting the application to a Storm cluster.<sup><a href="#fn1" id="ref1">1</a></sup></p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// transform Storm topology to Flink program
-</span></span></span><span class="line"><span class="cl"><span class="c1">// replaces: StormTopology topology = builder.createTopology();
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">FlinkTopology</span> <span class="n">topology</span> <span class="o">=</span> <span class="n">FlinkTopology</span><span class="o">.</span><span class="na">createTopology</span><span class="o">(</span><span class="n">builder</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">Config</span> <span class="n">conf</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Config</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"><span class="k">if</span><span class="o">(</span><span class="n">runLocal</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// use FlinkLocalCluster instead of LocalCluster
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">FlinkLocalCluster</span> <span class="n">cluster</span> <span class="o">=</span> <span class="n">FlinkLocalCluster</span><span class="o">.</span><span class="na">getLocalCluster</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="n">cluster</span><span class="o">.</span><span class="na">submitTopology</span><span class="o">(</span><span class="s">"WordCount"</span><span class="o">,</span> <span class="n">conf</span><span class="o">,</span> <span class="n">topology</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span> <span class="k">else</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// use FlinkSubmitter instead of StormSubmitter
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">FlinkSubmitter</span><span class="o">.</span><span class="na">submitTopology</span><span class="o">(</span><span class="s">"WordCount"</span><span class="o">,</span> <span class="n">conf</span><span class="o">,</span> <span class="n">topology</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>As a shorter Flink-style alternative that replaces the Storm-style submission code, you can also use context-based job execution:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// transform Storm topology to Flink program (as above)
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">FlinkTopology</span> <span class="n">topology</span> <span class="o">=</span> <span class="n">FlinkTopology</span><span class="o">.</span><span class="na">createTopology</span><span class="o">(</span><span class="n">builder</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// executes locally by default or remotely if submitted with Flink's command-line client
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">topology</span><span class="o">.</span><span class="na">execute</span><span class="o">()</span>
-</span></span></code></pre></div><p>After the code is packaged in a jar file (e.g., <code>StormWordCount.jar</code>), it can be easily submitted to Flink via</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// assemble topology, the Storm way</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">TopologyBuilder</span><span class="w"> </span><span class="n">builder</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">TopologyBuilder</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">builder</span><span class="p">.</span><span class="na">setSpout</span><span class="p">(</span><span class="s">"source"</span><span class="p">,</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">StormFileSpout</span><span class="p">(</span><span class="n">inputFilePath</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">builder</span><span class="p">.</span><span class="na">setBolt</span><span class="p">(</span><span class="s">"tokenizer"</span><span class="p">,</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">StormBoltTokenizer</span><span class="p">())</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">shuffleGrouping</span><span class="p">(</span><span class="s">"source"</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">builder</span><span class="p">.</span><span class="na">setBolt</span><span class="p">(</span><span class="s">"counter"</span><span class="p">,</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">StormBoltCounter</span><span class="p">())</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">fieldsGrouping</span><span class="p">(</span><span class="s">"tokenizer"</span><span class="p">,</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Fields</span><span class="p">(</span><span class="s">"word"</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">builder</span><span class="p">.</span><span class="na">setBolt</span><span class="p">(</span><span class="s">"sink"</span><span class="p">,</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">StormBoltFileSink</span><span class="p">(</span><span class="n">outputFilePath</span><span class="p">))</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">shuffleGrouping</span><span class="p">(</span><span class="s">"counter"</span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><p>In order to execute the topology, we need to translate it to a <code>FlinkTopology</code> and submit it to a local or remote Flink cluster, very similar to submitting the application to a Storm cluster.<sup><a href="#fn1" id="ref1">1</a></sup></p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// transform Storm topology to Flink program</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// replaces: StormTopology topology = builder.createTopology();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">FlinkTopology</span><span class="w"> </span><span class="n">topology</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">FlinkTopology</span><span class="p">.</span><span class="na">createTopology</span><span class="p">(</span><span class="n">builder</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">Config</span><span class="w"> </span><span class="n">conf</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Config</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="k">if</span><span class="p">(</span><span class="n">runLocal</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// use FlinkLocalCluster instead of LocalCluster</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">FlinkLocalCluster</span><span class="w"> </span><span class="n">cluster</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">FlinkLocalCluster</span><span class="p">.</span><span class="na">getLocalCluster</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">cluster</span><span class="p">.</span><span class="na">submitTopology</span><span class="p">(</span><span class="s">"WordCount"</span><span class="p">,</span><span class="w"> </span><span class="n">conf</span><span class="p">,</span><span class="w"> </span><span class="n">topology</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w"> </span><span class="k">else</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// use FlinkSubmitter instead of StormSubmitter</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">FlinkSubmitter</span><span class="p">.</span><span class="na">submitTopology</span><span class="p">(</span><span class="s">"WordCount"</span><span class="p">,</span><span class="w"> </span><span class="n">conf</span><span class="p">,</span><span class="w"> </span><span class="n">topology</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>As a shorter Flink-style alternative that replaces the Storm-style submission code, you can also use context-based job execution:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// transform Storm topology to Flink program (as above)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">FlinkTopology</span><span class="w"> </span><span class="n">topology</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">FlinkTopology</span><span class="p">.</span><span class="na">createTopology</span><span class="p">(</span><span class="n">builder</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// executes locally by default or remotely if submitted with Flink's command-line client</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">topology</span><span class="p">.</span><span class="na">execute</span><span class="p">()</span><span class="w">
+</span></span></span></code></pre></div><p>After the code is packaged in a jar file (e.g., <code>StormWordCount.jar</code>), it can be easily submitted to Flink via</p>
<pre tabindex="0"><code>bin/flink run StormWordCount.jar
</code></pre><p>The used Spouts and Bolts as well as the topology assemble code is not changed at all!
Only the translation and submission step have to be changed to the Storm-API compatible Flink pendants.
@@ -560,35 +559,35 @@ This allows for minimal code changes and easy adaption to Flink.</p>
The compatibility package provides wrapper classes for Spouts and Bolts which are implemented as a Flink <code>SourceFunction</code> and <code>StreamOperator</code> respectively.
Those wrappers automatically translate incoming Flink POJO and <code>TupleXX</code> records into Storm’s <code>Tuple</code> type and emitted <code>Values</code> back into either POJOs or <code>TupleXX</code> types for further processing by Flink operators.
As Storm is type agnostic, it is required to specify the output type of embedded Spouts/Bolts manually to get a fully typed Flink streaming program.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// use regular Flink streaming environment
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">StreamExecutionEnvironment</span> <span class="n">env</span> <span class="o">=</span> <span class="n">StreamExecutionEnvironment</span><span class="o">.</span><span class="na">getExecutionEnvironment</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// use Spout as source
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Tuple1</span><span class="o"><</span><span class="n">String</span><span class="o">>></span> <span class="n">source</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="n">env</span><span class="o">.</span><span class="na">addSource</span><span class="o">(</span><span class="c1">// Flink provided wrapper including original Spout
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">new</span> <span class="n">SpoutWrapper</span><span class="o"><</span><span class="n">String</span><span class="o">>(</span><span class="k">new</span> <span class="n">FileSpout</span><span class="o">(</span><span class="n">localFilePath</span><span class="o">)),</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// specify output type manually
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">TypeExtractor</span><span class="o">.</span><span class="na">getForObject</span><span class="o">(</span><span class="k">new</span> <span class="n">Tuple1</span><span class="o"><</span><span class="n">String</span><span class="o">>(</span><span class="s">""</span><span class="o">)));</span>
-</span></span><span class="line"><span class="cl"><span class="c1">// FileSpout cannot be parallelized
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Tuple1</span><span class="o"><</span><span class="n">String</span><span class="o">>></span> <span class="n">text</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="na">setParallelism</span><span class="o">(</span><span class="n">1</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// further processing with Flink
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="o">,</span><span class="n">Integer</span><span class="o">></span> <span class="n">tokens</span> <span class="o">=</span> <span class="n">text</span><span class="o">.</span><span class="na">flatMap</span><span class="o">(</span><span class="k">new</sp [...]
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// use Bolt for counting
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="o">,</span><span class="n">Integer</span><span class="o">></span> <span class="n">counts</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="n">tokens</span><span class="o">.</span><span class="na">transform</span><span class="o">(</span><span class="s">"Counter"</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// specify output type manually
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">TypeExtractor</span><span class="o">.</span><span class="na">getForObject</span><span class="o">(</span><span class="k">new</span> <span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="o">,</span><span class="n">Integer</span><span class="o">>(</span><span class="s">""</span><span class="o">,</span><span class="n">0</sp [...]
-</span></span><span class="line"><span class="cl"> <span class="c1">// Flink provided wrapper including original Bolt
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">new</span> <span class="n">BoltWrapper</span><span class="o"><</span><span class="n">String</span><span class="o">,</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="o">,</span><span class="n">Integer</span><span class="o">>>(</span><span class="k">new</span> <span class="n">BoltCounter</span><span class="o">()) [...]
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// write result to file via Flink sink
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">counts</span><span class="o">.</span><span class="na">writeAsText</span><span class="o">(</span><span class="n">outputPath</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// start Flink job
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">env</span><span class="o">.</span><span class="na">execute</span><span class="o">(</span><span class="s">"WordCount with Spout source and Bolt counter"</span><span class="o">);</span>
-</span></span></code></pre></div><p>Although some boilerplate code is needed (we plan to address this soon!), the actual embedded Spout and Bolt code can be used unmodified.
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// use regular Flink streaming environment</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">StreamExecutionEnvironment</span><span class="w"> </span><span class="n">env</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">StreamExecutionEnvironment</span><span class="p">.</span><span class="na">getExecutionEnvironment</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// use Spout as source</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Tuple1</span><span class="o"><</span><span class="n">String</span><span class="o">>></span><span class="w"> </span><span class="n">source</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">env</span><span class="p">.</span><span class="na">addSource</span><span class="p">(</span><span class="c1">// Flink provided wrapper including original Spout</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">SpoutWrapper</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">FileSpout</span><span class="p">(</span><span class="n">localFilePath</span><span class="p">)),</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// specify output type manually</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">TypeExtractor</span><span class="p">.</span><span class="na">getForObject</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">Tuple1</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="p">(</span><span class="s">""</span><span class="p">)));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// FileSpout cannot be parallelized</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Tuple1</span><span class="o"><</span><span class="n">String</span><span class="o">>></span><span class="w"> </span><span class="n">text</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">source</span><span class="p">.</span><span class="na">setParallelism</span><span class="p">(</spa [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// further processing with Flink</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="n">Integer</span><span class="o">></span><span class="w"> </span><span class="n">tokens</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">text</span><span class="p">.</span><span clas [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// use Bolt for counting</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="n">Integer</span><span class="o">></span><span class="w"> </span><span class="n">counts</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">tokens</span><span class="p">.</span><span class="na">transform</span><span class="p">(</span><span class="s">"Counter"</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// specify output type manually</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">TypeExtractor</span><span class="p">.</span><span class="na">getForObject</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="n">Integer</span><span class="o">></span><span class="p">(</span><span class="s">""</span> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// Flink provided wrapper including original Bolt</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">BoltWrapper</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="n">Integer</span><span class="o">>></span><span class="p">(</span><span class="k">new</span><span cla [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// write result to file via Flink sink</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">counts</span><span class="p">.</span><span class="na">writeAsText</span><span class="p">(</span><span class="n">outputPath</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// start Flink job</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">env</span><span class="p">.</span><span class="na">execute</span><span class="p">(</span><span class="s">"WordCount with Spout source and Bolt counter"</span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><p>Although some boilerplate code is needed (we plan to address this soon!), the actual embedded Spout and Bolt code can be used unmodified.
We also note that the resulting program is fully typed, and type errors will be found by Flink’s type extractor even if the original Spouts and Bolts are not.</p>
<h2 id="outlook">
Outlook
diff --git a/content/2015/12/18/flink-2015-a-year-in-review-and-a-lookout-to-2016/index.html b/content/2015/12/18/flink-2015-a-year-in-review-and-a-lookout-to-2016/index.html
index 8841e8ce8..f1dd00b12 100644
--- a/content/2015/12/18/flink-2015-a-year-in-review-and-a-lookout-to-2016/index.html
+++ b/content/2015/12/18/flink-2015-a-year-in-review-and-a-lookout-to-2016/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ Overall, we have seen Flink grow in terms of functionality from an engine to one
<meta property="og:url" content="https://flink.apache.org/2015/12/18/flink-2015-a-year-in-review-and-a-lookout-to-2016/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2015-12-18T10:00:00+00:00" />
<meta property="article:modified_time" content="2015-12-18T10:00:00+00:00" />
-
<title>Flink 2015: A year in review, and a lookout to 2016 | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2016/02/11/flink-0.10.2-released/index.html b/content/2016/02/11/flink-0.10.2-released/index.html
index d3a6ba860..04a945430 100644
--- a/content/2016/02/11/flink-0.10.2-released/index.html
+++ b/content/2016/02/11/flink-0.10.2-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -17,27 +17,26 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="Today, the Flink community released Flink version 0.10.2, the second bugfix release of the 0.10 series.
We recommend all users updating to this release by bumping the version of your Flink dependencies to 0.10.2 and updating the binaries on the server.
-Issues fixed # FLINK-3242: Adjust StateBackendITCase for 0.10 signatures of state backends FLINK-3236: Flink user code classloader as parent classloader from Flink core classes FLINK-2962: Cluster startup script refers to unused variable FLINK-3151: Downgrade to Netty version 4.">
+Issues fixed # FLINK-3242: Adjust StateBackendITCase for 0.10 signatures of state backends FLINK-3236: Flink user code classloader as parent classloader from Flink core classes FLINK-2962: Cluster startup script refers to unused variable FLINK-3151: Downgrade to Netty version 4.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Flink 0.10.2 Released" />
<meta property="og:description" content="Today, the Flink community released Flink version 0.10.2, the second bugfix release of the 0.10 series.
We recommend all users updating to this release by bumping the version of your Flink dependencies to 0.10.2 and updating the binaries on the server.
-Issues fixed # FLINK-3242: Adjust StateBackendITCase for 0.10 signatures of state backends FLINK-3236: Flink user code classloader as parent classloader from Flink core classes FLINK-2962: Cluster startup script refers to unused variable FLINK-3151: Downgrade to Netty version 4." />
+Issues fixed # FLINK-3242: Adjust StateBackendITCase for 0.10 signatures of state backends FLINK-3236: Flink user code classloader as parent classloader from Flink core classes FLINK-2962: Cluster startup script refers to unused variable FLINK-3151: Downgrade to Netty version 4." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2016/02/11/flink-0.10.2-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2016-02-11T08:00:00+00:00" />
<meta property="article:modified_time" content="2016-02-11T08:00:00+00:00" />
-
<title>Flink 0.10.2 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -60,7 +59,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -267,7 +266,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2016/03/08/announcing-apache-flink-1.0.0/index.html b/content/2016/03/08/announcing-apache-flink-1.0.0/index.html
index f298d6bb0..161604753 100644
--- a/content/2016/03/08/announcing-apache-flink-1.0.0/index.html
+++ b/content/2016/03/08/announcing-apache-flink-1.0.0/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -16,26 +16,25 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="The Apache Flink community is pleased to announce the availability of the 1.0.0 release. The community put significant effort into improving and extending Apache Flink since the last release, focusing on improving the experience of writing and executing data stream processing pipelines in production.
- Flink version 1.0.0 marks the beginning of the 1.X.X series of releases, which will maintain backwards compatibility with 1.0.0. This means that applications written against stable APIs of Flink 1.">
+Flink version 1.0.0 marks the beginning of the 1.X.X series of releases, which will maintain backwards compatibility with 1.0.0. This means that applications written against stable APIs of Flink 1.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Announcing Apache Flink 1.0.0" />
<meta property="og:description" content="The Apache Flink community is pleased to announce the availability of the 1.0.0 release. The community put significant effort into improving and extending Apache Flink since the last release, focusing on improving the experience of writing and executing data stream processing pipelines in production.
- Flink version 1.0.0 marks the beginning of the 1.X.X series of releases, which will maintain backwards compatibility with 1.0.0. This means that applications written against stable APIs of Flink 1." />
+Flink version 1.0.0 marks the beginning of the 1.X.X series of releases, which will maintain backwards compatibility with 1.0.0. This means that applications written against stable APIs of Flink 1." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2016/03/08/announcing-apache-flink-1.0.0/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2016-03-08T13:00:00+00:00" />
<meta property="article:modified_time" content="2016-03-08T13:00:00+00:00" />
-
<title>Announcing Apache Flink 1.0.0 | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2016/04/06/flink-1.0.1-released/index.html b/content/2016/04/06/flink-1.0.1-released/index.html
index f961643c2..9d027d2a5 100644
--- a/content/2016/04/06/flink-1.0.1-released/index.html
+++ b/content/2016/04/06/flink-1.0.1-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -17,27 +17,26 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="Today, the Flink community released Flink version 1.0.1, the first bugfix release of the 1.0 series.
We recommend all users updating to this release by bumping the version of your Flink dependencies to 1.0.1 and updating the binaries on the server. You can find the binaries on the updated Downloads page.
-Fixed Issues # Bug [FLINK-3179] - Combiner is not injected if Reduce or GroupReduce input is explicitly partitioned [FLINK-3472] - JDBCInputFormat.">
+Fixed Issues # Bug [FLINK-3179] - Combiner is not injected if Reduce or GroupReduce input is explicitly partitioned [FLINK-3472] - JDBCInputFormat.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Flink 1.0.1 Released" />
<meta property="og:description" content="Today, the Flink community released Flink version 1.0.1, the first bugfix release of the 1.0 series.
We recommend all users updating to this release by bumping the version of your Flink dependencies to 1.0.1 and updating the binaries on the server. You can find the binaries on the updated Downloads page.
-Fixed Issues # Bug [FLINK-3179] - Combiner is not injected if Reduce or GroupReduce input is explicitly partitioned [FLINK-3472] - JDBCInputFormat." />
+Fixed Issues # Bug [FLINK-3179] - Combiner is not injected if Reduce or GroupReduce input is explicitly partitioned [FLINK-3472] - JDBCInputFormat." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2016/04/06/flink-1.0.1-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2016-04-06T08:00:00+00:00" />
<meta property="article:modified_time" content="2016-04-06T08:00:00+00:00" />
-
<title>Flink 1.0.1 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -60,7 +59,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -267,7 +266,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2016/04/06/introducing-complex-event-processing-cep-with-apache-flink/index.html b/content/2016/04/06/introducing-complex-event-processing-cep-with-apache-flink/index.html
index 94c2a66b8..d42e0f303 100644
--- a/content/2016/04/06/introducing-complex-event-processing-cep-with-apache-flink/index.html
+++ b/content/2016/04/06/introducing-complex-event-processing-cep-with-apache-flink/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -22,18 +22,17 @@
<meta property="og:url" content="https://flink.apache.org/2016/04/06/introducing-complex-event-processing-cep-with-apache-flink/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2016-04-06T10:00:00+00:00" />
<meta property="article:modified_time" content="2016-04-06T10:00:00+00:00" />
-
<title>Introducing Complex Event Processing (CEP) with Apache Flink | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -530,104 +529,104 @@ This alert can then lead to countermeasures to cool the rack.</p>
Every monitoring message contains its originating rack ID.
The temperature event additionally contains the current temperature and the power consumption event contains the current voltage.
We model the events as POJOs:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">abstract</span> <span class="kd">class</span> <span class="nc">MonitoringEvent</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="kt">int</span> <span class="n">rackID</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">...</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">TemperatureEvent</span> <span class="kd">extends</span> <span class="n">MonitoringEvent</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="kt">double</span> <span class="n">temperature</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">...</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">PowerEvent</span> <span class="kd">extends</span> <span class="n">MonitoringEvent</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="kt">double</span> <span class="n">voltage</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">...</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>Now we can ingest the monitoring event stream using one of Flink’s connectors (e.g. Kafka, RabbitMQ, etc.).
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">abstract</span><span class="w"> </span><span class="kd">class</span> <span class="nc">MonitoringEvent</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">rackID</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">...</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">TemperatureEvent</span><span class="w"> </span><span class="kd">extends</span><span class="w"> </span><span class="n">MonitoringEvent</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="kt">double</span><span class="w"> </span><span class="n">temperature</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">...</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">PowerEvent</span><span class="w"> </span><span class="kd">extends</span><span class="w"> </span><span class="n">MonitoringEvent</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="kt">double</span><span class="w"> </span><span class="n">voltage</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">...</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>Now we can ingest the monitoring event stream using one of Flink’s connectors (e.g. Kafka, RabbitMQ, etc.).
This will give us a <code>DataStream<MonitoringEvent> inputEventStream</code> which we will use as the input for Flink’s CEP operator.
But first, we have to define the event pattern to detect temperature warnings.
The CEP library offers an intuitive <a href="//nightlies.apache.org/flink/flink-docs-master/apis/streaming/libs/cep.html#the-pattern-api">Pattern API</a> to easily define these complex patterns.</p>
<p>Every pattern consists of a sequence of events which can have optional filter conditions assigned.
A pattern always starts with a first event to which we will assign the name <code>“First Event”</code>.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">Pattern</span><span class="o">.<</span><span class="n">MonitoringEvent</span><span class="o">></span><span class="n">begin</span><span class="o">(</span><span class="s">"First Event"</span><span class="o">);</span>
-</span></span></code></pre></div><p>This pattern will match every monitoring event.
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">Pattern</span><span class="p">.</span><span class="o"><</span><span class="n">MonitoringEvent</span><span class="o">></span><span class="n">begin</span><span class="p">(</span><span class="s">"First Event"</span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><p>This pattern will match every monitoring event.
Since we are only interested in <code>TemperatureEvents</code> whose temperature is above a threshold value, we have to add an additional subtype constraint and a where clause:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">Pattern</span><span class="o">.<</span><span class="n">MonitoringEvent</span><span class="o">></span><span class="n">begin</span><span class="o">(</span><span class="s">"First Event"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">subtype</span><span class="o">(</span><span class="n">TemperatureEvent</span><span class="o">.</span><span class="na">class</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">where</span><span class="o">(</span><span class="n">evt</span> <span class="o">-></span> <span class="n">evt</span><span class="o">.</span><span class="na">getTemperature</span><span class="o">()</span> <span class="o">>=</span> <span class="n">TEMPERATURE_THRESHOLD</span><span class="o">);</span>
-</span></span></code></pre></div><p>As stated before, we want to generate a <code>TemperatureWarning</code> if and only if we see two consecutive <code>TemperatureEvents</code> for the same rack whose temperatures are too high.
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">Pattern</span><span class="p">.</span><span class="o"><</span><span class="n">MonitoringEvent</span><span class="o">></span><span class="n">begin</span><span class="p">(</span><span class="s">"First Event"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">subtype</span><span class="p">(</span><span class="n">TemperatureEvent</span><span class="p">.</span><span class="na">class</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">where</span><span class="p">(</span><span class="n">evt</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">evt</span><span class="p">.</span><span class="na">getTemperature</span><span class="p">()</span><span class="w"> </span><span class="o">>=</span><span class="w"> </span><span class="n">TEMPERATURE_THRESHOLD</s [...]
+</span></span></span></code></pre></div><p>As stated before, we want to generate a <code>TemperatureWarning</code> if and only if we see two consecutive <code>TemperatureEvents</code> for the same rack whose temperatures are too high.
The Pattern API offers the <code>next</code> call which allows us to add a new event to our pattern.
This event has to follow directly the first matching event in order for the whole pattern to match.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">Pattern</span><span class="o"><</span><span class="n">MonitoringEvent</span><span class="o">,</span> <span class="o">?></span> <span class="n">warningPattern</span> <span class="o">=</span> <span class="n">Pattern</span><span class="o">.<</span><span class="n">MonitoringEvent</span><span class="o">></span><span class="n">begi [...]
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">subtype</span><span class="o">(</span><span class="n">TemperatureEvent</span><span class="o">.</span><span class="na">class</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">where</span><span class="o">(</span><span class="n">evt</span> <span class="o">-></span> <span class="n">evt</span><span class="o">.</span><span class="na">getTemperature</span><span class="o">()</span> <span class="o">>=</span> <span class="n">TEMPERATURE_THRESHOLD</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">next</span><span class="o">(</span><span class="s">"Second Event"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">subtype</span><span class="o">(</span><span class="n">TemperatureEvent</span><span class="o">.</span><span class="na">class</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">where</span><span class="o">(</span><span class="n">evt</span> <span class="o">-></span> <span class="n">evt</span><span class="o">.</span><span class="na">getTemperature</span><span class="o">()</span> <span class="o">>=</span> <span class="n">TEMPERATURE_THRESHOLD</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">within</span><span class="o">(</span><span class="n">Time</span><span class="o">.</span><span class="na">seconds</span><span class="o">(</span><span class="n">10</span><span class="o">));</span>
-</span></span></code></pre></div><p>The final pattern definition also contains the <code>within</code> API call which defines that two consecutive <code>TemperatureEvents</code> have to occur within a time interval of 10 seconds for the pattern to match.
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">Pattern</span><span class="o"><</span><span class="n">MonitoringEvent</span><span class="p">,</span><span class="w"> </span><span class="o">?></span><span class="w"> </span><span class="n">warningPattern</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Pattern</span><span class="p">.</sp [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">subtype</span><span class="p">(</span><span class="n">TemperatureEvent</span><span class="p">.</span><span class="na">class</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">where</span><span class="p">(</span><span class="n">evt</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">evt</span><span class="p">.</span><span class="na">getTemperature</span><span class="p">()</span><span class="w"> </span><span class="o">>=</span><span class="w"> </span><span class="n">TEMPERATURE_THRESHOLD</s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">next</span><span class="p">(</span><span class="s">"Second Event"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">subtype</span><span class="p">(</span><span class="n">TemperatureEvent</span><span class="p">.</span><span class="na">class</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">where</span><span class="p">(</span><span class="n">evt</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">evt</span><span class="p">.</span><span class="na">getTemperature</span><span class="p">()</span><span class="w"> </span><span class="o">>=</span><span class="w"> </span><span class="n">TEMPERATURE_THRESHOLD</s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">within</span><span class="p">(</span><span class="n">Time</span><span class="p">.</span><span class="na">seconds</span><span class="p">(</span><span class="n">10</span><span class="p">));</span><span class="w">
+</span></span></span></code></pre></div><p>The final pattern definition also contains the <code>within</code> API call which defines that two consecutive <code>TemperatureEvents</code> have to occur within a time interval of 10 seconds for the pattern to match.
Depending on the time characteristic setting, this can either be processing, ingestion or event time.</p>
<p>Having defined the event pattern, we can now apply it on the <code>inputEventStream</code>.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">PatternStream</span><span class="o"><</span><span class="n">MonitoringEvent</span><span class="o">></span> <span class="n">tempPatternStream</span> <span class="o">=</span> <span class="n">CEP</span><span class="o">.</span><span class="na">pattern</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">inputEventStream</span><span class="o">.</span><span class="na">keyBy</span><span class="o">(</span><span class="s">"rackID"</span><span class="o">),</span>
-</span></span><span class="line"><span class="cl"> <span class="n">warningPattern</span><span class="o">);</span>
-</span></span></code></pre></div><p>Since we want to generate our warnings for each rack individually, we <code>keyBy</code> the input event stream by the <code>“rackID”</code> POJO field.
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">PatternStream</span><span class="o"><</span><span class="n">MonitoringEvent</span><span class="o">></span><span class="w"> </span><span class="n">tempPatternStream</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">CEP</span><span class="p">.</span><span class="na">pattern</span><span clas [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">inputEventStream</span><span class="p">.</span><span class="na">keyBy</span><span class="p">(</span><span class="s">"rackID"</span><span class="p">),</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">warningPattern</span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><p>Since we want to generate our warnings for each rack individually, we <code>keyBy</code> the input event stream by the <code>“rackID”</code> POJO field.
This enforces that matching events of our pattern will all have the same rack ID.</p>
<p>The <code>PatternStream<MonitoringEvent></code> gives us access to successfully matched event sequences.
They can be accessed using the <code>select</code> API call.
The <code>select</code> API call takes a <code>PatternSelectFunction</code> which is called for every matching event sequence.
The event sequence is provided as a <code>Map<String, MonitoringEvent></code> where each <code>MonitoringEvent</code> is identified by its assigned event name.
Our pattern select function generates for each matching pattern a <code>TemperatureWarning</code> event.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">TemperatureWarning</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="kt">int</span> <span class="n">rackID</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="kt">double</span> <span class="n">averageTemperature</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">...</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">TemperatureWarning</span><span class="o">></span> <span class="n">warnings</span> <span class="o">=</span> <span class="n">tempPatternStream</span><span class="o">.</span><span class="na">select</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="o">(</span><span class="n">Map</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">MonitoringEvent</span><span class="o">></span> <span class="n">pattern</span><span class="o">)</span> <span class="o">-></span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">TemperatureEvent</span> <span class="n">first</span> <span class="o">=</span> <span class="o">(</span><span class="n">TemperatureEvent</span><span class="o">)</span> <span class="n">pattern</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="s">"First Event"</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="n">TemperatureEvent</span> <span class="n">second</span> <span class="o">=</span> <span class="o">(</span><span class="n">TemperatureEvent</span><span class="o">)</span> <span class="n">pattern</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="s">"Second Event"</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="k">new</span> <span class="n">TemperatureWarning</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">first</span><span class="o">.</span><span class="na">getRackID</span><span class="o">(),</span>
-</span></span><span class="line"><span class="cl"> <span class="o">(</span><span class="n">first</span><span class="o">.</span><span class="na">getTemperature</span><span class="o">()</span> <span class="o">+</span> <span class="n">second</span><span class="o">.</span><span class="na">getTemperature</span><span class="o">())</span> <span class="o">/</span> <span class="n">2</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">);</span>
-</span></span></code></pre></div><p>Now we have generated a new complex event stream <code>DataStream<TemperatureWarning> warnings</code> from the initial monitoring event stream.
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">TemperatureWarning</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">rackID</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="kt">double</span><span class="w"> </span><span class="n">averageTemperature</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">...</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">TemperatureWarning</span><span class="o">></span><span class="w"> </span><span class="n">warnings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tempPatternStream</span><span class="p">.</span><span class="na">select</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">(</span><span class="n">Map</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">MonitoringEvent</span><span class="o">></span><span class="w"> </span><span class="n">pattern</span><span class="p">)</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">TemperatureEvent</span><span class="w"> </span><span class="n">first</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="n">TemperatureEvent</span><span class="p">)</span><span class="w"> </span><span class="n">pattern</span><span class="p">.</span><span class="na">get</span><span class="p">(</span><span class="s">"Firs [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">TemperatureEvent</span><span class="w"> </span><span class="n">second</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="n">TemperatureEvent</span><span class="p">)</span><span class="w"> </span><span class="n">pattern</span><span class="p">.</span><span class="na">get</span><span class="p">(</span><span class="s">"Sec [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">TemperatureWarning</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">first</span><span class="p">.</span><span class="na">getRackID</span><span class="p">(),</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">(</span><span class="n">first</span><span class="p">.</span><span class="na">getTemperature</span><span class="p">()</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">second</span><span class="p">.</span><span class="na">getTemperature</span><span class="p">())</span><span class="w"> </span><span class="o">/</span><span class="w"> </span> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><p>Now we have generated a new complex event stream <code>DataStream<TemperatureWarning> warnings</code> from the initial monitoring event stream.
This complex event stream can again be used as the input for another round of complex event processing.
We use the <code>TemperatureWarnings</code> to generate <code>TemperatureAlerts</code> whenever we see two consecutive <code>TemperatureWarnings</code> for the same rack with increasing temperatures.
The <code>TemperatureAlerts</code> have the following definition:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">TemperatureAlert</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="kt">int</span> <span class="n">rackID</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">...</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>At first, we have to define our alert event pattern:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">Pattern</span><span class="o"><</span><span class="n">TemperatureWarning</span><span class="o">,</span> <span class="o">?></span> <span class="n">alertPattern</span> <span class="o">=</span> <span class="n">Pattern</span><span class="o">.<</span><span class="n">TemperatureWarning</span><span class="o">></span><span class="n"> [...]
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">next</span><span class="o">(</span><span class="s">"Second Event"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">within</span><span class="o">(</span><span class="n">Time</span><span class="o">.</span><span class="na">seconds</span><span class="o">(</span><span class="n">20</span><span class="o">));</span>
-</span></span></code></pre></div><p>This definition says that we want to see two <code>TemperatureWarnings</code> within 20 seconds.
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">TemperatureAlert</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">rackID</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">...</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>At first, we have to define our alert event pattern:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">Pattern</span><span class="o"><</span><span class="n">TemperatureWarning</span><span class="p">,</span><span class="w"> </span><span class="o">?></span><span class="w"> </span><span class="n">alertPattern</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Pattern</span><span class="p">.</s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">next</span><span class="p">(</span><span class="s">"Second Event"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">within</span><span class="p">(</span><span class="n">Time</span><span class="p">.</span><span class="na">seconds</span><span class="p">(</span><span class="n">20</span><span class="p">));</span><span class="w">
+</span></span></span></code></pre></div><p>This definition says that we want to see two <code>TemperatureWarnings</code> within 20 seconds.
The first event has the name <code>“First Event”</code> and the second consecutive event has the name <code>“Second Event”</code>.
The individual events don’t have a where clause assigned, because we need access to both events in order to decide whether the temperature is increasing.
Therefore, we apply the filter condition in the select clause.
But first, we obtain again a <code>PatternStream</code>.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">PatternStream</span><span class="o"><</span><span class="n">TemperatureWarning</span><span class="o">></span> <span class="n">alertPatternStream</span> <span class="o">=</span> <span class="n">CEP</span><span class="o">.</span><span class="na">pattern</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">warnings</span><span class="o">.</span><span class="na">keyBy</span><span class="o">(</span><span class="s">"rackID"</span><span class="o">),</span>
-</span></span><span class="line"><span class="cl"> <span class="n">alertPattern</span><span class="o">);</span>
-</span></span></code></pre></div><p>Again, we <code>keyBy</code> the warnings input stream by the <code>"rackID"</code> so that we generate our alerts for each rack individually.
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">PatternStream</span><span class="o"><</span><span class="n">TemperatureWarning</span><span class="o">></span><span class="w"> </span><span class="n">alertPatternStream</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">CEP</span><span class="p">.</span><span class="na">pattern</span><span [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">warnings</span><span class="p">.</span><span class="na">keyBy</span><span class="p">(</span><span class="s">"rackID"</span><span class="p">),</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">alertPattern</span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><p>Again, we <code>keyBy</code> the warnings input stream by the <code>"rackID"</code> so that we generate our alerts for each rack individually.
Next we apply the <code>flatSelect</code> method which will give us access to matching event sequences and allows us to output an arbitrary number of complex events.
Thus, we will only generate a <code>TemperatureAlert</code> if and only if the temperature is increasing.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">TemperatureAlert</span><span class="o">></span> <span class="n">alerts</span> <span class="o">=</span> <span class="n">alertPatternStream</span><span class="o">.</span><span class="na">flatSelect</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="o">(</span><span class="n">Map</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">TemperatureWarning</span><span class="o">></span> <span class="n">pattern</span><span class="o">,</span> <span class="n">Collector</span><span class="o"><</span><span class="n">TemperatureAlert</span><span class="o">></span> <span class="n">out</span><span class="o">)</span> <span [...]
-</span></span><span class="line"><span class="cl"> <span class="n">TemperatureWarning</span> <span class="n">first</span> <span class="o">=</span> <span class="n">pattern</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="s">"First Event"</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="n">TemperatureWarning</span> <span class="n">second</span> <span class="o">=</span> <span class="n">pattern</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="s">"Second Event"</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">(</span><span class="n">first</span><span class="o">.</span><span class="na">getAverageTemperature</span><span class="o">()</span> <span class="o"><</span> <span class="n">second</span><span class="o">.</span><span class="na">getAverageTemperature</span><span class="o">())</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">out</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="k">new</span> <span class="n">TemperatureAlert</span><span class="o">(</span><span class="n">first</span><span class="o">.</span><span class="na">getRackID</span><span class="o">()));</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">});</span>
-</span></span></code></pre></div><p>The <code>DataStream<TemperatureAlert> alerts</code> is the data stream of temperature alerts for each rack.
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">TemperatureAlert</span><span class="o">></span><span class="w"> </span><span class="n">alerts</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">alertPatternStream</span><span class="p">.</span><span class="na">flatSelect</span><span [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">(</span><span class="n">Map</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">TemperatureWarning</span><span class="o">></span><span class="w"> </span><span class="n">pattern</span><span class="p">,</span><span class="w"> </span><span class="n">Collector</span><span class="o"><</span><span class="n">Temperatu [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">TemperatureWarning</span><span class="w"> </span><span class="n">first</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">pattern</span><span class="p">.</span><span class="na">get</span><span class="p">(</span><span class="s">"First Event"</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">TemperatureWarning</span><span class="w"> </span><span class="n">second</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">pattern</span><span class="p">.</span><span class="na">get</span><span class="p">(</span><span class="s">"Second Event"</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">first</span><span class="p">.</span><span class="na">getAverageTemperature</span><span class="p">()</span><span class="w"> </span><span class="o"><</span><span class="w"> </span><span class="n">second</span><span class="p">.</span><span class="na">getAverageTemperature</span><span class="p">())</span><span clas [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">out</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">TemperatureAlert</span><span class="p">(</span><span class="n">first</span><span class="p">.</span><span class="na">getRackID</span><span class="p">()));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">});</span><span class="w">
+</span></span></span></code></pre></div><p>The <code>DataStream<TemperatureAlert> alerts</code> is the data stream of temperature alerts for each rack.
Based on these alerts we can now adapt the workload or cooling for overheating racks.</p>
<p>The full source code for the presented example as well as an example data source which generates randomly monitoring events can be found in <a href="https://github.com/tillrohrmann/cep-monitoring">this repository</a>.</p>
<h2 id="conclusion">
diff --git a/content/2016/04/14/flink-forward-2016-call-for-submissions-is-now-open/index.html b/content/2016/04/14/flink-forward-2016-call-for-submissions-is-now-open/index.html
index 61138c6e3..1372f2847 100644
--- a/content/2016/04/14/flink-forward-2016-call-for-submissions-is-now-open/index.html
+++ b/content/2016/04/14/flink-forward-2016-call-for-submissions-is-now-open/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ The conference welcomes submissions on everything Flink-related, including exper
<meta property="og:url" content="https://flink.apache.org/2016/04/14/flink-forward-2016-call-for-submissions-is-now-open/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2016-04-14T10:00:00+00:00" />
<meta property="article:modified_time" content="2016-04-14T10:00:00+00:00" />
-
<title>Flink Forward 2016 Call for Submissions Is Now Open | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2016/04/22/flink-1.0.2-released/index.html b/content/2016/04/22/flink-1.0.2-released/index.html
index 7f5573c0c..8fd2756b8 100644
--- a/content/2016/04/22/flink-1.0.2-released/index.html
+++ b/content/2016/04/22/flink-1.0.2-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -17,27 +17,26 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="Today, the Flink community released Flink version 1.0.2, the second bugfix release of the 1.0 series.
We recommend all users updating to this release by bumping the version of your Flink dependencies to 1.0.2 and updating the binaries on the server. You can find the binaries on the updated Downloads page.
-Fixed Issues # Bug # [FLINK-3657] [dataSet] Change access of DataSetUtils.countElements() to ‘public’ [FLINK-3762] [core] Enable Kryo reference tracking [FLINK-3732] [core] Fix potential null deference in ExecutionConfig#equals() [FLINK-3760] Fix StateDescriptor.">
+Fixed Issues # Bug # [FLINK-3657] [dataSet] Change access of DataSetUtils.countElements() to ‘public’ [FLINK-3762] [core] Enable Kryo reference tracking [FLINK-3732] [core] Fix potential null deference in ExecutionConfig#equals() [FLINK-3760] Fix StateDescriptor.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Flink 1.0.2 Released" />
<meta property="og:description" content="Today, the Flink community released Flink version 1.0.2, the second bugfix release of the 1.0 series.
We recommend all users updating to this release by bumping the version of your Flink dependencies to 1.0.2 and updating the binaries on the server. You can find the binaries on the updated Downloads page.
-Fixed Issues # Bug # [FLINK-3657] [dataSet] Change access of DataSetUtils.countElements() to ‘public’ [FLINK-3762] [core] Enable Kryo reference tracking [FLINK-3732] [core] Fix potential null deference in ExecutionConfig#equals() [FLINK-3760] Fix StateDescriptor." />
+Fixed Issues # Bug # [FLINK-3657] [dataSet] Change access of DataSetUtils.countElements() to ‘public’ [FLINK-3762] [core] Enable Kryo reference tracking [FLINK-3732] [core] Fix potential null deference in ExecutionConfig#equals() [FLINK-3760] Fix StateDescriptor." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2016/04/22/flink-1.0.2-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2016-04-22T08:00:00+00:00" />
<meta property="article:modified_time" content="2016-04-22T08:00:00+00:00" />
-
<title>Flink 1.0.2 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -60,7 +59,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -267,7 +266,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2016/05/11/flink-1.0.3-released/index.html b/content/2016/05/11/flink-1.0.3-released/index.html
index 6f39bb1dc..db02349f5 100644
--- a/content/2016/05/11/flink-1.0.3-released/index.html
+++ b/content/2016/05/11/flink-1.0.3-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -17,27 +17,26 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="Today, the Flink community released Flink version 1.0.3, the third bugfix release of the 1.0 series.
We recommend all users updating to this release by bumping the version of your Flink dependencies to 1.0.3 and updating the binaries on the server. You can find the binaries on the updated Downloads page.
-Fixed Issues # Bug # [FLINK-3790] [streaming] Use proper hadoop config in rolling sink [FLINK-3840] Remove Testing Files in RocksDB Backend [FLINK-3835] [optimizer] Add input id to JSON plan to resolve ambiguous input names [hotfix] OptionSerializer.">
+Fixed Issues # Bug # [FLINK-3790] [streaming] Use proper hadoop config in rolling sink [FLINK-3840] Remove Testing Files in RocksDB Backend [FLINK-3835] [optimizer] Add input id to JSON plan to resolve ambiguous input names [hotfix] OptionSerializer.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Flink 1.0.3 Released" />
<meta property="og:description" content="Today, the Flink community released Flink version 1.0.3, the third bugfix release of the 1.0 series.
We recommend all users updating to this release by bumping the version of your Flink dependencies to 1.0.3 and updating the binaries on the server. You can find the binaries on the updated Downloads page.
-Fixed Issues # Bug # [FLINK-3790] [streaming] Use proper hadoop config in rolling sink [FLINK-3840] Remove Testing Files in RocksDB Backend [FLINK-3835] [optimizer] Add input id to JSON plan to resolve ambiguous input names [hotfix] OptionSerializer." />
+Fixed Issues # Bug # [FLINK-3790] [streaming] Use proper hadoop config in rolling sink [FLINK-3840] Remove Testing Files in RocksDB Backend [FLINK-3835] [optimizer] Add input id to JSON plan to resolve ambiguous input names [hotfix] OptionSerializer." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2016/05/11/flink-1.0.3-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2016-05-11T08:00:00+00:00" />
<meta property="article:modified_time" content="2016-05-11T08:00:00+00:00" />
-
<title>Flink 1.0.3 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -60,7 +59,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -267,7 +266,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2016/05/24/stream-processing-for-everyone-with-sql-and-apache-flink/index.html b/content/2016/05/24/stream-processing-for-everyone-with-sql-and-apache-flink/index.html
index 7a8c25e1e..7163b935d 100644
--- a/content/2016/05/24/stream-processing-for-everyone-with-sql-and-apache-flink/index.html
+++ b/content/2016/05/24/stream-processing-for-everyone-with-sql-and-apache-flink/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -22,18 +22,17 @@
<meta property="og:url" content="https://flink.apache.org/2016/05/24/stream-processing-for-everyone-with-sql-and-apache-flink/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2016-05-24T10:00:00+00:00" />
<meta property="article:modified_time" content="2016-05-24T10:00:00+00:00" />
-
<title>Stream Processing for Everyone with SQL and Apache Flink | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2016/08/04/announcing-apache-flink-1.1.0/index.html b/content/2016/08/04/announcing-apache-flink-1.1.0/index.html
index 7c7943e44..19338219e 100644
--- a/content/2016/08/04/announcing-apache-flink-1.1.0/index.html
+++ b/content/2016/08/04/announcing-apache-flink-1.1.0/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ This release is the first major release in the 1.X.X series of releases, which m
<meta property="og:url" content="https://flink.apache.org/2016/08/04/announcing-apache-flink-1.1.0/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2016-08-04T13:00:00+00:00" />
<meta property="article:modified_time" content="2016-08-04T13:00:00+00:00" />
-
<title>Announcing Apache Flink 1.1.0 | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -504,42 +503,42 @@ https://github.com/alex-shpak/hugo-book
<a class="anchor" href="#continuous-file-system-sources">#</a>
</h3>
<p>A frequently requested feature for Flink 1.0 was to be able to monitor directories and process files continuously. Flink 1.1 now adds support for this via <code>FileProcessingMode</code>s:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">stream</span> <span class="o">=</span> <span class="n">env</span><span class="o">.</span><span class="na">readFile</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">textInputFormat</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="s">"hdfs:///file-path"</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">FileProcessingMode</span><span class="o">.</span><span class="na">PROCESS_CONTINUOUSLY</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">5000</span><span class="o">,</span> <span class="c1">// monitoring interval (millis)
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">FilePathFilter</span><span class="o">.</span><span class="na">createDefaultFilter</span><span class="o">());</span> <span class="c1">// file path filter
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">stream</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">env</span><span class="p">.</span><span class="na">readFile</span><span class="p">(</span><span cl [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">textInputFormat</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="s">"hdfs:///file-path"</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">FileProcessingMode</span><span class="p">.</span><span class="na">PROCESS_CONTINUOUSLY</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">5000</span><span class="p">,</span><span class="w"> </span><span class="c1">// monitoring interval (millis)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">FilePathFilter</span><span class="p">.</span><span class="na">createDefaultFilter</span><span class="p">());</span><span class="w"> </span><span class="c1">// file path filter</span><span class="w">
</span></span></span></code></pre></div><p>This will monitor <code>hdfs:///file-path</code> every <code>5000</code> milliseconds. Check out the <a href="//nightlies.apache.org/flink/flink-docs-release-1.1/apis/streaming/index.html#data-sources">DataSource documentation for more details</a>.</p>
<h3 id="kinesis-source-and-sink">
Kinesis Source and Sink
<a class="anchor" href="#kinesis-source-and-sink">#</a>
</h3>
<p>Flink 1.1 adds a Kinesis connector for both consuming (<code>FlinkKinesisConsumer</code>) from and producing (<code>FlinkKinesisProduer</code>) to <a href="https://aws.amazon.com/kinesis/">Amazon Kinesis Streams</a>, which is a managed service purpose-built to make it easy to work with streaming data on AWS.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">kinesis</span> <span class="o">=</span> <span class="n">env</span><span class="o">.</span><span class="na">addSource</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">FlinkKinesisConsumer</span><span class="o"><>(</span><span class="s">"stream-name"</span><span class="o">,</span> <span class="n">schema</span><span class="o">,</span> <span class="n">config</span><span class="o">));</span>
-</span></span></code></pre></div><p>Check out the <a href="//nightlies.apache.org/flink/flink-docs-release-1.1/apis/streaming/connectors/kinesis.html">Kinesis connector documentation for more details</a>.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">kinesis</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">env</span><span class="p">.</span><span class="na">addSource</span><span class="p">(</span><span [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">FlinkKinesisConsumer</span><span class="o"><></span><span class="p">(</span><span class="s">"stream-name"</span><span class="p">,</span><span class="w"> </span><span class="n">schema</span><span class="p">,</span><span class="w"> </span><span class="n">config</span><span class="p">));</span><span class="w">
+</span></span></span></code></pre></div><p>Check out the <a href="//nightlies.apache.org/flink/flink-docs-release-1.1/apis/streaming/connectors/kinesis.html">Kinesis connector documentation for more details</a>.</p>
<h3 id="cassandra-sink">
Cassandra Sink
<a class="anchor" href="#cassandra-sink">#</a>
</h3>
<p>The <a href="http://wiki.apache.org/cassandra/GettingStarted">Apache Cassandra</a> sink allows you to write from Flink to Cassandra. Flink can provide exactly-once guarantees if the query is idempotent, meaning it can be applied multiple times without changing the result.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">CassandraSink</span><span class="o">.</span><span class="na">addSink</span><span class="o">(</span><span class="n">input</span><span class="o">)</span>
-</span></span></code></pre></div><p>Check out the <a href="//nightlies.apache.org/flink/flink-docs-release-1.1/apis/streaming/connectors/cassandra.html">Cassandra Sink documentation for more details</a>.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">CassandraSink</span><span class="p">.</span><span class="na">addSink</span><span class="p">(</span><span class="n">input</span><span class="p">)</span><span class="w">
+</span></span></span></code></pre></div><p>Check out the <a href="//nightlies.apache.org/flink/flink-docs-release-1.1/apis/streaming/connectors/cassandra.html">Cassandra Sink documentation for more details</a>.</p>
<h2 id="table-api-and-sql">
Table API and SQL
<a class="anchor" href="#table-api-and-sql">#</a>
</h2>
<p>The Table API is a SQL-like expression language for relational stream and batch processing that can be easily embedded in Flink’s DataSet and DataStream APIs (for both Java and Scala).</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">Table</span> <span class="n">custT</span> <span class="o">=</span> <span class="n">tableEnv</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">toTable</span><span class="o">(</span><span class="n">custDs</span><span class="o">,</span> <span class="s">"name, zipcode"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">where</span><span class="o">(</span><span class="s">"zipcode = '12345'"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">select</span><span class="o">(</span><span class="s">"name"</span><span class="o">)</span>
-</span></span></code></pre></div><p>An initial version of this API was already available in Flink 1.0. For Flink 1.1, the community put a lot of work into reworking the architecture of the Table API and integrating it with <a href="https://calcite.apache.org">Apache Calcite</a>.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">Table</span><span class="w"> </span><span class="n">custT</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tableEnv</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">toTable</span><span class="p">(</span><span class="n">custDs</span><span class="p">,</span><span class="w"> </span><span class="s">"name, zipcode"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">where</span><span class="p">(</span><span class="s">"zipcode = '12345'"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">select</span><span class="p">(</span><span class="s">"name"</span><span class="p">)</span><span class="w">
+</span></span></span></code></pre></div><p>An initial version of this API was already available in Flink 1.0. For Flink 1.1, the community put a lot of work into reworking the architecture of the Table API and integrating it with <a href="https://calcite.apache.org">Apache Calcite</a>.</p>
<p>In this first version, SQL (and Table API) queries on streams are limited to selection, filter, and union operators. Compared to Flink 1.0, the revised Table API supports many more scalar functions and is able to read tables from external sources and write them back to external sinks.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">Table</span> <span class="n">result</span> <span class="o">=</span> <span class="n">tableEnv</span><span class="o">.</span><span class="na">sql</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="s">"SELECT STREAM product, amount FROM Orders WHERE product LIKE '%Rubber%'"</span><span class="o">);</span>
-</span></span></code></pre></div><p>A more detailed introduction can be found in the <a href="http://flink.apache.org/news/2016/05/24/stream-sql.html">Flink blog</a> and the <a href="//nightlies.apache.org/flink/flink-docs-release-1.1/apis/table.html">Table API documentation</a>.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">Table</span><span class="w"> </span><span class="n">result</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tableEnv</span><span class="p">.</span><span class="na">sql</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="s">"SELECT STREAM product, amount FROM Orders WHERE product LIKE '%Rubber%'"</span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><p>A more detailed introduction can be found in the <a href="http://flink.apache.org/news/2016/05/24/stream-sql.html">Flink blog</a> and the <a href="//nightlies.apache.org/flink/flink-docs-release-1.1/apis/table.html">Table API documentation</a>.</p>
<h2 id="datastream-api">
DataStream API
<a class="anchor" href="#datastream-api">#</a>
@@ -553,18 +552,18 @@ https://github.com/alex-shpak/hugo-book
<center>
<img src="/img/blog/session-windows.svg" style="height:400px">
</center>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">input</span><span class="o">.</span><span class="na">keyBy</span><span class="o">(<</span><span class="n">key</span> <span class="n">selector</span><span class="o">>)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">window</span><span class="o">(</span><span class="n">EventTimeSessionWindows</span><span class="o">.</span><span class="na">withGap</span><span class="o">(</span><span class="n">Time</span><span class="o">.</span><span class="na">minutes</span><span class="o">(</span><span class="n">10</span><span class="o">)))</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.<</span><span class="n">windowed</span> <span class="n">transformation</span><span class="o">>(<</span><span class="n">window</span> <span class="n">function</span><span class="o">>);</span>
-</span></span></code></pre></div><h3 id="support-for-late-elements">
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">input</span><span class="p">.</span><span class="na">keyBy</span><span class="p">(</span><span class="o"><</span><span class="n">key</span><span class="w"> </span><span class="n">selector</span><span class="o">></span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">window</span><span class="p">(</span><span class="n">EventTimeSessionWindows</span><span class="p">.</span><span class="na">withGap</span><span class="p">(</span><span class="n">Time</span><span class="p">.</span><span class="na">minutes</span><span class="p">(</span><span class="n">10</span><span class="p">)))</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="o"><</span><span class="n">windowed</span><span class="w"> </span><span class="n">transformation</span><span class="o">></span><span class="p">(</span><span class="o"><</span><span class="n">window</span><span class="w"> </span><span class="n">function</span><span class="o">></span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><h3 id="support-for-late-elements">
Support for Late Elements
<a class="anchor" href="#support-for-late-elements">#</a>
</h3>
<p>You can now specify how a windowed transformation should deal with late elements and how much lateness is allowed. The parameter for this is called <em>allowed lateness</em>. This specifies by how much time elements can be late.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">input</span><span class="o">.</span><span class="na">keyBy</span><span class="o">(<</span><span class="n">key</span> <span class="n">selector</span><span class="o">>).</span><span class="na">window</span><span class="o">(<</span><span class="n">window</span> <span class="n">assigner</span><span class="o">>)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">allowedLateness</span><span class="o">(<</span><span class="n">time</span><span class="o">>)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.<</span><span class="n">windowed</span> <span class="n">transformation</span><span class="o">>(<</span><span class="n">window</span> <span class="n">function</span><span class="o">>);</span>
-</span></span></code></pre></div><p>Elements that arrive within the allowed lateness are still put into windows and are considered when computing window results. If elements arrive after the allowed lateness they will be dropped. Flink will also make sure that any state held by the windowing operation is garbage collected once the watermark passes the end of a window plus the allowed lateness.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">input</span><span class="p">.</span><span class="na">keyBy</span><span class="p">(</span><span class="o"><</span><span class="n">key</span><span class="w"> </span><span class="n">selector</span><span class="o">></span><span class="p">).</span><span class="na">window</span><span class="p">(</span><span class="o"><</span><span cla [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">allowedLateness</span><span class="p">(</span><span class="o"><</span><span class="n">time</span><span class="o">></span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="o"><</span><span class="n">windowed</span><span class="w"> </span><span class="n">transformation</span><span class="o">></span><span class="p">(</span><span class="o"><</span><span class="n">window</span><span class="w"> </span><span class="n">function</span><span class="o">></span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><p>Elements that arrive within the allowed lateness are still put into windows and are considered when computing window results. If elements arrive after the allowed lateness they will be dropped. Flink will also make sure that any state held by the windowing operation is garbage collected once the watermark passes the end of a window plus the allowed lateness.</p>
<p>Check out the <a href="//nightlies.apache.org/flink/flink-docs-release-1.1/apis/streaming/windows.html">Windows documentation for more details</a>.</p>
<h2 id="scala-api-for-complex-event-processing-cep">
Scala API for Complex Event Processing (CEP)
@@ -583,10 +582,10 @@ https://github.com/alex-shpak/hugo-book
<a class="anchor" href="#metrics">#</a>
</h2>
<p>Flink’s new metrics system allows you to easily gather and expose metrics from your user application to external systems. You can add counters, gauges, and histograms to your application via the runtime context:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">Counter</span> <span class="n">counter</span> <span class="o">=</span> <span class="n">getRuntimeContext</span><span class="o">()</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">getMetricGroup</span><span class="o">()</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">counter</span><span class="o">(</span><span class="s">"my-counter"</span><span class="o">);</span>
-</span></span></code></pre></div><p>All registered metrics will be exposed via reporters. Out of the box, Flinks comes with support for JMX, Ganglia, Graphite, and statsD. In addition to your custom metrics, Flink exposes many internal metrics like checkpoint sizes and JVM stats.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">Counter</span><span class="w"> </span><span class="n">counter</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">getRuntimeContext</span><span class="p">()</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">getMetricGroup</span><span class="p">()</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">counter</span><span class="p">(</span><span class="s">"my-counter"</span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><p>All registered metrics will be exposed via reporters. Out of the box, Flinks comes with support for JMX, Ganglia, Graphite, and statsD. In addition to your custom metrics, Flink exposes many internal metrics like checkpoint sizes and JVM stats.</p>
<p>Check out the <a href="//nightlies.apache.org/flink/flink-docs-release-1.1/apis/metrics.html">Metrics documentation for more details</a>.</p>
<h2 id="list-of-contributors">
List of Contributors
diff --git a/content/2016/08/04/flink-1.1.1-released/index.html b/content/2016/08/04/flink-1.1.1-released/index.html
index 0b62be5d5..96f833eb5 100644
--- a/content/2016/08/04/flink-1.1.1-released/index.html
+++ b/content/2016/08/04/flink-1.1.1-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -26,18 +26,17 @@ This was fixed with this release and we highly recommend all users to use this v
<meta property="og:url" content="https://flink.apache.org/2016/08/04/flink-1.1.1-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2016-08-04T09:00:00+00:00" />
<meta property="article:modified_time" content="2016-08-04T09:00:00+00:00" />
-
<title>Flink 1.1.1 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -60,7 +59,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -267,7 +266,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2016/08/24/flink-forward-2016-announcing-schedule-keynotes-and-panel-discussion/index.html b/content/2016/08/24/flink-forward-2016-announcing-schedule-keynotes-and-panel-discussion/index.html
index b0f970b99..3c5c8b9d4 100644
--- a/content/2016/08/24/flink-forward-2016-announcing-schedule-keynotes-and-panel-discussion/index.html
+++ b/content/2016/08/24/flink-forward-2016-announcing-schedule-keynotes-and-panel-discussion/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ Ted Dunning has been announced as a keynote speaker at the event. Ted is the VP
<meta property="og:url" content="https://flink.apache.org/2016/08/24/flink-forward-2016-announcing-schedule-keynotes-and-panel-discussion/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2016-08-24T09:00:00+00:00" />
<meta property="article:modified_time" content="2016-08-24T09:00:00+00:00" />
-
<title>Flink Forward 2016: Announcing Schedule, Keynotes, and Panel Discussion | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2016/09/05/apache-flink-1.1.2-released/index.html b/content/2016/09/05/apache-flink-1.1.2-released/index.html
index dfdd8d9e2..d856951a4 100644
--- a/content/2016/09/05/apache-flink-1.1.2-released/index.html
+++ b/content/2016/09/05/apache-flink-1.1.2-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -18,28 +18,27 @@
<meta name="description" content="The Apache Flink community released another bugfix version of the Apache Flink 1.1. series.
We recommend all users to upgrade to Flink 1.1.2.
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.1.2</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.10</artifactId> <version>1.1.2</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.10</artifactId> <versio [...]
-Release Notes - Flink - Version 1.1.2 [FLINK-4236] - Flink Dashboard stops showing list of uploaded jars if main method cannot be looked up [FLINK-4309] - Potential null pointer dereference in DelegatingConfiguration#keySet() [FLINK-4334] - Shaded Hadoop1 jar not fully excluded in Quickstart [FLINK-4341] - Kinesis connector does not emit maximum watermark properly [FLINK-4402] - Wrong metrics parameter names in documentation [FLINK-4409] - class conflict between jsr305-1.">
+Release Notes - Flink - Version 1.1.2 [FLINK-4236] - Flink Dashboard stops showing list of uploaded jars if main method cannot be looked up [FLINK-4309] - Potential null pointer dereference in DelegatingConfiguration#keySet() [FLINK-4334] - Shaded Hadoop1 jar not fully excluded in Quickstart [FLINK-4341] - Kinesis connector does not emit maximum watermark properly [FLINK-4402] - Wrong metrics parameter names in documentation [FLINK-4409] - class conflict between jsr305-1.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Apache Flink 1.1.2 Released" />
<meta property="og:description" content="The Apache Flink community released another bugfix version of the Apache Flink 1.1. series.
We recommend all users to upgrade to Flink 1.1.2.
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.1.2</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.10</artifactId> <version>1.1.2</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.10</artifactId> <versio [...]
-Release Notes - Flink - Version 1.1.2 [FLINK-4236] - Flink Dashboard stops showing list of uploaded jars if main method cannot be looked up [FLINK-4309] - Potential null pointer dereference in DelegatingConfiguration#keySet() [FLINK-4334] - Shaded Hadoop1 jar not fully excluded in Quickstart [FLINK-4341] - Kinesis connector does not emit maximum watermark properly [FLINK-4402] - Wrong metrics parameter names in documentation [FLINK-4409] - class conflict between jsr305-1." />
+Release Notes - Flink - Version 1.1.2 [FLINK-4236] - Flink Dashboard stops showing list of uploaded jars if main method cannot be looked up [FLINK-4309] - Potential null pointer dereference in DelegatingConfiguration#keySet() [FLINK-4334] - Shaded Hadoop1 jar not fully excluded in Quickstart [FLINK-4341] - Kinesis connector does not emit maximum watermark properly [FLINK-4402] - Wrong metrics parameter names in documentation [FLINK-4409] - class conflict between jsr305-1." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2016/09/05/apache-flink-1.1.2-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2016-09-05T09:00:00+00:00" />
<meta property="article:modified_time" content="2016-09-05T09:00:00+00:00" />
-
<title>Apache Flink 1.1.2 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -62,7 +61,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -269,7 +268,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2016/10/12/apache-flink-1.1.3-released/index.html b/content/2016/10/12/apache-flink-1.1.3-released/index.html
index 0d37bc00d..4a8d058c2 100644
--- a/content/2016/10/12/apache-flink-1.1.3-released/index.html
+++ b/content/2016/10/12/apache-flink-1.1.3-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -18,28 +18,27 @@
<meta name="description" content="The Apache Flink community released the next bugfix version of the Apache Flink 1.1. series.
We recommend all users to upgrade to Flink 1.1.3.
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.1.3</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.10</artifactId> <version>1.1.3</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.10</artifactId> <versio [...]
-Note for RocksDB Backend Users # It is highly recommended to use the “fully async” mode for the RocksDB state backend.">
+Note for RocksDB Backend Users # It is highly recommended to use the “fully async” mode for the RocksDB state backend. The “fully async” mode will most likely allow you to easily upgrade to Flink 1.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Apache Flink 1.1.3 Released" />
<meta property="og:description" content="The Apache Flink community released the next bugfix version of the Apache Flink 1.1. series.
We recommend all users to upgrade to Flink 1.1.3.
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.1.3</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.10</artifactId> <version>1.1.3</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.10</artifactId> <versio [...]
-Note for RocksDB Backend Users # It is highly recommended to use the “fully async” mode for the RocksDB state backend." />
+Note for RocksDB Backend Users # It is highly recommended to use the “fully async” mode for the RocksDB state backend. The “fully async” mode will most likely allow you to easily upgrade to Flink 1." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2016/10/12/apache-flink-1.1.3-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2016-10-12T09:00:00+00:00" />
<meta property="article:modified_time" content="2016-10-12T09:00:00+00:00" />
-
<title>Apache Flink 1.1.3 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -62,7 +61,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -269,7 +268,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -516,9 +515,9 @@ https://github.com/alex-shpak/hugo-book
<a class="anchor" href="#note-for-rocksdb-backend-users">#</a>
</h2>
<p>It is highly recommended to use the “fully async” mode for the RocksDB state backend. The “fully async” mode will most likely allow you to easily upgrade to Flink 1.2 (via <a href="//nightlies.apache.org/flink/flink-docs-release-1.1/apis/streaming/savepoints.html">savepoints</a>) when it is released. The “semi async” mode will no longer be supported by Flink 1.2.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">RocksDBStateBackend</span> <span class="n">backend</span> <span class="o">=</span> <span class="k">new</span> <span class="n">RocksDBStateBackend</span><span class="o">(</span><span class="s">"..."</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"><span class="n">backend</span><span class="o">.</span><span class="na">enableFullyAsyncSnapshots</span><span class="o">();</span>
-</span></span></code></pre></div><h2 id="release-notes---flink---version-113">
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">RocksDBStateBackend</span><span class="w"> </span><span class="n">backend</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">RocksDBStateBackend</span><span class="p">(</span><span class="s">"..."</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">backend</span><span class="p">.</span><span class="na">enableFullyAsyncSnapshots</span><span class="p">();</span><span class="w">
+</span></span></span></code></pre></div><h2 id="release-notes---flink---version-113">
Release Notes - Flink - Version 1.1.3
<a class="anchor" href="#release-notes---flink---version-113">#</a>
</h2>
diff --git a/content/2016/12/19/apache-flink-in-2016-year-in-review/index.html b/content/2016/12/19/apache-flink-in-2016-year-in-review/index.html
index 59f45742e..778b2f6d7 100644
--- a/content/2016/12/19/apache-flink-in-2016-year-in-review/index.html
+++ b/content/2016/12/19/apache-flink-in-2016-year-in-review/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ In this post, we’ll look back on the project’s progress over the course of 2
<meta property="og:url" content="https://flink.apache.org/2016/12/19/apache-flink-in-2016-year-in-review/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2016-12-19T09:00:00+00:00" />
<meta property="article:modified_time" content="2016-12-19T09:00:00+00:00" />
-
<title>Apache Flink in 2016: Year in Review | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2016/12/21/apache-flink-1.1.4-released/index.html b/content/2016/12/21/apache-flink-1.1.4-released/index.html
index 6a3bd4e35..3fec8a730 100644
--- a/content/2016/12/21/apache-flink-1.1.4-released/index.html
+++ b/content/2016/12/21/apache-flink-1.1.4-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -18,28 +18,27 @@
<meta name="description" content="The Apache Flink community released the next bugfix version of the Apache Flink 1.1 series.
This release includes major robustness improvements for checkpoint cleanup on failures and consumption of intermediate streams. We highly recommend all users to upgrade to Flink 1.1.4.
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.1.4</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.10</artifactId> <version>1.1.4</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.10</artifactId> <versio [...]
-Note for RocksDB Backend Users # We updated Flink’s RocksDB dependency version from 4.">
+Note for RocksDB Backend Users # We updated Flink’s RocksDB dependency version from 4.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Apache Flink 1.1.4 Released" />
<meta property="og:description" content="The Apache Flink community released the next bugfix version of the Apache Flink 1.1 series.
This release includes major robustness improvements for checkpoint cleanup on failures and consumption of intermediate streams. We highly recommend all users to upgrade to Flink 1.1.4.
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.1.4</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.10</artifactId> <version>1.1.4</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.10</artifactId> <versio [...]
-Note for RocksDB Backend Users # We updated Flink’s RocksDB dependency version from 4." />
+Note for RocksDB Backend Users # We updated Flink’s RocksDB dependency version from 4." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2016/12/21/apache-flink-1.1.4-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2016-12-21T09:00:00+00:00" />
<meta property="article:modified_time" content="2016-12-21T09:00:00+00:00" />
-
<title>Apache Flink 1.1.4 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -62,7 +61,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -269,7 +268,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -516,11 +515,11 @@ https://github.com/alex-shpak/hugo-book
<a class="anchor" href="#note-for-rocksdb-backend-users">#</a>
</h2>
<p>We updated Flink’s RocksDB dependency version from <code>4.5.1</code> to <code>4.11.2</code>. Between these versions some of RocksDB’s internal configuration defaults changed that would affect the memory footprint of running Flink with RocksDB. Therefore, we manually reset them to the previous defaults. If you want to run with the new Rocks 4.11.2 defaults, you can do this via:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">RocksDBStateBackend</span> <span class="n">backend</span> <span class="o">=</span> <span class="k">new</span> <span class="n">RocksDBStateBackend</span><span class="o">(</span><span class="s">"..."</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"><span class="c1">// Use the new default options. Otherwise, the default for RocksDB 4.5.1
-</span></span></span><span class="line"><span class="cl"><span class="c1">// `PredefinedOptions.DEFAULT_ROCKS_4_5_1` will be used.
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">backend</span><span class="o">.</span><span class="na">setPredefinedOptions</span><span class="o">(</span><span class="n">PredefinedOptions</span><span class="o">.</span><span class="na">DEFAULT</span><span class="o">);</span>
-</span></span></code></pre></div><h2 id="release-notes---flink---version-114">
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">RocksDBStateBackend</span><span class="w"> </span><span class="n">backend</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">RocksDBStateBackend</span><span class="p">(</span><span class="s">"..."</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// Use the new default options. Otherwise, the default for RocksDB 4.5.1</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// `PredefinedOptions.DEFAULT_ROCKS_4_5_1` will be used.</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">backend</span><span class="p">.</span><span class="na">setPredefinedOptions</span><span class="p">(</span><span class="n">PredefinedOptions</span><span class="p">.</span><span class="na">DEFAULT</span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><h2 id="release-notes---flink---version-114">
Release Notes - Flink - Version 1.1.4
<a class="anchor" href="#release-notes---flink---version-114">#</a>
</h2>
diff --git a/content/2017/02/06/announcing-apache-flink-1.2.0/index.html b/content/2017/02/06/announcing-apache-flink-1.2.0/index.html
index 1d6412e3e..c8a0cd8eb 100644
--- a/content/2017/02/06/announcing-apache-flink-1.2.0/index.html
+++ b/content/2017/02/06/announcing-apache-flink-1.2.0/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -26,18 +26,17 @@ We encourage everyone to download the release and check out the documentation."
<meta property="og:url" content="https://flink.apache.org/2017/02/06/announcing-apache-flink-1.2.0/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2017-02-06T12:00:00+00:00" />
<meta property="article:modified_time" content="2017-02-06T12:00:00+00:00" />
-
<title>Announcing Apache Flink 1.2.0 | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -60,7 +59,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -267,7 +266,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2017/03/23/apache-flink-1.1.5-released/index.html b/content/2017/03/23/apache-flink-1.1.5-released/index.html
index e0208ed36..f2c9ff225 100644
--- a/content/2017/03/23/apache-flink-1.1.5-released/index.html
+++ b/content/2017/03/23/apache-flink-1.1.5-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -26,18 +26,17 @@ This release includes critical fixes for HA recovery robustness, fault tolerance
<meta property="og:url" content="https://flink.apache.org/2017/03/23/apache-flink-1.1.5-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2017-03-23T18:00:00+00:00" />
<meta property="article:modified_time" content="2017-03-23T18:00:00+00:00" />
-
<title>Apache Flink 1.1.5 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -60,7 +59,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -267,7 +266,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2017/03/29/from-streams-to-tables-and-back-again-an-update-on-flinks-table-sql-api/index.html b/content/2017/03/29/from-streams-to-tables-and-back-again-an-update-on-flinks-table-sql-api/index.html
index 6e9bc8ff5..7a97c785a 100644
--- a/content/2017/03/29/from-streams-to-tables-and-back-again-an-update-on-flinks-table-sql-api/index.html
+++ b/content/2017/03/29/from-streams-to-tables-and-back-again-an-update-on-flinks-table-sql-api/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ Flink’s DataStream abstraction is a powerful API which lets you flexibly d
<meta property="og:url" content="https://flink.apache.org/2017/03/29/from-streams-to-tables-and-back-again-an-update-on-flinks-table-sql-api/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2017-03-29T12:00:00+00:00" />
<meta property="article:modified_time" content="2017-03-29T12:00:00+00:00" />
-
<title>From Streams to Tables and Back Again: An Update on Flink's Table & SQL API | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2017/03/30/continuous-queries-on-dynamic-tables/index.html b/content/2017/03/30/continuous-queries-on-dynamic-tables/index.html
index 4135ed0e3..4c5aae55c 100644
--- a/content/2017/03/30/continuous-queries-on-dynamic-tables/index.html
+++ b/content/2017/03/30/continuous-queries-on-dynamic-tables/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -15,27 +15,26 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
-<meta name="description" content="Analyzing Data Streams with SQL # More and more companies are adopting stream processing and are migrating existing batch applications to streaming or implementing streaming solutions for new use cases. Many of those applications focus on analyzing streaming data. The data streams that are analyzed come from a wide variety of sources such as database transactions, clicks, sensor measurements, or IoT devices.
- Apache Flink is very well suited to power streaming analytics applications because it provides support for event-time semantics, stateful exactly-once processing, and achieves high throughput and low latency at the same time.">
+<meta name="description" content="Analyzing Data Streams with SQL # More and more companies are adopting stream processing and are migrating existing batch applications to streaming or implementing streaming solutions for new use cases. Many of those applications focus on analyzing streaming data. The data streams that are analyzed come from a wide variety of sources such as database transactions, clicks, sensor measurements, or IoT devices.
+Apache Flink is very well suited to power streaming analytics applications because it provides support for event-time semantics, stateful exactly-once processing, and achieves high throughput and low latency at the same time.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Continuous Queries on Dynamic Tables" />
-<meta property="og:description" content="Analyzing Data Streams with SQL # More and more companies are adopting stream processing and are migrating existing batch applications to streaming or implementing streaming solutions for new use cases. Many of those applications focus on analyzing streaming data. The data streams that are analyzed come from a wide variety of sources such as database transactions, clicks, sensor measurements, or IoT devices.
- Apache Flink is very well suited to power streaming analytics applications because it provides support for event-time semantics, stateful exactly-once processing, and achieves high throughput and low latency at the same time." />
+<meta property="og:description" content="Analyzing Data Streams with SQL # More and more companies are adopting stream processing and are migrating existing batch applications to streaming or implementing streaming solutions for new use cases. Many of those applications focus on analyzing streaming data. The data streams that are analyzed come from a wide variety of sources such as database transactions, clicks, sensor measurements, or IoT devices.
+Apache Flink is very well suited to power streaming analytics applications because it provides support for event-time semantics, stateful exactly-once processing, and achieves high throughput and low latency at the same time." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2017/03/30/continuous-queries-on-dynamic-tables/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2017-03-30T12:00:00+00:00" />
<meta property="article:modified_time" content="2017-03-30T12:00:00+00:00" />
-
<title>Continuous Queries on Dynamic Tables | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2017/04/26/apache-flink-1.2.1-released/index.html b/content/2017/04/26/apache-flink-1.2.1-released/index.html
index ac652f7e1..67178d0ec 100644
--- a/content/2017/04/26/apache-flink-1.2.1-released/index.html
+++ b/content/2017/04/26/apache-flink-1.2.1-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -19,29 +19,28 @@
This release includes many critical fixes for Flink 1.2.0. The list below includes a detailed list of all fixes.
We highly recommend all users to upgrade to Flink 1.2.1.
Please note that there are two unresolved major issues in Flink 1.2.1 and 1.2.0:
- FLINK-6353 Restoring using CheckpointedRestoring does not work from 1.2 to 1.2 FLINK-6188 Some setParallelism() methods can’t cope with default parallelism <dependency> <groupId>org.">
+FLINK-6353 Restoring using CheckpointedRestoring does not work from 1.2 to 1.2 FLINK-6188 Some setParallelism() methods can’t cope with default parallelism <dependency> <groupId>org.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Apache Flink 1.2.1 Released" />
<meta property="og:description" content="The Apache Flink community released the first bugfix version of the Apache Flink 1.2 series.
This release includes many critical fixes for Flink 1.2.0. The list below includes a detailed list of all fixes.
We highly recommend all users to upgrade to Flink 1.2.1.
Please note that there are two unresolved major issues in Flink 1.2.1 and 1.2.0:
- FLINK-6353 Restoring using CheckpointedRestoring does not work from 1.2 to 1.2 FLINK-6188 Some setParallelism() methods can’t cope with default parallelism <dependency> <groupId>org." />
+FLINK-6353 Restoring using CheckpointedRestoring does not work from 1.2 to 1.2 FLINK-6188 Some setParallelism() methods can’t cope with default parallelism <dependency> <groupId>org." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2017/04/26/apache-flink-1.2.1-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2017-04-26T18:00:00+00:00" />
<meta property="article:modified_time" content="2017-04-26T18:00:00+00:00" />
-
<title>Apache Flink 1.2.1 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2017/05/16/introducing-docker-images-for-apache-flink/index.html b/content/2017/05/16/introducing-docker-images-for-apache-flink/index.html
index 57ae4567d..964984e6e 100644
--- a/content/2017/05/16/introducing-docker-images-for-apache-flink/index.html
+++ b/content/2017/05/16/introducing-docker-images-for-apache-flink/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ A community-maintained way to run Apache Flink on Docker and other container run
<meta property="og:url" content="https://flink.apache.org/2017/05/16/introducing-docker-images-for-apache-flink/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2017-05-16T09:00:00+00:00" />
<meta property="article:modified_time" content="2017-05-16T09:00:00+00:00" />
-
<title>Introducing Docker Images for Apache Flink | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2017/06/01/apache-flink-1.3.0-release-announcement/index.html b/content/2017/06/01/apache-flink-1.3.0-release-announcement/index.html
index 675b977f4..1477cac44 100644
--- a/content/2017/06/01/apache-flink-1.3.0-release-announcement/index.html
+++ b/content/2017/06/01/apache-flink-1.3.0-release-announcement/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -26,18 +26,17 @@ Users can expect Flink releases now in a 4 month cycle." />
<meta property="og:url" content="https://flink.apache.org/2017/06/01/apache-flink-1.3.0-release-announcement/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2017-06-01T12:00:00+00:00" />
<meta property="article:modified_time" content="2017-06-01T12:00:00+00:00" />
-
<title>Apache Flink 1.3.0 Release Announcement | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -60,7 +59,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -267,7 +266,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2017/06/23/apache-flink-1.3.1-released/index.html b/content/2017/06/23/apache-flink-1.3.1-released/index.html
index f8199314c..70bf28adf 100644
--- a/content/2017/06/23/apache-flink-1.3.1-released/index.html
+++ b/content/2017/06/23/apache-flink-1.3.1-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -28,18 +28,17 @@ We highly recommend all users to upgrade to Flink 1.3.1.
<meta property="og:url" content="https://flink.apache.org/2017/06/23/apache-flink-1.3.1-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2017-06-23T16:00:00+00:00" />
<meta property="article:modified_time" content="2017-06-23T16:00:00+00:00" />
-
<title>Apache Flink 1.3.1 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -62,7 +61,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -269,7 +268,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2017/07/04/a-deep-dive-into-rescalable-state-in-apache-flink/index.html b/content/2017/07/04/a-deep-dive-into-rescalable-state-in-apache-flink/index.html
index 477380561..7b8f69763 100644
--- a/content/2017/07/04/a-deep-dive-into-rescalable-state-in-apache-flink/index.html
+++ b/content/2017/07/04/a-deep-dive-into-rescalable-state-in-apache-flink/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -15,27 +15,26 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
-<meta name="description" content="Apache Flink 1.2.0, released in February 2017, introduced support for rescalable state. This post provides a detailed overview of stateful stream processing and rescalable state in Flink. An Intro to Stateful Stream Processing # At a high level, we can consider state in stream processing as memory in operators that remembers information about past input and can be used to influence the processing of future input.
+<meta name="description" content="Apache Flink 1.2.0, released in February 2017, introduced support for rescalable state. This post provides a detailed overview of stateful stream processing and rescalable state in Flink. An Intro to Stateful Stream Processing # At a high level, we can consider state in stream processing as memory in operators that remembers information about past input and can be used to influence the processing of future input.
In contrast, operators in stateless stream processing only consider their current inputs, without further context and knowledge about the past.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="A Deep Dive into Rescalable State in Apache Flink" />
-<meta property="og:description" content="Apache Flink 1.2.0, released in February 2017, introduced support for rescalable state. This post provides a detailed overview of stateful stream processing and rescalable state in Flink. An Intro to Stateful Stream Processing # At a high level, we can consider state in stream processing as memory in operators that remembers information about past input and can be used to influence the processing of future input.
+<meta property="og:description" content="Apache Flink 1.2.0, released in February 2017, introduced support for rescalable state. This post provides a detailed overview of stateful stream processing and rescalable state in Flink. An Intro to Stateful Stream Processing # At a high level, we can consider state in stream processing as memory in operators that remembers information about past input and can be used to influence the processing of future input.
In contrast, operators in stateless stream processing only consider their current inputs, without further context and knowledge about the past." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2017/07/04/a-deep-dive-into-rescalable-state-in-apache-flink/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2017-07-04T09:00:00+00:00" />
<meta property="article:modified_time" content="2017-07-04T09:00:00+00:00" />
-
<title>A Deep Dive into Rescalable State in Apache Flink | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -543,41 +542,41 @@ https://github.com/alex-shpak/hugo-book
<img src="/img/blog/list-checkpointed.svg" style="width:70%;margin:10px">
</center><br>
<p>Our approach provides a simple API with which implementing operators can encode domain-specific knowledge about how to partition and merge units of state. With our new checkpointing interface, the Kafka source makes individual partition offsets explicit, and state reassignment becomes as easy as splitting and merging lists.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">FlinkKafkaConsumer</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="kd">extends</span> <span class="n">RichParallelSourceFunction</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class=" [...]
-</span></span><span class="line"><span class="cl"> <span class="c1">// ...
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="kd">transient</span> <span class="n">ListState</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">KafkaTopicPartition</span><span class="o">,</span> <span class="n">Long</span><span class="o">>></span> <span class="n">offsetsOperatorState</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">initializeState</span><span class="o">(</span><span class="n">FunctionInitializationContext</span> <span class="n">context</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="n">OperatorStateStore</span> <span class="n">stateStore</span> <span class="o">=</span> <span class="n">context</span><span class="o">.</span><span class="na">getOperatorStateStore</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// register the state with the backend
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">this</span><span class="o">.</span><span class="na">offsetsOperatorState</span> <span class="o">=</span> <span class="n">stateStore</span><span class="o">.</span><span class="na">getSerializableListState</span><span class="o">(</span><span class="s">"kafka-offsets"</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">// if the job was restarted, we set the restored offsets
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">if</span> <span class="o">(</span><span class="n">context</span><span class="o">.</span><span class="na">isRestored</span><span class="o">())</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="o">(</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">KafkaTopicPartition</span><span class="o">,</span> <span class="n">Long</span><span class="o">></span> <span class="n">kafkaOffset</span> <span class="o">:</span> <span class="n">offsetsOperatorState</span><span class="o">.</span><span class="na">get</span><span class="o">())</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// ... restore logic
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">snapshotState</span><span class="o">(</span><span class="n">FunctionSnapshotContext</span> <span class="n">context</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="k">this</span><span class="o">.</span><span class="na">offsetsOperatorState</span><span class="o">.</span><span class="na">clear</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">// write the partition offsets to the list of operator states
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">for</span> <span class="o">(</span><span class="n">Map</span><span class="o">.</span><span class="na">Entry</span><span class="o"><</span><span class="n">KafkaTopicPartition</span><span class="o">,</span> <span class="n">Long</span><span class="o">></span> <span class="n">partition</span> <span class="o">:</span> <span class="k">this</span><span class="o">.</span><span class="na"> [...]
-</span></span><span class="line"><span class="cl"> <span class="k">this</span><span class="o">.</span><span class="na">offsetsOperatorState</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="n">Tuple2</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">partition</span><span class="o">.</span><span class="na">getKey</span><span class="o">(),</span> <span class="n">partition</span><span class="o" [...]
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">// ...
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><h2 id="reassigning-keyed-state-when-rescaling">
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">FlinkKafkaConsumer</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="w"> </span><span class="kd">extends</span><span class="w"> </span><span class="n">RichParallelSourceFunction</span><span class="o"><</s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// ...</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="kd">transient</span><span class="w"> </span><span class="n">ListState</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">KafkaTopicPartition</span><span class="p">,</span><span class="w"> </span><span class="n">Long</span><span class="o">>></span><span class="w"> </span><span clas [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">initializeState</span><span class="p">(</span><span class="n">FunctionInitializationContext</span><span class="w"> </span><span class="n">context</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">OperatorStateStore</span><span class="w"> </span><span class="n">stateStore</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">context</span><span class="p">.</span><span class="na">getOperatorStateStore</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// register the state with the backend</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">offsetsOperatorState</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">stateStore</span><span class="p">.</span><span class="na">getSerializableListState</span><span class="p">(</span><span class="s">"kafka-offsets"</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// if the job was restarted, we set the restored offsets</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">context</span><span class="p">.</span><span class="na">isRestored</span><span class="p">())</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">KafkaTopicPartition</span><span class="p">,</span><span class="w"> </span><span class="n">Long</span><span class="o">></span><span class="w"> </span><span class="n">kafkaOffset</span><span class="w"> </span><span class="p">:</span><span class="w"> </span [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// ... restore logic</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">snapshotState</span><span class="p">(</span><span class="n">FunctionSnapshotContext</span><span class="w"> </span><span class="n">context</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span><span cl [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">offsetsOperatorState</span><span class="p">.</span><span class="na">clear</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// write the partition offsets to the list of operator states</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="n">Map</span><span class="p">.</span><span class="na">Entry</span><span class="o"><</span><span class="n">KafkaTopicPartition</span><span class="p">,</span><span class="w"> </span><span class="n">Long</span><span class="o">></span><span class="w"> </span><span class="n">partition</span><span class="w"> </span><s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">offsetsOperatorState</span><span class="p">.</span><span class="na">add</span><span class="p">(</span><span class="n">Tuple2</span><span class="p">.</span><span class="na">of</span><span class="p">(</span><span class="n">partition</span><span class="p">.</span><span class="na">getKey</span><span class="p">(),</span><span class="w"> < [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// ...</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><h2 id="reassigning-keyed-state-when-rescaling">
Reassigning Keyed State When Rescaling
<a class="anchor" href="#reassigning-keyed-state-when-rescaling">#</a>
</h2>
diff --git a/content/2017/08/05/apache-flink-1.3.2-released/index.html b/content/2017/08/05/apache-flink-1.3.2-released/index.html
index 96066f601..d0c9cd79d 100644
--- a/content/2017/08/05/apache-flink-1.3.2-released/index.html
+++ b/content/2017/08/05/apache-flink-1.3.2-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -28,18 +28,17 @@ Important Notice: A user reported a bug in the FlinkKafkaConsumer (FLINK-7143) t
<meta property="og:url" content="https://flink.apache.org/2017/08/05/apache-flink-1.3.2-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2017-08-05T09:00:00+00:00" />
<meta property="article:modified_time" content="2017-08-05T09:00:00+00:00" />
-
<title>Apache Flink 1.3.2 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -62,7 +61,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -269,7 +268,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2017/11/21/looking-ahead-to-apache-flink-1.4.0-and-1.5.0/index.html b/content/2017/11/21/looking-ahead-to-apache-flink-1.4.0-and-1.5.0/index.html
index cd245e0d2..d08e24528 100644
--- a/content/2017/11/21/looking-ahead-to-apache-flink-1.4.0-and-1.5.0/index.html
+++ b/content/2017/11/21/looking-ahead-to-apache-flink-1.4.0-and-1.5.0/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ Both releases include ambitious features that we believe will move Flink to an e
<meta property="og:url" content="https://flink.apache.org/2017/11/21/looking-ahead-to-apache-flink-1.4.0-and-1.5.0/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2017-11-21T10:00:00+00:00" />
<meta property="article:modified_time" content="2017-11-21T10:00:00+00:00" />
-
<title>Looking Ahead to Apache Flink 1.4.0 and 1.5.0 | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2017/12/12/apache-flink-1.4.0-release-announcement/index.html b/content/2017/12/12/apache-flink-1.4.0-release-announcement/index.html
index b5a5267a6..a936f01b7 100644
--- a/content/2017/12/12/apache-flink-1.4.0-release-announcement/index.html
+++ b/content/2017/12/12/apache-flink-1.4.0-release-announcement/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -26,18 +26,17 @@ We encourage everyone to download the release and check out the documentation."
<meta property="og:url" content="https://flink.apache.org/2017/12/12/apache-flink-1.4.0-release-announcement/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2017-12-12T10:00:00+00:00" />
<meta property="article:modified_time" content="2017-12-12T10:00:00+00:00" />
-
<title>Apache Flink 1.4.0 Release Announcement | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -60,7 +59,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -267,7 +266,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2017/12/21/apache-flink-in-2017-year-in-review/index.html b/content/2017/12/21/apache-flink-in-2017-year-in-review/index.html
index e6ea0c58c..adea8b125 100644
--- a/content/2017/12/21/apache-flink-in-2017-year-in-review/index.html
+++ b/content/2017/12/21/apache-flink-in-2017-year-in-review/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -22,18 +22,17 @@
<meta property="og:url" content="https://flink.apache.org/2017/12/21/apache-flink-in-2017-year-in-review/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2017-12-21T09:00:00+00:00" />
<meta property="article:modified_time" content="2017-12-21T09:00:00+00:00" />
-
<title>Apache Flink in 2017: Year in Review | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2018/01/30/managing-large-state-in-apache-flink-an-intro-to-incremental-checkpointing/index.html b/content/2018/01/30/managing-large-state-in-apache-flink-an-intro-to-incremental-checkpointing/index.html
index f5a523467..a1e04c0dd 100644
--- a/content/2018/01/30/managing-large-state-in-apache-flink-an-intro-to-incremental-checkpointing/index.html
+++ b/content/2018/01/30/managing-large-state-in-apache-flink-an-intro-to-incremental-checkpointing/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ State is a fundamental, enabling concept in stream processing required for a maj
<meta property="og:url" content="https://flink.apache.org/2018/01/30/managing-large-state-in-apache-flink-an-intro-to-incremental-checkpointing/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-01-30T12:00:00+00:00" />
<meta property="article:modified_time" content="2018-01-30T12:00:00+00:00" />
-
<title>Managing Large State in Apache Flink: An Intro to Incremental Checkpointing | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -521,17 +520,17 @@ https://github.com/alex-shpak/hugo-book
Java Example
<a class="anchor" href="#java-example">#</a>
</h4>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">StreamExecutionEnvironment</span> <span class="n">env</span> <span class="o">=</span> <span class="n">StreamExecutionEnvironment</span><span class="o">.</span><span class="na">getExecutionEnvironment</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"><span class="n">env</span><span class="o">.</span><span class="na">setStateBackend</span><span class="o">(</span><span class="k">new</span> <span class="n">RocksDBStateBackend</span><span class="o">(</span><span class="n">filebackend</span><span class="o">,</span> <span class="kc">true</span><span class="o">));</span>
-</span></span></code></pre></div><h4 id="scala-example">
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">StreamExecutionEnvironment</span><span class="w"> </span><span class="n">env</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">StreamExecutionEnvironment</span><span class="p">.</span><span class="na">getExecutionEnvironment</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">env</span><span class="p">.</span><span class="na">setStateBackend</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">RocksDBStateBackend</span><span class="p">(</span><span class="n">filebackend</span><span class="p">,</span><span class="w"> </span><span class="kc">true</span><span class="p">));</span><span class="w">
+</span></span></span></code></pre></div><h4 id="scala-example">
Scala Example
<a class="anchor" href="#scala-example">#</a>
</h4>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-scala" data-lang="scala"><span class="line"><span class="cl"><span class="k">val</span> <span class="n">env</span> <span class="k">=</span> <span class="nc">StreamExecutionEnvironment</span><span class="o">.</span><span class="n">getExecutionEnvironment</span><span class="o">()</span>
</span></span><span class="line"><span class="cl"><span class="n">env</span><span class="o">.</span><span class="n">setStateBackend</span><span class="o">(</span><span class="k">new</span> <span class="nc">RocksDBStateBackend</span><span class="o">(</span><span class="n">filebackend</span><span class="o">,</span> <span class="kc">true</span><span class="o">))</span>
</span></span></code></pre></div><p>By default, Flink retains 1 completed checkpoint, so if you need a higher number, <a href="//nightlies.apache.org/flink/flink-docs-master/dev/stream/state/checkpointing.html#related-config-options">you can configure it with the following flag</a>:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">state</span><span class="o">.</span><span class="na">checkpoints</span><span class="o">.</span><span class="na">num</span><span class="o">-</span><span class="n">retained</span>
-</span></span></code></pre></div><h3 id="how-it-works">
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">state</span><span class="p">.</span><span class="na">checkpoints</span><span class="p">.</span><span class="na">num</span><span class="o">-</span><span class="n">retained</span><span class="w">
+</span></span></span></code></pre></div><h3 id="how-it-works">
How it Works
<a class="anchor" href="#how-it-works">#</a>
</h3>
diff --git a/content/2018/02/15/apache-flink-1.4.1-released/index.html b/content/2018/02/15/apache-flink-1.4.1-released/index.html
index 26d2ce47f..fb3dd48e1 100644
--- a/content/2018/02/15/apache-flink-1.4.1-released/index.html
+++ b/content/2018/02/15/apache-flink-1.4.1-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2018/02/15/apache-flink-1.4.1-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-02-15T12:00:00+00:00" />
<meta property="article:modified_time" content="2018-02-15T12:00:00+00:00" />
-
<title>Apache Flink 1.4.1 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2018/02/28/an-overview-of-end-to-end-exactly-once-processing-in-apache-flink-with-apache-kafka-too/index.html b/content/2018/02/28/an-overview-of-end-to-end-exactly-once-processing-in-apache-flink-with-apache-kafka-too/index.html
index 8dc1087e2..fd9cbbc9b 100644
--- a/content/2018/02/28/an-overview-of-end-to-end-exactly-once-processing-in-apache-flink-with-apache-kafka-too/index.html
+++ b/content/2018/02/28/an-overview-of-end-to-end-exactly-once-processing-in-apache-flink-with-apache-kafka-too/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ Apache Flink 1.4.0, released in December 2017, introduced a significant mileston
<meta property="og:url" content="https://flink.apache.org/2018/02/28/an-overview-of-end-to-end-exactly-once-processing-in-apache-flink-with-apache-kafka-too/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-02-28T12:00:00+00:00" />
<meta property="article:modified_time" content="2018-02-28T12:00:00+00:00" />
-
<title>An Overview of End-to-End Exactly-Once Processing in Apache Flink (with Apache Kafka, too!) | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2018/03/08/apache-flink-1.4.2-released/index.html b/content/2018/03/08/apache-flink-1.4.2-released/index.html
index 4c70e985b..9d24f37f9 100644
--- a/content/2018/03/08/apache-flink-1.4.2-released/index.html
+++ b/content/2018/03/08/apache-flink-1.4.2-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2018/03/08/apache-flink-1.4.2-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-03-08T12:00:00+00:00" />
<meta property="article:modified_time" content="2018-03-08T12:00:00+00:00" />
-
<title>Apache Flink 1.4.2 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2018/03/15/apache-flink-1.3.3-released/index.html b/content/2018/03/15/apache-flink-1.3.3-released/index.html
index 55cfb1e80..7b80bae8f 100644
--- a/content/2018/03/15/apache-flink-1.3.3-released/index.html
+++ b/content/2018/03/15/apache-flink-1.3.3-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2018/03/15/apache-flink-1.3.3-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-03-15T12:00:00+00:00" />
<meta property="article:modified_time" content="2018-03-15T12:00:00+00:00" />
-
<title>Apache Flink 1.3.3 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2018/05/18/apache-flink-1.5.0-release-announcement/index.html b/content/2018/05/18/apache-flink-1.5.0-release-announcement/index.html
index 440f119b9..448946abe 100644
--- a/content/2018/05/18/apache-flink-1.5.0-release-announcement/index.html
+++ b/content/2018/05/18/apache-flink-1.5.0-release-announcement/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -26,18 +26,17 @@ We encourage everyone to download the release and check out the documentation."
<meta property="og:url" content="https://flink.apache.org/2018/05/18/apache-flink-1.5.0-release-announcement/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-05-18T10:00:00+00:00" />
<meta property="article:modified_time" content="2018-05-18T10:00:00+00:00" />
-
<title>Apache Flink 1.5.0 Release Announcement | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -60,7 +59,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -267,7 +266,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2018/07/12/apache-flink-1.5.1-released/index.html b/content/2018/07/12/apache-flink-1.5.1-released/index.html
index 3961f600c..6cc322fd4 100644
--- a/content/2018/07/12/apache-flink-1.5.1-released/index.html
+++ b/content/2018/07/12/apache-flink-1.5.1-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2018/07/12/apache-flink-1.5.1-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-07-12T12:00:00+00:00" />
<meta property="article:modified_time" content="2018-07-12T12:00:00+00:00" />
-
<title>Apache Flink 1.5.1 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2018/07/31/apache-flink-1.5.2-released/index.html b/content/2018/07/31/apache-flink-1.5.2-released/index.html
index b7fb29696..f9f39aa56 100644
--- a/content/2018/07/31/apache-flink-1.5.2-released/index.html
+++ b/content/2018/07/31/apache-flink-1.5.2-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2018/07/31/apache-flink-1.5.2-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-07-31T12:00:00+00:00" />
<meta property="article:modified_time" content="2018-07-31T12:00:00+00:00" />
-
<title>Apache Flink 1.5.2 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2018/08/09/apache-flink-1.6.0-release-announcement/index.html b/content/2018/08/09/apache-flink-1.6.0-release-announcement/index.html
index 85f0a4796..197be1bef 100644
--- a/content/2018/08/09/apache-flink-1.6.0-release-announcement/index.html
+++ b/content/2018/08/09/apache-flink-1.6.0-release-announcement/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -26,18 +26,17 @@ We encourage everyone to download the release and check out the documentation."
<meta property="og:url" content="https://flink.apache.org/2018/08/09/apache-flink-1.6.0-release-announcement/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-08-09T12:00:00+00:00" />
<meta property="article:modified_time" content="2018-08-09T12:00:00+00:00" />
-
<title>Apache Flink 1.6.0 Release Announcement | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -60,7 +59,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -267,7 +266,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2018/08/21/apache-flink-1.5.3-released/index.html b/content/2018/08/21/apache-flink-1.5.3-released/index.html
index 7113c1185..ec19ec746 100644
--- a/content/2018/08/21/apache-flink-1.5.3-released/index.html
+++ b/content/2018/08/21/apache-flink-1.5.3-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2018/08/21/apache-flink-1.5.3-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-08-21T12:00:00+00:00" />
<meta property="article:modified_time" content="2018-08-21T12:00:00+00:00" />
-
<title>Apache Flink 1.5.3 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2018/09/20/apache-flink-1.5.4-released/index.html b/content/2018/09/20/apache-flink-1.5.4-released/index.html
index 83759ab2b..5c09481c4 100644
--- a/content/2018/09/20/apache-flink-1.5.4-released/index.html
+++ b/content/2018/09/20/apache-flink-1.5.4-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2018/09/20/apache-flink-1.5.4-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-09-20T12:00:00+00:00" />
<meta property="article:modified_time" content="2018-09-20T12:00:00+00:00" />
-
<title>Apache Flink 1.5.4 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2018/09/20/apache-flink-1.6.1-released/index.html b/content/2018/09/20/apache-flink-1.6.1-released/index.html
index a75b22130..71daa421f 100644
--- a/content/2018/09/20/apache-flink-1.6.1-released/index.html
+++ b/content/2018/09/20/apache-flink-1.6.1-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2018/09/20/apache-flink-1.6.1-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-09-20T12:00:00+00:00" />
<meta property="article:modified_time" content="2018-09-20T12:00:00+00:00" />
-
<title>Apache Flink 1.6.1 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2018/10/29/apache-flink-1.5.5-released/index.html b/content/2018/10/29/apache-flink-1.5.5-released/index.html
index dfc4d76d4..4839f21ab 100644
--- a/content/2018/10/29/apache-flink-1.5.5-released/index.html
+++ b/content/2018/10/29/apache-flink-1.5.5-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2018/10/29/apache-flink-1.5.5-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-10-29T12:00:00+00:00" />
<meta property="article:modified_time" content="2018-10-29T12:00:00+00:00" />
-
<title>Apache Flink 1.5.5 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2018/10/29/apache-flink-1.6.2-released/index.html b/content/2018/10/29/apache-flink-1.6.2-released/index.html
index 1a3804144..330674183 100644
--- a/content/2018/10/29/apache-flink-1.6.2-released/index.html
+++ b/content/2018/10/29/apache-flink-1.6.2-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2018/10/29/apache-flink-1.6.2-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-10-29T12:00:00+00:00" />
<meta property="article:modified_time" content="2018-10-29T12:00:00+00:00" />
-
<title>Apache Flink 1.6.2 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2018/11/30/apache-flink-1.7.0-release-announcement/index.html b/content/2018/11/30/apache-flink-1.7.0-release-announcement/index.html
index cc072249f..49704257e 100644
--- a/content/2018/11/30/apache-flink-1.7.0-release-announcement/index.html
+++ b/content/2018/11/30/apache-flink-1.7.0-release-announcement/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ Flink 1.7.0 is API-compatible with previous 1.x.y releases for APIs annotated wi
<meta property="og:url" content="https://flink.apache.org/2018/11/30/apache-flink-1.7.0-release-announcement/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-11-30T12:00:00+00:00" />
<meta property="article:modified_time" content="2018-11-30T12:00:00+00:00" />
-
<title>Apache Flink 1.7.0 Release Announcement | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2018/12/21/apache-flink-1.7.1-released/index.html b/content/2018/12/21/apache-flink-1.7.1-released/index.html
index 58596ac77..402fed418 100644
--- a/content/2018/12/21/apache-flink-1.7.1-released/index.html
+++ b/content/2018/12/21/apache-flink-1.7.1-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2018/12/21/apache-flink-1.7.1-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-12-21T12:00:00+00:00" />
<meta property="article:modified_time" content="2018-12-21T12:00:00+00:00" />
-
<title>Apache Flink 1.7.1 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2018/12/22/apache-flink-1.6.3-released/index.html b/content/2018/12/22/apache-flink-1.6.3-released/index.html
index f41973982..5fe8df62f 100644
--- a/content/2018/12/22/apache-flink-1.6.3-released/index.html
+++ b/content/2018/12/22/apache-flink-1.6.3-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2018/12/22/apache-flink-1.6.3-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-12-22T12:00:00+00:00" />
<meta property="article:modified_time" content="2018-12-22T12:00:00+00:00" />
-
<title>Apache Flink 1.6.3 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2018/12/26/apache-flink-1.5.6-released/index.html b/content/2018/12/26/apache-flink-1.5.6-released/index.html
index 525f39a4c..4ef1d849e 100644
--- a/content/2018/12/26/apache-flink-1.5.6-released/index.html
+++ b/content/2018/12/26/apache-flink-1.5.6-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2018/12/26/apache-flink-1.5.6-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2018-12-26T12:00:00+00:00" />
<meta property="article:modified_time" content="2018-12-26T12:00:00+00:00" />
-
<title>Apache Flink 1.5.6 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2019/02/13/batch-as-a-special-case-of-streaming-and-alibabas-contribution-of-blink/index.html b/content/2019/02/13/batch-as-a-special-case-of-streaming-and-alibabas-contribution-of-blink/index.html
index 2ce353fcf..4eba388fc 100644
--- a/content/2019/02/13/batch-as-a-special-case-of-streaming-and-alibabas-contribution-of-blink/index.html
+++ b/content/2019/02/13/batch-as-a-special-case-of-streaming-and-alibabas-contribution-of-blink/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -16,26 +16,25 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="Last week, we broke the news that Alibaba decided to contribute its Flink-fork, called Blink, back to the Apache Flink project. Why is that a big thing for Flink, what will it mean for users and the community, and how does it fit into Flink’s overall vision? Let’s take a step back to understand this better…
-A Unified Approach to Batch and Streaming # Since its early days, Apache Flink has followed the philosophy of taking a unified approach to batch and streaming data processing.">
+A Unified Approach to Batch and Streaming # Since its early days, Apache Flink has followed the philosophy of taking a unified approach to batch and streaming data processing.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Batch as a Special Case of Streaming and Alibaba's contribution of Blink" />
<meta property="og:description" content="Last week, we broke the news that Alibaba decided to contribute its Flink-fork, called Blink, back to the Apache Flink project. Why is that a big thing for Flink, what will it mean for users and the community, and how does it fit into Flink’s overall vision? Let’s take a step back to understand this better…
-A Unified Approach to Batch and Streaming # Since its early days, Apache Flink has followed the philosophy of taking a unified approach to batch and streaming data processing." />
+A Unified Approach to Batch and Streaming # Since its early days, Apache Flink has followed the philosophy of taking a unified approach to batch and streaming data processing." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2019/02/13/batch-as-a-special-case-of-streaming-and-alibabas-contribution-of-blink/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-02-13T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-02-13T12:00:00+00:00" />
-
<title>Batch as a Special Case of Streaming and Alibaba's contribution of Blink | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2019/02/15/apache-flink-1.7.2-released/index.html b/content/2019/02/15/apache-flink-1.7.2-released/index.html
index 835926b91..6ca100f66 100644
--- a/content/2019/02/15/apache-flink-1.7.2-released/index.html
+++ b/content/2019/02/15/apache-flink-1.7.2-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2019/02/15/apache-flink-1.7.2-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-02-15T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-02-15T12:00:00+00:00" />
-
<title>Apache Flink 1.7.2 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2019/02/21/monitoring-apache-flink-applications-101/index.html b/content/2019/02/21/monitoring-apache-flink-applications-101/index.html
index 16f72967a..998d218d1 100644
--- a/content/2019/02/21/monitoring-apache-flink-applications-101/index.html
+++ b/content/2019/02/21/monitoring-apache-flink-applications-101/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -15,25 +15,24 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
-<meta name="description" content="table { border: 0px solid black; table-layout: auto; width: 800px; } th, td { border: 1px solid black; padding: 5px; padding-left: 10px; padding-right: 10px; } th { text-align: center } td { vertical-align: top } This blog post provides an introduction to Apache Flink’s built-in monitoring and metrics system, that allows developers to effectively monitor their Flink jobs. Oftentimes, the task of picking the relevant metrics to monitor a Flink applicatio [...]
+<meta name="description" content="This blog post provides an introduction to Apache Flink’s built-in monitoring and metrics system, that allows developers to effectively monitor their Flink jobs. Oftentimes, the task of picking the relevant metrics to monitor a Flink application can be overwhelming for a DevOps team that is just starting with stream processing and Apache Flink. Having worked with many organizations that deploy Flink at scale, I would like to share my experience and some [...]
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Monitoring Apache Flink Applications 101" />
-<meta property="og:description" content="table { border: 0px solid black; table-layout: auto; width: 800px; } th, td { border: 1px solid black; padding: 5px; padding-left: 10px; padding-right: 10px; } th { text-align: center } td { vertical-align: top } This blog post provides an introduction to Apache Flink’s built-in monitoring and metrics system, that allows developers to effectively monitor their Flink jobs. Oftentimes, the task of picking the relevant metrics to monitor a Flink app [...]
+<meta property="og:description" content="This blog post provides an introduction to Apache Flink’s built-in monitoring and metrics system, that allows developers to effectively monitor their Flink jobs. Oftentimes, the task of picking the relevant metrics to monitor a Flink application can be overwhelming for a DevOps team that is just starting with stream processing and Apache Flink. Having worked with many organizations that deploy Flink at scale, I would like to share my experience an [...]
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2019/02/21/monitoring-apache-flink-applications-101/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-02-21T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-02-21T12:00:00+00:00" />
-
<title>Monitoring Apache Flink Applications 101 | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2019/02/25/apache-flink-1.6.4-released/index.html b/content/2019/02/25/apache-flink-1.6.4-released/index.html
index 2be204580..fc4b49e7b 100644
--- a/content/2019/02/25/apache-flink-1.6.4-released/index.html
+++ b/content/2019/02/25/apache-flink-1.6.4-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2019/02/25/apache-flink-1.6.4-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-02-25T00:00:00+00:00" />
<meta property="article:modified_time" content="2019-02-25T00:00:00+00:00" />
-
<title>Apache Flink 1.6.4 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2019/03/06/what-to-expect-from-flink-forward-san-francisco-2019/index.html b/content/2019/03/06/what-to-expect-from-flink-forward-san-francisco-2019/index.html
index 7d1b8b500..a7b1675ed 100644
--- a/content/2019/03/06/what-to-expect-from-flink-forward-san-francisco-2019/index.html
+++ b/content/2019/03/06/what-to-expect-from-flink-forward-san-francisco-2019/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -22,18 +22,17 @@
<meta property="og:url" content="https://flink.apache.org/2019/03/06/what-to-expect-from-flink-forward-san-francisco-2019/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-03-06T11:00:00+00:00" />
<meta property="article:modified_time" content="2019-03-06T11:00:00+00:00" />
-
<title>What to expect from Flink Forward San Francisco 2019 | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2019/03/11/flink-and-prometheus-cloud-native-monitoring-of-streaming-applications/index.html b/content/2019/03/11/flink-and-prometheus-cloud-native-monitoring-of-streaming-applications/index.html
index e0b1d4bdb..e55617dc6 100644
--- a/content/2019/03/11/flink-and-prometheus-cloud-native-monitoring-of-streaming-applications/index.html
+++ b/content/2019/03/11/flink-and-prometheus-cloud-native-monitoring-of-streaming-applications/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -22,18 +22,17 @@
<meta property="og:url" content="https://flink.apache.org/2019/03/11/flink-and-prometheus-cloud-native-monitoring-of-streaming-applications/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-03-11T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-03-11T12:00:00+00:00" />
-
<title>Flink and Prometheus: Cloud-native monitoring of streaming applications | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -528,21 +527,21 @@ https://github.com/alex-shpak/hugo-book
</center>
<br/>
<p>The <code>PrometheusExampleJob</code> has three operators: Random numbers up to 10,000 are generated, then a map counts the events and creates a histogram of the values passed through. Finally, the events are discarded without further output. The very simple code below is from the second operator. It illustrates how easy it is to add custom metrics relevant to your business logic into your Flink job.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">class</span> <span class="nc">FlinkMetricsExposingMapFunction</span> <span class="kd">extends</span> <span class="n">RichMapFunction</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Integer</span><span class="o">></span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="kd">transient</span> <span class="n">Counter</span> <span class="n">eventCounter</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">open</span><span class="o">(</span><span class="n">Configuration</span> <span class="n">parameters</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">eventCounter</span> <span class="o">=</span> <span class="n">getRuntimeContext</span><span class="o">().</span><span class="na">getMetricGroup</span><span class="o">().</span><span class="na">counter</span><span class="o">(</span><span class="s">"events"</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">Integer</span> <span class="nf">map</span><span class="o">(</span><span class="n">Integer</span> <span class="n">value</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">eventCounter</span><span class="o">.</span><span class="na">inc</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">value</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><center><i><small>Excerpt from <a href="https://github.com/mbode/flink-prometheus-example/blob/master/src/main/java/com/github/mbode/flink_prometheus_example/FlinkMetricsExposingMapFunction.java">FlinkMetricsExposingMapFunction.java</a> demonstrating custom Flink metric.</small></i></center>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">class</span> <span class="nc">FlinkMetricsExposingMapFunction</span><span class="w"> </span><span class="kd">extends</span><span class="w"> </span><span class="n">RichMapFunction</span><span class="o"><</span><span class="n">Integer</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="o">&g [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="kd">transient</span><span class="w"> </span><span class="n">Counter</span><span class="w"> </span><span class="n">eventCounter</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">open</span><span class="p">(</span><span class="n">Configuration</span><span class="w"> </span><span class="n">parameters</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">eventCounter</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">getRuntimeContext</span><span class="p">().</span><span class="na">getMetricGroup</span><span class="p">().</span><span class="na">counter</span><span class="p">(</span><span class="s">"events"</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">Integer</span><span class="w"> </span><span class="nf">map</span><span class="p">(</span><span class="n">Integer</span><span class="w"> </span><span class="n">value</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">eventCounter</span><span class="p">.</span><span class="na">inc</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">value</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><center><i><small>Excerpt from <a href="https://github.com/mbode/flink-prometheus-example/blob/master/src/main/java/com/github/mbode/flink_prometheus_example/FlinkMetricsExposingMapFunction.java">FlinkMetricsExposingMapFunction.java</a> demonstrating custom Flink metric.</small></i></center>
<h2 id="configuring-prometheus-with-flink">
Configuring Prometheus with Flink
<a class="anchor" href="#configuring-prometheus-with-flink">#</a>
diff --git a/content/2019/04/09/apache-flink-1.8.0-release-announcement/index.html b/content/2019/04/09/apache-flink-1.8.0-release-announcement/index.html
index ec80bcef3..9def1fb6b 100644
--- a/content/2019/04/09/apache-flink-1.8.0-release-announcement/index.html
+++ b/content/2019/04/09/apache-flink-1.8.0-release-announcement/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ Flink 1.8.0 is API-compatible with previous 1.x.y releases for APIs annotated wi
<meta property="og:url" content="https://flink.apache.org/2019/04/09/apache-flink-1.8.0-release-announcement/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-04-09T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-04-09T12:00:00+00:00" />
-
<title>Apache Flink 1.8.0 Release Announcement | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2019/04/17/apache-flinks-application-to-season-of-docs/index.html b/content/2019/04/17/apache-flinks-application-to-season-of-docs/index.html
index 878a60697..0cbe23500 100644
--- a/content/2019/04/17/apache-flinks-application-to-season-of-docs/index.html
+++ b/content/2019/04/17/apache-flinks-application-to-season-of-docs/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -22,18 +22,17 @@
<meta property="og:url" content="https://flink.apache.org/2019/04/17/apache-flinks-application-to-season-of-docs/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-04-17T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-04-17T12:00:00+00:00" />
-
<title>Apache Flink's Application to Season of Docs | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2019/05/03/pulsar-flink.html b/content/2019/05/03/pulsar-flink.html
index ae7e35b3b..86c922a65 100644
--- a/content/2019/05/03/pulsar-flink.html
+++ b/content/2019/05/03/pulsar-flink.html
@@ -1 +1,10 @@
-<!DOCTYPE html><html><head><title>https://flink.apache.org/2019/05/03/when-flink-pulsar-come-together/</title><link rel="canonical" href="https://flink.apache.org/2019/05/03/when-flink-pulsar-come-together/"/><meta name="robots" content="noindex"><meta charset="utf-8" /><meta http-equiv="refresh" content="0; url=https://flink.apache.org/2019/05/03/when-flink-pulsar-come-together/" /></head></html>
\ No newline at end of file
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <title>https://flink.apache.org/2019/05/03/when-flink-pulsar-come-together/</title>
+ <link rel="canonical" href="https://flink.apache.org/2019/05/03/when-flink-pulsar-come-together/">
+ <meta name="robots" content="noindex">
+ <meta charset="utf-8">
+ <meta http-equiv="refresh" content="0; url=https://flink.apache.org/2019/05/03/when-flink-pulsar-come-together/">
+ </head>
+</html>
diff --git a/content/2019/05/03/when-flink-pulsar-come-together/index.html b/content/2019/05/03/when-flink-pulsar-come-together/index.html
index 37f4fd34a..143da52be 100644
--- a/content/2019/05/03/when-flink-pulsar-come-together/index.html
+++ b/content/2019/05/03/when-flink-pulsar-come-together/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -22,18 +22,17 @@
<meta property="og:url" content="https://flink.apache.org/2019/05/03/when-flink-pulsar-come-together/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-05-03T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-05-03T12:00:00+00:00" />
-
<title>When Flink & Pulsar Come Together | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -545,76 +544,76 @@ https://github.com/alex-shpak/hugo-book
<a class="anchor" href="#existing-integrations">#</a>
</h3>
<p>Integration between the two frameworks is ongoing and developers can already use Pulsar with Flink in multiple ways. For example, Pulsar can be used as a streaming source and streaming sink in Flink DataStream applications. Developers can ingest data from Pulsar into a Flink job that makes computations and processes real-time data, to then send the data back to a Pulsar topic as a streaming sink. Such an example is shown below:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// create and configure Pulsar consumer
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">PulsarSourceBuilder</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="n">builder</span> <span class="o">=</span> <span class="n">PulsarSourceBuilder</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">builder</span><span class="o">(</span><span class="k">new</span> <span class="n">SimpleStringSchema</span><span class="o">())</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">serviceUrl</span><span class="o">(</span><span class="n">serviceUrl</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">topic</span><span class="o">(</span><span class="n">inputTopic</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">subscriptionName</span><span class="o">(</span><span class="n">subscription</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"><span class="n">SourceFunction</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">src</span> <span class="o">=</span> <span class="n">builder</span><span class="o">.</span><span class="na">build</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"><span class="c1">// ingest DataStream with Pulsar consumer
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">words</span> <span class="o">=</span> <span class="n">env</span><span class="o">.</span><span class="na">addSource</span><span class="o">(</span><span class="n">src</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// perform computation on DataStream (here a simple WordCount)
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">WordWithCount</span><span class="o">></span> <span class="n">wc</span> <span class="o">=</span> <span class="n">words</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">flatMap</span><span class="o">((</span><span class="n">FlatMapFunction</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">WordWithCount</span><span class="o">>)</span> <span class="o">(</span><span class="n">word</span><span class="o">,</span> <span class="n">collector</span><span class="o">)</span> <span class="o">-></span> <span class=" [...]
-</span></span><span class="line"><span class="cl"> <span class="n">collector</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="k">new</span> <span class="n">WordWithCount</span><span class="o">(</span><span class="n">word</span><span class="o">,</span> <span class="n">1</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"> <span class="o">})</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">returns</span><span class="o">(</span><span class="n">WordWithCount</span><span class="o">.</span><span class="na">class</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">keyBy</span><span class="o">(</span><span class="s">"word"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">timeWindow</span><span class="o">(</span><span class="n">Time</span><span class="o">.</span><span class="na">seconds</span><span class="o">(</span><span class="n">5</span><span class="o">))</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">reduce</span><span class="o">((</span><span class="n">ReduceFunction</span><span class="o"><</span><span class="n">WordWithCount</span><span class="o">>)</span> <span class="o">(</span><span class="n">c1</span><span class="o">,</span> <span class="n">c2</span><span class="o">)</span> <span class="o">-></span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">WordWithCount</span><span class="o">(</span><span class="n">c1</span><span class="o">.</span><span class="na">word</span><span class="o">,</span> <span class="n">c1</span><span class="o">.</span><span class="na">count</span> <span class="o">+</span> <span class="n">c2</span><span class="o">.</span><span class="na">count</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// emit result via Pulsar producer
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">wc</span><span class="o">.</span><span class="na">addSink</span><span class="o">(</span><span class="k">new</span> <span class="n">FlinkPulsarProducer</span><span class="o"><>(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">serviceUrl</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">outputTopic</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">AuthenticationDisabled</span><span class="o">(),</span>
-</span></span><span class="line"><span class="cl"> <span class="n">wordWithCount</span> <span class="o">-></span> <span class="n">wordWithCount</span><span class="o">.</span><span class="na">toString</span><span class="o">().</span><span class="na">getBytes</span><span class="o">(</span><span class="n">UTF_8</span><span class="o">),</span>
-</span></span><span class="line"><span class="cl"> <span class="n">wordWithCount</span> <span class="o">-></span> <span class="n">wordWithCount</span><span class="o">.</span><span class="na">word</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="o">);</span>
-</span></span></code></pre></div><p>Another integration between the two frameworks that developers can take advantage of includes using Pulsar as both a streaming source and a streaming table sink for Flink SQL or Table API queries as shown in the example below:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// obtain a DataStream with words
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">words</span> <span class="o">=</span> <span class="o">...</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// register DataStream as Table "words" with two attributes ("word", "ts").
-</span></span></span><span class="line"><span class="cl"><span class="c1">// "ts" is an event-time timestamp.
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">tableEnvironment</span><span class="o">.</span><span class="na">registerDataStream</span><span class="o">(</span><span class="s">"words"</span><span class="o">,</span> <span class="n">words</span><span class="o">,</span> <span class="s">"word, ts.rowtime"</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// create a TableSink that produces to Pulsar
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">TableSink</span> <span class="n">sink</span> <span class="o">=</span> <span class="k">new</span> <span class="n">PulsarJsonTableSink</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">serviceUrl</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">outputTopic</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">AuthenticationDisabled</span><span class="o">(),</span>
-</span></span><span class="line"><span class="cl"> <span class="n">ROUTING_KEY</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// register Pulsar TableSink as table "wc"
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">tableEnvironment</span><span class="o">.</span><span class="na">registerTableSink</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="s">"wc"</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">sink</span><span class="o">.</span><span class="na">configure</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">String</span><span class="o">[]{</span><span class="s">"word"</span><span class="o">,</span> <span class="s">"cnt"</span><span class="o">},</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">TypeInformation</span><span class="o">[]{</span><span class="n">Types</span><span class="o">.</span><span class="na">STRING</span><span class="o">,</span> <span class="n">Types</span><span class="o">.</span><span class="na">LONG</span><span class="o">}));</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// count words per 5 seconds and write result to table "wc"
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">tableEnvironment</span><span class="o">.</span><span class="na">sqlUpdate</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="s">"INSERT INTO wc "</span> <span class="o">+</span>
-</span></span><span class="line"><span class="cl"> <span class="s">"SELECT word, COUNT(*) AS cnt "</span> <span class="o">+</span>
-</span></span><span class="line"><span class="cl"> <span class="s">"FROM words "</span> <span class="o">+</span>
-</span></span><span class="line"><span class="cl"> <span class="s">"GROUP BY word, TUMBLE(ts, INTERVAL '5' SECOND)"</span><span class="o">);</span>
-</span></span></code></pre></div><p>Finally, Flink integrates with Pulsar for batch workloads as a batch sink where all results get pushed to Pulsar after Apache Flink has completed the computation in a static data set. Such an example is shown below:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// obtain DataSet from arbitrary computation
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataSet</span><span class="o"><</span><span class="n">WordWithCount</span><span class="o">></span> <span class="n">wc</span> <span class="o">=</span> <span class="o">...</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// create PulsarOutputFormat instance
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">OutputFormat</span> <span class="n">pulsarOutputFormat</span> <span class="o">=</span> <span class="k">new</span> <span class="n">PulsarOutputFormat</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">serviceUrl</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">topic</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">AuthenticationDisabled</span><span class="o">(),</span>
-</span></span><span class="line"><span class="cl"> <span class="n">wordWithCount</span> <span class="o">-></span> <span class="n">wordWithCount</span><span class="o">.</span><span class="na">toString</span><span class="o">().</span><span class="na">getBytes</span><span class="o">());</span>
-</span></span><span class="line"><span class="cl"><span class="c1">// write DataSet to Pulsar
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">wc</span><span class="o">.</span><span class="na">output</span><span class="o">(</span><span class="n">pulsarOutputFormat</span><span class="o">);</span>
-</span></span></code></pre></div><h2 id="conclusion">
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// create and configure Pulsar consumer</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">PulsarSourceBuilder</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="n">builder</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">PulsarSourceBuilder</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">builder</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">SimpleStringSchema</span><span class="p">())</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">serviceUrl</span><span class="p">(</span><span class="n">serviceUrl</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">topic</span><span class="p">(</span><span class="n">inputTopic</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">subscriptionName</span><span class="p">(</span><span class="n">subscription</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">SourceFunction</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">src</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">builder</span><span class="p">.</span><span class="na">build</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// ingest DataStream with Pulsar consumer</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">words</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">env</span><span class="p">.</span><span class="na">addSource</span><span class="p">(</span><span class="n">src</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// perform computation on DataStream (here a simple WordCount)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">WordWithCount</span><span class="o">></span><span class="w"> </span><span class="n">wc</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">words</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">flatMap</span><span class="p">((</span><span class="n">FlatMapFunction</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">WordWithCount</span><span class="o">></span><span class="p">)</span><span class="w"> </span><span class="p">(</span><span class="n">word</span><span class="p">,</span>< [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">collector</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">WordWithCount</span><span class="p">(</span><span class="n">word</span><span class="p">,</span><span class="w"> </span><span class="n">1</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">})</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">returns</span><span class="p">(</span><span class="n">WordWithCount</span><span class="p">.</span><span class="na">class</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">keyBy</span><span class="p">(</span><span class="s">"word"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">timeWindow</span><span class="p">(</span><span class="n">Time</span><span class="p">.</span><span class="na">seconds</span><span class="p">(</span><span class="n">5</span><span class="p">))</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">reduce</span><span class="p">((</span><span class="n">ReduceFunction</span><span class="o"><</span><span class="n">WordWithCount</span><span class="o">></span><span class="p">)</span><span class="w"> </span><span class="p">(</span><span class="n">c1</span><span class="p">,</span><span class="w"> </span><span class="n">c2</span><span class="p">)</span><span cla [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">WordWithCount</span><span class="p">(</span><span class="n">c1</span><span class="p">.</span><span class="na">word</span><span class="p">,</span><span class="w"> </span><span class="n">c1</span><span class="p">.</span><span class="na">count</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">c2</ [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// emit result via Pulsar producer</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">wc</span><span class="p">.</span><span class="na">addSink</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">FlinkPulsarProducer</span><span class="o"><></span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">serviceUrl</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">outputTopic</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">AuthenticationDisabled</span><span class="p">(),</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">wordWithCount</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">wordWithCount</span><span class="p">.</span><span class="na">toString</span><span class="p">().</span><span class="na">getBytes</span><span class="p">(</span><span class="n">UTF_8</span><span class="p">),</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">wordWithCount</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">wordWithCount</span><span class="p">.</span><span class="na">word</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><p>Another integration between the two frameworks that developers can take advantage of includes using Pulsar as both a streaming source and a streaming table sink for Flink SQL or Table API queries as shown in the example below:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// obtain a DataStream with words</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">words</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">...</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// register DataStream as Table "words" with two attributes ("word", "ts"). </span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// "ts" is an event-time timestamp.</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">tableEnvironment</span><span class="p">.</span><span class="na">registerDataStream</span><span class="p">(</span><span class="s">"words"</span><span class="p">,</span><span class="w"> </span><span class="n">words</span><span class="p">,</span><span class="w"> </span><span class="s">"word, ts.rowtime"</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// create a TableSink that produces to Pulsar</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">TableSink</span><span class="w"> </span><span class="n">sink</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">PulsarJsonTableSink</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">serviceUrl</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">outputTopic</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">AuthenticationDisabled</span><span class="p">(),</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">ROUTING_KEY</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// register Pulsar TableSink as table "wc"</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">tableEnvironment</span><span class="p">.</span><span class="na">registerTableSink</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="s">"wc"</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">sink</span><span class="p">.</span><span class="na">configure</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">String</span><span class="o">[]</span><span class="p">{</span><span class="s">"word"</span><span class="p">,</span><span class="w"> </span><span class="s">"cnt"</span><span class="p">},</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">TypeInformation</span><span class="o">[]</span><span class="p">{</span><span class="n">Types</span><span class="p">.</span><span class="na">STRING</span><span class="p">,</span><span class="w"> </span><span class="n">Types</span><span class="p">.</span><span class="na">LONG</span><span class="p">}));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// count words per 5 seconds and write result to table "wc"</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">tableEnvironment</span><span class="p">.</span><span class="na">sqlUpdate</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="s">"INSERT INTO wc "</span><span class="w"> </span><span class="o">+</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="s">"SELECT word, COUNT(*) AS cnt "</span><span class="w"> </span><span class="o">+</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="s">"FROM words "</span><span class="w"> </span><span class="o">+</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="s">"GROUP BY word, TUMBLE(ts, INTERVAL '5' SECOND)"</span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><p>Finally, Flink integrates with Pulsar for batch workloads as a batch sink where all results get pushed to Pulsar after Apache Flink has completed the computation in a static data set. Such an example is shown below:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// obtain DataSet from arbitrary computation</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataSet</span><span class="o"><</span><span class="n">WordWithCount</span><span class="o">></span><span class="w"> </span><span class="n">wc</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">...</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// create PulsarOutputFormat instance</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">OutputFormat</span><span class="w"> </span><span class="n">pulsarOutputFormat</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">PulsarOutputFormat</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">serviceUrl</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">topic</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">AuthenticationDisabled</span><span class="p">(),</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">wordWithCount</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">wordWithCount</span><span class="p">.</span><span class="na">toString</span><span class="p">().</span><span class="na">getBytes</span><span class="p">());</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// write DataSet to Pulsar</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">wc</span><span class="p">.</span><span class="na">output</span><span class="p">(</span><span class="n">pulsarOutputFormat</span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><h2 id="conclusion">
Conclusion
<a class="anchor" href="#conclusion">#</a>
</h2>
diff --git a/content/2019/05/14/flux-capacitor-huh-temporal-tables-and-joins-in-streaming-sql/index.html b/content/2019/05/14/flux-capacitor-huh-temporal-tables-and-joins-in-streaming-sql/index.html
index 90c5e8c8a..c8f928d79 100644
--- a/content/2019/05/14/flux-capacitor-huh-temporal-tables-and-joins-in-streaming-sql/index.html
+++ b/content/2019/05/14/flux-capacitor-huh-temporal-tables-and-joins-in-streaming-sql/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -22,18 +22,17 @@
<meta property="og:url" content="https://flink.apache.org/2019/05/14/flux-capacitor-huh-temporal-tables-and-joins-in-streaming-sql/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-05-14T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-05-14T12:00:00+00:00" />
-
<title>Flux capacitor, huh? Temporal Tables and Joins in Streaming SQL | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -512,38 +511,38 @@ https://github.com/alex-shpak/hugo-book
</li>
</ul>
<p>Going back to our example use case, a temporal table is just what we need to model the conversion rate data such as to make it useful for point-in-time querying. Temporal table functions are implemented as an extension of Flink’s generic <a href="//nightlies.apache.org/flink/flink-docs-release-1.8/dev/table/udfs.html#table-functions">table function</a> class and can be defined in the same straightforward way to be used with the Table API or SQL parser.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">org.apache.flink.table.functions.TemporalTableFunction</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="o">(...)</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// Get the stream and table environments.
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">StreamExecutionEnvironment</span> <span class="n">env</span> <span class="o">=</span> <span class="n">StreamExecutionEnvironment</span><span class="o">.</span><span class="na">getExecutionEnvironment</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"><span class="n">StreamTableEnvironment</span> <span class="n">tEnv</span> <span class="o">=</span> <span class="n">StreamTableEnvironment</span><span class="o">.</span><span class="na">getTableEnvironment</span><span class="o">(</span><span class="n">env</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// Provide a sample static data set of the rates history table.
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">List</span> <span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Long</span><span class="o">>></span><span class="n">ratesHistoryData</span> <span class="o">=</span><span class="k">new</span> <span class="n">ArrayList</span><span class="o"><>();</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">ratesHistoryData</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="n">Tuple2</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"USD"</span><span class="o">,</span> <span class="n">102L</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"><span class="n">ratesHistoryData</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="n">Tuple2</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"EUR"</span><span class="o">,</span> <span class="n">114L</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"><span class="n">ratesHistoryData</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="n">Tuple2</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"YEN"</span><span class="o">,</span> <span class="n">1L</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"><span class="n">ratesHistoryData</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="n">Tuple2</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"EUR"</span><span class="o">,</span> <span class="n">116L</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"><span class="n">ratesHistoryData</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="n">Tuple2</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"USD"</span><span class="o">,</span> <span class="n">105L</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// Create and register an example table using the sample data set.
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Long</span><span class="o">>></span> <span class="n">ratesHistoryStream</span> <span class="o">=</span> <span class="n">env</span><span class="o">.</span><span class="na">fromCollection</span><span class="o">(</span><sp [...]
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">Table</span> <span class="n">ratesHistory</span> <span class="o">=</span> <span class="n">tEnv</span><span class="o">.</span><span class="na">fromDataStream</span><span class="o">(</span><span class="n">ratesHistoryStream</span><span class="o">,</span> <span class="s">"r_currency, r_rate, r_proctime.proctime"</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">tEnv</span><span class="o">.</span><span class="na">registerTable</span><span class="o">(</span><span class="s">"RatesHistory"</span><span class="o">,</span> <span class="n">ratesHistory</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// Create and register the temporal table function "rates".
-</span></span></span><span class="line"><span class="cl"><span class="c1">// Define "r_proctime" as the versioning field and "r_currency" as the primary key.
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">TemporalTableFunction</span> <span class="n">rates</span> <span class="o">=</span> <span class="n">ratesHistory</span><span class="o">.</span><span class="na">createTemporalTableFunction</span><span class="o">(</span><span class="s">"r_proctime"</span><span class="o">,</span> <span class="s">"r_currency"</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">tEnv</span><span class="o">.</span><span class="na">registerFunction</span><span class="o">(</span><span class="s">"Rates"</span><span class="o">,</span> <span class="n">rates</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="o">(...)</span>
-</span></span></code></pre></div><p>What does this <strong>Rates</strong> function do, in practice? Imagine we would like to check what the conversion rates looked like at a given time — say, 11:00. We could simply do something like:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kn">import</span><span class="w"> </span><span class="nn">org.apache.flink.table.functions.TemporalTableFunction</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">(...)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// Get the stream and table environments.</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">StreamExecutionEnvironment</span><span class="w"> </span><span class="n">env</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">StreamExecutionEnvironment</span><span class="p">.</span><span class="na">getExecutionEnvironment</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">StreamTableEnvironment</span><span class="w"> </span><span class="n">tEnv</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">StreamTableEnvironment</span><span class="p">.</span><span class="na">getTableEnvironment</span><span class="p">(</span><span class="n">env</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// Provide a sample static data set of the rates history table.</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">List</span><span class="w"> </span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">Long</span><span class="o">>></span><span class="n">ratesHistoryData</span><span class="w"> </span><span class="o">=</span><span class="k">new</span><span class="w"> </span><span [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">ratesHistoryData</span><span class="p">.</span><span class="na">add</span><span class="p">(</span><span class="n">Tuple2</span><span class="p">.</span><span class="na">of</span><span class="p">(</span><span class="s">"USD"</span><span class="p">,</span><span class="w"> </span><span class="n">102L</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">ratesHistoryData</span><span class="p">.</span><span class="na">add</span><span class="p">(</span><span class="n">Tuple2</span><span class="p">.</span><span class="na">of</span><span class="p">(</span><span class="s">"EUR"</span><span class="p">,</span><span class="w"> </span><span class="n">114L</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">ratesHistoryData</span><span class="p">.</span><span class="na">add</span><span class="p">(</span><span class="n">Tuple2</span><span class="p">.</span><span class="na">of</span><span class="p">(</span><span class="s">"YEN"</span><span class="p">,</span><span class="w"> </span><span class="n">1L</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">ratesHistoryData</span><span class="p">.</span><span class="na">add</span><span class="p">(</span><span class="n">Tuple2</span><span class="p">.</span><span class="na">of</span><span class="p">(</span><span class="s">"EUR"</span><span class="p">,</span><span class="w"> </span><span class="n">116L</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">ratesHistoryData</span><span class="p">.</span><span class="na">add</span><span class="p">(</span><span class="n">Tuple2</span><span class="p">.</span><span class="na">of</span><span class="p">(</span><span class="s">"USD"</span><span class="p">,</span><span class="w"> </span><span class="n">105L</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// Create and register an example table using the sample data set.</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">Long</span><span class="o">>></span><span class="w"> </span><span class="n">ratesHistoryStream</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">env</spa [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">Table</span><span class="w"> </span><span class="n">ratesHistory</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tEnv</span><span class="p">.</span><span class="na">fromDataStream</span><span class="p">(</span><span class="n">ratesHistoryStream</span><span class="p">,</span><span class="w"> </span><span class="s">"r_currency, r_rate, r_proctime [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">tEnv</span><span class="p">.</span><span class="na">registerTable</span><span class="p">(</span><span class="s">"RatesHistory"</span><span class="p">,</span><span class="w"> </span><span class="n">ratesHistory</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// Create and register the temporal table function "rates".</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// Define "r_proctime" as the versioning field and "r_currency" as the primary key.</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">TemporalTableFunction</span><span class="w"> </span><span class="n">rates</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">ratesHistory</span><span class="p">.</span><span class="na">createTemporalTableFunction</span><span class="p">(</span><span class="s">"r_proctime"</span><span class="p">,</span><span class="w"> </span><span class="s"> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">tEnv</span><span class="p">.</span><span class="na">registerFunction</span><span class="p">(</span><span class="s">"Rates"</span><span class="p">,</span><span class="w"> </span><span class="n">rates</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">(...)</span><span class="w">
+</span></span></span></code></pre></div><p>What does this <strong>Rates</strong> function do, in practice? Imagine we would like to check what the conversion rates looked like at a given time — say, 11:00. We could simply do something like:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="cl"><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">Rates</span><span class="p">(</span><span class="s1">'11:00'</span><span class="p">);</span><span class="w">
</span></span></span></code></pre></div><center>
<img src="/img/blog/2019-05-13-temporal-tables/TemporalTables2.png" width="650px" alt="Point-in-time Querying"/>
diff --git a/content/2019/05/14/temporal-tables.html b/content/2019/05/14/temporal-tables.html
index 19d9ced84..d4a5c8d32 100644
--- a/content/2019/05/14/temporal-tables.html
+++ b/content/2019/05/14/temporal-tables.html
@@ -1 +1,10 @@
-<!DOCTYPE html><html><head><title>https://flink.apache.org/2019/05/14/flux-capacitor-huh-temporal-tables-and-joins-in-streaming-sql/</title><link rel="canonical" href="https://flink.apache.org/2019/05/14/flux-capacitor-huh-temporal-tables-and-joins-in-streaming-sql/"/><meta name="robots" content="noindex"><meta charset="utf-8" /><meta http-equiv="refresh" content="0; url=https://flink.apache.org/2019/05/14/flux-capacitor-huh-temporal-tables-and-joins-in-streaming-sql/" /></head></html>
\ No newline at end of file
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <title>https://flink.apache.org/2019/05/14/flux-capacitor-huh-temporal-tables-and-joins-in-streaming-sql/</title>
+ <link rel="canonical" href="https://flink.apache.org/2019/05/14/flux-capacitor-huh-temporal-tables-and-joins-in-streaming-sql/">
+ <meta name="robots" content="noindex">
+ <meta charset="utf-8">
+ <meta http-equiv="refresh" content="0; url=https://flink.apache.org/2019/05/14/flux-capacitor-huh-temporal-tables-and-joins-in-streaming-sql/">
+ </head>
+</html>
diff --git a/content/2019/05/17/state-ttl-in-flink-1.8.0-how-to-automatically-cleanup-application-state-in-apache-flink/index.html b/content/2019/05/17/state-ttl-in-flink-1.8.0-how-to-automatically-cleanup-application-state-in-apache-flink/index.html
index deddc754b..ad7975126 100644
--- a/content/2019/05/17/state-ttl-in-flink-1.8.0-how-to-automatically-cleanup-application-state-in-apache-flink/index.html
+++ b/content/2019/05/17/state-ttl-in-flink-1.8.0-how-to-automatically-cleanup-application-state-in-apache-flink/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ In this post, we motivate the State TTL feature and discuss its use cases." />
<meta property="og:url" content="https://flink.apache.org/2019/05/17/state-ttl-in-flink-1.8.0-how-to-automatically-cleanup-application-state-in-apache-flink/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-05-17T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-05-17T12:00:00+00:00" />
-
<title>State TTL in Flink 1.8.0: How to Automatically Cleanup Application State in Apache Flink | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -521,21 +520,21 @@ Recent developments around data privacy regulations, such as the General Data Pr
</h1>
<p>The 1.6.0 release of Apache Flink introduced the State TTL feature. It enabled developers of stream processing applications to configure the state of operators to expire and be cleaned up after a defined timeout (time-to-live). In Flink 1.8.0 the feature was extended, including continuous cleanup of old entries for both the RocksDB and the heap state backends (FSStateBackend and MemoryStateBackend), enabling a continuous cleanup process of old entries (according to the TTL setting).</p>
<p>In Flink’s DataStream API, application state is defined by a <a href="//nightlies.apache.org/flink/flink-docs-release-1.8/dev/stream/state/state.html#using-managed-keyed-state">state descriptor</a>. State TTL is configured by passing a <code>StateTtlConfiguration</code> object to a state descriptor. The following Java example shows how to create a state TTL configuration and provide it to the state descriptor that holds the last login time of a user as a <code>Long</code> value:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">org.apache.flink.api.common.state.StateTtlConfig</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">org.apache.flink.api.common.time.Time</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">org.apache.flink.api.common.state.ValueStateDescriptor</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">StateTtlConfig</span> <span class="n">ttlConfig</span> <span class="o">=</span> <span class="n">StateTtlConfig</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">newBuilder</span><span class="o">(</span><span class="n">Time</span><span class="o">.</span><span class="na">days</span><span class="o">(</span><span class="n">7</span><span class="o">))</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">setUpdateType</span><span class="o">(</span><span class="n">StateTtlConfig</span><span class="o">.</span><span class="na">UpdateType</span><span class="o">.</span><span class="na">OnCreateAndWrite</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">setStateVisibility</span><span class="o">(</span><span class="n">StateTtlConfig</span><span class="o">.</span><span class="na">StateVisibility</span><span class="o">.</span><span class="na">NeverReturnExpired</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">build</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">ValueStateDescriptor</span><span class="o"><</span><span class="n">Long</span><span class="o">></span> <span class="n">lastUserLogin</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">ValueStateDescriptor</span><span class="o"><>(</span><span class="s">"lastUserLogin"</span><span class="o">,</span> <span class="n">Long</span><span class="o">.</span><span class="na">class</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">lastUserLogin</span><span class="o">.</span><span class="na">enableTimeToLive</span><span class="o">(</span><span class="n">ttlConfig</span><span class="o">);</span>
-</span></span></code></pre></div><p>Flink provides multiple options to configure the behavior of the state TTL functionality.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kn">import</span><span class="w"> </span><span class="nn">org.apache.flink.api.common.state.StateTtlConfig</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kn">import</span><span class="w"> </span><span class="nn">org.apache.flink.api.common.time.Time</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kn">import</span><span class="w"> </span><span class="nn">org.apache.flink.api.common.state.ValueStateDescriptor</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">StateTtlConfig</span><span class="w"> </span><span class="n">ttlConfig</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">StateTtlConfig</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">newBuilder</span><span class="p">(</span><span class="n">Time</span><span class="p">.</span><span class="na">days</span><span class="p">(</span><span class="n">7</span><span class="p">))</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">setUpdateType</span><span class="p">(</span><span class="n">StateTtlConfig</span><span class="p">.</span><span class="na">UpdateType</span><span class="p">.</span><span class="na">OnCreateAndWrite</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">setStateVisibility</span><span class="p">(</span><span class="n">StateTtlConfig</span><span class="p">.</span><span class="na">StateVisibility</span><span class="p">.</span><span class="na">NeverReturnExpired</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">build</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">ValueStateDescriptor</span><span class="o"><</span><span class="n">Long</span><span class="o">></span><span class="w"> </span><span class="n">lastUserLogin</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">ValueStateDescriptor</span><span class="o"><></span><span class="p">(</span><span class="s">"lastUserLogin"</span><span class="p">,</span><span class="w"> </span><span class="n">Long</span><span class="p">.</span><span class="na">class</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">lastUserLogin</span><span class="p">.</span><span class="na">enableTimeToLive</span><span class="p">(</span><span class="n">ttlConfig</span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><p>Flink provides multiple options to configure the behavior of the state TTL functionality.</p>
<ul>
<li>
<p><strong>When is the Time-to-Live reset?</strong>
@@ -563,23 +562,23 @@ With Flink 1.8.0, users can only define a state TTL in terms of processing time.
<a class="anchor" href="#keep-full-state-snapshots-clean">#</a>
</h2>
<p>Flink 1.6.0 already supported automatic eviction of the expired state when a full snapshot for a checkpoint or savepoint is taken. Note that state eviction is not applied for incremental checkpoints. State eviction on full snapshots must be explicitly enabled as shown in the following example:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">StateTtlConfig</span> <span class="n">ttlConfig</span> <span class="o">=</span> <span class="n">StateTtlConfig</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">newBuilder</span><span class="o">(</span><span class="n">Time</span><span class="o">.</span><span class="na">days</span><span class="o">(</span><span class="n">7</span><span class="o">))</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">cleanupFullSnapshot</span><span class="o">()</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">build</span><span class="o">();</span>
-</span></span></code></pre></div><p>The local storage stays untouched but the size of the stored snapshot is reduced. The local state of an operator will only be cleaned up when the operator reloads its state from a snapshot, i.e. in case of recovery or when starting from a savepoint.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">StateTtlConfig</span><span class="w"> </span><span class="n">ttlConfig</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">StateTtlConfig</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">newBuilder</span><span class="p">(</span><span class="n">Time</span><span class="p">.</span><span class="na">days</span><span class="p">(</span><span class="n">7</span><span class="p">))</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">cleanupFullSnapshot</span><span class="p">()</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">build</span><span class="p">();</span><span class="w">
+</span></span></span></code></pre></div><p>The local storage stays untouched but the size of the stored snapshot is reduced. The local state of an operator will only be cleaned up when the operator reloads its state from a snapshot, i.e. in case of recovery or when starting from a savepoint.</p>
<p>Due to these limitations, applications still need to actively remove state after it expired in Flink 1.6.0. To improve the user experience, Flink 1.8.0 introduces two more autonomous cleanup strategies, one for each of Flink’s two state backend types. We describe them below.</p>
<h2 id="incremental-cleanup-in-heap-state-backends">
Incremental cleanup in Heap state backends
<a class="anchor" href="#incremental-cleanup-in-heap-state-backends">#</a>
</h2>
<p>This approach is specific to the Heap state backends (FSStateBackend and MemoryStateBackend). The idea is that the storage backend keeps a lazy global iterator over all state entries. Certain events, for instance state access, trigger an incremental cleanup. Every time an incremental cleanup is triggered, the iterator is advanced. The traversed state entries are checked and expired once are removed. The following code example shows how to enable incremental cleanup:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">StateTtlConfig</span> <span class="n">ttlConfig</span> <span class="o">=</span> <span class="n">StateTtlConfig</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">newBuilder</span><span class="o">(</span><span class="n">Time</span><span class="o">.</span><span class="na">days</span><span class="o">(</span><span class="n">7</span><span class="o">))</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// check 10 keys for every state access
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="o">.</span><span class="na">cleanupIncrementally</span><span class="o">(</span><span class="n">10</span><span class="o">,</span> <span class="kc">false</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">build</span><span class="o">();</span>
-</span></span></code></pre></div><p>If enabled, every state access triggers a cleanup step. For every clean up step, a certain number of state entries are checked for expiration. There are two tuning parameters. The first defines the number of state entries to check for each cleanup step. The second parameter is a flag to trigger a cleanup step after each processed record, additionally to each state access.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">StateTtlConfig</span><span class="w"> </span><span class="n">ttlConfig</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">StateTtlConfig</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">newBuilder</span><span class="p">(</span><span class="n">Time</span><span class="p">.</span><span class="na">days</span><span class="p">(</span><span class="n">7</span><span class="p">))</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// check 10 keys for every state access</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">cleanupIncrementally</span><span class="p">(</span><span class="n">10</span><span class="p">,</span><span class="w"> </span><span class="kc">false</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">build</span><span class="p">();</span><span class="w">
+</span></span></span></code></pre></div><p>If enabled, every state access triggers a cleanup step. For every clean up step, a certain number of state entries are checked for expiration. There are two tuning parameters. The first defines the number of state entries to check for each cleanup step. The second parameter is a flag to trigger a cleanup step after each processed record, additionally to each state access.</p>
<p>There are two important caveats about this approach:</p>
<ul>
<li>The first one is that the time spent for the incremental cleanup increases the record processing latency.</li>
@@ -591,11 +590,11 @@ With Flink 1.8.0, users can only define a state TTL in terms of processing time.
</h2>
<p>If your application uses the RocksDB state backend, you can enable another cleanup strategy which is based on a Flink specific compaction filter. RocksDB periodically runs asynchronous compactions to merge state updates and reduce storage. The Flink compaction filter checks the expiration timestamp of state entries with TTL and discards all expired values.</p>
<p>The first step to activate this feature is to configure the RocksDB state backend by setting the following Flink configuration option: <code>state.backend.rocksdb.ttl.compaction.filter.enabled</code>. Once the RocksDB state backend is configured, the compaction cleanup strategy is enabled for a state as shown in the following code example:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">StateTtlConfig</span> <span class="n">ttlConfig</span> <span class="o">=</span> <span class="n">StateTtlConfig</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">newBuilder</span><span class="o">(</span><span class="n">Time</span><span class="o">.</span><span class="na">days</span><span class="o">(</span><span class="n">7</span><span class="o">))</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">cleanupInRocksdbCompactFilter</span><span class="o">()</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">build</span><span class="o">();</span>
-</span></span></code></pre></div><p>Keep in mind that calling the Flink TTL filter slows down the RocksDB compaction.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">StateTtlConfig</span><span class="w"> </span><span class="n">ttlConfig</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">StateTtlConfig</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">newBuilder</span><span class="p">(</span><span class="n">Time</span><span class="p">.</span><span class="na">days</span><span class="p">(</span><span class="n">7</span><span class="p">))</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">cleanupInRocksdbCompactFilter</span><span class="p">()</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">build</span><span class="p">();</span><span class="w">
+</span></span></span></code></pre></div><p>Keep in mind that calling the Flink TTL filter slows down the RocksDB compaction.</p>
<h2 id="eager-state-cleanup-with-timers">
Eager State Cleanup with Timers
<a class="anchor" href="#eager-state-cleanup-with-timers">#</a>
diff --git a/content/2019/05/19/state-ttl.html b/content/2019/05/19/state-ttl.html
index 4ff0566f9..f02c581ad 100644
--- a/content/2019/05/19/state-ttl.html
+++ b/content/2019/05/19/state-ttl.html
@@ -1 +1,10 @@
-<!DOCTYPE html><html><head><title>https://flink.apache.org/2019/05/17/state-ttl-in-flink-1.8.0-how-to-automatically-cleanup-application-state-in-apache-flink/</title><link rel="canonical" href="https://flink.apache.org/2019/05/17/state-ttl-in-flink-1.8.0-how-to-automatically-cleanup-application-state-in-apache-flink/"/><meta name="robots" content="noindex"><meta charset="utf-8" /><meta http-equiv="refresh" content="0; url=https://flink.apache.org/2019/05/17/state-ttl-in-flink-1.8.0-how-t [...]
\ No newline at end of file
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <title>https://flink.apache.org/2019/05/17/state-ttl-in-flink-1.8.0-how-to-automatically-cleanup-application-state-in-apache-flink/</title>
+ <link rel="canonical" href="https://flink.apache.org/2019/05/17/state-ttl-in-flink-1.8.0-how-to-automatically-cleanup-application-state-in-apache-flink/">
+ <meta name="robots" content="noindex">
+ <meta charset="utf-8">
+ <meta http-equiv="refresh" content="0; url=https://flink.apache.org/2019/05/17/state-ttl-in-flink-1.8.0-how-to-automatically-cleanup-application-state-in-apache-flink/">
+ </head>
+</html>
diff --git a/content/2019/06/05/a-deep-dive-into-flinks-network-stack/index.html b/content/2019/06/05/a-deep-dive-into-flinks-network-stack/index.html
index 9a55acb35..8df24fde7 100644
--- a/content/2019/06/05/a-deep-dive-into-flinks-network-stack/index.html
+++ b/content/2019/06/05/a-deep-dive-into-flinks-network-stack/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -15,25 +15,24 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
-<meta name="description" content=".tg {border-collapse:collapse;border-spacing:0;} .tg td{padding:10px 20px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;} .tg th{padding:10px 20px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;background-color:#eff0f1;} .tg .tg-wide{padding:10px 30px;} .tg .tg-top{vertical-align:top} .tg .tg-center{text-align:center;vertical-align:center} Flink’s network stack is one of the core components that make up the [...]
+<meta name="description" content="Flink’s network stack is one of the core components that make up the flink-runtime module and sit at the heart of every Flink job. It connects individual work units (subtasks) from all TaskManagers. This is where your streamed-in data flows through and it is therefore crucial to the performance of your Flink job for both the throughput as well as latency you observe. In contrast to the coordination channels between TaskManagers and JobManagers which are [...]
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="A Deep-Dive into Flink's Network Stack" />
-<meta property="og:description" content=".tg {border-collapse:collapse;border-spacing:0;} .tg td{padding:10px 20px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;} .tg th{padding:10px 20px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;background-color:#eff0f1;} .tg .tg-wide{padding:10px 30px;} .tg .tg-top{vertical-align:top} .tg .tg-center{text-align:center;vertical-align:center} Flink’s network stack is one of the core components that make [...]
+<meta property="og:description" content="Flink’s network stack is one of the core components that make up the flink-runtime module and sit at the heart of every Flink job. It connects individual work units (subtasks) from all TaskManagers. This is where your streamed-in data flows through and it is therefore crucial to the performance of your Flink job for both the throughput as well as latency you observe. In contrast to the coordination channels between TaskManagers and JobManagers whi [...]
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2019/06/05/a-deep-dive-into-flinks-network-stack/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-06-05T08:45:00+00:00" />
<meta property="article:modified_time" content="2019-06-05T08:45:00+00:00" />
-
<title>A Deep-Dive into Flink's Network Stack | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2019/06/05/flink-network-stack.html b/content/2019/06/05/flink-network-stack.html
index 51686f7e6..ce7a22af4 100644
--- a/content/2019/06/05/flink-network-stack.html
+++ b/content/2019/06/05/flink-network-stack.html
@@ -1 +1,10 @@
-<!DOCTYPE html><html><head><title>https://flink.apache.org/2019/06/05/a-deep-dive-into-flinks-network-stack/</title><link rel="canonical" href="https://flink.apache.org/2019/06/05/a-deep-dive-into-flinks-network-stack/"/><meta name="robots" content="noindex"><meta charset="utf-8" /><meta http-equiv="refresh" content="0; url=https://flink.apache.org/2019/06/05/a-deep-dive-into-flinks-network-stack/" /></head></html>
\ No newline at end of file
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <title>https://flink.apache.org/2019/06/05/a-deep-dive-into-flinks-network-stack/</title>
+ <link rel="canonical" href="https://flink.apache.org/2019/06/05/a-deep-dive-into-flinks-network-stack/">
+ <meta name="robots" content="noindex">
+ <meta charset="utf-8">
+ <meta http-equiv="refresh" content="0; url=https://flink.apache.org/2019/06/05/a-deep-dive-into-flinks-network-stack/">
+ </head>
+</html>
diff --git a/content/2019/06/26/a-practical-guide-to-broadcast-state-in-apache-flink/index.html b/content/2019/06/26/a-practical-guide-to-broadcast-state-in-apache-flink/index.html
index 6fec2db73..7b67f7794 100644
--- a/content/2019/06/26/a-practical-guide-to-broadcast-state-in-apache-flink/index.html
+++ b/content/2019/06/26/a-practical-guide-to-broadcast-state-in-apache-flink/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -16,26 +16,25 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="Since version 1.5.0, Apache Flink features a new type of state which is called Broadcast State. In this post, we explain what Broadcast State is, and show an example of how it can be applied to an application that evaluates dynamic patterns on an event stream. We walk you through the processing steps and the source code to implement this application in practice.
-What is Broadcast State? # The Broadcast State can be used to combine and jointly process two streams of events in a specific way.">
+What is Broadcast State? # The Broadcast State can be used to combine and jointly process two streams of events in a specific way.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="A Practical Guide to Broadcast State in Apache Flink" />
<meta property="og:description" content="Since version 1.5.0, Apache Flink features a new type of state which is called Broadcast State. In this post, we explain what Broadcast State is, and show an example of how it can be applied to an application that evaluates dynamic patterns on an event stream. We walk you through the processing steps and the source code to implement this application in practice.
-What is Broadcast State? # The Broadcast State can be used to combine and jointly process two streams of events in a specific way." />
+What is Broadcast State? # The Broadcast State can be used to combine and jointly process two streams of events in a specific way." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2019/06/26/a-practical-guide-to-broadcast-state-in-apache-flink/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-06-26T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-06-26T12:00:00+00:00" />
-
<title>A Practical Guide to Broadcast State in Apache Flink | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -552,9 +551,9 @@ The website implements a streaming application that detects a pattern on the str
</h2>
<p>Until now, we conceptually discussed the application and explained how it uses broadcast state to evaluate dynamic patterns over event streams. Next, we’ll show how to implement the example application with Flink’s DataStream API and the broadcast state feature.</p>
<p>Let’s start with the input data of the application. We have two data streams, actions, and patterns. At this point, we don’t really care where the streams come from. The streams could be ingested from Apache Kafka or Kinesis or any other system. Action and Pattern are Pojos with two fields each:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">Action</span><span class="o">></span> <span class="n">actions</span> <span class="o">=</span> <span class="o">???</span>
-</span></span><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">Pattern</span><span class="o">></span> <span class="n">patterns</span> <span class="o">=</span> <span class="o">???</span>
-</span></span></code></pre></div><p><code>Action</code> and <code>Pattern</code> are Pojos with two fields each:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">Action</span><span class="o">></span><span class="w"> </span><span class="n">actions</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">???</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Pattern</span><span class="o">></span><span class="w"> </span><span class="n">patterns</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">???</span><span class="w">
+</span></span></span></code></pre></div><p><code>Action</code> and <code>Pattern</code> are Pojos with two fields each:</p>
<ul>
<li>
<p><code>Action: Long userId, String action</code></p>
@@ -564,80 +563,80 @@ The website implements a streaming application that detects a pattern on the str
</li>
</ul>
<p>As a first step, we key the action stream on the <code>userId</code> attribute.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">KeyedStream</span><span class="o"><</span><span class="n">Action</span><span class="o">,</span> <span class="n">Long</span><span class="o">></span> <span class="n">actionsByUser</span> <span class="o">=</span> <span class="n">actions</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">keyBy</span><span class="o">((</span><span class="n">KeySelector</span><span class="o"><</span><span class="n">Action</span><span class="o">,</span> <span class="n">Long</span><span class="o">>)</span> <span class="n">action</span> <span class="o">-></span> <span class="n">action</span><span class="o">.</span><span class="na">userId</span><span class="o">);</span>
-</span></span></code></pre></div><p>Next, we prepare the broadcast state. Broadcast state is always represented as <code>MapState</code>, the most versatile state primitive that Flink provides.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">MapStateDescriptor</span><span class="o"><</span><span class="n">Void</span><span class="o">,</span> <span class="n">Pattern</span><span class="o">></span> <span class="n">bcStateDescriptor</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">MapStateDescriptor</span><span class="o"><>(</span><span class="s">"patterns"</span><span class="o">,</span> <span class="n">Types</span><span class="o">.</span><span class="na">VOID</span><span class="o">,</span> <span class="n">Types</span><span class="o">.</span><span class="na">POJO</span><span class="o">(</span><span class="n">Pattern</span><span class="o">.</span><span class [...]
-</span></span></code></pre></div><p>Since our application only evaluates and stores a single <code>Pattern</code> at a time, we configure the broadcast state as a <code>MapState</code> with key type <code>Void</code> and value type <code>Pattern</code>. The <code>Pattern</code> is always stored in the <code>MapState</code> with <code>null</code> as key.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">BroadcastStream</span><span class="o"><</span><span class="n">Pattern</span><span class="o">></span> <span class="n">bcedPatterns</span> <span class="o">=</span> <span class="n">patterns</span><span class="o">.</span><span class="na">broadcast</span><span class="o">(</span><span class="n">bcStateDescriptor</span><span class="o">);</span>
-</span></span></code></pre></div><p>Using the <code>MapStateDescriptor</code> for the broadcast state, we apply the <code>broadcast()</code> transformation on the patterns stream and receive a <code>BroadcastStream bcedPatterns</code>.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Long</span><span class="o">,</span> <span class="n">Pattern</span><span class="o">>></span> <span class="n">matches</span> <span class="o">=</span> <span class="n">actionsByUser</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">connect</span><span class="o">(</span><span class="n">bcedPatterns</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">process</span><span class="o">(</span><span class="k">new</span> <span class="n">PatternEvaluator</span><span class="o">());</span>
-</span></span></code></pre></div><p>After we obtained the keyed <code>actionsByUser</code> stream and the broadcasted <code>bcedPatterns</code> stream, we <code>connect()</code> both streams and apply a <code>PatternEvaluator</code> on the connected streams. <code>PatternEvaluator</code> is a custom function that implements the <code>KeyedBroadcastProcessFunction</code> interface. It applies the pattern matching logic that we discussed before and emits <code>Tuple2<Long, Pattern></ [...]
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">PatternEvaluator</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">extends</span> <span class="n">KeyedBroadcastProcessFunction</span><span class="o"><</span><span class="n">Long</span><span class="o">,</span> <span class="n">Action</span><span class="o">,</span> <span class="n">Pattern</span><span class="o">,</span> <span class="n">Tuple2</span><span class="o"><</span><span class="n">Long</span><span class="o">,</span> <span class="n">Pattern</span><span class="o">>></sp [...]
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">// handle for keyed state (per user)
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">ValueState</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">prevActionState</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// broadcast state descriptor
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">MapStateDescriptor</span><span class="o"><</span><span class="n">Void</span><span class="o">,</span> <span class="n">Pattern</span><span class="o">></span> <span class="n">patternDesc</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">open</span><span class="o">(</span><span class="n">Configuration</span> <span class="n">conf</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// initialize keyed state
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">prevActionState</span> <span class="o">=</span> <span class="n">getRuntimeContext</span><span class="o">().</span><span class="na">getState</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">ValueStateDescriptor</span><span class="o"><>(</span><span class="s">"lastAction"</span><span class="o">,</span> <span class="n">Types</span><span class="o">.</span><span class="na">STRING</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"> <span class="n">patternDesc</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">MapStateDescriptor</span><span class="o"><>(</span><span class="s">"patterns"</span><span class="o">,</span> <span class="n">Types</span><span class="o">.</span><span class="na">VOID</span><span class="o">,</span> <span class="n">Types</span><span class="o">.</span><span class="na">POJO</span><span class="o">(</span><span class="n">Pattern</span><span class="o">.</span><span c [...]
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="cm">/**
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">KeyedStream</span><span class="o"><</span><span class="n">Action</span><span class="p">,</span><span class="w"> </span><span class="n">Long</span><span class="o">></span><span class="w"> </span><span class="n">actionsByUser</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">actions</span>< [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">keyBy</span><span class="p">((</span><span class="n">KeySelector</span><span class="o"><</span><span class="n">Action</span><span class="p">,</span><span class="w"> </span><span class="n">Long</span><span class="o">></span><span class="p">)</span><span class="w"> </span><span class="n">action</span><span class="w"> </span><span class="o">-></span><span class [...]
+</span></span></span></code></pre></div><p>Next, we prepare the broadcast state. Broadcast state is always represented as <code>MapState</code>, the most versatile state primitive that Flink provides.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">MapStateDescriptor</span><span class="o"><</span><span class="n">Void</span><span class="p">,</span><span class="w"> </span><span class="n">Pattern</span><span class="o">></span><span class="w"> </span><span class="n">bcStateDescriptor</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">MapStateDescriptor</span><span class="o"><></span><span class="p">(</span><span class="s">"patterns"</span><span class="p">,</span><span class="w"> </span><span class="n">Types</span><span class="p">.</span><span class="na">VOID</span><span class="p">,</span><span class="w"> </span><span class="n">Types</span><span class="p">.</ [...]
+</span></span></span></code></pre></div><p>Since our application only evaluates and stores a single <code>Pattern</code> at a time, we configure the broadcast state as a <code>MapState</code> with key type <code>Void</code> and value type <code>Pattern</code>. The <code>Pattern</code> is always stored in the <code>MapState</code> with <code>null</code> as key.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">BroadcastStream</span><span class="o"><</span><span class="n">Pattern</span><span class="o">></span><span class="w"> </span><span class="n">bcedPatterns</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">patterns</span><span class="p">.</span><span class="na">broadcast</span><span class="p [...]
+</span></span></span></code></pre></div><p>Using the <code>MapStateDescriptor</code> for the broadcast state, we apply the <code>broadcast()</code> transformation on the patterns stream and receive a <code>BroadcastStream bcedPatterns</code>.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Long</span><span class="p">,</span><span class="w"> </span><span class="n">Pattern</span><span class="o">>></span><span class="w"> </span><span class="n">matches</span><span class="w"> </span><span class="o">=</span>< [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">connect</span><span class="p">(</span><span class="n">bcedPatterns</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">process</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">PatternEvaluator</span><span class="p">());</span><span class="w">
+</span></span></span></code></pre></div><p>After we obtained the keyed <code>actionsByUser</code> stream and the broadcasted <code>bcedPatterns</code> stream, we <code>connect()</code> both streams and apply a <code>PatternEvaluator</code> on the connected streams. <code>PatternEvaluator</code> is a custom function that implements the <code>KeyedBroadcastProcessFunction</code> interface. It applies the pattern matching logic that we discussed before and emits <code>Tuple2<Long, Patter [...]
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">static</span><span class="w"> </span><span class="kd">class</span> <span class="nc">PatternEvaluator</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">extends</span><span class="w"> </span><span class="n">KeyedBroadcastProcessFunction</span><span class="o"><</span><span class="n">Long</span><span class="p">,</span><span class="w"> </span><span class="n">Action</span><span class="p">,</span><span class="w"> </span><span class="n">Pattern</span><span class="p">,</span><span class="w"> </span><span class="n">Tuple2</span><span class="o" [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// handle for keyed state (per user)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">ValueState</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">prevActionState</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// broadcast state descriptor</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">MapStateDescriptor</span><span class="o"><</span><span class="n">Void</span><span class="p">,</span><span class="w"> </span><span class="n">Pattern</span><span class="o">></span><span class="w"> </span><span class="n">patternDesc</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">open</span><span class="p">(</span><span class="n">Configuration</span><span class="w"> </span><span class="n">conf</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// initialize keyed state</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">prevActionState</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">getRuntimeContext</span><span class="p">().</span><span class="na">getState</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">ValueStateDescriptor</span><span class="o"><></span><span class="p">(</span><span class="s">"lastAction"</span><span class="p">,</span><span class="w"> </span><span class="n">Types</span><span class="p">.</span><span class="na">STRING</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">patternDesc</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">MapStateDescriptor</span><span class="o"><></span><span class="p">(</span><span class="s">"patterns"</span><span class="p">,</span><span class="w"> </span><span class="n">Types</span><span class="p">.</span><span class="na">VOID</span><span class="p">,</span><span class="w"> </span><span class="n">Types</span><span class="p" [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="cm">/**
</span></span></span><span class="line"><span class="cl"><span class="cm"> * Called for each user action.
</span></span></span><span class="line"><span class="cl"><span class="cm"> * Evaluates the current pattern against the previous and
</span></span></span><span class="line"><span class="cl"><span class="cm"> * current action of the user.
-</span></span></span><span class="line"><span class="cl"><span class="cm"> */</span>
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">processElement</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Action</span> <span class="n">action</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">ReadOnlyContext</span> <span class="n">ctx</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Collector</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Long</span><span class="o">,</span> <span class="n">Pattern</span><span class="o">>></span> <span class="n">out</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// get current pattern from broadcast state
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">Pattern</span> <span class="n">pattern</span> <span class="o">=</span> <span class="n">ctx</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">getBroadcastState</span><span class="o">(</span><span class="k">this</span><span class="o">.</span><span class="na">patternDesc</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// access MapState with null as VOID default value
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="kc">null</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// get previous action of current user from keyed state
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">String</span> <span class="n">prevAction</span> <span class="o">=</span> <span class="n">prevActionState</span><span class="o">.</span><span class="na">value</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">(</span><span class="n">pattern</span> <span class="o">!=</span> <span class="kc">null</span> <span class="o">&&</span> <span class="n">prevAction</span> <span class="o">!=</span> <span class="kc">null</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// user had an action before, check if pattern matches
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">if</span> <span class="o">(</span><span class="n">pattern</span><span class="o">.</span><span class="na">firstAction</span><span class="o">.</span><span class="na">equals</span><span class="o">(</span><span class="n">prevAction</span><span class="o">)</span> <span class="o">&&</span>
-</span></span><span class="line"><span class="cl"> <span class="n">pattern</span><span class="o">.</span><span class="na">secondAction</span><span class="o">.</span><span class="na">equals</span><span class="o">(</span><span class="n">action</span><span class="o">.</span><span class="na">action</span><span class="o">))</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// MATCH
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">out</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="k">new</span> <span class="n">Tuple2</span><span class="o"><>(</span><span class="n">ctx</span><span class="o">.</span><span class="na">getCurrentKey</span><span class="o">(),</span> <span class="n">pattern</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// update keyed state and remember action for next pattern evaluation
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">prevActionState</span><span class="o">.</span><span class="na">update</span><span class="o">(</span><span class="n">action</span><span class="o">.</span><span class="na">action</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="cm">/**
+</span></span></span><span class="line"><span class="cl"><span class="cm"> */</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">processElement</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Action</span><span class="w"> </span><span class="n">action</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">ReadOnlyContext</span><span class="w"> </span><span class="n">ctx</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Collector</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Long</span><span class="p">,</span><span class="w"> </span><span class="n">Pattern</span><span class="o">>></span><span class="w"> </span><span class="n">out</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><spa [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// get current pattern from broadcast state</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Pattern</span><span class="w"> </span><span class="n">pattern</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">ctx</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">getBroadcastState</span><span class="p">(</span><span class="k">this</span><span class="p">.</span><span class="na">patternDesc</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// access MapState with null as VOID default value</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">get</span><span class="p">(</span><span class="kc">null</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// get previous action of current user from keyed state</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">prevAction</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">prevActionState</span><span class="p">.</span><span class="na">value</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">pattern</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="kc">null</span><span class="w"> </span><span class="o">&&</span><span class="w"> </span><span class="n">prevAction</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="kc">n [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// user had an action before, check if pattern matches</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">pattern</span><span class="p">.</span><span class="na">firstAction</span><span class="p">.</span><span class="na">equals</span><span class="p">(</span><span class="n">prevAction</span><span class="p">)</span><span class="w"> </span><span class="o">&&</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">pattern</span><span class="p">.</span><span class="na">secondAction</span><span class="p">.</span><span class="na">equals</span><span class="p">(</span><span class="n">action</span><span class="p">.</span><span class="na">action</span><span class="p">))</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// MATCH</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">out</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">Tuple2</span><span class="o"><></span><span class="p">(</span><span class="n">ctx</span><span class="p">.</span><span class="na">getCurrentKey</span><span class="p">(),</span><span class="w"> </span><span class="n">pattern</span [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// update keyed state and remember action for next pattern evaluation</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">prevActionState</span><span class="p">.</span><span class="na">update</span><span class="p">(</span><span class="n">action</span><span class="p">.</span><span class="na">action</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="cm">/**
</span></span></span><span class="line"><span class="cl"><span class="cm"> * Called for each new pattern.
</span></span></span><span class="line"><span class="cl"><span class="cm"> * Overwrites the current pattern with the new pattern.
-</span></span></span><span class="line"><span class="cl"><span class="cm"> */</span>
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">processBroadcastElement</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Pattern</span> <span class="n">pattern</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Context</span> <span class="n">ctx</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Collector</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Long</span><span class="o">,</span> <span class="n">Pattern</span><span class="o">>></span> <span class="n">out</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// store the new pattern by updating the broadcast state
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">BroadcastState</span><span class="o"><</span><span class="n">Void</span><span class="o">,</span> <span class="n">Pattern</span><span class="o">></span> <span class="n">bcState</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="na">getBroadcastState</span><span class="o">(</span><span class="n">patternDesc</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// storing in MapState with null as VOID default value
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">bcState</span><span class="o">.</span><span class="na">put</span><span class="o">(</span><span class="kc">null</span><span class="o">,</span> <span class="n">pattern</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>The <code>KeyedBroadcastProcessFunction</code> interface provides three methods to process records and emit results.</p>
+</span></span></span><span class="line"><span class="cl"><span class="cm"> */</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">processBroadcastElement</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Pattern</span><span class="w"> </span><span class="n">pattern</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Context</span><span class="w"> </span><span class="n">ctx</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Collector</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Long</span><span class="p">,</span><span class="w"> </span><span class="n">Pattern</span><span class="o">>></span><span class="w"> </span><span class="n">out</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><spa [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// store the new pattern by updating the broadcast state</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">BroadcastState</span><span class="o"><</span><span class="n">Void</span><span class="p">,</span><span class="w"> </span><span class="n">Pattern</span><span class="o">></span><span class="w"> </span><span class="n">bcState</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">ctx</span><span class="p">.</span><span class="na">getBroadcastState</s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// storing in MapState with null as VOID default value</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">bcState</span><span class="p">.</span><span class="na">put</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span><span class="w"> </span><span class="n">pattern</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>The <code>KeyedBroadcastProcessFunction</code> interface provides three methods to process records and emit results.</p>
<ul>
<li><code>processBroadcastElement()</code> is called for each record of the broadcasted stream. In our <code>PatternEvaluator</code> function, we simply put the received <code>Pattern</code> record in to the broadcast state using the <code>null</code> key (remember, we only store a single pattern in the <code>MapState</code>).</li>
<li><code>processElement()</code> is called for each record of the keyed stream. It provides read-only access to the broadcast state to prevent modification that result in different broadcast states across the parallel instances of the function. The <code>processElement()</code> method of the <code>PatternEvaluator</code> retrieves the current pattern from the broadcast state and the previous action of the user from the keyed state. If both are present, it checks whether the previous and [...]
diff --git a/content/2019/06/26/broadcast-state.html b/content/2019/06/26/broadcast-state.html
index 20cbd573d..271c1b2cd 100644
--- a/content/2019/06/26/broadcast-state.html
+++ b/content/2019/06/26/broadcast-state.html
@@ -1 +1,10 @@
-<!DOCTYPE html><html><head><title>https://flink.apache.org/2019/06/26/a-practical-guide-to-broadcast-state-in-apache-flink/</title><link rel="canonical" href="https://flink.apache.org/2019/06/26/a-practical-guide-to-broadcast-state-in-apache-flink/"/><meta name="robots" content="noindex"><meta charset="utf-8" /><meta http-equiv="refresh" content="0; url=https://flink.apache.org/2019/06/26/a-practical-guide-to-broadcast-state-in-apache-flink/" /></head></html>
\ No newline at end of file
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <title>https://flink.apache.org/2019/06/26/a-practical-guide-to-broadcast-state-in-apache-flink/</title>
+ <link rel="canonical" href="https://flink.apache.org/2019/06/26/a-practical-guide-to-broadcast-state-in-apache-flink/">
+ <meta name="robots" content="noindex">
+ <meta charset="utf-8">
+ <meta http-equiv="refresh" content="0; url=https://flink.apache.org/2019/06/26/a-practical-guide-to-broadcast-state-in-apache-flink/">
+ </head>
+</html>
diff --git a/content/2019/07/02/apache-flink-1.8.1-released/index.html b/content/2019/07/02/apache-flink-1.8.1-released/index.html
index 20ea509a2..7af21b51b 100644
--- a/content/2019/07/02/apache-flink-1.8.1-released/index.html
+++ b/content/2019/07/02/apache-flink-1.8.1-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2019/07/02/apache-flink-1.8.1-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-07-02T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-07-02T12:00:00+00:00" />
-
<title>Apache Flink 1.8.1 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2019/07/23/flink-network-stack-2.html b/content/2019/07/23/flink-network-stack-2.html
index b2e84daa6..c64580429 100644
--- a/content/2019/07/23/flink-network-stack-2.html
+++ b/content/2019/07/23/flink-network-stack-2.html
@@ -1 +1,10 @@
-<!DOCTYPE html><html><head><title>https://flink.apache.org/2019/07/23/flink-network-stack-vol.-2-monitoring-metrics-and-that-backpressure-thing/</title><link rel="canonical" href="https://flink.apache.org/2019/07/23/flink-network-stack-vol.-2-monitoring-metrics-and-that-backpressure-thing/"/><meta name="robots" content="noindex"><meta charset="utf-8" /><meta http-equiv="refresh" content="0; url=https://flink.apache.org/2019/07/23/flink-network-stack-vol.-2-monitoring-metrics-and-that-bac [...]
\ No newline at end of file
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <title>https://flink.apache.org/2019/07/23/flink-network-stack-vol.-2-monitoring-metrics-and-that-backpressure-thing/</title>
+ <link rel="canonical" href="https://flink.apache.org/2019/07/23/flink-network-stack-vol.-2-monitoring-metrics-and-that-backpressure-thing/">
+ <meta name="robots" content="noindex">
+ <meta charset="utf-8">
+ <meta http-equiv="refresh" content="0; url=https://flink.apache.org/2019/07/23/flink-network-stack-vol.-2-monitoring-metrics-and-that-backpressure-thing/">
+ </head>
+</html>
diff --git a/content/2019/07/23/flink-network-stack-vol.-2-monitoring-metrics-and-that-backpressure-thing/index.html b/content/2019/07/23/flink-network-stack-vol.-2-monitoring-metrics-and-that-backpressure-thing/index.html
index d34b59609..dec5860a4 100644
--- a/content/2019/07/23/flink-network-stack-vol.-2-monitoring-metrics-and-that-backpressure-thing/index.html
+++ b/content/2019/07/23/flink-network-stack-vol.-2-monitoring-metrics-and-that-backpressure-thing/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -15,25 +15,24 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
-<meta name="description" content=".tg {border-collapse:collapse;border-spacing:0;} .tg td{padding:10px 10px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;} .tg th{padding:10px 10px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;background-color:#eff0f1;} .tg .tg-wide{padding:10px 30px;} .tg .tg-top{vertical-align:top} .tg .tg-topcenter{text-align:center;vertical-align:top} .tg .tg-center{text-align:center;vertical-align:center} In a previou [...]
+<meta name="description" content="In a previous blog post, we presented how Flink’s network stack works from the high-level abstractions to the low-level details. This second blog post in the series of network stack posts extends on this knowledge and discusses monitoring network-related metrics to identify effects such as backpressure or bottlenecks in throughput and latency. Although this post briefly covers what to do with backpressure, the topic of tuning the network stack will be fu [...]
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Flink Network Stack Vol. 2: Monitoring, Metrics, and that Backpressure Thing" />
-<meta property="og:description" content=".tg {border-collapse:collapse;border-spacing:0;} .tg td{padding:10px 10px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;} .tg th{padding:10px 10px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;background-color:#eff0f1;} .tg .tg-wide{padding:10px 30px;} .tg .tg-top{vertical-align:top} .tg .tg-topcenter{text-align:center;vertical-align:top} .tg .tg-center{text-align:center;vertical-align:center} In a [...]
+<meta property="og:description" content="In a previous blog post, we presented how Flink’s network stack works from the high-level abstractions to the low-level details. This second blog post in the series of network stack posts extends on this knowledge and discusses monitoring network-related metrics to identify effects such as backpressure or bottlenecks in throughput and latency. Although this post briefly covers what to do with backpressure, the topic of tuning the network stack wil [...]
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2019/07/23/flink-network-stack-vol.-2-monitoring-metrics-and-that-backpressure-thing/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-07-23T15:30:00+00:00" />
<meta property="article:modified_time" content="2019-07-23T15:30:00+00:00" />
-
<title>Flink Network Stack Vol. 2: Monitoring, Metrics, and that Backpressure Thing | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2019/08/22/apache-flink-1.9.0-release-announcement/index.html b/content/2019/08/22/apache-flink-1.9.0-release-announcement/index.html
index 2e97bf5ed..ff7187f41 100644
--- a/content/2019/08/22/apache-flink-1.9.0-release-announcement/index.html
+++ b/content/2019/08/22/apache-flink-1.9.0-release-announcement/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ The Apache Flink project’s goal is to develop a stream processing system t
<meta property="og:url" content="https://flink.apache.org/2019/08/22/apache-flink-1.9.0-release-announcement/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-08-22T02:30:00+00:00" />
<meta property="article:modified_time" content="2019-08-22T02:30:00+00:00" />
-
<title>Apache Flink 1.9.0 Release Announcement | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2019/09/05/flink-community-update-september19/index.html b/content/2019/09/05/flink-community-update-september19/index.html
index 8a5080614..ca2d654ed 100644
--- a/content/2019/09/05/flink-community-update-september19/index.html
+++ b/content/2019/09/05/flink-community-update-september19/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ The first post in the series takes you on an little detour across the year, to f
<meta property="og:url" content="https://flink.apache.org/2019/09/05/flink-community-update-september19/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-09-05T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-09-05T12:00:00+00:00" />
-
<title>Flink Community Update - September'19 | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2019/09/11/apache-flink-1.8.2-released/index.html b/content/2019/09/11/apache-flink-1.8.2-released/index.html
index 64942265a..5b2d35f64 100644
--- a/content/2019/09/11/apache-flink-1.8.2-released/index.html
+++ b/content/2019/09/11/apache-flink-1.8.2-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2019/09/11/apache-flink-1.8.2-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-09-11T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-09-11T12:00:00+00:00" />
-
<title>Apache Flink 1.8.2 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2019/09/13/the-state-processor-api-how-to-read-write-and-modify-the-state-of-flink-applications/index.html b/content/2019/09/13/the-state-processor-api-how-to-read-write-and-modify-the-state-of-flink-applications/index.html
index 81c21ddc8..e7d40758b 100644
--- a/content/2019/09/13/the-state-processor-api-how-to-read-write-and-modify-the-state-of-flink-applications/index.html
+++ b/content/2019/09/13/the-state-processor-api-how-to-read-write-and-modify-the-state-of-flink-applications/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ In this post, we explain why this feature is a big step for Flink, what you can
<meta property="og:url" content="https://flink.apache.org/2019/09/13/the-state-processor-api-how-to-read-write-and-modify-the-state-of-flink-applications/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-09-13T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-09-13T12:00:00+00:00" />
-
<title>The State Processor API: How to Read, write and modify the state of Flink applications | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2019/10/18/apache-flink-1.9.1-released/index.html b/content/2019/10/18/apache-flink-1.9.1-released/index.html
index 87f61d80e..c5a5cd410 100644
--- a/content/2019/10/18/apache-flink-1.9.1-released/index.html
+++ b/content/2019/10/18/apache-flink-1.9.1-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2019/10/18/apache-flink-1.9.1-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-10-18T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-10-18T12:00:00+00:00" />
-
<title>Apache Flink 1.9.1 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2019/11/06/running-apache-flink-on-kubernetes-with-kudo/index.html b/content/2019/11/06/running-apache-flink-on-kubernetes-with-kudo/index.html
index a749407e7..4ddbd5f62 100644
--- a/content/2019/11/06/running-apache-flink-on-kubernetes-with-kudo/index.html
+++ b/content/2019/11/06/running-apache-flink-on-kubernetes-with-kudo/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -22,18 +22,17 @@
<meta property="og:url" content="https://flink.apache.org/2019/11/06/running-apache-flink-on-kubernetes-with-kudo/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-11-06T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-11-06T12:00:00+00:00" />
-
<title>Running Apache Flink on Kubernetes with KUDO | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2019/11/25/how-to-query-pulsar-streams-using-apache-flink/index.html b/content/2019/11/25/how-to-query-pulsar-streams-using-apache-flink/index.html
index 7796f858d..ea58d4308 100644
--- a/content/2019/11/25/how-to-query-pulsar-streams-using-apache-flink/index.html
+++ b/content/2019/11/25/how-to-query-pulsar-streams-using-apache-flink/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -16,26 +16,25 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="In a previous story on the Flink blog, we explained the different ways that Apache Flink and Apache Pulsar can integrate to provide elastic data processing at large scale. This blog post discusses the new developments and integrations between the two frameworks and showcases how you can leverage Pulsar’s built-in schema to query Pulsar streams in real time using Apache Flink.
-A short intro to Apache Pulsar # Apache Pulsar is a flexible pub/sub messaging system, backed by durable log storage.">
+A short intro to Apache Pulsar # Apache Pulsar is a flexible pub/sub messaging system, backed by durable log storage.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="How to query Pulsar Streams using Apache Flink" />
<meta property="og:description" content="In a previous story on the Flink blog, we explained the different ways that Apache Flink and Apache Pulsar can integrate to provide elastic data processing at large scale. This blog post discusses the new developments and integrations between the two frameworks and showcases how you can leverage Pulsar’s built-in schema to query Pulsar streams in real time using Apache Flink.
-A short intro to Apache Pulsar # Apache Pulsar is a flexible pub/sub messaging system, backed by durable log storage." />
+A short intro to Apache Pulsar # Apache Pulsar is a flexible pub/sub messaging system, backed by durable log storage." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2019/11/25/how-to-query-pulsar-streams-using-apache-flink/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-11-25T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-11-25T12:00:00+00:00" />
-
<title>How to query Pulsar Streams using Apache Flink | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -510,37 +509,37 @@ https://github.com/alex-shpak/hugo-book
<a class="anchor" href="#existing-pulsar--flink-integration-apache-flink-16">#</a>
</h1>
<p>The existing integration between Pulsar and Flink exploits Pulsar as a message queue in a Flink application. Flink developers can utilize Pulsar as a streaming source and streaming sink for their Flink applications by selecting a specific Pulsar source and connecting to their desired Pulsar cluster and topic:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// create and configure Pulsar consumer
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">PulsarSourceBuilder</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="n">builder</span> <span class="o">=</span> <span class="n">PulsarSourceBuilder</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">builder</span><span class="o">(</span><span class="k">new</span> <span class="n">SimpleStringSchema</span><span class="o">())</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">serviceUrl</span><span class="o">(</span><span class="n">serviceUrl</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">topic</span><span class="o">(</span><span class="n">inputTopic</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">subsciptionName</span><span class="o">(</span><span class="n">subscription</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"><span class="n">SourceFunction</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">src</span> <span class="o">=</span> <span class="n">builder</span><span class="o">.</span><span class="na">build</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"><span class="c1">// ingest DataStream with Pulsar consumer
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">words</span> <span class="o">=</span> <span class="n">env</span><span class="o">.</span><span class="na">addSource</span><span class="o">(</span><span class="n">src</span><span class="o">);</span>
-</span></span></code></pre></div><p>Pulsar streams can then get connected to the Flink processing logic…</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// perform computation on DataStream (here a simple WordCount)
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">WordWithCount</span><span class="o">></span> <span class="n">wc</span> <span class="o">=</span> <span class="n">words</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">flatmap</span><span class="o">((</span><span class="n">FlatMapFunction</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">WordWithCount</span><span class="o">>)</span> <span class="o">(</span><span class="n">word</span><span class="o">,</span> <span class="n">collector</span><span class="o">)</span> <span class="o">-></span> <span class="o [...]
-</span></span><span class="line"><span class="cl"> <span class="n">collector</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="k">new</span> <span class="n">WordWithCount</span><span class="o">(</span><span class="n">word</span><span class="o">,</span> <span class="n">1</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"> <span class="o">})</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">returns</span><span class="o">(</span><span class="n">WordWithCount</span><span class="o">.</span><span class="na">class</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">keyBy</span><span class="o">(</span><span class="s">"word"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">timeWindow</span><span class="o">(</span><span class="n">Time</span><span class="o">.</span><span class="na">seconds</span><span class="o">(</span><span class="n">5</span><span class="o">))</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">reduce</span><span class="o">((</span><span class="n">ReduceFunction</span><span class="o"><</span><span class="n">WordWithCount</span><span class="o">>)</span> <span class="o">(</span><span class="n">c1</span><span class="o">,</span> <span class="n">c2</span><span class="o">)</span> <span class="o">-></span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">WordWithCount</span><span class="o">(</span><span class="n">c1</span><span class="o">.</span><span class="na">word</span><span class="o">,</span> <span class="n">c1</span><span class="o">.</span><span class="na">count</span> <span class="o">+</span> <span class="n">c2</span><span class="o">.</span><span class="na">count</span><span class="o">));</span>
-</span></span></code></pre></div><p>…and then get emitted back to Pulsar (used now as a sink), sending one’s computation results downstream, back to a Pulsar topic:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// emit result via Pulsar producer
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">wc</span><span class="o">.</span><span class="na">addSink</span><span class="o">(</span><span class="k">new</span> <span class="n">FlinkPulsarProducer</span><span class="o"><>(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">serviceUrl</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">outputTopic</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">AuthentificationDisabled</span><span class="o">(),</span>
-</span></span><span class="line"><span class="cl"> <span class="n">wordWithCount</span> <span class="o">-></span> <span class="n">wordWithCount</span><span class="o">.</span><span class="na">toString</span><span class="o">().</span><span class="na">getBytes</span><span class="o">(</span><span class="n">UTF_8</span><span class="o">),</span>
-</span></span><span class="line"><span class="cl"> <span class="n">wordWithCount</span> <span class="o">-></span> <span class="n">wordWithCount</span><span class="o">.</span><span class="na">word</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="o">);</span>
-</span></span></code></pre></div><p>Although this is a great first integration step, the existing design is not leveraging the full power of Pulsar. Some shortcomings of the integration with Flink 1.6.0 relate to Pulsar neither being utilized as durable storage nor having schema integration with Flink, resulting in manual input when describing an application’s schema registry.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// create and configure Pulsar consumer</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">PulsarSourceBuilder</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="n">builder</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">PulsarSourceBuilder</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">builder</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">SimpleStringSchema</span><span class="p">())</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">serviceUrl</span><span class="p">(</span><span class="n">serviceUrl</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">topic</span><span class="p">(</span><span class="n">inputTopic</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">subsciptionName</span><span class="p">(</span><span class="n">subscription</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">SourceFunction</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">src</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">builder</span><span class="p">.</span><span class="na">build</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// ingest DataStream with Pulsar consumer</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">words</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">env</span><span class="p">.</span><span class="na">addSource</span><span class="p">(</span><span class="n">src</span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><p>Pulsar streams can then get connected to the Flink processing logic…</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// perform computation on DataStream (here a simple WordCount)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">WordWithCount</span><span class="o">></span><span class="w"> </span><span class="n">wc</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">words</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">flatmap</span><span class="p">((</span><span class="n">FlatMapFunction</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">WordWithCount</span><span class="o">></span><span class="p">)</span><span class="w"> </span><span class="p">(</span><span class="n">word</span><span class="p">,</span><s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">collector</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">WordWithCount</span><span class="p">(</span><span class="n">word</span><span class="p">,</span><span class="w"> </span><span class="n">1</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">})</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">returns</span><span class="p">(</span><span class="n">WordWithCount</span><span class="p">.</span><span class="na">class</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">keyBy</span><span class="p">(</span><span class="s">"word"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">timeWindow</span><span class="p">(</span><span class="n">Time</span><span class="p">.</span><span class="na">seconds</span><span class="p">(</span><span class="n">5</span><span class="p">))</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">reduce</span><span class="p">((</span><span class="n">ReduceFunction</span><span class="o"><</span><span class="n">WordWithCount</span><span class="o">></span><span class="p">)</span><span class="w"> </span><span class="p">(</span><span class="n">c1</span><span class="p">,</span><span class="w"> </span><span class="n">c2</span><span class="p">)</span><span clas [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">WordWithCount</span><span class="p">(</span><span class="n">c1</span><span class="p">.</span><span class="na">word</span><span class="p">,</span><span class="w"> </span><span class="n">c1</span><span class="p">.</span><span class="na">count</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">c2</spa [...]
+</span></span></span></code></pre></div><p>…and then get emitted back to Pulsar (used now as a sink), sending one’s computation results downstream, back to a Pulsar topic:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// emit result via Pulsar producer </span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">wc</span><span class="p">.</span><span class="na">addSink</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">FlinkPulsarProducer</span><span class="o"><></span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">serviceUrl</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">outputTopic</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">AuthentificationDisabled</span><span class="p">(),</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">wordWithCount</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">wordWithCount</span><span class="p">.</span><span class="na">toString</span><span class="p">().</span><span class="na">getBytes</span><span class="p">(</span><span class="n">UTF_8</span><span class="p">),</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">wordWithCount</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">wordWithCount</span><span class="p">.</span><span class="na">word</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><p>Although this is a great first integration step, the existing design is not leveraging the full power of Pulsar. Some shortcomings of the integration with Flink 1.6.0 relate to Pulsar neither being utilized as durable storage nor having schema integration with Flink, resulting in manual input when describing an application’s schema registry.</p>
<h1 id="pulsars-integration-with-flink-19-using-pulsar-as-a-flink-catalog">
Pulsar’s integration with Flink 1.9: Using Pulsar as a Flink catalog
<a class="anchor" href="#pulsars-integration-with-flink-19-using-pulsar-as-a-flink-catalog">#</a>
@@ -552,18 +551,18 @@ https://github.com/alex-shpak/hugo-book
</h1>
<p>Before delving into the integration details and how you can use Pulsar schema with Flink, let us describe how schema in Pulsar works. Schema in Apache Pulsar already co-exists and serves as the representation of the data on the broker side of the framework, something that makes schema registry with external systems obsolete. Additionally, the data schema in Pulsar is associated with each topic so both producers and consumers send data with predefined schema information, while the brok [...]
<p>Below you can find an example of Pulsar’s schema on both the producer and consumer side. On the producer side, you can specify which schema you want to use and Pulsar then sends a POJO class without the need to perform any serialization/deserialization. Similarly, on the consumer end, you can also specify the data schema and upon receiving the data, Pulsar will automatically validate the schema information, fetch the schema of the given version and then deserialize the data back to a [...]
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// Create producer with Struct schema and send messages
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">Producer</span><span class="o"><</span><span class="n">User</span><span class="o">></span> <span class="n">producer</span> <span class="o">=</span> <span class="n">client</span><span class="o">.</span><span class="na">newProducer</span><span class="o">(</span><span class="n">Schema</span><span class="o">.</span><span class="na">AVRO</span><span class="o">(</span><span class="n">User</sp [...]
-</span></span><span class="line"><span class="cl"><span class="n">producer</span><span class="o">.</span><span class="na">newMessage</span><span class="o">()</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">value</span><span class="o">(</span><span class="n">User</span><span class="o">.</span><span class="na">builder</span><span class="o">()</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">userName</span><span class="o">(</span><span class="err">“</span><span class="n">pulsar</span><span class="o">-</span><span class="n">user</span><span class="err">”</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">userId</span><span class="o">(</span><span class="n">1L</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">build</span><span class="o">())</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">send</span><span class="o">();</span>
-</span></span></code></pre></div><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// Create consumer with Struct schema and receive messages
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">Consumer</span><span class="o"><</span><span class="n">User</span><span class="o">></span> <span class="n">consumer</span> <span class="o">=</span> <span class="n">client</span><span class="o">.</span><span class="na">newCOnsumer</span><span class="o">(</span><span class="n">Schema</span><span class="o">.</span><span class="na">AVRO</span><span class="o">(</span><span class="n">User</sp [...]
-</span></span><span class="line"><span class="cl"><span class="n">consumer</span><span class="o">.</span><span class="na">receive</span><span class="o">();</span>
-</span></span></code></pre></div><p>Let’s assume we have an application that specifies a schema to the producer and/or consumer. Upon receiving the schema information, the producer (or consumer) — that is connected to the broker — will transfer such information so that the broker can then perform schema registration, validations and schema compatibility checks before returning or rejecting the schema as illustrated in the diagram below:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// Create producer with Struct schema and send messages</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">Producer</span><span class="o"><</span><span class="n">User</span><span class="o">></span><span class="w"> </span><span class="n">producer</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">client</span><span class="p">.</span><span class="na">newProducer</span><span class="p">(</span><span class="n">Schema</span><span class="p">.</span><span cl [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">producer</span><span class="p">.</span><span class="na">newMessage</span><span class="p">()</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">value</span><span class="p">(</span><span class="n">User</span><span class="p">.</span><span class="na">builder</span><span class="p">()</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">userName</span><span class="p">(</span><span class="err">“</span><span class="n">pulsar</span><span class="o">-</span><span class="n">user</span><span class="err">”</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">userId</span><span class="p">(</span><span class="n">1L</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">build</span><span class="p">())</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">send</span><span class="p">();</span><span class="w">
+</span></span></span></code></pre></div><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// Create consumer with Struct schema and receive messages</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">Consumer</span><span class="o"><</span><span class="n">User</span><span class="o">></span><span class="w"> </span><span class="n">consumer</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">client</span><span class="p">.</span><span class="na">newCOnsumer</span><span class="p">(</span><span class="n">Schema</span><span class="p">.</span><span cl [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">consumer</span><span class="p">.</span><span class="na">receive</span><span class="p">();</span><span class="w">
+</span></span></span></code></pre></div><p>Let’s assume we have an application that specifies a schema to the producer and/or consumer. Upon receiving the schema information, the producer (or consumer) — that is connected to the broker — will transfer such information so that the broker can then perform schema registration, validations and schema compatibility checks before returning or rejecting the schema as illustrated in the diagram below:</p>
<center>
<img src="/img/blog/flink-pulsar-sql-blog-post-visual.png" width="600px" alt="Pulsar Schema"/>
</center>
@@ -583,83 +582,83 @@ https://github.com/alex-shpak/hugo-book
<ul>
<li>Create a Pulsar source for streaming queries</li>
</ul>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">val</span> <span class="n">env</span> <span class="o">=</span> <span class="n">StreamExecutionEnvironment</span><span class="o">.</span><span class="na">getExecutionEnvironment</span>
-</span></span><span class="line"><span class="cl"><span class="n">val</span> <span class="n">props</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Properties</span><span class="o">()</span>
-</span></span><span class="line"><span class="cl"><span class="n">props</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"service.url"</span><span class="o">,</span> <span class="s">"pulsar://..."</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">props</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"admin.url"</span><span class="o">,</span> <span class="s">"http://..."</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">props</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"partitionDiscoveryIntervalMillis"</span><span class="o">,</span> <span class="s">"5000"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">props</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"startingOffsets"</span><span class="o">,</span> <span class="s">"earliest"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">props</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"topic"</span><span class="o">,</span> <span class="s">"test-source-topic"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">val</span> <span class="n">source</span> <span class="o">=</span> <span class="k">new</span> <span class="n">FlinkPulsarSource</span><span class="o">(</span><span class="n">props</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="c1">// you don't need to provide a type information to addSource since FlinkPulsarSource is ResultTypeQueryable
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">val</span> <span class="n">dataStream</span> <span class="o">=</span> <span class="n">env</span><span class="o">.</span><span class="na">addSource</span><span class="o">(</span><span class="n">source</span><span class="o">)(</span><span class="kc">null</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// chain operations on dataStream of Row and sink the output
-</span></span></span><span class="line"><span class="cl"><span class="c1">// end method chaining
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span>
-</span></span><span class="line"><span class="cl"><span class="n">env</span><span class="o">.</span><span class="na">execute</span><span class="o">()</span>
-</span></span></code></pre></div><ul>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">val</span><span class="w"> </span><span class="n">env</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">StreamExecutionEnvironment</span><span class="p">.</span><span class="na">getExecutionEnvironment</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">val</span><span class="w"> </span><span class="n">props</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Properties</span><span class="p">()</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">props</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"service.url"</span><span class="p">,</span><span class="w"> </span><span class="s">"pulsar://..."</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">props</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"admin.url"</span><span class="p">,</span><span class="w"> </span><span class="s">"http://..."</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">props</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"partitionDiscoveryIntervalMillis"</span><span class="p">,</span><span class="w"> </span><span class="s">"5000"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">props</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"startingOffsets"</span><span class="p">,</span><span class="w"> </span><span class="s">"earliest"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">props</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"topic"</span><span class="p">,</span><span class="w"> </span><span class="s">"test-source-topic"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">val</span><span class="w"> </span><span class="n">source</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">FlinkPulsarSource</span><span class="p">(</span><span class="n">props</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// you don't need to provide a type information to addSource since FlinkPulsarSource is ResultTypeQueryable</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">val</span><span class="w"> </span><span class="n">dataStream</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">env</span><span class="p">.</span><span class="na">addSource</span><span class="p">(</span><span class="n">source</span><span class="p">)(</span><span class="kc">null</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// chain operations on dataStream of Row and sink the output</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// end method chaining</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">env</span><span class="p">.</span><span class="na">execute</span><span class="p">()</span><span class="w">
+</span></span></span></code></pre></div><ul>
<li>Register topics in Pulsar as streaming tables</li>
</ul>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">val</span> <span class="n">env</span> <span class="o">=</span> <span class="n">StreamExecutionEnvironment</span><span class="o">.</span><span class="na">getExecutionEnvironment</span>
-</span></span><span class="line"><span class="cl"><span class="n">val</span> <span class="n">tEnv</span> <span class="o">=</span> <span class="n">StreamTableEnvironment</span><span class="o">.</span><span class="na">create</span><span class="o">(</span><span class="n">env</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">val</span> <span class="n">prop</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Properties</span><span class="o">()</span>
-</span></span><span class="line"><span class="cl"><span class="n">prop</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"service.url"</span><span class="o">,</span> <span class="n">serviceUrl</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">prop</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"admin.url"</span><span class="o">,</span> <span class="n">adminUrl</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">prop</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"flushOnCheckpoint"</span><span class="o">,</span> <span class="s">"true"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">prop</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"failOnWrite"</span><span class="o">,</span> <span class="s">"true"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">props</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"topic"</span><span class="o">,</span> <span class="s">"test-sink-topic"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">tEnv</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">connect</span><span class="o">(</span><span class="k">new</span> <span class="n">Pulsar</span><span class="o">().</span><span class="na">properties</span><span class="o">(</span><span class="n">props</span><span class="o">))</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">inAppendMode</span><span class="o">()</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">registerTableSource</span><span class="o">(</span><span class="s">"sink-table"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">val</span> <span class="n">sql</span> <span class="o">=</span> <span class="s">"INSERT INTO sink-table ....."</span>
-</span></span><span class="line"><span class="cl"><span class="n">tEnv</span><span class="o">.</span><span class="na">sqlUpdate</span><span class="o">(</span><span class="n">sql</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">env</span><span class="o">.</span><span class="na">execute</span><span class="o">()</span>
-</span></span></code></pre></div><h1 id="flink--pulsar-write-data-to-pulsar">
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">val</span><span class="w"> </span><span class="n">env</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">StreamExecutionEnvironment</span><span class="p">.</span><span class="na">getExecutionEnvironment</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">val</span><span class="w"> </span><span class="n">tEnv</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">StreamTableEnvironment</span><span class="p">.</span><span class="na">create</span><span class="p">(</span><span class="n">env</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">val</span><span class="w"> </span><span class="n">prop</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Properties</span><span class="p">()</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">prop</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"service.url"</span><span class="p">,</span><span class="w"> </span><span class="n">serviceUrl</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">prop</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"admin.url"</span><span class="p">,</span><span class="w"> </span><span class="n">adminUrl</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">prop</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"flushOnCheckpoint"</span><span class="p">,</span><span class="w"> </span><span class="s">"true"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">prop</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"failOnWrite"</span><span class="p">,</span><span class="w"> </span><span class="s">"true"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">props</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"topic"</span><span class="p">,</span><span class="w"> </span><span class="s">"test-sink-topic"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">tEnv</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">connect</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">Pulsar</span><span class="p">().</span><span class="na">properties</span><span class="p">(</span><span class="n">props</span><span class="p">))</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">inAppendMode</span><span class="p">()</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">registerTableSource</span><span class="p">(</span><span class="s">"sink-table"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">val</span><span class="w"> </span><span class="n">sql</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">"INSERT INTO sink-table ....."</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">tEnv</span><span class="p">.</span><span class="na">sqlUpdate</span><span class="p">(</span><span class="n">sql</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">env</span><span class="p">.</span><span class="na">execute</span><span class="p">()</span><span class="w">
+</span></span></span></code></pre></div><h1 id="flink--pulsar-write-data-to-pulsar">
Flink & Pulsar: Write data to Pulsar
<a class="anchor" href="#flink--pulsar-write-data-to-pulsar">#</a>
</h1>
<ul>
<li>Create a Pulsar sink for streaming queries</li>
</ul>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">val</span> <span class="n">env</span> <span class="o">=</span> <span class="n">StreamExecutionEnvironment</span><span class="o">.</span><span class="na">getExecutionEnvironment</span>
-</span></span><span class="line"><span class="cl"><span class="n">val</span> <span class="n">stream</span> <span class="o">=</span> <span class="o">.....</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">val</span> <span class="n">prop</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Properties</span><span class="o">()</span>
-</span></span><span class="line"><span class="cl"><span class="n">prop</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"service.url"</span><span class="o">,</span> <span class="n">serviceUrl</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">prop</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"admin.url"</span><span class="o">,</span> <span class="n">adminUrl</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">prop</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"flushOnCheckpoint"</span><span class="o">,</span> <span class="s">"true"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">prop</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"failOnWrite"</span><span class="o">,</span> <span class="s">"true"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">props</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"topic"</span><span class="o">,</span> <span class="s">"test-sink-topic"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">stream</span><span class="o">.</span><span class="na">addSink</span><span class="o">(</span><span class="k">new</span> <span class="n">FlinkPulsarSink</span><span class="o">(</span><span class="n">prop</span><span class="o">,</span> <span class="n">DummyTopicKeyExtractor</span><span class="o">))</span>
-</span></span><span class="line"><span class="cl"><span class="n">env</span><span class="o">.</span><span class="na">execute</span><span class="o">()</span>
-</span></span></code></pre></div><ul>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">val</span><span class="w"> </span><span class="n">env</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">StreamExecutionEnvironment</span><span class="p">.</span><span class="na">getExecutionEnvironment</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">val</span><span class="w"> </span><span class="n">stream</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">.....</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">val</span><span class="w"> </span><span class="n">prop</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Properties</span><span class="p">()</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">prop</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"service.url"</span><span class="p">,</span><span class="w"> </span><span class="n">serviceUrl</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">prop</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"admin.url"</span><span class="p">,</span><span class="w"> </span><span class="n">adminUrl</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">prop</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"flushOnCheckpoint"</span><span class="p">,</span><span class="w"> </span><span class="s">"true"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">prop</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"failOnWrite"</span><span class="p">,</span><span class="w"> </span><span class="s">"true"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">props</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"topic"</span><span class="p">,</span><span class="w"> </span><span class="s">"test-sink-topic"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">stream</span><span class="p">.</span><span class="na">addSink</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">FlinkPulsarSink</span><span class="p">(</span><span class="n">prop</span><span class="p">,</span><span class="w"> </span><span class="n">DummyTopicKeyExtractor</span><span class="p">))</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">env</span><span class="p">.</span><span class="na">execute</span><span class="p">()</span><span class="w">
+</span></span></span></code></pre></div><ul>
<li>Write a streaming table to Pulsar</li>
</ul>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">val</span> <span class="n">env</span> <span class="o">=</span> <span class="n">StreamExecutionEnvironment</span><span class="o">.</span><span class="na">getExecutionEnvironment</span>
-</span></span><span class="line"><span class="cl"><span class="n">val</span> <span class="n">tEnv</span> <span class="o">=</span> <span class="n">StreamTableEnvironment</span><span class="o">.</span><span class="na">create</span><span class="o">(</span><span class="n">env</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">val</span> <span class="n">prop</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Properties</span><span class="o">()</span>
-</span></span><span class="line"><span class="cl"><span class="n">prop</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"service.url"</span><span class="o">,</span> <span class="n">serviceUrl</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">prop</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"admin.url"</span><span class="o">,</span> <span class="n">adminUrl</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">prop</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"flushOnCheckpoint"</span><span class="o">,</span> <span class="s">"true"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">prop</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"failOnWrite"</span><span class="o">,</span> <span class="s">"true"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">props</span><span class="o">.</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"topic"</span><span class="o">,</span> <span class="s">"test-sink-topic"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">tEnv</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">connect</span><span class="o">(</span><span class="k">new</span> <span class="n">Pulsar</span><span class="o">().</span><span class="na">properties</span><span class="o">(</span><span class="n">props</span><span class="o">))</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">inAppendMode</span><span class="o">()</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">registerTableSource</span><span class="o">(</span><span class="s">"sink-table"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">val</span> <span class="n">sql</span> <span class="o">=</span> <span class="s">"INSERT INTO sink-table ....."</span>
-</span></span><span class="line"><span class="cl"><span class="n">tEnv</span><span class="o">.</span><span class="na">sqlUpdate</span><span class="o">(</span><span class="n">sql</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"><span class="n">env</span><span class="o">.</span><span class="na">execute</span><span class="o">()</span>
-</span></span></code></pre></div><p>In every instance, Flink developers only need to specify the properties of how Flink will connect to a Pulsar cluster without worrying about any schema registry, or serialization/deserialization actions and register the Pulsar cluster as a source, sink or streaming table in Flink. Once all three elements are put together, Pulsar can then be registered as a catalog in Flink, something that drastically simplifies how you process and query data like, for [...]
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">val</span><span class="w"> </span><span class="n">env</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">StreamExecutionEnvironment</span><span class="p">.</span><span class="na">getExecutionEnvironment</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">val</span><span class="w"> </span><span class="n">tEnv</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">StreamTableEnvironment</span><span class="p">.</span><span class="na">create</span><span class="p">(</span><span class="n">env</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">val</span><span class="w"> </span><span class="n">prop</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Properties</span><span class="p">()</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">prop</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"service.url"</span><span class="p">,</span><span class="w"> </span><span class="n">serviceUrl</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">prop</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"admin.url"</span><span class="p">,</span><span class="w"> </span><span class="n">adminUrl</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">prop</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"flushOnCheckpoint"</span><span class="p">,</span><span class="w"> </span><span class="s">"true"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">prop</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"failOnWrite"</span><span class="p">,</span><span class="w"> </span><span class="s">"true"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">props</span><span class="p">.</span><span class="na">setProperty</span><span class="p">(</span><span class="s">"topic"</span><span class="p">,</span><span class="w"> </span><span class="s">"test-sink-topic"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">tEnv</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">connect</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">Pulsar</span><span class="p">().</span><span class="na">properties</span><span class="p">(</span><span class="n">props</span><span class="p">))</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">inAppendMode</span><span class="p">()</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">registerTableSource</span><span class="p">(</span><span class="s">"sink-table"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">val</span><span class="w"> </span><span class="n">sql</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">"INSERT INTO sink-table ....."</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">tEnv</span><span class="p">.</span><span class="na">sqlUpdate</span><span class="p">(</span><span class="n">sql</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">env</span><span class="p">.</span><span class="na">execute</span><span class="p">()</span><span class="w">
+</span></span></span></code></pre></div><p>In every instance, Flink developers only need to specify the properties of how Flink will connect to a Pulsar cluster without worrying about any schema registry, or serialization/deserialization actions and register the Pulsar cluster as a source, sink or streaming table in Flink. Once all three elements are put together, Pulsar can then be registered as a catalog in Flink, something that drastically simplifies how you process and query data lik [...]
<h1 id="next-steps--future-integration">
Next Steps & Future Integration
<a class="anchor" href="#next-steps--future-integration">#</a>
diff --git a/content/2019/12/11/apache-flink-1.8.3-released/index.html b/content/2019/12/11/apache-flink-1.8.3-released/index.html
index f110f4ccf..1ad3a1905 100644
--- a/content/2019/12/11/apache-flink-1.8.3-released/index.html
+++ b/content/2019/12/11/apache-flink-1.8.3-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2019/12/11/apache-flink-1.8.3-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2019-12-11T12:00:00+00:00" />
<meta property="article:modified_time" content="2019-12-11T12:00:00+00:00" />
-
<title>Apache Flink 1.8.3 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2020/01/15/advanced-flink-application-patterns-vol.1-case-study-of-a-fraud-detection-system/index.html b/content/2020/01/15/advanced-flink-application-patterns-vol.1-case-study-of-a-fraud-detection-system/index.html
index 1c8661511..601b73c0e 100644
--- a/content/2020/01/15/advanced-flink-application-patterns-vol.1-case-study-of-a-fraud-detection-system/index.html
+++ b/content/2020/01/15/advanced-flink-application-patterns-vol.1-case-study-of-a-fraud-detection-system/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -16,28 +16,27 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="In this series of blog posts you will learn about three powerful Flink patterns for building streaming applications:
- Dynamic updates of application logic Dynamic data partitioning (shuffle), controlled at runtime Low latency alerting based on custom windowing logic (without using the window API) These patterns expand the possibilities of what is achievable with statically defined data flows and provide the building blocks to fulfill complex business requirements.
+Dynamic updates of application logic Dynamic data partitioning (shuffle), controlled at runtime Low latency alerting based on custom windowing logic (without using the window API) These patterns expand the possibilities of what is achievable with statically defined data flows and provide the building blocks to fulfill complex business requirements.
Dynamic updates of application logic allow Flink jobs to change at runtime, without downtime from stopping and resubmitting the code.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Advanced Flink Application Patterns Vol.1: Case Study of a Fraud Detection System" />
<meta property="og:description" content="In this series of blog posts you will learn about three powerful Flink patterns for building streaming applications:
- Dynamic updates of application logic Dynamic data partitioning (shuffle), controlled at runtime Low latency alerting based on custom windowing logic (without using the window API) These patterns expand the possibilities of what is achievable with statically defined data flows and provide the building blocks to fulfill complex business requirements.
+Dynamic updates of application logic Dynamic data partitioning (shuffle), controlled at runtime Low latency alerting based on custom windowing logic (without using the window API) These patterns expand the possibilities of what is achievable with statically defined data flows and provide the building blocks to fulfill complex business requirements.
Dynamic updates of application logic allow Flink jobs to change at runtime, without downtime from stopping and resubmitting the code." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2020/01/15/advanced-flink-application-patterns-vol.1-case-study-of-a-fraud-detection-system/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2020-01-15T12:00:00+00:00" />
<meta property="article:modified_time" content="2020-01-15T12:00:00+00:00" />
-
<title>Advanced Flink Application Patterns Vol.1: Case Study of a Fraud Detection System | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -60,7 +59,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -267,7 +266,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -556,11 +555,11 @@ We hope that this series will place these powerful approaches into your tool bel
<p>The first pattern we will look into is Dynamic Data Partitioning.</p>
<p>If you have used Flink’s DataStream API in the past, you are undoubtedly familiar with the <strong>keyBy</strong> method. Keying a stream shuffles all the records such that elements with the same key are assigned to the same partition. This means all records with the same key are processed by the same physical instance of the next operator.</p>
<p>In a typical streaming application, the choice of key is fixed, determined by some static field within the elements. For instance, when building a simple window-based aggregation of a stream of transactions, we might always group by the transactions account id.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">Transaction</span><span class="o">></span> <span class="n">input</span> <span class="o">=</span> <span class="c1">// [...]
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><...></span> <span class="n">windowed</span> <span class="o">=</span> <span class="n">input</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">keyBy</span><span class="o">(</span><span class="n">Transaction</span><span class="o">::</span><span class="n">getAccountId</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">window</span><span class="o">(</span><span class="cm">/*window specification*/</span><span class="o">);</span>
-</span></span></code></pre></div><p>This approach is the main building block for achieving horizontal scalability in a wide range of use cases. However, in the case of an application striving to provide flexibility in business logic at runtime, this is not enough.
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">Transaction</span><span class="o">></span><span class="w"> </span><span class="n">input</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="c1">// [...]</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="p">...</span><span class="o">></span><span class="w"> </span><span class="n">windowed</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">input</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">keyBy</span><span class="p">(</span><span class="n">Transaction</span><span class="p">::</span><span class="n">getAccountId</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">window</span><span class="p">(</span><span class="cm">/*window specification*/</span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><p>This approach is the main building block for achieving horizontal scalability in a wide range of use cases. However, in the case of an application striving to provide flexibility in business logic at runtime, this is not enough.
To understand why this is the case, let us start with articulating a realistic sample rule definition for our fraud detection system in the form of a functional requirement:</p>
<p><em>“Whenever the <strong>sum</strong> of the accumulated <strong>payment amount</strong> from the same <strong>payer</strong> to the same <strong>beneficiary</strong> within the <strong>duration of a week</strong> is <strong>greater</strong> than <strong>1 000 000 $</strong> - fire an alert.”</em></p>
<p>In this formulation we can spot a number of parameters that we would like to be able to specify in a newly-submitted rule and possibly even later modify or tweak at runtime:</p>
@@ -586,12 +585,12 @@ To understand why this is the case, let us start with articulating a realistic s
</span></span></code></pre></div><p>At this point, it is important to understand that <strong><code>groupingKeyNames</code></strong> determine the actual physical grouping of events - all Transactions with the same values of specified parameters (e.g. <em>payer #25 -> beneficiary #12</em>) have to be aggregated in the same physical instance of the evaluating operator. Naturally, the process of distributing data in such a way in Flink’s API is realised by a <code>keyBy()</code> f [...]
<p>Most examples in Flink’s <code>keyBy()</code><a href="//nightlies.apache.org/flink/flink-docs-stable/dev/api_concepts.html#define-keys-using-field-expressions">documentation</a> use a hard-coded <code>KeySelector</code>, which extracts specific fixed events’ fields. However, to support the desired flexibility, we have to extract them in a more dynamic fashion based on the specifications of the rules. For this, we will have to use one additional operator that prepares every [...]
<p>On a high level, our main processing pipeline looks like this:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">Alert</span><span class="o">></span> <span class="n">alerts</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="n">transactions</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">process</span><span class="o">(</span><span class="k">new</span> <span class="n">DynamicKeyFunction</span><span class="o">())</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">keyBy</span><span class="o">(</span><span class="cm">/* some key selector */</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">process</span><span class="o">(</span><span class="cm">/* actual calculations and alerting */</span><span class="o">)</span>
-</span></span></code></pre></div><p>We have previously established that each rule defines a <strong><code>groupingKeyNames</code></strong> parameter that specifies which combination of fields will be used for the incoming events’ grouping. Each rule might use an arbitrary combination of these fields. At the same time, every incoming event potentially needs to be evaluated against multiple rules. This implies that events might simultaneously need to be present at multiple parallel i [...]
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">Alert</span><span class="o">></span><span class="w"> </span><span class="n">alerts</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">transactions</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">process</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">DynamicKeyFunction</span><span class="p">())</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">keyBy</span><span class="p">(</span><span class="cm">/* some key selector */</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">process</span><span class="p">(</span><span class="cm">/* actual calculations and alerting */</span><span class="p">)</span><span class="w">
+</span></span></span></code></pre></div><p>We have previously established that each rule defines a <strong><code>groupingKeyNames</code></strong> parameter that specifies which combination of fields will be used for the incoming events’ grouping. Each rule might use an arbitrary combination of these fields. At the same time, every incoming event potentially needs to be evaluated against multiple rules. This implies that events might simultaneously need to be present at multiple par [...]
<center>
<img src="/img/blog/2019-11-19-demo-fraud-detection/shuffle_function_1.png" width="800px" alt="Figure 3: Forking events with Dynamic Key Function"/>
<br/>
@@ -599,49 +598,49 @@ To understand why this is the case, let us start with articulating a realistic s
</center>
<br/>
<p><code>DynamicKeyFunction</code> iterates over a set of defined rules and prepares every event to be processed by a <code>keyBy()</code> function by extracting the required grouping keys:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">DynamicKeyFunction</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">extends</span> <span class="n">ProcessFunction</span><span class="o"><</span><span class="n">Transaction</span><span class="o">,</span> <span class="n">Keyed</span><span class="o"><</span><span class="n">Transaction</span><span class="o">,</span> <span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="o">...</span>
-</span></span><span class="line"><span class="cl"> <span class="cm">/* Simplified */</span>
-</span></span><span class="line"><span class="cl"> <span class="n">List</span><span class="o"><</span><span class="n">Rule</span><span class="o">></span> <span class="n">rules</span> <span class="o">=</span> <span class="cm">/* Rules that are initialized somehow.
-</span></span></span><span class="line"><span class="cl"><span class="cm"> Details will be discussed in a future blog post. */</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">processElement</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Transaction</span> <span class="n">event</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Context</span> <span class="n">ctx</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Collector</span><span class="o"><</span><span class="n">Keyed</span><span class="o"><</span><span class="n">Transaction</span><span class="o">,</span> <span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span> <span class="n">out</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="o">(</span><span class="n">Rule</span> <span class="n">rule</span> <span class="o">:</span><span class="n">rules</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">out</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">Keyed</span><span class="o"><>(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">event</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">KeysExtractor</span><span class="o">.</span><span class="na">getKey</span><span class="o">(</span><span class="n">rule</span><span class="o">.</span><span class="na">getGroupingKeyNames</span><span class="o">(),</span> <span class="n">event</span><span class="o">),</span>
-</span></span><span class="line"><span class="cl"> <span class="n">rule</span><span class="o">.</span><span class="na">getRuleId</span><span class="o">()));</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">...</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p><code>KeysExtractor.getKey()</code> uses reflection to extract the required values of <code>groupingKeyNames</code> fields from events and combines them as a single concatenated String key, e.g <code>"{payerId=25;beneficiaryId=12}"</code>. Flink will calculate the hash of this key and assign the processing of this particular combination to a specific server in the cluster. This will allow tracking all transactions between <em>payer #25</em> a [...]
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">DynamicKeyFunction</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">extends</span><span class="w"> </span><span class="n">ProcessFunction</span><span class="o"><</span><span class="n">Transaction</span><span class="p">,</span><span class="w"> </span><span class="n">Keyed</span><span class="o"><</span><span class="n">Transaction</span><span class="p">,</span><span class="w"> </span><span class="n">String</span><span class="p">,</span><span class="w"> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">...</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="cm">/* Simplified */</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">List</span><span class="o"><</span><span class="n">Rule</span><span class="o">></span><span class="w"> </span><span class="n">rules</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="cm">/* Rules that are initialized somehow.
+</span></span></span><span class="line"><span class="cl"><span class="cm"> Details will be discussed in a future blog post. */</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">processElement</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Transaction</span><span class="w"> </span><span class="n">event</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Context</span><span class="w"> </span><span class="n">ctx</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Collector</span><span class="o"><</span><span class="n">Keyed</span><span class="o"><</span><span class="n">Transaction</span><span class="p">,</span><span class="w"> </span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="o">>></span><span class="w"> </span><span class="n">out</span><span class="p">)</spa [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="n">Rule</span><span class="w"> </span><span class="n">rule</span><span class="w"> </span><span class="p">:</span><span class="n">rules</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">out</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Keyed</span><span class="o"><></span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">event</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">KeysExtractor</span><span class="p">.</span><span class="na">getKey</span><span class="p">(</span><span class="n">rule</span><span class="p">.</span><span class="na">getGroupingKeyNames</span><span class="p">(),</span><span class="w"> </span><span class="n">event</span><span class="p">),</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">rule</span><span class="p">.</span><span class="na">getRuleId</span><span class="p">()));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">...</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p><code>KeysExtractor.getKey()</code> uses reflection to extract the required values of <code>groupingKeyNames</code> fields from events and combines them as a single concatenated String key, e.g <code>"{payerId=25;beneficiaryId=12}"</code>. Flink will calculate the hash of this key and assign the processing of this particular combination to a specific server in the cluster. This will allow tracking all transactions between <em>payer #25 [...]
<p>Notice that a wrapper class <code>Keyed</code> with the following signature was introduced as the output type of <code>DynamicKeyFunction</code>:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">Keyed</span><span class="o"><</span><span class="n">IN</span><span class="o">,</span> <span class="n">KEY</span><span class="o">,</span> <span class="n">ID</span><span class="o">></span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">IN</span> <span class="n">wrapped</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">KEY</span> <span class="n">key</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">ID</span> <span class="n">id</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="o">...</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">KEY</span> <span class="nf">getKey</span><span class="o">(){</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">key</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>Fields of this POJO carry the following information: <code>wrapped</code> is the original transaction event, <code>key</code> is the result of using <code>KeysExtractor</code> and <code>id</code> is the ID of the Rule that caused the dispatch of the event (according to the rule-specific grouping logic).</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">Keyed</span><span class="o"><</span><span class="n">IN</span><span class="p">,</span><span class="w"> </span><span class="n">KEY</span><span class="p">,</span><span class="w"> </span><span class="n">ID</span><span class="o">></span><span class="w"> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">IN</span><span class="w"> </span><span class="n">wrapped</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">KEY</span><span class="w"> </span><span class="n">key</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">ID</span><span class="w"> </span><span class="n">id</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">...</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">KEY</span><span class="w"> </span><span class="nf">getKey</span><span class="p">(){</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">key</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>Fields of this POJO carry the following information: <code>wrapped</code> is the original transaction event, <code>key</code> is the result of using <code>KeysExtractor</code> and <code>id</code> is the ID of the Rule that caused the dispatch of the event (according to the rule-specific grouping logic).</p>
<p>Events of this type will be the input to the <code>keyBy()</code> function in the main processing pipeline and allow the use of a simple lambda-expression as a <a href="//nightlies.apache.org/flink/flink-docs-stable/dev/api_concepts.html#define-keys-using-key-selector-functions"><code>KeySelector</code></a> for the final step of implementing dynamic data shuffle.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">Alert</span><span class="o">></span> <span class="n">alerts</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="n">transactions</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">process</span><span class="o">(</span><span class="k">new</span> <span class="n">DynamicKeyFunction</span><span class="o">())</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">keyBy</span><span class="o">((</span><span class="n">keyed</span><span class="o">)</span> <span class="o">-></span> <span class="n">keyed</span><span class="o">.</span><span class="na">getKey</span><span class="o">());</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">process</span><span class="o">(</span><span class="k">new</span> <span class="n">DynamicAlertFunction</span><span class="o">())</span>
-</span></span></code></pre></div><p>By applying <code>DynamicKeyFunction</code> we are implicitly copying events for performing parallel per-rule evaluation within a Flink cluster. By doing so, we achieve an important property - horizontal scalability of rules’ processing. Our system will be capable of handling more rules by adding more servers to the cluster, i.e. increasing the parallelism. This property is achieved at the cost of data duplication, which might become an issue dep [...]
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">Alert</span><span class="o">></span><span class="w"> </span><span class="n">alerts</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">transactions</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">process</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">DynamicKeyFunction</span><span class="p">())</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">keyBy</span><span class="p">((</span><span class="n">keyed</span><span class="p">)</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">keyed</span><span class="p">.</span><span class="na">getKey</span><span class="p">());</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">process</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">DynamicAlertFunction</span><span class="p">())</span><span class="w">
+</span></span></span></code></pre></div><p>By applying <code>DynamicKeyFunction</code> we are implicitly copying events for performing parallel per-rule evaluation within a Flink cluster. By doing so, we achieve an important property - horizontal scalability of rules’ processing. Our system will be capable of handling more rules by adding more servers to the cluster, i.e. increasing the parallelism. This property is achieved at the cost of data duplication, which might become an is [...]
<h3 id="summary">
Summary:
<a class="anchor" href="#summary">#</a>
diff --git a/content/2020/01/29/state-unlocked-interacting-with-state-in-apache-flink/index.html b/content/2020/01/29/state-unlocked-interacting-with-state-in-apache-flink/index.html
index f1e7a671d..b4d00aa8b 100644
--- a/content/2020/01/29/state-unlocked-interacting-with-state-in-apache-flink/index.html
+++ b/content/2020/01/29/state-unlocked-interacting-with-state-in-apache-flink/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -15,27 +15,26 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
-<meta name="description" content="Introduction # With stateful stream-processing becoming the norm for complex event-driven applications and real-time analytics, Apache Flink is often the backbone for running business logic and managing an organization’s most valuable asset — its data — as application state in Flink.
+<meta name="description" content="Introduction # With stateful stream-processing becoming the norm for complex event-driven applications and real-time analytics, Apache Flink is often the backbone for running business logic and managing an organization’s most valuable asset — its data — as application state in Flink.
In order to provide a state-of-the-art experience to Flink developers, the Apache Flink community makes significant efforts to provide the safety and future-proof guarantees organizations need while managing state in Flink.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="State Unlocked: Interacting with State in Apache Flink" />
-<meta property="og:description" content="Introduction # With stateful stream-processing becoming the norm for complex event-driven applications and real-time analytics, Apache Flink is often the backbone for running business logic and managing an organization’s most valuable asset — its data — as application state in Flink.
+<meta property="og:description" content="Introduction # With stateful stream-processing becoming the norm for complex event-driven applications and real-time analytics, Apache Flink is often the backbone for running business logic and managing an organization’s most valuable asset — its data — as application state in Flink.
In order to provide a state-of-the-art experience to Flink developers, the Apache Flink community makes significant efforts to provide the safety and future-proof guarantees organizations need while managing state in Flink." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2020/01/29/state-unlocked-interacting-with-state-in-apache-flink/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2020-01-29T12:00:00+00:00" />
<meta property="article:modified_time" content="2020-01-29T12:00:00+00:00" />
-
<title>State Unlocked: Interacting with State in Apache Flink | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -535,18 +534,18 @@ https://github.com/alex-shpak/hugo-book
State registration with built-in serialization in Apache Flink
<a class="anchor" href="#state-registration-with-built-in-serialization-in-apache-flink">#</a>
</h2>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">MyFunction</span> <span class="kd">extends</span> <span class="n">KeyedProcessFunction</span><span class="o"><</span><span class="n">Key</span><span class="o">,</span> <span class="n">Input</span><span class="o">,</span> <span class="n">Output</span><span class="o">></sp [...]
-</span></span><span class="line"><span class="cl"><span class="err"></span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="kd">transient</span> <span class="n">ValueState</span><span class="o"><</span><span class="n">MyState</span><span class="o">></span> <span class="n">valueState</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"><span class="err"></span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">open</span><span class="o">(</span><span class="n">Configuration</span> <span class="n">parameters</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">ValueStateDescriptor</span><span class="o"><</span><span class="n">MyState</span><span class="o">></span> <span class="n">descriptor</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">ValueStateDescriptor</span><span class="o"><>(</span><span class="s">"my-state"</span><span class="o">,</span> <span class="n">TypeInformation</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">MyState</span><span class="o">.</span><span class="na">class</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"><span class="err"></span>
-</span></span><span class="line"><span class="cl"> <span class="n">valueState</span> <span class="o">=</span> <span class="n">getRuntimeContext</span><span class="o">().</span><span class="na">getState</span><span class="o">(</span><span class="n">descriptor</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>Typically, evolving the schema of an application’s state happens because of some business logic change (adding or dropping fields or changing data types). In all cases, the schema is determined by means of its serializer, and can be thought of in terms of an alter table statement when compared with a database. When a state variable is first introduced it is like running a <code>CREATE_TABLE</code> command, there is a lot of freedom with its execution. [...]
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">MyFunction</span><span class="w"> </span><span class="kd">extends</span><span class="w"> </span><span class="n">KeyedProcessFunction</span><span class="o"><</span><span class="n">Key</span><span class="p">,</span><span class="w"> </span><span class="n [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="err"></span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="kd">transient</span><span class="w"> </span><span class="n">ValueState</span><span class="o"><</span><span class="n">MyState</span><span class="o">></span><span class="w"> </span><span class="n">valueState</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="err"></span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">open</span><span class="p">(</span><span class="n">Configuration</span><span class="w"> </span><span class="n">parameters</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">ValueStateDescriptor</span><span class="o"><</span><span class="n">MyState</span><span class="o">></span><span class="w"> </span><span class="n">descriptor</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">ValueStateDescriptor</span><span class="o"><></span><span class="p">(</span><span class="s">"my-state"</span><span class="p">,</span><span class="w"> </span><span class="n">TypeInformation</span><span class="p">.</span><span class="na">of</span><span class="p">(</span><span class="n">MyState</span><span class="p">.</span><sp [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="err"></span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">valueState</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">getRuntimeContext</span><span class="p">().</span><span class="na">getState</span><span class="p">(</span><span class="n">descriptor</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>Typically, evolving the schema of an application’s state happens because of some business logic change (adding or dropping fields or changing data types). In all cases, the schema is determined by means of its serializer, and can be thought of in terms of an alter table statement when compared with a database. When a state variable is first introduced it is like running a <code>CREATE_TABLE</code> command, there is a lot of freedom with its exec [...]
<p><a href="https://flink.apache.org/downloads.html#apache-flink-182">Flink 1.8</a> comes with built-in support for <a href="https://avro.apache.org/">Apache Avro</a> (specifically the <a href="https://avro.apache.org/docs/1.7.7/spec.html">1.7.7 specification</a>) and evolves state schema according to Avro specifications by adding and removing types or even by swapping between generic and specific Avro record types.</p>
<p>In <a href="https://flink.apache.org/downloads.html#apache-flink-191">Flink 1.9</a> the community added support for schema evolution for POJOs, including the ability to remove existing fields from POJO types or add new fields. The POJO schema evolution tends to be less flexible — when compared to Avro — since it is not possible to change neither the declared field types nor the class name of a POJO type, including its namespace.</p>
<p>With the community’s efforts related to schema evolution, Flink developers can now expect out-of-the-box support for both Avro and POJO formats, with backwards compatibility for all Flink state backends. Future work revolves around adding support for Scala Case Classes, Tuples and other formats. Make sure to subscribe to the <a href="https://flink.apache.org/community.html">Flink mailing list</a> to contribute and stay on top of any upcoming additions in this space.</p>
@@ -565,98 +564,98 @@ https://github.com/alex-shpak/hugo-book
<li>Check if both items above are compatible</li>
<li>Consume the bytes from the snapshot and move forward or alert the user otherwise</li>
</ul>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">interface</span> <span class="nc">TypeSerializerSnapshot</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"><span class="err"></span>
-</span></span><span class="line"><span class="cl"> <span class="kt">int</span> <span class="nf">getCurrentVersion</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"><span class="err"></span>
-</span></span><span class="line"><span class="cl"> <span class="kt">void</span> <span class="nf">writeSnapshot</span><span class="o">(</span><span class="n">DataOutputView</span> <span class="n">out</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">IOException</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"><span class="err"></span>
-</span></span><span class="line"><span class="cl"> <span class="kt">void</span> <span class="nf">readSnapshot</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="kt">int</span> <span class="n">readVersion</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">DataInputView</span> <span class="n">in</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">ClassLoader</span> <span class="n">userCodeClassLoader</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">IOException</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"><span class="err"></span>
-</span></span><span class="line"><span class="cl"> <span class="n">TypeSerializer</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="nf">restoreSerializer</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"><span class="err"></span>
-</span></span><span class="line"><span class="cl"> <span class="n">TypeSerializerSchemaCompatibility</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="nf">resolveSchemaCompatibility</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">TypeSerializer</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="n">newSerializer</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><h2 id="implementing-apache-avro-serialization-in-flink">
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">interface</span> <span class="nc">TypeSerializerSnapshot</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="err"></span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="nf">getCurrentVersion</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="err"></span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">writeSnapshot</span><span class="p">(</span><span class="n">DataOutputView</span><span class="w"> </span><span class="n">out</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">IOException</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="err"></span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">readSnapshot</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">readVersion</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">DataInputView</span><span class="w"> </span><span class="n">in</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">ClassLoader</span><span class="w"> </span><span class="n">userCodeClassLoader</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">IOException</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="err"></span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">TypeSerializer</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="w"> </span><span class="nf">restoreSerializer</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="err"></span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">TypeSerializerSchemaCompatibility</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="w"> </span><span class="nf">resolveSchemaCompatibility</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">TypeSerializer</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="w"> </span><span class="n">newSerializer</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><h2 id="implementing-apache-avro-serialization-in-flink">
Implementing Apache Avro Serialization in Flink
<a class="anchor" href="#implementing-apache-avro-serialization-in-flink">#</a>
</h2>
<p>Apache Avro is a data serialization format that has very well-defined schema migration semantics and supports both reader and writer schemas. During normal Flink execution the reader and writer schemas will be the same. However, when upgrading an application they may be different and with schema evolution, Flink will be able to migrate objects with their schemas.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">AvroSerializerSnapshot</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="kd">implements</span> <span class="n">TypeSerializerSnapshot</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span clas [...]
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">Schema</span> <span class="n">runtimeSchema</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">Schema</span> <span class="n">previousSchema</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"><span class="err"></span>
-</span></span><span class="line"><span class="cl"> <span class="nd">@SuppressWarnings</span><span class="o">(</span><span class="s">"WeakerAccess"</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="nf">AvroSerializerSnapshot</span><span class="o">()</span> <span class="o">{</span> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="err"></span>
-</span></span><span class="line"><span class="cl"> <span class="n">AvroSerializerSnapshot</span><span class="o">(</span><span class="n">Schema</span> <span class="n">schema</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">this</span><span class="o">.</span><span class="na">runtimeSchema</span> <span class="o">=</span> <span class="n">schema</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span></code></pre></div><p>This is a sketch of our Avro serializer. It uses the provided schemas and delegates to Apache Avro for all (de)-serialization. Let’s take a look at one possible implementation of a <code>TypeSerializerSnapshot</code> that supports schema migration for Avro.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">AvroSerializerSnapshot</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="w"> </span><span class="kd">implements</span><span class="w"> </span><span class="n">TypeSerializerSnapshot</span><span class="o">< [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">Schema</span><span class="w"> </span><span class="n">runtimeSchema</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">Schema</span><span class="w"> </span><span class="n">previousSchema</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="err"></span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@SuppressWarnings</span><span class="p">(</span><span class="s">"WeakerAccess"</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="nf">AvroSerializerSnapshot</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="err"></span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">AvroSerializerSnapshot</span><span class="p">(</span><span class="n">Schema</span><span class="w"> </span><span class="n">schema</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">runtimeSchema</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">schema</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>This is a sketch of our Avro serializer. It uses the provided schemas and delegates to Apache Avro for all (de)-serialization. Let’s take a look at one possible implementation of a <code>TypeSerializerSnapshot</code> that supports schema migration for Avro.</p>
<h1 id="writing-out-the-snapshot">
Writing out the snapshot
<a class="anchor" href="#writing-out-the-snapshot">#</a>
</h1>
<p>When serializing out the snapshot, the snapshot configuration will write two pieces of information; the current snapshot configuration version and the serializer configuration.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">int</span> <span class="nf">getCurrentVersion</span><span class="o">()</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">1</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="err"></span>
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">writeSnapshot</span><span class="o">(</span><span class="n">DataOutputView</span> <span class="n">out</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">IOException</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">out</span><span class="o">.</span><span class="na">writeUTF</span><span class="o">(</span><span class="n">runtimeSchema</span><span class="o">.</span><span class="na">toString</span><span class="o">(</span><span class="kc">false</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span></code></pre></div><p>The version is used to version the snapshot configuration object itself while the <code>writeSnapshot</code> method writes out all the information we need to understand the current format; the runtime schema.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">readSnapshot</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="kt">int</span> <span class="n">readVersion</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">DataInputView</span> <span class="n">in</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">ClassLoader</span> <span class="n">userCodeClassLoader</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">IOException</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="k">assert</span> <span class="n">readVersion</span> <span class="o">==</span> <span class="n">1</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">final</span> <span class="n">String</span> <span class="n">previousSchemaDefinition</span> <span class="o">=</span> <span class="n">in</span><span class="o">.</span><span class="na">readUTF</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="k">this</span><span class="o">.</span><span class="na">previousSchema</span> <span class="o">=</span> <span class="n">parseAvroSchema</span><span class="o">(</span><span class="n">previousSchemaDefinition</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="k">this</span><span class="o">.</span><span class="na">runtimeType</span> <span class="o">=</span> <span class="n">findClassOrFallbackToGeneric</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">userCodeClassLoader</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">previousSchema</span><span class="o">.</span><span class="na">getFullName</span><span class="o">());</span>
-</span></span><span class="line"><span class="cl"><span class="err"></span>
-</span></span><span class="line"><span class="cl"> <span class="k">this</span><span class="o">.</span><span class="na">runtimeSchema</span> <span class="o">=</span> <span class="n">tryExtractAvroSchema</span><span class="o">(</span><span class="n">userCodeClassLoader</span><span class="o">,</span> <span class="n">runtimeType</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span></code></pre></div><p>Now when Flink restores it is able to read back in the writer schema used to serialize the data. The current runtime schema is discovered on the class path using some Java reflection magic.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="nf">getCurrentVersion</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">1</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="err"></span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">writeSnapshot</span><span class="p">(</span><span class="n">DataOutputView</span><span class="w"> </span><span class="n">out</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">IOException</span><span class="w"> </s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">out</span><span class="p">.</span><span class="na">writeUTF</span><span class="p">(</span><span class="n">runtimeSchema</span><span class="p">.</span><span class="na">toString</span><span class="p">(</span><span class="kc">false</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>The version is used to version the snapshot configuration object itself while the <code>writeSnapshot</code> method writes out all the information we need to understand the current format; the runtime schema.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">readSnapshot</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">readVersion</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">DataInputView</span><span class="w"> </span><span class="n">in</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">ClassLoader</span><span class="w"> </span><span class="n">userCodeClassLoader</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">IOException</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">assert</span><span class="w"> </span><span class="n">readVersion</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="n">1</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">previousSchemaDefinition</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">in</span><span class="p">.</span><span class="na">readUTF</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">previousSchema</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">parseAvroSchema</span><span class="p">(</span><span class="n">previousSchemaDefinition</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">runtimeType</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">findClassOrFallbackToGeneric</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">userCodeClassLoader</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">previousSchema</span><span class="p">.</span><span class="na">getFullName</span><span class="p">());</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="err"></span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">runtimeSchema</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">tryExtractAvroSchema</span><span class="p">(</span><span class="n">userCodeClassLoader</span><span class="p">,</span><span class="w"> </span><span class="n">runtimeType</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>Now when Flink restores it is able to read back in the writer schema used to serialize the data. The current runtime schema is discovered on the class path using some Java reflection magic.</p>
<p>Once we have both of these we can compare them for compatibility. Perhaps nothing has changed and the schemas are compatible as is.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">TypeSerializerSchemaCompatibility</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="nf">resolveSchemaCompatibility</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">TypeSerializer</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="n">newSerializer</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"><span class="err"></span>
-</span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">(!(</span><span class="n">newSerializer</span> <span class="k">instanceof</span> <span class="n">AvroSerializer</span><span class="o">))</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">TypeSerializerSchemaCompatibility</span><span class="o">.</span><span class="na">incompatible</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="err"></span>
-</span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">(</span><span class="n">Objects</span><span class="o">.</span><span class="na">equals</span><span class="o">(</span><span class="n">previousSchema</span><span class="o">,</span> <span class="n">runtimeSchema</span><span class="o">))</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">TypeSerializerSchemaCompatibility</span><span class="o">.</span><span class="na">compatibleAsIs</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span></code></pre></div><p>Otherwise, the schemas are compared using Avro’s compatibility checks and they may either be compatible with a migration or incompatible.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"> <span class="kd">final</span> <span class="n">SchemaPairCompatibility</span> <span class="n">compatibility</span> <span class="o">=</span> <span class="n">SchemaCompatibility</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">checkReaderWriterCompatibility</span><span class="o">(</span><span class="n">previousSchema</span><span class="o">,</span> <span class="n">runtimeSchema</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"><span class="err"></span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">avroCompatibilityToFlinkCompatibility</span><span class="o">(</span><span class="n">compatibility</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span></code></pre></div><p>If they are compatible with migration then Flink will restore a new serializer that can read the old schema and deserialize into the new runtime type which is in effect a migration.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">TypeSerializer</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="nf">restoreSerializer</span><span class="o">()</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">(</span><span class="n">previousSchema</span> <span class="o">!=</span> <span class="kc">null</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="k">new</span> <span class="n">AvroSerializer</span><span class="o"><>(</span><span class="n">runtimeType</span><span class="o">,</span> <span class="n">runtimeSchema</span><span class="o">,</span> <span class="n">previousSchema</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span> <span class="k">else</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="k">new</span> <span class="n">AvroSerializer</span><span class="o"><>(</span><span class="n">runtimeType</span><span class="o">,</span> <span class="n">runtimeSchema</span><span class="o">,</span> <span class="n">runtimeSchema</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><h1 id="the-state-processor-api-reading-writing-and-modifying-flink-state">
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">TypeSerializerSchemaCompatibility</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="w"> </span><span class="nf">resolveSchemaCompatibility</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">TypeSerializer</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="w"> </span><span class="n">newSerializer</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="err"></span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="o">!</span><span class="p">(</span><span class="n">newSerializer</span><span class="w"> </span><span class="k">instanceof</span><span class="w"> </span><span class="n">AvroSerializer</span><span class="p">))</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">TypeSerializerSchemaCompatibility</span><span class="p">.</span><span class="na">incompatible</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="err"></span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">Objects</span><span class="p">.</span><span class="na">equals</span><span class="p">(</span><span class="n">previousSchema</span><span class="p">,</span><span class="w"> </span><span class="n">runtimeSchema</span><span class="p">))</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">TypeSerializerSchemaCompatibility</span><span class="p">.</span><span class="na">compatibleAsIs</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>Otherwise, the schemas are compared using Avro’s compatibility checks and they may either be compatible with a migration or incompatible.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="n">SchemaPairCompatibility</span><span class="w"> </span><span class="n">compatibility</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">SchemaCompatibility</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">checkReaderWriterCompatibility</span><span class="p">(</span><span class="n">previousSchema</span><span class="p">,</span><span class="w"> </span><span class="n">runtimeSchema</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="err"></span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">avroCompatibilityToFlinkCompatibility</span><span class="p">(</span><span class="n">compatibility</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>If they are compatible with migration then Flink will restore a new serializer that can read the old schema and deserialize into the new runtime type which is in effect a migration.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">TypeSerializer</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="w"> </span><span class="nf">restoreSerializer</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">previousSchema</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="kc">null</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">AvroSerializer</span><span class="o"><></span><span class="p">(</span><span class="n">runtimeType</span><span class="p">,</span><span class="w"> </span><span class="n">runtimeSchema</span><span class="p">,</span><span class="w"> </span><span class="n">previousSchema</span><span cl [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="k">else</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">AvroSerializer</span><span class="o"><></span><span class="p">(</span><span class="n">runtimeType</span><span class="p">,</span><span class="w"> </span><span class="n">runtimeSchema</span><span class="p">,</span><span class="w"> </span><span class="n">runtimeSchema</span><span cla [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><h1 id="the-state-processor-api-reading-writing-and-modifying-flink-state">
The State Processor API: Reading, writing and modifying Flink state
<a class="anchor" href="#the-state-processor-api-reading-writing-and-modifying-flink-state">#</a>
</h1>
diff --git a/content/2020/01/30/apache-flink-1.9.2-released/index.html b/content/2020/01/30/apache-flink-1.9.2-released/index.html
index 9bbd02f2c..636132fdd 100644
--- a/content/2020/01/30/apache-flink-1.9.2-released/index.html
+++ b/content/2020/01/30/apache-flink-1.9.2-released/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -30,18 +30,17 @@ Updated Maven dependencies:
<meta property="og:url" content="https://flink.apache.org/2020/01/30/apache-flink-1.9.2-released/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2020-01-30T12:00:00+00:00" />
<meta property="article:modified_time" content="2020-01-30T12:00:00+00:00" />
-
<title>Apache Flink 1.9.2 Released | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -64,7 +63,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -271,7 +270,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2020/02/03/a-guide-for-unit-testing-in-apache-flink/index.html b/content/2020/02/03/a-guide-for-unit-testing-in-apache-flink/index.html
index 8d5eaa472..3c27d6b10 100644
--- a/content/2020/02/03/a-guide-for-unit-testing-in-apache-flink/index.html
+++ b/content/2020/02/03/a-guide-for-unit-testing-in-apache-flink/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -22,18 +22,17 @@
<meta property="og:url" content="https://flink.apache.org/2020/02/03/a-guide-for-unit-testing-in-apache-flink/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2020-02-03T12:00:00+00:00" />
<meta property="article:modified_time" content="2020-02-03T12:00:00+00:00" />
-
<title>A Guide for Unit Testing in Apache Flink | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -525,106 +524,106 @@ Apache Flink provides a robust unit testing framework to make sure your applicat
<a class="anchor" href="#stateless-operators">#</a>
</h1>
<p>Writing unit tests for a stateless operator is a breeze. You need to follow the basic norm of writing a test case, i.e., create an instance of the function class and test the appropriate methods. Let’s take an example of a simple <code>Map</code> operator.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">MyStatelessMap</span> <span class="kd">implements</span> <span class="n">MapFunction</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">String</span><span class="o">></span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">String</span> <span class="nf">map</span><span class="o">(</span><span class="n">String</span> <span class="n">in</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">String</span> <span class="n">out</span> <span class="o">=</span> <span class="s">"hello "</span> <span class="o">+</span> <span class="n">in</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">out</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>The test case for the above operator should look like</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="nd">@Test</span>
-</span></span><span class="line"><span class="cl"><span class="kd">public</span> <span class="kt">void</span> <span class="nf">testMap</span><span class="o">()</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">MyStatelessMap</span> <span class="n">statelessMap</span> <span class="o">=</span> <span class="k">new</span> <span class="n">MyStatelessMap</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="n">String</span> <span class="n">out</span> <span class="o">=</span> <span class="n">statelessMap</span><span class="o">.</span><span class="na">map</span><span class="o">(</span><span class="s">"world"</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Assert</span><span class="o">.</span><span class="na">assertEquals</span><span class="o">(</span><span class="s">"hello world"</span><span class="o">,</span> <span class="n">out</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>Pretty simple, right? Let’s take a look at one for the <code>FlatMap</code> operator.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">MyStatelessFlatMap</span> <span class="kd">implements</span> <span class="n">FlatMapFunction</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">String</span><span class="o">></span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">flatMap</span><span class="o">(</span><span class="n">String</span> <span class="n">in</span><span class="o">,</span> <span class="n">Collector</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">collector</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span cl [...]
-</span></span><span class="line"><span class="cl"> <span class="n">String</span> <span class="n">out</span> <span class="o">=</span> <span class="s">"hello "</span> <span class="o">+</span> <span class="n">in</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="n">collector</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="n">out</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p><code>FlatMap</code> operators require a <code>Collector</code> object along with the input. For the test case, we have two options:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">MyStatelessMap</span><span class="w"> </span><span class="kd">implements</span><span class="w"> </span><span class="n">MapFunction</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class=" [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="nf">map</span><span class="p">(</span><span class="n">String</span><span class="w"> </span><span class="n">in</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span><span class="w"> </span><span class="p"> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">out</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">"hello "</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">in</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">out</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>The test case for the above operator should look like</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="nd">@Test</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">testMap</span><span class="p">()</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">MyStatelessMap</span><span class="w"> </span><span class="n">statelessMap</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">MyStatelessMap</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">out</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">statelessMap</span><span class="p">.</span><span class="na">map</span><span class="p">(</span><span class="s">"world"</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Assert</span><span class="p">.</span><span class="na">assertEquals</span><span class="p">(</span><span class="s">"hello world"</span><span class="p">,</span><span class="w"> </span><span class="n">out</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>Pretty simple, right? Let’s take a look at one for the <code>FlatMap</code> operator.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">MyStatelessFlatMap</span><span class="w"> </span><span class="kd">implements</span><span class="w"> </span><span class="n">FlatMapFunction</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">flatMap</span><span class="p">(</span><span class="n">String</span><span class="w"> </span><span class="n">in</span><span class="p">,</span><span class="w"> </span><span class="n">Collector</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span cl [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">out</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">"hello "</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">in</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">collector</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="n">out</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p><code>FlatMap</code> operators require a <code>Collector</code> object along with the input. For the test case, we have two options:</p>
<ol>
<li>Mock the <code>Collector</code> object using Mockito</li>
<li>Use the <code>ListCollector</code> provided by Flink</li>
</ol>
<p>I prefer the second method as it requires fewer lines of code and is suitable for most of the cases.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="nd">@Test</span>
-</span></span><span class="line"><span class="cl"><span class="kd">public</span> <span class="kt">void</span> <span class="nf">testFlatMap</span><span class="o">()</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">MyStatelessFlatMap</span> <span class="n">statelessFlatMap</span> <span class="o">=</span> <span class="k">new</span> <span class="n">MyStatelessFlatMap</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="n">List</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">out</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ArrayList</span><span class="o"><>();</span>
-</span></span><span class="line"><span class="cl"> <span class="n">ListCollector</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">listCollector</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ListCollector</span><span class="o"><>(</span><span class="n">out</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="n">statelessFlatMap</span><span class="o">.</span><span class="na">flatMap</span><span class="o">(</span><span class="s">"world"</span><span class="o">,</span> <span class="n">listCollector</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Assert</span><span class="o">.</span><span class="na">assertEquals</span><span class="o">(</span><span class="n">Lists</span><span class="o">.</span><span class="na">newArrayList</span><span class="o">(</span><span class="s">"hello world"</span><span class="o">),</span> <span class="n">out</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><h1 id="stateful-operators">
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="nd">@Test</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">testFlatMap</span><span class="p">()</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">MyStatelessFlatMap</span><span class="w"> </span><span class="n">statelessFlatMap</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">MyStatelessFlatMap</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">List</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">out</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">ArrayList</span><span class="o"><></span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">ListCollector</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">listCollector</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">ListCollector</span><span class="o"><></span><span class="p">(</span><span class="n">ou [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">statelessFlatMap</span><span class="p">.</span><span class="na">flatMap</span><span class="p">(</span><span class="s">"world"</span><span class="p">,</span><span class="w"> </span><span class="n">listCollector</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Assert</span><span class="p">.</span><span class="na">assertEquals</span><span class="p">(</span><span class="n">Lists</span><span class="p">.</span><span class="na">newArrayList</span><span class="p">(</span><span class="s">"hello world"</span><span class="p">),</span><span class="w"> </span><span class="n">out</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><h1 id="stateful-operators">
Stateful Operators
<a class="anchor" href="#stateful-operators">#</a>
</h1>
<p>Writing test cases for stateful operators requires more effort. You need to check whether the operator state is updated correctly and if it is cleaned up properly along with the output of the operator.</p>
<p>Let’s take an example of stateful <code>FlatMap</code> function</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">StatefulFlatMap</span> <span class="kd">extends</span> <span class="n">RichFlatMapFunction</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">String</span><span class="o">></span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">ValueState</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">previousInput</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">open</span><span class="o">(</span><span class="n">Configuration</span> <span class="n">parameters</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">previousInput</span> <span class="o">=</span> <span class="n">getRuntimeContext</span><span class="o">().</span><span class="na">getState</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">ValueStateDescriptor</span><span class="o"><</span><span class="n">String</span><span class="o">>(</span><span class="s">"previousInput"</span><span class="o">,</span> <span class="n">Types</span><span class="o">.</span><span class="na">STRING</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">flatMap</span><span class="o">(</span><span class="n">String</span> <span class="n">in</span><span class="o">,</span> <span class="n">Collector</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">collector</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span cl [...]
-</span></span><span class="line"><span class="cl"> <span class="n">String</span> <span class="n">out</span> <span class="o">=</span> <span class="s">"hello "</span> <span class="o">+</span> <span class="n">in</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="k">if</span><span class="o">(</span><span class="n">previousInput</span><span class="o">.</span><span class="na">value</span><span class="o">()</span> <span class="o">!=</span> <span class="kc">null</span><span class="o">){</span>
-</span></span><span class="line"><span class="cl"> <span class="n">out</span> <span class="o">=</span> <span class="n">out</span> <span class="o">+</span> <span class="s">" "</span> <span class="o">+</span> <span class="n">previousInput</span><span class="o">.</span><span class="na">value</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="n">previousInput</span><span class="o">.</span><span class="na">update</span><span class="o">(</span><span class="n">in</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="n">collector</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="n">out</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>The intricate part of writing tests for the above class is to mock the configuration as well as the runtime context of the application. Flink provides TestHarness classes so that users don’t have to create the mock objects themselves. Using the <code>KeyedOperatorHarness</code>, the test looks like:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">org.apache.flink.streaming.api.operators.StreamFlatMap</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">org.apache.flink.streaming.runtime.streamrecord.StreamRecord</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">org.apache.flink.streaming.util.KeyedOneInputStreamOperatorTestHarness</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">org.apache.flink.streaming.util.OneInputStreamOperatorTestHarness</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="nd">@Test</span>
-</span></span><span class="line"><span class="cl"><span class="kd">public</span> <span class="kt">void</span> <span class="nf">testFlatMap</span><span class="o">()</span> <span class="kd">throws</span> <span class="n">Exception</span><span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">StatefulFlatMap</span> <span class="n">statefulFlatMap</span> <span class="o">=</span> <span class="k">new</span> <span class="n">StatefulFlatMap</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">// OneInputStreamOperatorTestHarness takes the input and output types as type parameters
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">OneInputStreamOperatorTestHarness</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">String</span><span class="o">></span> <span class="n">testHarness</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="c1">// KeyedOneInputStreamOperatorTestHarness takes three arguments:
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="c1">// Flink operator object, key selector and key type
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">new</span> <span class="n">KeyedOneInputStreamOperatorTestHarness</span><span class="o"><>(</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">StreamFlatMap</span><span class="o"><>(</span><span class="n">statefulFlatMap</span><span class="o">),</span> <span class="n">x</span> <span class="o">-></span> <span class="s">"1"</span><span class="o">,</span> <span class="n">Types</span><span class="o">.</span><span class="na">STRING</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="n">testHarness</span><span class="o">.</span><span class="na">open</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">// test first record
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">testHarness</span><span class="o">.</span><span class="na">processElement</span><span class="o">(</span><span class="s">"world"</span><span class="o">,</span> <span class="n">10</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="n">ValueState</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">previousInput</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="n">statefulFlatMap</span><span class="o">.</span><span class="na">getRuntimeContext</span><span class="o">().</span><span class="na">getState</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">ValueStateDescriptor</span><span class="o"><>(</span><span class="s">"previousInput"</span><span class="o">,</span> <span class="n">Types</span><span class="o">.</span><span class="na">STRING</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"> <span class="n">String</span> <span class="n">stateValue</span> <span class="o">=</span> <span class="n">previousInput</span><span class="o">.</span><span class="na">value</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Assert</span><span class="o">.</span><span class="na">assertEquals</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Lists</span><span class="o">.</span><span class="na">newArrayList</span><span class="o">(</span><span class="k">new</span> <span class="n">StreamRecord</span><span class="o"><>(</span><span class="s">"hello world"</span><span class="o">,</span> <span class="n">10</span><span class="o">)),</span>
-</span></span><span class="line"><span class="cl"> <span class="n">testHarness</span><span class="o">.</span><span class="na">extractOutputStreamRecords</span><span class="o">());</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Assert</span><span class="o">.</span><span class="na">assertEquals</span><span class="o">(</span><span class="s">"world"</span><span class="o">,</span> <span class="n">stateValue</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">// test second record
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">testHarness</span><span class="o">.</span><span class="na">processElement</span><span class="o">(</span><span class="s">"parallel"</span><span class="o">,</span> <span class="n">20</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Assert</span><span class="o">.</span><span class="na">assertEquals</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Lists</span><span class="o">.</span><span class="na">newArrayList</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">StreamRecord</span><span class="o"><>(</span><span class="s">"hello world"</span><span class="o">,</span> <span class="n">10</span><span class="o">),</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">StreamRecord</span><span class="o"><>(</span><span class="s">"hello parallel world"</span><span class="o">,</span> <span class="n">20</span><span class="o">)),</span>
-</span></span><span class="line"><span class="cl"> <span class="n">testHarness</span><span class="o">.</span><span class="na">extractOutputStreamRecords</span><span class="o">());</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Assert</span><span class="o">.</span><span class="na">assertEquals</span><span class="o">(</span><span class="s">"parallel"</span><span class="o">,</span> <span class="n">previousInput</span><span class="o">.</span><span class="na">value</span><span class="o">());</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>The test harness provides many helper methods, three of which are being used here:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">StatefulFlatMap</span><span class="w"> </span><span class="kd">extends</span><span class="w"> </span><span class="n">RichFlatMapFunction</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span c [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">ValueState</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">previousInput</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">open</span><span class="p">(</span><span class="n">Configuration</span><span class="w"> </span><span class="n">parameters</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span><span class="w"> </span>< [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">previousInput</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">getRuntimeContext</span><span class="p">().</span><span class="na">getState</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">ValueStateDescriptor</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="p">(</span><span class="s">"previousInput"</span><span class="p">,</span><span class="w"> </span><span class="n">Types</span><span class="p">.</span><span class="na">STRING</span><span class="p">));</span><spa [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">flatMap</span><span class="p">(</span><span class="n">String</span><span class="w"> </span><span class="n">in</span><span class="p">,</span><span class="w"> </span><span class="n">Collector</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span cl [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">out</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">"hello "</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">in</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">if</span><span class="p">(</span><span class="n">previousInput</span><span class="p">.</span><span class="na">value</span><span class="p">()</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="kc">null</span><span class="p">){</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">out</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">out</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="s">" "</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">previousInput</span><span class="p">.</span><span class="na">value</span><span class="p" [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">previousInput</span><span class="p">.</span><span class="na">update</span><span class="p">(</span><span class="n">in</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">collector</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="n">out</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>The intricate part of writing tests for the above class is to mock the configuration as well as the runtime context of the application. Flink provides TestHarness classes so that users don’t have to create the mock objects themselves. Using the <code>KeyedOperatorHarness</code>, the test looks like:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kn">import</span><span class="w"> </span><span class="nn">org.apache.flink.streaming.api.operators.StreamFlatMap</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kn">import</span><span class="w"> </span><span class="nn">org.apache.flink.streaming.runtime.streamrecord.StreamRecord</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kn">import</span><span class="w"> </span><span class="nn">org.apache.flink.streaming.util.KeyedOneInputStreamOperatorTestHarness</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kn">import</span><span class="w"> </span><span class="nn">org.apache.flink.streaming.util.OneInputStreamOperatorTestHarness</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nd">@Test</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">testFlatMap</span><span class="p">()</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">StatefulFlatMap</span><span class="w"> </span><span class="n">statefulFlatMap</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">StatefulFlatMap</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// OneInputStreamOperatorTestHarness takes the input and output types as type parameters </span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">OneInputStreamOperatorTestHarness</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">testHarness</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// KeyedOneInputStreamOperatorTestHarness takes three arguments:</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// Flink operator object, key selector and key type</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">KeyedOneInputStreamOperatorTestHarness</span><span class="o"><></span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">StreamFlatMap</span><span class="o"><></span><span class="p">(</span><span class="n">statefulFlatMap</span><span class="p">),</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="s">"1"</span><span class="p">,</span><span class="w"> </s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">testHarness</span><span class="p">.</span><span class="na">open</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// test first record</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">testHarness</span><span class="p">.</span><span class="na">processElement</span><span class="p">(</span><span class="s">"world"</span><span class="p">,</span><span class="w"> </span><span class="n">10</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">ValueState</span><span class="o"><</span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">previousInput</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">statefulFlatMap</span><span class="p">.</span><span class="na">getRuntimeContext</span><span class="p">().</span><span class="na">getState</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">ValueStateDescriptor</span><span class="o"><></span><span class="p">(</span><span class="s">"previousInput"</span><span class="p">,</span><span class="w"> </span><span class="n">Types</span><span class="p">.</span><span class="na">STRING</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">stateValue</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">previousInput</span><span class="p">.</span><span class="na">value</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Assert</span><span class="p">.</span><span class="na">assertEquals</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Lists</span><span class="p">.</span><span class="na">newArrayList</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">StreamRecord</span><span class="o"><></span><span class="p">(</span><span class="s">"hello world"</span><span class="p">,</span><span class="w"> </span><span class="n">10</span><span class="p">)),</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">testHarness</span><span class="p">.</span><span class="na">extractOutputStreamRecords</span><span class="p">());</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Assert</span><span class="p">.</span><span class="na">assertEquals</span><span class="p">(</span><span class="s">"world"</span><span class="p">,</span><span class="w"> </span><span class="n">stateValue</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// test second record</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">testHarness</span><span class="p">.</span><span class="na">processElement</span><span class="p">(</span><span class="s">"parallel"</span><span class="p">,</span><span class="w"> </span><span class="n">20</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Assert</span><span class="p">.</span><span class="na">assertEquals</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Lists</span><span class="p">.</span><span class="na">newArrayList</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">StreamRecord</span><span class="o"><></span><span class="p">(</span><span class="s">"hello world"</span><span class="p">,</span><span class="w"> </span><span class="n">10</span><span class="p">),</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">StreamRecord</span><span class="o"><></span><span class="p">(</span><span class="s">"hello parallel world"</span><span class="p">,</span><span class="w"> </span><span class="n">20</span><span class="p">)),</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">testHarness</span><span class="p">.</span><span class="na">extractOutputStreamRecords</span><span class="p">());</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Assert</span><span class="p">.</span><span class="na">assertEquals</span><span class="p">(</span><span class="s">"parallel"</span><span class="p">,</span><span class="w"> </span><span class="n">previousInput</span><span class="p">.</span><span class="na">value</span><span class="p">());</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>The test harness provides many helper methods, three of which are being used here:</p>
<ol>
<li><code>open</code>: calls the open of the <code>FlatMap</code> function with relevant parameters. It also initializes the context.</li>
<li><code>processElement</code>: allows users to pass an input element as well as the timestamp associated with the element.</li>
@@ -638,57 +637,57 @@ Apache Flink provides a robust unit testing framework to make sure your applicat
</h1>
<p>Writing tests for process functions, that work with time, is quite similar to writing tests for stateful functions because you can also use test harness.
However, you need to take care of another aspect, which is providing timestamps for events and controlling the current time of the application. By setting the current (processing or event) time, you can trigger registered timers, which will call the <code>onTimer</code> method of the function</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">MyProcessFunction</span> <span class="kd">extends</span> <span class="n">KeyedProcessFunction</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">String</span><span class="o">,</span> <span class="n">String</span><span class=" [...]
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">processElement</span><span class="o">(</span><span class="n">String</span> <span class="n">in</span><span class="o">,</span> <span class="n">Context</span> <span class="n">context</span><span class="o">,</span> <span class="n">Collector</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">collector</span><spa [...]
-</span></span><span class="line"><span class="cl"> <span class="n">context</span><span class="o">.</span><span class="na">timerService</span><span class="o">().</span><span class="na">registerProcessingTimeTimer</span><span class="o">(</span><span class="n">50</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="n">String</span> <span class="n">out</span> <span class="o">=</span> <span class="s">"hello "</span> <span class="o">+</span> <span class="n">in</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="n">collector</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="n">out</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">onTimer</span><span class="o">(</span><span class="kt">long</span> <span class="n">timestamp</span><span class="o">,</span> <span class="n">OnTimerContext</span> <span class="n">ctx</span><span class="o">,</span> <span class="n">Collector</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">out</span><span cl [...]
-</span></span><span class="line"><span class="cl"> <span class="n">out</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="n">String</span><span class="o">.</span><span class="na">format</span><span class="o">(</span><span class="s">"Timer triggered at timestamp %d"</span><span class="o">,</span> <span class="n">timestamp</span><span class="o">));</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>We need to test both the methods in the <code>KeyedProcessFunction</code>, i.e., <code>processElement</code> as well as <code>onTimer</code>. Using a test harness, we can control the current time of the function. Thus, we can trigger the timer at will rather than waiting for a specific time.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">MyProcessFunction</span><span class="w"> </span><span class="kd">extends</span><span class="w"> </span><span class="n">KeyedProcessFunction</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><spa [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">processElement</span><span class="p">(</span><span class="n">String</span><span class="w"> </span><span class="n">in</span><span class="p">,</span><span class="w"> </span><span class="n">Context</span><span class="w"> </span><span class="n">context</span><span class="p">,</span><span cl [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">context</span><span class="p">.</span><span class="na">timerService</span><span class="p">().</span><span class="na">registerProcessingTimeTimer</span><span class="p">(</span><span class="n">50</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">out</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">"hello "</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">in</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">collector</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="n">out</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">onTimer</span><span class="p">(</span><span class="kt">long</span><span class="w"> </span><span class="n">timestamp</span><span class="p">,</span><span class="w"> </span><span class="n">OnTimerContext</span><span class="w"> </span><span class="n">ctx</span><span class="p">,</span><span [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">out</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="n">String</span><span class="p">.</span><span class="na">format</span><span class="p">(</span><span class="s">"Timer triggered at timestamp %d"</span><span class="p">,</span><span class="w"> </span><span class="n">timestamp</span><span class="p">));</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>We need to test both the methods in the <code>KeyedProcessFunction</code>, i.e., <code>processElement</code> as well as <code>onTimer</code>. Using a test harness, we can control the current time of the function. Thus, we can trigger the timer at will rather than waiting for a specific time.</p>
<p>Let’s take a look at the test case</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="nd">@Test</span>
-</span></span><span class="line"><span class="cl"><span class="kd">public</span> <span class="kt">void</span> <span class="nf">testProcessElement</span><span class="o">()</span> <span class="kd">throws</span> <span class="n">Exception</span><span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">MyProcessFunction</span> <span class="n">myProcessFunction</span> <span class="o">=</span> <span class="k">new</span> <span class="n">MyProcessFunction</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="n">OneInputStreamOperatorTestHarness</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">String</span><span class="o">></span> <span class="n">testHarness</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">KeyedOneInputStreamOperatorTestHarness</span><span class="o"><>(</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">KeyedProcessOperator</span><span class="o"><>(</span><span class="n">myProcessFunction</span><span class="o">),</span> <span class="n">x</span> <span class="o">-></span> <span class="s">"1"</span><span class="o">,</span> <span class="n">Types</span><span class="o">.</span><span class="na">STRING</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">// Function time is initialized to 0
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">testHarness</span><span class="o">.</span><span class="na">open</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="n">testHarness</span><span class="o">.</span><span class="na">processElement</span><span class="o">(</span><span class="s">"world"</span><span class="o">,</span> <span class="n">10</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="n">Assert</span><span class="o">.</span><span class="na">assertEquals</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Lists</span><span class="o">.</span><span class="na">newArrayList</span><span class="o">(</span><span class="k">new</span> <span class="n">StreamRecord</span><span class="o"><>(</span><span class="s">"hello world"</span><span class="o">,</span> <span class="n">10</span><span class="o">)),</span>
-</span></span><span class="line"><span class="cl"> <span class="n">testHarness</span><span class="o">.</span><span class="na">extractOutputStreamRecords</span><span class="o">());</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="nd">@Test</span>
-</span></span><span class="line"><span class="cl"><span class="kd">public</span> <span class="kt">void</span> <span class="nf">testOnTimer</span><span class="o">()</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">MyProcessFunction</span> <span class="n">myProcessFunction</span> <span class="o">=</span> <span class="k">new</span> <span class="n">MyProcessFunction</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="n">OneInputStreamOperatorTestHarness</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">String</span><span class="o">></span> <span class="n">testHarness</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">KeyedOneInputStreamOperatorTestHarness</span><span class="o"><>(</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">KeyedProcessOperator</span><span class="o"><>(</span><span class="n">myProcessFunction</span><span class="o">),</span> <span class="n">x</span> <span class="o">-></span> <span class="s">"1"</span><span class="o">,</span> <span class="n">Types</span><span class="o">.</span><span class="na">STRING</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="n">testHarness</span><span class="o">.</span><span class="na">open</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="n">testHarness</span><span class="o">.</span><span class="na">processElement</span><span class="o">(</span><span class="s">"world"</span><span class="o">,</span> <span class="n">10</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Assert</span><span class="o">.</span><span class="na">assertEquals</span><span class="o">(</span><span class="n">1</span><span class="o">,</span> <span class="n">testHarness</span><span class="o">.</span><span class="na">numProcessingTimeTimers</span><span class="o">());</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="c1">// Function time is set to 50
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">testHarness</span><span class="o">.</span><span class="na">setProcessingTime</span><span class="o">(</span><span class="n">50</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Assert</span><span class="o">.</span><span class="na">assertEquals</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Lists</span><span class="o">.</span><span class="na">newArrayList</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">StreamRecord</span><span class="o"><>(</span><span class="s">"hello world"</span><span class="o">,</span> <span class="n">10</span><span class="o">),</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">StreamRecord</span><span class="o"><>(</span><span class="s">"Timer triggered at timestamp 50"</span><span class="o">)),</span>
-</span></span><span class="line"><span class="cl"> <span class="n">testHarness</span><span class="o">.</span><span class="na">extractOutputStreamRecords</span><span class="o">());</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>The mechanism to test the multi-input stream operators such as CoProcess functions is similar to the ones described in this article. You should use the TwoInput variant of the harness for these operators, such as <code>TwoInputStreamOperatorTestHarness</code>.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="nd">@Test</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">testProcessElement</span><span class="p">()</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">MyProcessFunction</span><span class="w"> </span><span class="n">myProcessFunction</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">MyProcessFunction</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">OneInputStreamOperatorTestHarness</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">testHarness</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">KeyedOneInputStreamOperatorTestHarness</span><span class="o"><></span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">KeyedProcessOperator</span><span class="o"><></span><span class="p">(</span><span class="n">myProcessFunction</span><span class="p">),</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="s">"1"</span><span class="p">,</span><span class [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// Function time is initialized to 0</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">testHarness</span><span class="p">.</span><span class="na">open</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">testHarness</span><span class="p">.</span><span class="na">processElement</span><span class="p">(</span><span class="s">"world"</span><span class="p">,</span><span class="w"> </span><span class="n">10</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Assert</span><span class="p">.</span><span class="na">assertEquals</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Lists</span><span class="p">.</span><span class="na">newArrayList</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">StreamRecord</span><span class="o"><></span><span class="p">(</span><span class="s">"hello world"</span><span class="p">,</span><span class="w"> </span><span class="n">10</span><span class="p">)),</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">testHarness</span><span class="p">.</span><span class="na">extractOutputStreamRecords</span><span class="p">());</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nd">@Test</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">testOnTimer</span><span class="p">()</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">MyProcessFunction</span><span class="w"> </span><span class="n">myProcessFunction</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">MyProcessFunction</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">OneInputStreamOperatorTestHarness</span><span class="o"><</span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">String</span><span class="o">></span><span class="w"> </span><span class="n">testHarness</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">KeyedOneInputStreamOperatorTestHarness</span><span class="o"><></span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">KeyedProcessOperator</span><span class="o"><></span><span class="p">(</span><span class="n">myProcessFunction</span><span class="p">),</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="s">"1"</span><span class="p">,</span><span class [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">testHarness</span><span class="p">.</span><span class="na">open</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">testHarness</span><span class="p">.</span><span class="na">processElement</span><span class="p">(</span><span class="s">"world"</span><span class="p">,</span><span class="w"> </span><span class="n">10</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Assert</span><span class="p">.</span><span class="na">assertEquals</span><span class="p">(</span><span class="n">1</span><span class="p">,</span><span class="w"> </span><span class="n">testHarness</span><span class="p">.</span><span class="na">numProcessingTimeTimers</span><span class="p">());</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// Function time is set to 50</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">testHarness</span><span class="p">.</span><span class="na">setProcessingTime</span><span class="p">(</span><span class="n">50</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Assert</span><span class="p">.</span><span class="na">assertEquals</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Lists</span><span class="p">.</span><span class="na">newArrayList</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">StreamRecord</span><span class="o"><></span><span class="p">(</span><span class="s">"hello world"</span><span class="p">,</span><span class="w"> </span><span class="n">10</span><span class="p">),</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">StreamRecord</span><span class="o"><></span><span class="p">(</span><span class="s">"Timer triggered at timestamp 50"</span><span class="p">)),</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">testHarness</span><span class="p">.</span><span class="na">extractOutputStreamRecords</span><span class="p">());</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>The mechanism to test the multi-input stream operators such as CoProcess functions is similar to the ones described in this article. You should use the TwoInput variant of the harness for these operators, such as <code>TwoInputStreamOperatorTestHarness</code>.</p>
<h1 id="summary">
Summary
<a class="anchor" href="#summary">#</a>
diff --git a/content/2020/02/11/apache-flink-1.10.0-release-announcement/index.html b/content/2020/02/11/apache-flink-1.10.0-release-announcement/index.html
index 28fe9b90c..68af8a016 100644
--- a/content/2020/02/11/apache-flink-1.10.0-release-announcement/index.html
+++ b/content/2020/02/11/apache-flink-1.10.0-release-announcement/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ Flink 1.10 also marks the completion of the Blink integration, hardening streami
<meta property="og:url" content="https://flink.apache.org/2020/02/11/apache-flink-1.10.0-release-announcement/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2020-02-11T02:30:00+00:00" />
<meta property="article:modified_time" content="2020-02-11T02:30:00+00:00" />
-
<title>Apache Flink 1.10.0 Release Announcement | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2020/02/20/no-java-required-configuring-sources-and-sinks-in-sql/index.html b/content/2020/02/20/no-java-required-configuring-sources-and-sinks-in-sql/index.html
index 59a93e97a..70097f21f 100644
--- a/content/2020/02/20/no-java-required-configuring-sources-and-sinks-in-sql/index.html
+++ b/content/2020/02/20/no-java-required-configuring-sources-and-sinks-in-sql/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -15,25 +15,24 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
-<meta name="description" content="Introduction # The recent Apache Flink 1.10 release includes many exciting features. In particular, it marks the end of the community’s year-long effort to merge in the Blink SQL contribution from Alibaba. The reason the community chose to spend so much time on the contribution is that SQL works. It allows Flink to offer a truly unified interface over batch and streaming and makes stream processing accessible to a broad audience of developers and [...]
+<meta name="description" content="Introduction # The recent Apache Flink 1.10 release includes many exciting features. In particular, it marks the end of the community’s year-long effort to merge in the Blink SQL contribution from Alibaba. The reason the community chose to spend so much time on the contribution is that SQL works. It allows Flink to offer a truly unified interface over batch and streaming and makes stream processing accessible to a broad audience of developers and a [...]
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="No Java Required: Configuring Sources and Sinks in SQL" />
-<meta property="og:description" content="Introduction # The recent Apache Flink 1.10 release includes many exciting features. In particular, it marks the end of the community’s year-long effort to merge in the Blink SQL contribution from Alibaba. The reason the community chose to spend so much time on the contribution is that SQL works. It allows Flink to offer a truly unified interface over batch and streaming and makes stream processing accessible to a broad audience of develope [...]
+<meta property="og:description" content="Introduction # The recent Apache Flink 1.10 release includes many exciting features. In particular, it marks the end of the community’s year-long effort to merge in the Blink SQL contribution from Alibaba. The reason the community chose to spend so much time on the contribution is that SQL works. It allows Flink to offer a truly unified interface over batch and streaming and makes stream processing accessible to a broad audience of developer [...]
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2020/02/20/no-java-required-configuring-sources-and-sinks-in-sql/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2020-02-20T12:00:00+00:00" />
<meta property="article:modified_time" content="2020-02-20T12:00:00+00:00" />
-
<title>No Java Required: Configuring Sources and Sinks in SQL | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -597,14 +596,14 @@ However, we are not ones to rest on our laurels and, the community is committed
The community is looking to add more catalogs, such as JDBC and Apache Pulsar.
We encourage you to sign up for the <a href="https://flink.apache.org/community.html">mailing list</a> and stay on top of the announcements and new features in upcoming releases.</p>
<hr>
-<section class="footnotes" role="doc-endnotes">
+<div class="footnotes" role="doc-endnotes">
<hr>
<ol>
-<li id="fn:1" role="doc-endnote">
+<li id="fn:1">
<p>My colleague Timo, whose worked on Flink SQL from the beginning, has the entire SQL standard printed on his desk and references it before any changes are merged. It’s enormous. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">↩︎</a></p>
</li>
</ol>
-</section>
+</div>
</p>
</article>
diff --git a/content/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink/index.html b/content/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink/index.html
index fd2840e00..9b4ffe7f0 100644
--- a/content/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink/index.html
+++ b/content/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ Apache Flink and Apache Beam are open-source frameworks for parallel, distribute
<meta property="og:url" content="https://flink.apache.org/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2020-02-22T12:00:00+00:00" />
<meta property="article:modified_time" content="2020-02-22T12:00:00+00:00" />
-
<title>Apache Beam: How Beam Runs on Top of Flink | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2020/03/24/advanced-flink-application-patterns-vol.2-dynamic-updates-of-application-logic/index.html b/content/2020/03/24/advanced-flink-application-patterns-vol.2-dynamic-updates-of-application-logic/index.html
index bbaaa46cd..06b458795 100644
--- a/content/2020/03/24/advanced-flink-application-patterns-vol.2-dynamic-updates-of-application-logic/index.html
+++ b/content/2020/03/24/advanced-flink-application-patterns-vol.2-dynamic-updates-of-application-logic/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ We intentionally omitted details of how the applied rules are initialized and wh
<meta property="og:url" content="https://flink.apache.org/2020/03/24/advanced-flink-application-patterns-vol.2-dynamic-updates-of-application-logic/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2020-03-24T12:00:00+00:00" />
<meta property="article:modified_time" content="2020-03-24T12:00:00+00:00" />
-
<title>Advanced Flink Application Patterns Vol.2: Dynamic Updates of Application Logic | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -501,21 +500,21 @@ https://github.com/alex-shpak/hugo-book
<a class="anchor" href="#rules-broadcasting">#</a>
</h2>
<p>Let’s first have a look at the <a href="https://flink.apache.org/news/2020/01/15/demo-fraud-detection.html#dynamic-data-partitioning">previously-defined</a> data-processing pipeline:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">Alert</span><span class="o">></span> <span class="n">alerts</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="n">transactions</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">process</span><span class="o">(</span><span class="k">new</span> <span class="n">DynamicKeyFunction</span><span class="o">())</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">keyBy</span><span class="o">((</span><span class="n">keyed</span><span class="o">)</span> <span class="o">-></span> <span class="n">keyed</span><span class="o">.</span><span class="na">getKey</span><span class="o">());</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">process</span><span class="o">(</span><span class="k">new</span> <span class="n">DynamicAlertFunction</span><span class="o">())</span>
-</span></span></code></pre></div><p><code>DynamicKeyFunction</code> provides dynamic data partitioning while <code>DynamicAlertFunction</code> is responsible for executing the main logic of processing transactions and sending alert messages according to defined rules.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">Alert</span><span class="o">></span><span class="w"> </span><span class="n">alerts</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">transactions</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">process</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">DynamicKeyFunction</span><span class="p">())</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">keyBy</span><span class="p">((</span><span class="n">keyed</span><span class="p">)</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">keyed</span><span class="p">.</span><span class="na">getKey</span><span class="p">());</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">process</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">DynamicAlertFunction</span><span class="p">())</span><span class="w">
+</span></span></span></code></pre></div><p><code>DynamicKeyFunction</code> provides dynamic data partitioning while <code>DynamicAlertFunction</code> is responsible for executing the main logic of processing transactions and sending alert messages according to defined rules.</p>
<p>Vol.1 of this series simplified the use case and assumed that the applied set of rules is pre-initialized and accessible via the <code>List<Rules></code> within <code>DynamicKeyFunction</code>.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">DynamicKeyFunction</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">extends</span> <span class="n">ProcessFunction</span><span class="o"><</span><span class="n">Transaction</span><span class="o">,</span> <span class="n">Keyed</span><span class="o"><</span><span class="n">Transaction</span><span class="o">,</span> <span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="cm">/* Simplified */</span>
-</span></span><span class="line"><span class="cl"> <span class="n">List</span><span class="o"><</span><span class="n">Rule</span><span class="o">></span> <span class="n">rules</span> <span class="o">=</span> <span class="cm">/* Rules that are initialized somehow.*/</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">...</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>Adding rules to this list is obviously possible directly inside the code of the Flink Job at the stage of its initialization (Create a <code>List</code> object; use it’s <code>add</code> method). A major drawback of doing so is that it will require recompilation of the job with each rule modification. In a real Fraud Detection system, rules are expected to change on a frequent basis, making this approach unacceptable from the point of view of bus [...]
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">DynamicKeyFunction</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">extends</span><span class="w"> </span><span class="n">ProcessFunction</span><span class="o"><</span><span class="n">Transaction</span><span class="p">,</span><span class="w"> </span><span class="n">Keyed</span><span class="o"><</span><span class="n">Transaction</span><span class="p">,</span><span class="w"> </span><span class="n">String</span><span class="p">,</span><span class="w"> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="cm">/* Simplified */</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">List</span><span class="o"><</span><span class="n">Rule</span><span class="o">></span><span class="w"> </span><span class="n">rules</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="cm">/* Rules that are initialized somehow.*/</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">...</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>Adding rules to this list is obviously possible directly inside the code of the Flink Job at the stage of its initialization (Create a <code>List</code> object; use it’s <code>add</code> method). A major drawback of doing so is that it will require recompilation of the job with each rule modification. In a real Fraud Detection system, rules are expected to change on a frequent basis, making this approach unacceptable from the point of view [...]
<p>Next, let’s take a look at a sample rule definition that we introduced in the previous post of the series:</p>
<center>
<img src="/img/blog/patterns-blog-2/rule-dsl.png" width="800px" alt="Figure 1: Rule definition"/>
@@ -591,63 +590,63 @@ There are actually a few more specialized data partitioning schemes in Flink whi
<a class="anchor" href="#broadcast-state-pattern">#</a>
</h2>
<p>In order to make use of the Rules Source, we need to “connect” it to the main data stream:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// Streams setup
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Transaction</span><span class="o">></span> <span class="n">transactions</span> <span class="o">=</span> <span class="o">[...]</span>
-</span></span><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">Rule</span><span class="o">></span> <span class="n">rulesUpdateStream</span> <span class="o">=</span> <span class="o">[...]</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="n">BroadcastStream</span><span class="o"><</span><span class="n">Rule</span><span class="o">></span> <span class="n">rulesStream</span> <span class="o">=</span> <span class="n">rulesUpdateStream</span><span class="o">.</span><span class="na">broadcast</span><span class="o">(</span><span class="n">RULES_STATE_DESCRIPTOR</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="c1">// Processing pipeline setup
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="n">DataStream</span><span class="o"><</span><span class="n">Alert</span><span class="o">></span> <span class="n">alerts</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="n">transactions</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">connect</span><span class="o">(</span><span class="n">rulesStream</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">process</span><span class="o">(</span><span class="k">new</span> <span class="n">DynamicKeyFunction</span><span class="o">())</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">keyBy</span><span class="o">((</span><span class="n">keyed</span><span class="o">)</span> <span class="o">-></span> <span class="n">keyed</span><span class="o">.</span><span class="na">getKey</span><span class="o">())</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">connect</span><span class="o">(</span><span class="n">rulesStream</span><span class="o">)</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">process</span><span class="o">(</span><span class="k">new</span> <span class="n">DynamicAlertFunction</span><span class="o">())</span>
-</span></span></code></pre></div><p>As you can see, the broadcast stream can be created from any regular stream by calling the <code>broadcast</code> method and specifying a state descriptor. Flink assumes that broadcasted data needs to be stored and retrieved while processing events of the main data flow and, therefore, always automatically creates a corresponding <em>broadcast state</em> from this state descriptor. This is different from any other Apache Flink state type in which you n [...]
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">MapStateDescriptor</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Rule</span><span class="o">></span> <span class="n">RULES_STATE_DESCRIPTOR</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">MapStateDescriptor</span><span class="o"><>(</span><span class="s">"rules"</span><span class="o">,</span> <span class="n">Integer</span><span class="o">.</span><span class="na">class</span><span class="o">,</span> <span class="n">Rule</span><span class="o">.</span><span class="na">class</span><span class="o">);</span>
-</span></span></code></pre></div><p>Connecting to <code>rulesStream</code> causes some changes in the signature of the processing functions. The previous article presented it in a slightly simplified way as a <code>ProcessFunction</code>. However, <code>DynamicKeyFunction</code> is actually a <code>BroadcastProcessFunction</code>.</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">abstract</span> <span class="kd">class</span> <span class="nc">BroadcastProcessFunction</span><span class="o"><</span><span class="n">IN1</span><span class="o">,</span> <span class="n">IN2</span><span class="o">,</span> <span class="n">OUT</span><span class="o">></span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kd">abstract</span> <span class="kt">void</span> <span class="nf">processElement</span><span class="o">(</span><span class="n">IN1</span> <span class="n">value</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">ReadOnlyContext</span> <span class="n">ctx</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Collector</span><span class="o"><</span><span class="n">OUT</span><span class="o">></span> <span class="n">out</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kd">abstract</span> <span class="kt">void</span> <span class="nf">processBroadcastElement</span><span class="o">(</span><span class="n">IN2</span> <span class="n">value</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Context</span> <span class="n">ctx</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Collector</span><span class="o"><</span><span class="n">OUT</span><span class="o">></span> <span class="n">out</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>The difference is the addition of the <code>processBroadcastElement</code> method through which messages of the rules stream will arrive. The following new version of <code>DynamicKeyFunction</code> allows modifying the list of data-distribution keys at runtime through this stream:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">DynamicKeyFunction</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">extends</span> <span class="n">BroadcastProcessFunction</span><span class="o"><</span><span class="n">Transaction</span><span class="o">,</span> <span class="n">Rule</span><span class="o">,</span> <span class="n">Keyed</span><span class="o"><</span><span class="n">Transaction</span><span class="o">,</span> <span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>> [...]
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">processBroadcastElement</span><span class="o">(</span><span class="n">Rule</span> <span class="n">rule</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Context</span> <span class="n">ctx</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Collector</span><span class="o"><</span><span class="n">Keyed</span><span class="o"><</span><span class="n">Transaction</span><span class="o">,</span> <span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span> <span class="n">out</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="n">BroadcastState</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Rule</span><span class="o">></span> <span class="n">broadcastState</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="na">getBroadcastState</span><span class="o">(</span><span class="n">RULES_STATE_DESCRIPTOR</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="n">broadcastState</span><span class="o">.</span><span class="na">put</span><span class="o">(</span><span class="n">rule</span><span class="o">.</span><span class="na">getRuleId</span><span class="o">(),</span> <span class="n">rule</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">processElement</span><span class="o">(</span><span class="n">Transaction</span> <span class="n">event</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">ReadOnlyContext</span> <span class="n">ctx</span><span class="o">,</span>
-</span></span><span class="line"><span class="cl"> <span class="n">Collector</span><span class="o"><</span><span class="n">Keyed</span><span class="o"><</span><span class="n">Transaction</span><span class="o">,</span> <span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span> <span class="n">out</span><span class="o">){</span>
-</span></span><span class="line"><span class="cl"> <span class="n">ReadOnlyBroadcastState</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Rule</span><span class="o">></span> <span class="n">rulesState</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="n">ctx</span><span class="o">.</span><span class="na">getBroadcastState</span><span class="o">(</span><span class="n">RULES_STATE_DESCRIPTOR</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="o">(</span><span class="n">Map</span><span class="o">.</span><span class="na">Entry</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Rule</span><span class="o">></span> <span class="n">entry</span> <span class="o">:</span> <span class="n">rulesState</span><span class="o">.</span><span class="na">immutableEntries</span><span class="o">())< [...]
-</span></span><span class="line"><span class="cl"> <span class="kd">final</span> <span class="n">Rule</span> <span class="n">rule</span> <span class="o">=</span> <span class="n">entry</span><span class="o">.</span><span class="na">getValue</span><span class="o">();</span>
-</span></span><span class="line"><span class="cl"> <span class="n">out</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span>
-</span></span><span class="line"><span class="cl"> <span class="k">new</span> <span class="n">Keyed</span><span class="o"><>(</span>
-</span></span><span class="line"><span class="cl"> <span class="n">event</span><span class="o">,</span> <span class="n">KeysExtractor</span><span class="o">.</span><span class="na">getKey</span><span class="o">(</span><span class="n">rule</span><span class="o">.</span><span class="na">getGroupingKeyNames</span><span class="o">(),</span> <span class="n">event</span><span class="o">),</span> <span class="n">rule</span><span class="o">.</span><span class="na">getRuleId</span><spa [...]
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>In the above code, <code>processElement()</code> receives Transactions, and <code>processBroadcastElement()</code> receives Rule updates. When a new rule is created, it is distributed as depicted in Figure 6 and saved in all parallel instances of the operator using <code>processBroadcastState</code>. We use a Rule’s ID as the key to store and reference individual rules. Instead of iterating over a hardcoded <code>List<Rules></code>, we iter [...]
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="c1">// Streams setup</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Transaction</span><span class="o">></span><span class="w"> </span><span class="n">transactions</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">[</span><span class="p">...</span><span class="o">]</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">DataStream</span><span class="o"><</span><span class="n">Rule</span><span class="o">></span><span class="w"> </span><span class="n">rulesUpdateStream</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">[</span><span class="p">...</span><span class="o">]</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">BroadcastStream</span><span class="o"><</span><span class="n">Rule</span><span class="o">></span><span class="w"> </span><span class="n">rulesStream</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">rulesUpdateStream</span><span class="p">.</span><span class="na">broadcast</span><span class="p">(</span><span class="n">RULES_STATE_DESCRIPTOR</sp [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="c1">// Processing pipeline setup</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">DataStream</span><span class="o"><</span><span class="n">Alert</span><span class="o">></span><span class="w"> </span><span class="n">alerts</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">transactions</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">connect</span><span class="p">(</span><span class="n">rulesStream</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">process</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">DynamicKeyFunction</span><span class="p">())</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">keyBy</span><span class="p">((</span><span class="n">keyed</span><span class="p">)</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">keyed</span><span class="p">.</span><span class="na">getKey</span><span class="p">())</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">connect</span><span class="p">(</span><span class="n">rulesStream</span><span class="p">)</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">process</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">DynamicAlertFunction</span><span class="p">())</span><span class="w">
+</span></span></span></code></pre></div><p>As you can see, the broadcast stream can be created from any regular stream by calling the <code>broadcast</code> method and specifying a state descriptor. Flink assumes that broadcasted data needs to be stored and retrieved while processing events of the main data flow and, therefore, always automatically creates a corresponding <em>broadcast state</em> from this state descriptor. This is different from any other Apache Flink state type in whic [...]
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">static</span><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="n">MapStateDescriptor</span><span class="o"><</span><span class="n">Integer</span><span class="p">,</span><span class="w"> </span><span class="n">Rule</span><span class="o">>< [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">MapStateDescriptor</span><span class="o"><></span><span class="p">(</span><span class="s">"rules"</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="p">.</span><span class="na">class</span><span class="p">,</span><span class="w"> </span><span class="n">Rule</span><span class="p [...]
+</span></span></span></code></pre></div><p>Connecting to <code>rulesStream</code> causes some changes in the signature of the processing functions. The previous article presented it in a slightly simplified way as a <code>ProcessFunction</code>. However, <code>DynamicKeyFunction</code> is actually a <code>BroadcastProcessFunction</code>.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">abstract</span><span class="w"> </span><span class="kd">class</span> <span class="nc">BroadcastProcessFunction</span><span class="o"><</span><span class="n">IN1</span><span class="p">,</span><span class="w"> </span><span class="n">IN2</span><span class="p">,</span><span class="w"> [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kd">abstract</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">processElement</span><span class="p">(</span><span class="n">IN1</span><span class="w"> </span><span class="n">value</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">ReadOnlyContext</span><span class="w"> </span><span class="n">ctx</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Collector</span><span class="o"><</span><span class="n">OUT</span><span class="o">></span><span class="w"> </span><span class="n">out</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kd">abstract</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">processBroadcastElement</span><span class="p">(</span><span class="n">IN2</span><span class="w"> </span><span class="n">value</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Context</span><span class="w"> </span><span class="n">ctx</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Collector</span><span class="o"><</span><span class="n">OUT</span><span class="o">></span><span class="w"> </span><span class="n">out</span><span class="p">)</span><span class="w"> </span><span class="kd">throws</span><span class="w"> </span><span class="n">Exception</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>The difference is the addition of the <code>processBroadcastElement</code> method through which messages of the rules stream will arrive. The following new version of <code>DynamicKeyFunction</code> allows modifying the list of data-distribution keys at runtime through this stream:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">DynamicKeyFunction</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">extends</span><span class="w"> </span><span class="n">BroadcastProcessFunction</span><span class="o"><</span><span class="n">Transaction</span><span class="p">,</span><span class="w"> </span><span class="n">Rule</span><span class="p">,</span><span class="w"> </span><span class="n">Keyed</span><span class="o"><</span><span class="n">Transaction</span><span class="p">,</span><span cla [...]
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">processBroadcastElement</span><span class="p">(</span><span class="n">Rule</span><span class="w"> </span><span class="n">rule</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Context</span><span class="w"> </span><span class="n">ctx</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Collector</span><span class="o"><</span><span class="n">Keyed</span><span class="o"><</span><span class="n">Transaction</span><span class="p">,</span><span class="w"> </span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="o">>></span><span class="w"> </span><span class="n">o [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">BroadcastState</span><span class="o"><</span><span class="n">Integer</span><span class="p">,</span><span class="w"> </span><span class="n">Rule</span><span class="o">></span><span class="w"> </span><span class="n">broadcastState</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">ctx</span><span class="p">.</span><span class="na">getBroadcast [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">broadcastState</span><span class="p">.</span><span class="na">put</span><span class="p">(</span><span class="n">rule</span><span class="p">.</span><span class="na">getRuleId</span><span class="p">(),</span><span class="w"> </span><span class="n">rule</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="nf">processElement</span><span class="p">(</span><span class="n">Transaction</span><span class="w"> </span><span class="n">event</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">ReadOnlyContext</span><span class="w"> </span><span class="n">ctx</span><span class="p">,</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">Collector</span><span class="o"><</span><span class="n">Keyed</span><span class="o"><</span><span class="n">Transaction</span><span class="p">,</span><span class="w"> </span><span class="n">String</span><span class="p">,</span><span class="w"> </span><span class="n">Integer</span><span class="o">>></span><span class="w"> </span><span class="n">out</span>< [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">ReadOnlyBroadcastState</span><span class="o"><</span><span class="n">Integer</span><span class="p">,</span><span class="w"> </span><span class="n">Rule</span><span class="o">></span><span class="w"> </span><span class="n">rulesState</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">ctx</span><span class="p">.</span><span class="na">getBroadcastState</span><span class="p">(</span><span class="n">RULES_STATE_DESCRIPTOR</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="n">Map</span><span class="p">.</span><span class="na">Entry</span><span class="o"><</span><span class="n">Integer</span><span class="p">,</span><span class="w"> </span><span class="n">Rule</span><span class="o">></span><span class="w"> </span><span class="n">entry</span><span class="w"> </span><span class="p">:</s [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="n">Rule</span><span class="w"> </span><span class="n">rule</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">entry</span><span class="p">.</span><span class="na">getValue</span><span class="p">();</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">out</span><span class="p">.</span><span class="na">collect</span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Keyed</span><span class="o"><></span><span class="p">(</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">event</span><span class="p">,</span><span class="w"> </span><span class="n">KeysExtractor</span><span class="p">.</span><span class="na">getKey</span><span class="p">(</span><span class="n">rule</span><span class="p">.</span><span class="na">getGroupingKeyNames</span><span class="p">(),</span><span class="w"> </span><span class="n">event</span><span class="p">),</span><span class=" [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>In the above code, <code>processElement()</code> receives Transactions, and <code>processBroadcastElement()</code> receives Rule updates. When a new rule is created, it is distributed as depicted in Figure 6 and saved in all parallel instances of the operator using <code>processBroadcastState</code>. We use a Rule’s ID as the key to store and reference individual rules. Instead of iterating over a hardcoded <code>List<Rules></code>, [...]
<p><code>DynamicAlertFunction</code> follows the same logic with respect to storing the rules in the broadcast <code>MapState</code>. As described in <a href="https://flink.apache.org/news/2020/01/15/demo-fraud-detection.html">Part 1</a>, each message in the <code>processElement</code> input is intended to be processed by one specific rule and comes “pre-marked” with a corresponding ID by <code>DynamicKeyFunction</code>. All we need to do is retrieve the definition of the co [...]
<h1 id="summary">
Summary
diff --git a/content/2020/03/27/flink-as-unified-engine-for-modern-data-warehousing-production-ready-hive-integration/index.html b/content/2020/03/27/flink-as-unified-engine-for-modern-data-warehousing-production-ready-hive-integration/index.html
index cc33bbcd0..3b0d0ecbf 100644
--- a/content/2020/03/27/flink-as-unified-engine-for-modern-data-warehousing-production-ready-hive-integration/index.html
+++ b/content/2020/03/27/flink-as-unified-engine-for-modern-data-warehousing-production-ready-hive-integration/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -16,30 +16,29 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="In this blog post, you will learn our motivation behind the Flink-Hive integration, and how Flink 1.10 can help modernize your data warehouse.
-Introduction # What are some of the latest requirements for your data warehouse and data infrastructure in 2020?
+Introduction # What are some of the latest requirements for your data warehouse and data infrastructure in 2020?
We’ve came up with some for you.
Firstly, today’s business is shifting to a more real-time fashion, and thus demands abilities to process online streaming data with low latency for near-real-time or even real-time analytics.">
<meta name="theme-color" content="#FFFFFF"><meta property="og:title" content="Flink as Unified Engine for Modern Data Warehousing: Production-Ready Hive Integration" />
<meta property="og:description" content="In this blog post, you will learn our motivation behind the Flink-Hive integration, and how Flink 1.10 can help modernize your data warehouse.
-Introduction # What are some of the latest requirements for your data warehouse and data infrastructure in 2020?
+Introduction # What are some of the latest requirements for your data warehouse and data infrastructure in 2020?
We’ve came up with some for you.
Firstly, today’s business is shifting to a more real-time fashion, and thus demands abilities to process online streaming data with low latency for near-real-time or even real-time analytics." />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://flink.apache.org/2020/03/27/flink-as-unified-engine-for-modern-data-warehousing-production-ready-hive-integration/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2020-03-27T02:30:00+00:00" />
<meta property="article:modified_time" content="2020-03-27T02:30:00+00:00" />
-
<title>Flink as Unified Engine for Modern Data Warehousing: Production-Ready Hive Integration | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -62,7 +61,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -269,7 +268,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2020/03/30/flink-community-update-april20/index.html b/content/2020/03/30/flink-community-update-april20/index.html
index 439f39102..f4679b4b1 100644
--- a/content/2020/03/30/flink-community-update-april20/index.html
+++ b/content/2020/03/30/flink-community-update-april20/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ And since now it’s more important than ever to keep up the spirits, we’d
<meta property="og:url" content="https://flink.apache.org/2020/03/30/flink-community-update-april20/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2020-03-30T08:00:00+00:00" />
<meta property="article:modified_time" content="2020-03-30T08:00:00+00:00" />
-
<title>Flink Community Update - April'20 | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2020/04/07/stateful-functions-2.0-an-event-driven-database-on-apache-flink/index.html b/content/2020/04/07/stateful-functions-2.0-an-event-driven-database-on-apache-flink/index.html
index 30f6d6850..95752e1a6 100644
--- a/content/2020/04/07/stateful-functions-2.0-an-event-driven-database-on-apache-flink/index.html
+++ b/content/2020/04/07/stateful-functions-2.0-an-event-driven-database-on-apache-flink/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ Stateful Functions 2.0 makes it possible to combine StateFun’s powerful approa
<meta property="og:url" content="https://flink.apache.org/2020/04/07/stateful-functions-2.0-an-event-driven-database-on-apache-flink/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2020-04-07T08:00:00+00:00" />
<meta property="article:modified_time" content="2020-04-07T08:00:00+00:00" />
-
<title>Stateful Functions 2.0 - An Event-driven Database on Apache Flink | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2020/04/09/pyflink-introducing-python-support-for-udfs-in-flinks-table-api/index.html b/content/2020/04/09/pyflink-introducing-python-support-for-udfs-in-flinks-table-api/index.html
index c0f58b12d..c0e9dbede 100644
--- a/content/2020/04/09/pyflink-introducing-python-support-for-udfs-in-flinks-table-api/index.html
+++ b/content/2020/04/09/pyflink-introducing-python-support-for-udfs-in-flinks-table-api/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -24,18 +24,17 @@ In Flink 1.10, the community further extended the support for Python by adding P
<meta property="og:url" content="https://flink.apache.org/2020/04/09/pyflink-introducing-python-support-for-udfs-in-flinks-table-api/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2020-04-09T12:00:00+00:00" />
<meta property="article:modified_time" content="2020-04-09T12:00:00+00:00" />
-
<title>PyFlink: Introducing Python Support for UDFs in Flink's Table API | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -58,7 +57,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -265,7 +264,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2020/04/09/pyflink-udf-support-flink.html b/content/2020/04/09/pyflink-udf-support-flink.html
index 1ec868d91..52828fdfe 100644
--- a/content/2020/04/09/pyflink-udf-support-flink.html
+++ b/content/2020/04/09/pyflink-udf-support-flink.html
@@ -1 +1,10 @@
-<!DOCTYPE html><html><head><title>https://flink.apache.org/2020/04/09/pyflink-introducing-python-support-for-udfs-in-flinks-table-api/</title><link rel="canonical" href="https://flink.apache.org/2020/04/09/pyflink-introducing-python-support-for-udfs-in-flinks-table-api/"/><meta name="robots" content="noindex"><meta charset="utf-8" /><meta http-equiv="refresh" content="0; url=https://flink.apache.org/2020/04/09/pyflink-introducing-python-support-for-udfs-in-flinks-table-api/" /></head></html>
\ No newline at end of file
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <title>https://flink.apache.org/2020/04/09/pyflink-introducing-python-support-for-udfs-in-flinks-table-api/</title>
+ <link rel="canonical" href="https://flink.apache.org/2020/04/09/pyflink-introducing-python-support-for-udfs-in-flinks-table-api/">
+ <meta name="robots" content="noindex">
+ <meta charset="utf-8">
+ <meta http-equiv="refresh" content="0; url=https://flink.apache.org/2020/04/09/pyflink-introducing-python-support-for-udfs-in-flinks-table-api/">
+ </head>
+</html>
diff --git a/content/2020/04/15/flink-serialization-tuning-vol.-1-choosing-your-serializer-if-you-can/index.html b/content/2020/04/15/flink-serialization-tuning-vol.-1-choosing-your-serializer-if-you-can/index.html
index 1086dec11..e7f8fbad0 100644
--- a/content/2020/04/15/flink-serialization-tuning-vol.-1-choosing-your-serializer-if-you-can/index.html
+++ b/content/2020/04/15/flink-serialization-tuning-vol.-1-choosing-your-serializer-if-you-can/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -22,18 +22,17 @@
<meta property="og:url" content="https://flink.apache.org/2020/04/15/flink-serialization-tuning-vol.-1-choosing-your-serializer-if-you-can/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2020-04-15T08:00:00+00:00" />
<meta property="article:modified_time" content="2020-04-15T08:00:00+00:00" />
-
<title>Flink Serialization Tuning Vol. 1: Choosing your Serializer — if you can | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
@@ -543,21 +542,21 @@ https://github.com/alex-shpak/hugo-book
<ul>
<li>you can have your source or operator implement <code>ResultTypeQueryable<Row></code>:</li>
</ul>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">RowSource</span> <span class="kd">implements</span> <span class="n">SourceFunction</span><span class="o"><</span><span class="n">Row</span><span class="o">>,</span> <span class="n">ResultTypeQueryable</span><span class="o"><</span><span [...]
-</span></span><span class="line"><span class="cl"> <span class="c1">// ...
-</span></span></span><span class="line"><span class="cl"><span class="c1"></span>
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">TypeInformation</span><span class="o"><</span><span class="n">Row</span><span class="o">></span> <span class="nf">getProducedType</span><span class="o">()</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">Types</span><span class="o">.</span><span class="na">ROW</span><span class="o">(</span><span class="n">Types</span><span class="o">.</span><span class="na">INT</span><span class="o">,</span> <span class="n">Types</span><span class="o">.</span><span class="na">STRING</span><span class="o">,</span> <span class="n">Types</span><span class="o">.</span><span class="na">OBJECT_ARRAY</span><span [...]
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><ul>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">static</span><span class="w"> </span><span class="kd">class</span> <span class="nc">RowSource</span><span class="w"> </span><span class="kd">implements</span><span class="w"> </span><span class="n">SourceFunction</span><span class="o"><</span><span class="n">Row</span><span class= [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c1">// ...</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">TypeInformation</span><span class="o"><</span><span class="n">Row</span><span class="o">></span><span class="w"> </span><span class="nf">getProducedType</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Types</span><span class="p">.</span><span class="na">ROW</span><span class="p">(</span><span class="n">Types</span><span class="p">.</span><span class="na">INT</span><span class="p">,</span><span class="w"> </span><span class="n">Types</span><span class="p">.</span><span class="na">STRING</span><span class="p">,</span><span class="w"> </ [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><ul>
<li>you can provide the types when building the job graph by using <code>SingleOutputStreamOperator#returns()</code></li>
</ul>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">Row</span><span class="o">></span> <span class="n">sourceStream</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="n">env</span><span class="o">.</span><span class="na">addSource</span><span class="o">(</span><span class="k">new</span> <span class="n">RowSource</span><span class="o">())</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">returns</span><span class="o">(</span><span class="n">Types</span><span class="o">.</span><span class="na">ROW</span><span class="o">(</span><span class="n">Types</span><span class="o">.</span><span class="na">INT</span><span class="o">,</span> <span class="n">Types</span><span class="o">.</span><span class="na">STRING</span><span class="o">,</span> <span class="n">Types</span><span class=" [...]
-</span></span></code></pre></div><div class="alert alert-warning" markdown="1">
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">Row</span><span class="o">></span><span class="w"> </span><span class="n">sourceStream</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">env</span><span class="p">.</span><span class="na">addSource</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">RowSource</span><span class="p">())</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">returns</span><span class="p">(</span><span class="n">Types</span><span class="p">.</span><span class="na">ROW</span><span class="p">(</span><span class="n">Types</span><span class="p">.</span><span class="na">INT</span><span class="p">,</span><span class="w"> </span><span class="n">Types</span><span class="p">.</span><span class="na">STRING</span><span class=" [...]
+</span></span></span></code></pre></div><div class="alert alert-warning" markdown="1">
<span class="label label-warning" style="display: inline-block"><span class="glyphicon glyphicon-warning-sign" aria-hidden="true"></span> Warning</span>
If you fail to provide the type information for a `Row`, Flink identifies that `Row` is not a valid POJO type according to the rules above and falls back to Kryo serialization (see below) which you will also see in the logs as:
<p><code>13:10:11,148 INFO org.apache.flink.api.java.typeutils.TypeExtractor - Class class org.apache.flink.types.Row cannot be used as a POJO type because not all fields are valid POJO fields, and must be processed as GenericType. Please read the Flink documentation on "Data Types & Serialization" for details of the effect on performance.</code></p>
@@ -583,25 +582,25 @@ If you fail to provide the type information for a `Row`, Flink identifies that `
<ul>
<li>implement <code>ResultTypeQueryable<GenericRecord></code>:</li>
</ul>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">AvroGenericSource</span> <span class="kd">implements</span> <span class="n">SourceFunction</span><span class="o"><</span><span class="n">GenericRecord</span><span class="o">>,</span> <span class="n">ResultTypeQueryable</span><span class="o" [...]
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="kd">final</span> <span class="n">GenericRecordAvroTypeInfo</span> <span class="n">producedType</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="nf">AvroGenericSource</span><span class="o">(</span><span class="n">Schema</span> <span class="n">schema</span><span class="o">)</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">this</span><span class="o">.</span><span class="na">producedType</span> <span class="o">=</span> <span class="k">new</span> <span class="n">GenericRecordAvroTypeInfo</span><span class="o">(</span><span class="n">schema</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl">
-</span></span><span class="line"><span class="cl"> <span class="nd">@Override</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="n">TypeInformation</span><span class="o"><</span><span class="n">GenericRecord</span><span class="o">></span> <span class="nf">getProducedType</span><span class="o">()</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">producedType</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><ul>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">static</span><span class="w"> </span><span class="kd">class</span> <span class="nc">AvroGenericSource</span><span class="w"> </span><span class="kd">implements</span><span class="w"> </span><span class="n">SourceFunction</span><span class="o"><</span><span class="n">GenericRecord< [...]
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="kd">final</span><span class="w"> </span><span class="n">GenericRecordAvroTypeInfo</span><span class="w"> </span><span class="n">producedType</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="nf">AvroGenericSource</span><span class="p">(</span><span class="n">Schema</span><span class="w"> </span><span class="n">schema</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">this</span><span class="p">.</span><span class="na">producedType</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">GenericRecordAvroTypeInfo</span><span class="p">(</span><span class="n">schema</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nd">@Override</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="n">TypeInformation</span><span class="o"><</span><span class="n">GenericRecord</span><span class="o">></span><span class="w"> </span><span class="nf">getProducedType</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">producedType</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><ul>
<li>provide type information when building the job graph by using <code>SingleOutputStreamOperator#returns()</code></li>
</ul>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">GenericRecord</span><span class="o">></span> <span class="n">sourceStream</span> <span class="o">=</span>
-</span></span><span class="line"><span class="cl"> <span class="n">env</span><span class="o">.</span><span class="na">addSource</span><span class="o">(</span><span class="k">new</span> <span class="n">AvroGenericSource</span><span class="o">())</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">returns</span><span class="o">(</span><span class="k">new</span> <span class="n">GenericRecordAvroTypeInfo</span><span class="o">(</span><span class="n">schema</span><span class="o">));</span>
-</span></span></code></pre></div><p>Without this type information, Flink will fall back to Kryo for serialization which would serialize the schema into every record, over and over again. As a result, the serialized form will be bigger and more costly to create.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">DataStream</span><span class="o"><</span><span class="n">GenericRecord</span><span class="o">></span><span class="w"> </span><span class="n">sourceStream</span><span class="w"> </span><span class="o">=</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="n">env</span><span class="p">.</span><span class="na">addSource</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">AvroGenericSource</span><span class="p">())</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">returns</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">GenericRecordAvroTypeInfo</span><span class="p">(</span><span class="n">schema</span><span class="p">));</span><span class="w">
+</span></span></span></code></pre></div><p>Without this type information, Flink will fall back to Kryo for serialization which would serialize the schema into every record, over and over again. As a result, the serialized form will be bigger and more costly to create.</p>
<div class="alert alert-info" markdown="1">
<span class="label label-info" style="display: inline-block"><span class="glyphicon glyphicon-info-sign" aria-hidden="true"></span> Note</span>
Since Avro’s `Schema` class is not serializable, it can not be sent around as is. You can work around this by converting it to a String and parsing it back when needed. If you only do this once on initialization, there is practically no difference to sending it directly.
@@ -611,14 +610,14 @@ If you fail to provide the type information for a `Row`, Flink identifies that `
<a class="anchor" href="#avro-reflect">#</a>
</h3>
<p>The third way of using Avro is to exchange Flink’s PojoSerializer (for POJOs according to the rules above) for Avro’s reflection-based serializer. This can be enabled by calling</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">env</span><span class="o">.</span><span class="na">getConfig</span><span class="o">().</span><span class="na">enableForceAvro</span><span class="o">();</span>
-</span></span></code></pre></div><h2 id="kryo">
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">env</span><span class="p">.</span><span class="na">getConfig</span><span class="p">().</span><span class="na">enableForceAvro</span><span class="p">();</span><span class="w">
+</span></span></span></code></pre></div><h2 id="kryo">
Kryo
<a class="anchor" href="#kryo">#</a>
</h2>
<p>Any class or object which does not fall into the categories above or is covered by a Flink-provided special serializer is de/serialized with a fallback to <a href="https://github.com/EsotericSoftware/kryo">Kryo</a> (currently version 2.24.0) which is a powerful and generic serialization framework in Java. Flink calls such a type a <em>generic type</em> and you may stumble upon <code>GenericTypeInfo</code> when debugging code. If you are using Kryo serialization, make sure to register [...]
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">env</span><span class="o">.</span><span class="na">getConfig</span><span class="o">().</span><span class="na">registerKryoType</span><span class="o">(</span><span class="n">MyCustomType</span><span class="o">.</span><span class="na">class</span><span class="o">);</span>
-</span></span></code></pre></div><p>Registering types adds them to an internal map of classes to tags so that, during serialization, Kryo does not have to add the fully qualified class names as a prefix into the serialized form. Instead, Kryo uses these (integer) tags to identify the underlying classes and reduce serialization overhead.</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">env</span><span class="p">.</span><span class="na">getConfig</span><span class="p">().</span><span class="na">registerKryoType</span><span class="p">(</span><span class="n">MyCustomType</span><span class="p">.</span><span class="na">class</span><span class="p">);</span><span class="w">
+</span></span></span></code></pre></div><p>Registering types adds them to an internal map of classes to tags so that, during serialization, Kryo does not have to add the fully qualified class names as a prefix into the serialized form. Instead, Kryo uses these (integer) tags to identify the underlying classes and reduce serialization overhead.</p>
<div class="alert alert-info" markdown="1">
<span class="label label-info" style="display: inline-block"><span class="glyphicon glyphicon-info-sign" aria-hidden="true"></span> Note</span>
Flink will store Kryo serializer mappings from type registrations in its checkpoints and savepoints and will retain them across job (re)starts.
@@ -628,8 +627,8 @@ Flink will store Kryo serializer mappings from type registrations in its checkpo
<a class="anchor" href="#disabling-kryo">#</a>
</h3>
<p>If desired, you can disable the Kryo fallback, i.e. the ability to serialize generic types, by calling</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">env</span><span class="o">.</span><span class="na">getConfig</span><span class="o">().</span><span class="na">disableGenericTypes</span><span class="o">();</span>
-</span></span></code></pre></div><p>This is mostly useful for finding out where these fallbacks are applied and replacing them with better serializers. If your job has any generic types with this configuration, it will fail with</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">env</span><span class="p">.</span><span class="na">getConfig</span><span class="p">().</span><span class="na">disableGenericTypes</span><span class="p">();</span><span class="w">
+</span></span></span></code></pre></div><p>This is mostly useful for finding out where these fallbacks are applied and replacing them with better serializers. If your job has any generic types with this configuration, it will fail with</p>
<blockquote>
<p>Exception in thread “main” java.lang.UnsupportedOperationException: Generic types have been disabled in the ExecutionConfig and type … is treated as a generic type.</p>
</blockquote>
@@ -639,8 +638,8 @@ Flink will store Kryo serializer mappings from type registrations in its checkpo
<a class="anchor" href="#apache-thrift-via-kryo">#</a>
</h2>
<p>In addition to the variants above, Flink also allows you to <a href="//nightlies.apache.org/flink/flink-docs-release-1.10/dev/custom_serializers.html#register-a-custom-serializer-for-your-flink-program">register other type serialization frameworks</a> with Kryo. After adding the appropriate dependencies from the <a href="//nightlies.apache.org/flink/flink-docs-release-1.10/dev/custom_serializers.html#register-a-custom-serializer-for-your-flink-program">documentation</a> (<code>com.twi [...]
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">env</span><span class="o">.</span><span class="na">getConfig</span><span class="o">().</span><span class="na">addDefaultKryoSerializer</span><span class="o">(</span><span class="n">MyCustomType</span><span class="o">.</span><span class="na">class</span><span class="o">,</span> <span class="n">TBaseSerializer</span><span class="o">.</span [...]
-</span></span></code></pre></div><p>This only works if generic types are not disabled and <code>MyCustomType</code> is a Thrift-generated data type. If the data type is not generated by Thrift, Flink will fail at runtime with an exception like this:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">env</span><span class="p">.</span><span class="na">getConfig</span><span class="p">().</span><span class="na">addDefaultKryoSerializer</span><span class="p">(</span><span class="n">MyCustomType</span><span class="p">.</span><span class="na">class</span><span class="p">,</span><span class="w"> </span><span class="n">TBaseSerializer</span> [...]
+</span></span></span></code></pre></div><p>This only works if generic types are not disabled and <code>MyCustomType</code> is a Thrift-generated data type. If the data type is not generated by Thrift, Flink will fail at runtime with an exception like this:</p>
<blockquote>
<p>java.lang.ClassCastException: class MyCustomType cannot be cast to class org.apache.thrift.TBase (MyCustomType and org.apache.thrift.TBase are in unnamed module of loader ‘app’)</p>
</blockquote>
@@ -653,8 +652,8 @@ Please note that `TBaseSerializer` can be registered as a default Kryo serialize
<a class="anchor" href="#protobuf-via-kryo">#</a>
</h2>
<p>In a way similar to Apache Thrift, <a href="https://developers.google.com/protocol-buffers/">Google Protobuf</a> may be <a href="//nightlies.apache.org/flink/flink-docs-release-1.10/dev/custom_serializers.html#register-a-custom-serializer-for-your-flink-program">registered as a custom serializer</a> after adding the right dependencies (<code>com.twitter:chill-protobuf</code> and <code>com.google.protobuf:protobuf-java</code>):</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">env</span><span class="o">.</span><span class="na">getConfig</span><span class="o">().</span><span class="na">registerTypeWithKryoSerializer</span><span class="o">(</span><span class="n">MyCustomType</span><span class="o">.</span><span class="na">class</span><span class="o">,</span> <span class="n">ProtobufSerializer</span><span class="o [...]
-</span></span></code></pre></div><p>This will work as long as generic types have not been disabled (this would disable Kryo for good). If <code>MyCustomType</code> is not a Protobuf-generated class, your Flink job will fail at runtime with the following exception:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">env</span><span class="p">.</span><span class="na">getConfig</span><span class="p">().</span><span class="na">registerTypeWithKryoSerializer</span><span class="p">(</span><span class="n">MyCustomType</span><span class="p">.</span><span class="na">class</span><span class="p">,</span><span class="w"> </span><span class="n">ProtobufSerializ [...]
+</span></span></span></code></pre></div><p>This will work as long as generic types have not been disabled (this would disable Kryo for good). If <code>MyCustomType</code> is not a Protobuf-generated class, your Flink job will fail at runtime with the following exception:</p>
<blockquote>
<p>java.lang.ClassCastException: class <code>MyCustomType</code> cannot be cast to class com.google.protobuf.Message (<code>MyCustomType</code> and com.google.protobuf.Message are in unnamed module of loader ‘app’)</p>
</blockquote>
@@ -672,26 +671,26 @@ Please note that `ProtobufSerializer` can be registered as a default Kryo serial
<a class="anchor" href="#performance-comparison">#</a>
</h1>
<p>With so many options for serialization, it is actually not easy to make the right choice. We already saw some technical advantages and disadvantages of each of them outlined above. Since serializers are at the core of your Flink jobs and usually also sit on the hot path (per record invocations), let us actually take a deeper look into their performance with the help of the Flink benchmarks project at <a href="https://github.com/dataArtisans/flink-benchmarks">https://github.com/dataArt [...]
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">MyPojo</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">public</span> <span class="kt">int</span> <span class="n">id</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">String</span> <span class="n">name</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">String</span><span class="o">[]</span> <span class="n">operationNames</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">MyOperation</span><span class="o">[]</span> <span class="n">operations</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="kt">int</span> <span class="n">otherId1</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="kt">int</span> <span class="n">otherId2</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="kt">int</span> <span class="n">otherId3</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">private</span> <span class="n">Object</span> <span class="n">someObject</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span><span class="line"><span class="cl"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">MyOperation</span> <span class="o">{</span>
-</span></span><span class="line"><span class="cl"> <span class="kt">int</span> <span class="n">id</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"> <span class="kd">protected</span> <span class="n">String</span> <span class="n">name</span><span class="o">;</span>
-</span></span><span class="line"><span class="cl"><span class="o">}</span>
-</span></span></code></pre></div><p>This is mapped to tuples, rows, Avro specific records, Thrift and Protobuf representations appropriately and sent through a simple Flink job at parallelism 4 where the data type is used during network communication like this:</p>
-<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">env</span><span class="o">.</span><span class="na">setParallelism</span><span class="o">(</span><span class="n">4</span><span class="o">);</span>
-</span></span><span class="line"><span class="cl"><span class="n">env</span><span class="o">.</span><span class="na">addSource</span><span class="o">(</span><span class="k">new</span> <span class="n">PojoSource</span><span class="o">(</span><span class="n">RECORDS_PER_INVOCATION</span><span class="o">,</span> <span class="n">10</span><span class="o">))</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">rebalance</span><span class="o">()</span>
-</span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="na">addSink</span><span class="o">(</span><span class="k">new</span> <span class="n">DiscardingSink</span><span class="o"><>());</span>
-</span></span></code></pre></div><p>After running this through the <a href="http://openjdk.java.net/projects/code-tools/jmh/">jmh</a> micro-benchmarks defined in <a href="https://github.com/dataArtisans/flink-benchmarks/blob/master/src/main/java/org/apache/flink/benchmark/full/SerializationFrameworkAllBenchmarks.java">SerializationFrameworkAllBenchmarks.java</a>, I retrieved the following performance results for Flink 1.10 on my machine (in number of operations per millisecond):
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">MyPojo</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">public</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">id</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">name</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">String</span><span class="o">[]</span><span class="w"> </span><span class="n">operationNames</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">MyOperation</span><span class="o">[]</span><span class="w"> </span><span class="n">operations</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">otherId1</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">otherId2</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">otherId3</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">private</span><span class="w"> </span><span class="n">Object</span><span class="w"> </span><span class="n">someObject</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="kd">public</span><span class="w"> </span><span class="kd">class</span> <span class="nc">MyOperation</span><span class="w"> </span><span class="p">{</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">id</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="kd">protected</span><span class="w"> </span><span class="n">String</span><span class="w"> </span><span class="n">name</span><span class="p">;</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="p">}</span><span class="w">
+</span></span></span></code></pre></div><p>This is mapped to tuples, rows, Avro specific records, Thrift and Protobuf representations appropriately and sent through a simple Flink job at parallelism 4 where the data type is used during network communication like this:</p>
+<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-java" data-lang="java"><span class="line"><span class="cl"><span class="n">env</span><span class="p">.</span><span class="na">setParallelism</span><span class="p">(</span><span class="n">4</span><span class="p">);</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="n">env</span><span class="p">.</span><span class="na">addSource</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">PojoSource</span><span class="p">(</span><span class="n">RECORDS_PER_INVOCATION</span><span class="p">,</span><span class="w"> </span><span class="n">10</span><span class="p">))</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">rebalance</span><span class="p">()</span><span class="w">
+</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="p">.</span><span class="na">addSink</span><span class="p">(</span><span class="k">new</span><span class="w"> </span><span class="n">DiscardingSink</span><span class="o"><></span><span class="p">());</span><span class="w">
+</span></span></span></code></pre></div><p>After running this through the <a href="http://openjdk.java.net/projects/code-tools/jmh/">jmh</a> micro-benchmarks defined in <a href="https://github.com/dataArtisans/flink-benchmarks/blob/master/src/main/java/org/apache/flink/benchmark/full/SerializationFrameworkAllBenchmarks.java">SerializationFrameworkAllBenchmarks.java</a>, I retrieved the following performance results for Flink 1.10 on my machine (in number of operations per millisecond):
<br></p>
<center>
<img src="/img/blog/2020-04-15-flink-serialization-performance-results.svg" width="800px" alt="Communication between the Flink operator and the Python execution environment"/>
diff --git a/content/2020/04/21/memory-management-improvements-with-apache-flink-1.10/index.html b/content/2020/04/21/memory-management-improvements-with-apache-flink-1.10/index.html
index 60e6a102e..42cd870f9 100644
--- a/content/2020/04/21/memory-management-improvements-with-apache-flink-1.10/index.html
+++ b/content/2020/04/21/memory-management-improvements-with-apache-flink-1.10/index.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
-<html lang="en" dir=>
+<html lang="en" dir=ZgotmplZ>
<head>
@@ -22,18 +22,17 @@
<meta property="og:url" content="https://flink.apache.org/2020/04/21/memory-management-improvements-with-apache-flink-1.10/" /><meta property="article:section" content="posts" />
<meta property="article:published_time" content="2020-04-21T12:00:00+00:00" />
<meta property="article:modified_time" content="2020-04-21T12:00:00+00:00" />
-
<title>Memory Management Improvements with Apache Flink 1.10 | Apache Flink</title>
<link rel="manifest" href="/manifest.json">
<link rel="icon" href="/favicon.png" type="image/x-icon">
<link rel="stylesheet" href="/book.min.22eceb4d17baa9cdc0f57345edd6f215a40474022dfee39b63befb5fb3c596b5.css" integrity="sha256-IuzrTRe6qc3A9XNF7dbyFaQEdAIt/uObY777X7PFlrU=">
-<script defer src="/en.search.min.95c5da0c520ed751207d02953b4bf93943ebf987a6be7878a5e741e13409b8b0.js" integrity="sha256-lcXaDFIO11EgfQKVO0v5OUPr+Yemvnh4pedB4TQJuLA="></script>
+<script defer src="/en.search.min.ebb1173488ef8f9f289ff87e668f44e7ef1a2c9f9594d2c615a5e332479a47c4.js" integrity="sha256-67EXNIjvj58on/h+Zo9E5+8aLJ+VlNLGFaXjMkeaR8Q="></script>
<!--
Made with Book Theme
https://github.com/alex-shpak/hugo-book
-->
- <meta name="generator" content="Hugo 0.93.1" />
+ <meta name="generator" content="Hugo 0.124.1">
<script>
@@ -56,7 +55,7 @@ https://github.com/alex-shpak/hugo-book
</head>
-<body dir=>
+<body dir=ZgotmplZ>
@@ -263,7 +262,7 @@ https://github.com/alex-shpak/hugo-book
<li>
- <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.9 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
+ <a class="dropdown-item" href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">Kubernetes Operator 1.10 (latest)<i class="link fa fa-external-link title" aria-hidden="true"></i>
</a>
diff --git a/content/2020/04/24/apache-flink-1.9.3-released/index.html b/content/2020/04/24/apache-flink-1.9.3-released/index.html
index a9b2c8c07..a862c70be 100644
--- a/content/2020/04/24/apache-flink-1.9.3-released/index.html
... 40889 lines suppressed ...