You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by tr...@apache.org on 2020/03/23 09:58:27 UTC
[flink] branch master updated: [hotfix][doc] Document union list
metadata gotcha
This is an automated email from the ASF dual-hosted git repository.
trohrmann pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push:
new a2b669f [hotfix][doc] Document union list metadata gotcha
a2b669f is described below
commit a2b669faabf372da7d5c8c0f100dac808f9b0821
Author: Jacob Sevart <js...@uber.com>
AuthorDate: Sat Mar 21 13:03:49 2020 -0700
[hotfix][doc] Document union list metadata gotcha
This closes #11475.
---
docs/dev/stream/state/state.md | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/docs/dev/stream/state/state.md b/docs/dev/stream/state/state.md
index 028ad8f..e9869e7 100644
--- a/docs/dev/stream/state/state.md
+++ b/docs/dev/stream/state/state.md
@@ -567,7 +567,9 @@ the following redistribution schemes are defined:
while `element2` will go to operator instance 1.
- **Union redistribution:** Each operator returns a List of state elements. The whole state is logically a concatenation of
- all lists. On restore/redistribution, each operator gets the complete list of state elements.
+ all lists. On restore/redistribution, each operator gets the complete list of state elements. Do not use this feature if
+ your list may have high cardinality. Checkpoint metadata will store an offset to each list entry, which could lead to RPC
+ framesize or out-of-memory errors.
Below is an example of a stateful `SinkFunction` that uses `CheckpointedFunction`
to buffer elements before sending them to the outside world. It demonstrates