You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by ch...@apache.org on 2018/08/20 15:57:48 UTC
[flink] branch master updated (7ae291c -> 1c2b01b)
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git.
from 7ae291c [FLINK-10082][metrics][slf4j] Provide initial size to StringBuilder
new aaa3e21 [hotfix][streaming][docs] Remove ambiguous remark about KeyedBroadcastProcessFunction#onTimer()
new 1c2b01b [FLINK-10072][streaming][docs] Fix syntax/formatting issues in broadcast docs
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/dev/stream/state/broadcast_state.md | 52 +++++++++++++++-----------------
1 file changed, 25 insertions(+), 27 deletions(-)
[flink] 02/02: [FLINK-10072][streaming][docs] Fix syntax/formatting
issues in broadcast docs
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 1c2b01be9458641a8b873ad37fe781569dd572b5
Author: Rick Hofstede <ri...@users.noreply.github.com>
AuthorDate: Mon Aug 6 14:33:52 2018 +0200
[FLINK-10072][streaming][docs] Fix syntax/formatting issues in broadcast docs
---
docs/dev/stream/state/broadcast_state.md | 50 ++++++++++++++++----------------
1 file changed, 25 insertions(+), 25 deletions(-)
diff --git a/docs/dev/stream/state/broadcast_state.md b/docs/dev/stream/state/broadcast_state.md
index b649221..f336a85 100644
--- a/docs/dev/stream/state/broadcast_state.md
+++ b/docs/dev/stream/state/broadcast_state.md
@@ -76,8 +76,8 @@ BroadcastStream<Rule> ruleBroadcastStream = ruleStream
{% endhighlight %}
Finally, in order to evaluate the `Rules` against the incoming elements from the `Item` stream, we need to:
- 1) connect the two streams and
- 2) specify our match detecting logic.
+ 1. connect the two streams, and
+ 2. specify our match detecting logic.
Connecting a stream (keyed or non-keyed) with a `BroadcastStream` can be done by calling `connect()` on the
non-broadcasted stream, with the `BroadcastStream` as an argument. This will return a `BroadcastConnectedStream`, on
@@ -161,7 +161,7 @@ across all tasks. Ignoring this rule would break the consistency guarantees of t
often difficult to debug results.
<div class="alert alert-info">
- <strong>Attention:</strong> The logic implemented in `processBroadcast()` must have the same determinstic behavior
+ <strong>Attention:</strong> The logic implemented in `processBroadcast()` must have the same deterministic behavior
across all parallel instances!
</div>
@@ -190,34 +190,34 @@ new KeyedBroadcastProcessFunction<Color, Item, Rule, String>() {
// store partial matches, i.e. first elements of the pair waiting for their second element
// we keep a list as we may have many first elements waiting
private final MapStateDescriptor<String, List<Item>> mapStateDesc =
- new MapStateDescriptor<>(
- "items",
- BasicTypeInfo.STRING_TYPE_INFO,
- new ListTypeInfo<>(Item.class));
+ new MapStateDescriptor<>(
+ "items",
+ BasicTypeInfo.STRING_TYPE_INFO,
+ new ListTypeInfo<>(Item.class));
// identical to our ruleStateDescriptor above
private final MapStateDescriptor<String, Rule> ruleStateDescriptor =
new MapStateDescriptor<>(
- "RulesBroadcastState",
- BasicTypeInfo.STRING_TYPE_INFO,
- TypeInformation.of(new TypeHint<Rule>() {}));
-
- @Override
- public void processBroadcastElement(Rule value,
- Context ctx,
- Collector<String> out) throws Exception {
- ctx.getBroadcastState(ruleStateDescriptor).put(value.name, value);
- }
-
- @Override
- public void processElement(Item value,
- ReadOnlyContext ctx,
- Collector<String> out) throws Exception {
+ "RulesBroadcastState",
+ BasicTypeInfo.STRING_TYPE_INFO,
+ TypeInformation.of(new TypeHint<Rule>() {}));
+
+ @Override
+ public void processBroadcastElement(Rule value,
+ Context ctx,
+ Collector<String> out) throws Exception {
+ ctx.getBroadcastState(ruleStateDescriptor).put(value.name, value);
+ }
+
+ @Override
+ public void processElement(Item value,
+ ReadOnlyContext ctx,
+ Collector<String> out) throws Exception {
final MapState<String, List<Item>> state = getRuntimeContext().getMapState(mapStateDesc);
final Shape shape = value.getShape();
- for (Map.Entry<String, Rule> entry:
+ for (Map.Entry<String, Rule> entry :
ctx.getBroadcastState(ruleStateDescriptor).immutableEntries()) {
final String ruleName = entry.getKey();
final Rule rule = entry.getValue();
@@ -234,7 +234,7 @@ new KeyedBroadcastProcessFunction<Color, Item, Rule, String>() {
stored.clear();
}
- // there is no else{} to cover if rule.first == rule.second
+ // there is no else{} to cover if rule.first == rule.second
if (shape.equals(rule.first)) {
stored.add(value);
}
@@ -245,7 +245,7 @@ new KeyedBroadcastProcessFunction<Color, Item, Rule, String>() {
state.put(ruleName, stored);
}
}
- }
+ }
}
{% endhighlight %}
[flink] 01/02: [hotfix][streaming][docs] Remove ambiguous remark
about KeyedBroadcastProcessFunction#onTimer()
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
commit aaa3e215b782b57a1f67f792e853ff25f7864131
Author: Rick Hofstede <ri...@users.noreply.github.com>
AuthorDate: Thu Aug 16 09:32:15 2018 +0200
[hotfix][streaming][docs] Remove ambiguous remark about KeyedBroadcastProcessFunction#onTimer()
It is hard to tell whether "This is aligned with the onTimer()
method of the KeyedProcessFunction" applies to "the ReadOnlyContext
in the processElement() method" or either/both of the sub-items.
Someone not deeply familiar with process functions would simply
assume that `KeyedBroadcastProcessFunction#onTimer` works exactly
like `KeyedProcessFunction#onTimer` anyway, so the removed statement
would be superfluous anyway.
---
docs/dev/stream/state/broadcast_state.md | 2 --
1 file changed, 2 deletions(-)
diff --git a/docs/dev/stream/state/broadcast_state.md b/docs/dev/stream/state/broadcast_state.md
index 593b6a2..b649221 100644
--- a/docs/dev/stream/state/broadcast_state.md
+++ b/docs/dev/stream/state/broadcast_state.md
@@ -172,8 +172,6 @@ exposes some functionality which is not available to the `BroadcastProcessFuncti
`OnTimerContext` which exposes the same functionality as the `ReadOnlyContext` plus
- the ability to ask if the timer that fired was an event or processing time one and
- to query the key associated with the timer.
-
- This is aligned with the `onTimer()` method of the `KeyedProcessFunction`.
2. the `Context` in the `processBroadcastElement()` method contains the method
`applyToKeyedState(StateDescriptor<S, VS> stateDescriptor, KeyedStateFunction<KS, S> function)`. This allows to
register a `KeyedStateFunction` to be **applied to all states of all keys** associated with the provided `stateDescriptor`.