You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by ch...@apache.org on 2018/08/20 15:57:48 UTC

[flink] branch master updated (7ae291c -> 1c2b01b)

This is an automated email from the ASF dual-hosted git repository.

chesnay pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git.


    from 7ae291c  [FLINK-10082][metrics][slf4j] Provide initial size to StringBuilder
     new aaa3e21  [hotfix][streaming][docs] Remove ambiguous remark about KeyedBroadcastProcessFunction#onTimer()
     new 1c2b01b  [FLINK-10072][streaming][docs] Fix syntax/formatting issues in broadcast docs

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 docs/dev/stream/state/broadcast_state.md | 52 +++++++++++++++-----------------
 1 file changed, 25 insertions(+), 27 deletions(-)


[flink] 02/02: [FLINK-10072][streaming][docs] Fix syntax/formatting issues in broadcast docs

Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

chesnay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git

commit 1c2b01be9458641a8b873ad37fe781569dd572b5
Author: Rick Hofstede <ri...@users.noreply.github.com>
AuthorDate: Mon Aug 6 14:33:52 2018 +0200

    [FLINK-10072][streaming][docs] Fix syntax/formatting issues in broadcast docs
---
 docs/dev/stream/state/broadcast_state.md | 50 ++++++++++++++++----------------
 1 file changed, 25 insertions(+), 25 deletions(-)

diff --git a/docs/dev/stream/state/broadcast_state.md b/docs/dev/stream/state/broadcast_state.md
index b649221..f336a85 100644
--- a/docs/dev/stream/state/broadcast_state.md
+++ b/docs/dev/stream/state/broadcast_state.md
@@ -76,8 +76,8 @@ BroadcastStream<Rule> ruleBroadcastStream = ruleStream
 {% endhighlight %}
 
 Finally, in order to evaluate the `Rules` against the incoming elements from the `Item` stream, we need to:
-    1) connect the two streams and 
-    2) specify our match detecting logic. 
+ 1. connect the two streams, and
+ 2. specify our match detecting logic.
 
 Connecting a stream (keyed or non-keyed) with a `BroadcastStream` can be done by calling `connect()` on the 
 non-broadcasted stream, with the `BroadcastStream` as an argument. This will return a `BroadcastConnectedStream`, on 
@@ -161,7 +161,7 @@ across all tasks. Ignoring this rule would break the consistency guarantees of t
 often difficult to debug results.
 
 <div class="alert alert-info">
-  <strong>Attention:</strong> The logic implemented in `processBroadcast()` must have the same determinstic behavior 
+  <strong>Attention:</strong> The logic implemented in `processBroadcast()` must have the same deterministic behavior
   across all parallel instances!
 </div>
 
@@ -190,34 +190,34 @@ new KeyedBroadcastProcessFunction<Color, Item, Rule, String>() {
     // store partial matches, i.e. first elements of the pair waiting for their second element
     // we keep a list as we may have many first elements waiting
     private final MapStateDescriptor<String, List<Item>> mapStateDesc =
-	    new MapStateDescriptor<>(
-	        "items",
-	        BasicTypeInfo.STRING_TYPE_INFO, 
-	        new ListTypeInfo<>(Item.class));
+        new MapStateDescriptor<>(
+            "items",
+            BasicTypeInfo.STRING_TYPE_INFO,
+            new ListTypeInfo<>(Item.class));
 
     // identical to our ruleStateDescriptor above
     private final MapStateDescriptor<String, Rule> ruleStateDescriptor = 
         new MapStateDescriptor<>(
-    	    "RulesBroadcastState",
-    		BasicTypeInfo.STRING_TYPE_INFO,
-    		TypeInformation.of(new TypeHint<Rule>() {}));
-
-	@Override
-	public void processBroadcastElement(Rule value, 
-	                                    Context ctx, 
-	                                    Collector<String> out) throws Exception {
-	    ctx.getBroadcastState(ruleStateDescriptor).put(value.name, value);
-	}
-
-	@Override
-	public void processElement(Item value, 
-	                           ReadOnlyContext ctx, 
-	                           Collector<String> out) throws Exception {
+            "RulesBroadcastState",
+            BasicTypeInfo.STRING_TYPE_INFO,
+            TypeInformation.of(new TypeHint<Rule>() {}));
+
+    @Override
+    public void processBroadcastElement(Rule value,
+                                        Context ctx,
+                                        Collector<String> out) throws Exception {
+        ctx.getBroadcastState(ruleStateDescriptor).put(value.name, value);
+    }
+
+    @Override
+    public void processElement(Item value,
+                               ReadOnlyContext ctx,
+                               Collector<String> out) throws Exception {
 
         final MapState<String, List<Item>> state = getRuntimeContext().getMapState(mapStateDesc);
         final Shape shape = value.getShape();
     
-        for (Map.Entry<String, Rule> entry: 
+        for (Map.Entry<String, Rule> entry :
                 ctx.getBroadcastState(ruleStateDescriptor).immutableEntries()) {
             final String ruleName = entry.getKey();
             final Rule rule = entry.getValue();
@@ -234,7 +234,7 @@ new KeyedBroadcastProcessFunction<Color, Item, Rule, String>() {
                 stored.clear();
             }
     
-            // there is  no else{} to cover if rule.first == rule.second
+            // there is no else{} to cover if rule.first == rule.second
             if (shape.equals(rule.first)) {
                 stored.add(value);
             }
@@ -245,7 +245,7 @@ new KeyedBroadcastProcessFunction<Color, Item, Rule, String>() {
                 state.put(ruleName, stored);
             }
         }
-	}
+    }
 }
 {% endhighlight %}
 


[flink] 01/02: [hotfix][streaming][docs] Remove ambiguous remark about KeyedBroadcastProcessFunction#onTimer()

Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

chesnay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git

commit aaa3e215b782b57a1f67f792e853ff25f7864131
Author: Rick Hofstede <ri...@users.noreply.github.com>
AuthorDate: Thu Aug 16 09:32:15 2018 +0200

    [hotfix][streaming][docs] Remove ambiguous remark about KeyedBroadcastProcessFunction#onTimer()
    
    It is hard to tell whether "This is aligned with the onTimer()
    method of the KeyedProcessFunction" applies to "the ReadOnlyContext
    in the processElement() method" or either/both of the sub-items.
    
    Someone not deeply familiar with process functions would simply
    assume that `KeyedBroadcastProcessFunction#onTimer` works exactly
    like `KeyedProcessFunction#onTimer` anyway, so the removed statement
    would be superfluous anyway.
---
 docs/dev/stream/state/broadcast_state.md | 2 --
 1 file changed, 2 deletions(-)

diff --git a/docs/dev/stream/state/broadcast_state.md b/docs/dev/stream/state/broadcast_state.md
index 593b6a2..b649221 100644
--- a/docs/dev/stream/state/broadcast_state.md
+++ b/docs/dev/stream/state/broadcast_state.md
@@ -172,8 +172,6 @@ exposes some functionality which is not available to the `BroadcastProcessFuncti
   `OnTimerContext` which exposes the same functionality as the `ReadOnlyContext` plus 
    - the ability to ask if the timer that fired was an event or processing time one and 
    - to query the key associated with the timer.
-
-  This is aligned with the `onTimer()` method of the `KeyedProcessFunction`. 
  2. the `Context` in the `processBroadcastElement()` method contains the method 
  `applyToKeyedState(StateDescriptor<S, VS> stateDescriptor, KeyedStateFunction<KS, S> function)`. This allows to 
   register a `KeyedStateFunction` to be **applied to all states of all keys** associated with the provided `stateDescriptor`.