You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@edgent.apache.org by dl...@apache.org on 2016/05/09 19:09:29 UTC
incubator-quarks-website git commit: [QUARKS-173] add load balanced
parallel recipe
Repository: incubator-quarks-website
Updated Branches:
refs/heads/master 93f231df8 -> d6bf44821
[QUARKS-173] add load balanced parallel recipe
Project: http://git-wip-us.apache.org/repos/asf/incubator-quarks-website/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-quarks-website/commit/d6bf4482
Tree: http://git-wip-us.apache.org/repos/asf/incubator-quarks-website/tree/d6bf4482
Diff: http://git-wip-us.apache.org/repos/asf/incubator-quarks-website/diff/d6bf4482
Branch: refs/heads/master
Commit: d6bf448218acb1170c389acf694a138eb049f141
Parents: 93f231d
Author: Dale LaBossiere <dl...@us.ibm.com>
Authored: Mon May 9 15:07:59 2016 -0400
Committer: Dale LaBossiere <dl...@us.ibm.com>
Committed: Mon May 9 15:07:59 2016 -0400
----------------------------------------------------------------------
site/recipes/recipe_parallel_analytics.md | 12 ++++++++++++
1 file changed, 12 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-quarks-website/blob/d6bf4482/site/recipes/recipe_parallel_analytics.md
----------------------------------------------------------------------
diff --git a/site/recipes/recipe_parallel_analytics.md b/site/recipes/recipe_parallel_analytics.md
index 965f455..4a3b13f 100644
--- a/site/recipes/recipe_parallel_analytics.md
+++ b/site/recipes/recipe_parallel_analytics.md
@@ -52,6 +52,8 @@ int width = 5; // number of parallel channels
ToIntFunction<Double> splitter = PlumbingStreams.roundRobinSplitter(width);
```
+Another possibility is to use a "load balanced splitter" configuration. That is covered below.
+
## Define the pipeline to run in parallel
Define a `BiFunction<TStream<T>, Integer, TStream<R>>` that builds the pipeline. That is, define a function that receives a `TStream<T>` and an integer `channel` and creates a pipeline for that channel that returns a `TStream<R>`.
@@ -85,6 +87,16 @@ Given a width, splitter and pipeline function it just takes a single call:
TStream<String> results = PlumbingStreams.parallel(readings, width, splitter, pipeline());
```
+## Load balanced parallel flow
+
+A load balanced parallel flow allocates an incoming tuple to the first available parallel channel. When tuple processing times are variable, using a load balanced parallel flow can result in greater overall throughput.
+
+To create a load balanced parallel flow simply use the `parallelBalanced()` method instead of `parallel()`. Everything is the same except you don't supply a splitter:
+
+```java
+TStream<String> results = PlumbingStreams.parallelBalanced(readings, width, pipeline());
+```
+
## The final application
When the application is run it prints out 5 (width) tuples every second. Without the parallel channels, it would only print one tuple each second.