You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by dh...@apache.org on 2017/05/25 21:16:09 UTC

[1/4] beam-site git commit: Closes #248

Repository: beam-site
Updated Branches:
  refs/heads/asf-site 34524776a -> af94ee216


Closes #248


Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/c2afb992
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/c2afb992
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/c2afb992

Branch: refs/heads/asf-site
Commit: c2afb992a7297cc7241b52ad2e9418078387c4ff
Parents: 3452477 0fee1fb
Author: Dan Halperin <dh...@google.com>
Authored: Thu May 25 14:05:13 2017 -0700
Committer: Dan Halperin <dh...@google.com>
Committed: Thu May 25 14:05:13 2017 -0700

----------------------------------------------------------------------
 src/_data/capability-matrix.yml | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
----------------------------------------------------------------------



[3/4] beam-site git commit: fixups

Posted by dh...@apache.org.
fixups


Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/d7582677
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/d7582677
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/d7582677

Branch: refs/heads/asf-site
Commit: d758267705617abb74d033fdf6f8d3212b96cea8
Parents: c2afb99
Author: Dan Halperin <dh...@google.com>
Authored: Thu May 25 14:15:17 2017 -0700
Committer: Dan Halperin <dh...@google.com>
Committed: Thu May 25 14:15:17 2017 -0700

----------------------------------------------------------------------
 src/_data/capability-matrix.yml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/beam-site/blob/d7582677/src/_data/capability-matrix.yml
----------------------------------------------------------------------
diff --git a/src/_data/capability-matrix.yml b/src/_data/capability-matrix.yml
index cd7c9b6..516033c 100644
--- a/src/_data/capability-matrix.yml
+++ b/src/_data/capability-matrix.yml
@@ -210,8 +210,8 @@ categories:
             l3: Allow transforms to gather simple metrics across bundles in a <tt>PTransform</tt>. Provide a mechanism to obtain both committed and attempted metrics. Semantically similar to using an additional output, but support partial results as the transform executes, and support both committed and attempted values. Will likely want to augment <tt>Metrics</tt> to be more useful for processing unbounded data by making them windowed.
           - class: dataflow
             l1: 'Partially'
-            l2: Batch supports committed Counters and Distributions. No streamming support.
-            l3: Gauge metrics are not supported.
+            l2: In batch mode, Dataflow supports committed and attempted Counters and Distributions.
+            l3: Gauge metrics are not supported in batch mode. Metrics are not yet supported at all in streaming mode, but this support is coming soon ([BEAM-2059](https://issues.apache.org/jira/browse/BEAM-2059)).
           - class: flink
             l1: 'Partially'
             l2: All metrics types are supported.


[4/4] beam-site git commit: Regenerate website after merge

Posted by dh...@apache.org.
Regenerate website after merge


Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/af94ee21
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/af94ee21
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/af94ee21

Branch: refs/heads/asf-site
Commit: af94ee21689d1e50bda93e5a46c1f5528866b9b7
Parents: d758267
Author: Dan Halperin <dh...@google.com>
Authored: Thu May 25 14:16:04 2017 -0700
Committer: Dan Halperin <dh...@google.com>
Committed: Thu May 25 14:16:04 2017 -0700

----------------------------------------------------------------------
 .../documentation/runners/capability-matrix/index.html  | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/beam-site/blob/af94ee21/content/documentation/runners/capability-matrix/index.html
----------------------------------------------------------------------
diff --git a/content/documentation/runners/capability-matrix/index.html b/content/documentation/runners/capability-matrix/index.html
index 1a31a2f..6467c66 100644
--- a/content/documentation/runners/capability-matrix/index.html
+++ b/content/documentation/runners/capability-matrix/index.html
@@ -438,7 +438,7 @@
   </tr>
   
   <tr class="cap-summary">
-    <th class="cap-summary color-capability format-capability" style="color:#ec3">Aggregators</th>
+    <th class="cap-summary color-capability format-capability" style="color:#ec3">Metrics</th>
     
     
 
@@ -1493,26 +1493,26 @@
   </tr>
   
   <tr class="cap">
-    <th class="cap color-capability format-capability" style="color:#ec3">Aggregators</th>
+    <th class="cap color-capability format-capability" style="color:#ec3">Metrics</th>
     
     
 
-    <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><center><b>Partially: user-provided metrics</b></center><br />Allow transforms to aggregate simple metrics across bundles in a <tt>DoFn</tt>. Semantically equivalent to using an additional output, but support partial results as the transform executes. Will likely want to augment <tt>Aggregators</tt> to be more useful for processing unbounded data by making them windowed.
+    <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><center><b>Partially: user-provided metrics</b></center><br />Allow transforms to gather simple metrics across bundles in a <tt>PTransform</tt>. Provide a mechanism to obtain both committed and attempted metrics. Semantically similar to using an additional output, but support partial results as the transform executes, and support both committed and attempted values. Will likely want to augment <tt>Metrics</tt> to be more useful for processing unbounded data by making them windowed.
 </td>
     
     
 
-    <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><center><b>Partially: may miscount in streaming mode</b></center><br />Current model is fully supported in batch mode. In streaming mode, <tt>Aggregators</tt> may under or overcount when bundles are retried.
+    <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><center><b>Partially: In batch mode, Dataflow supports committed and attempted Counters and Distributions.</b></center><br />Gauge metrics are not supported in batch mode. Metrics are not yet supported at all in streaming mode, but this support is coming soon ([BEAM-2059](https://issues.apache.org/jira/browse/BEAM-2059)).
 </td>
     
     
 
-    <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><center><b>Partially: may undercount in streaming</b></center><br />Current model is fully supported in batch. In streaming mode, <tt>Aggregators</tt> may undercount.
+    <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><center><b>Partially: All metrics types are supported.</b></center><br />Only attempted values are supported. No committed values for metrics.
 </td>
     
     
 
-    <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><center><b>Partially: may overcount when tasks are retried in transformations.</b></center><br />supported via <tt>AccumulatorParam</tt> mechanism. If a task retries, and the accumulator is not within a Spark "Action", an overcount is possible.
+    <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><center><b>Partially: All metric types are supported.</b></center><br />Only attempted values are supported. No committed values for metrics.
 </td>
     
     


[2/4] beam-site git commit: Changing Aggregators to Metrics in capability matrix

Posted by dh...@apache.org.
Changing Aggregators to Metrics in capability matrix


Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/0fee1fb1
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/0fee1fb1
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/0fee1fb1

Branch: refs/heads/asf-site
Commit: 0fee1fb1ea966cf69d79481d200e16ae1cf05cab
Parents: 3452477
Author: Pablo <pa...@google.com>
Authored: Fri May 19 10:12:52 2017 -0700
Committer: Dan Halperin <dh...@google.com>
Committed: Thu May 25 14:05:13 2017 -0700

----------------------------------------------------------------------
 src/_data/capability-matrix.yml | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/beam-site/blob/0fee1fb1/src/_data/capability-matrix.yml
----------------------------------------------------------------------
diff --git a/src/_data/capability-matrix.yml b/src/_data/capability-matrix.yml
index 565ffba..cd7c9b6 100644
--- a/src/_data/capability-matrix.yml
+++ b/src/_data/capability-matrix.yml
@@ -202,24 +202,24 @@ categories:
             l1: 'Yes'
             l2: fully supported
             l3: ''
-      - name: Aggregators
+      - name: Metrics
         values:
           - class: model
             l1: 'Partially'
             l2: user-provided metrics
-            l3: Allow transforms to aggregate simple metrics across bundles in a <tt>DoFn</tt>. Semantically equivalent to using an additional output, but support partial results as the transform executes. Will likely want to augment <tt>Aggregators</tt> to be more useful for processing unbounded data by making them windowed.
+            l3: Allow transforms to gather simple metrics across bundles in a <tt>PTransform</tt>. Provide a mechanism to obtain both committed and attempted metrics. Semantically similar to using an additional output, but support partial results as the transform executes, and support both committed and attempted values. Will likely want to augment <tt>Metrics</tt> to be more useful for processing unbounded data by making them windowed.
           - class: dataflow
             l1: 'Partially'
-            l2: may miscount in streaming mode
-            l3: Current model is fully supported in batch mode. In streaming mode, <tt>Aggregators</tt> may under or overcount when bundles are retried.
+            l2: Batch supports committed Counters and Distributions. No streamming support.
+            l3: Gauge metrics are not supported.
           - class: flink
             l1: 'Partially'
-            l2: may undercount in streaming
-            l3: Current model is fully supported in batch. In streaming mode, <tt>Aggregators</tt> may undercount.
+            l2: All metrics types are supported.
+            l3: Only attempted values are supported. No committed values for metrics.
           - class: spark
             l1: 'Partially'
-            l2: may overcount when tasks are retried in transformations.
-            l3: 'supported via <tt>AccumulatorParam</tt> mechanism. If a task retries, and the accumulator is not within a Spark "Action", an overcount is possible.'
+            l2: All metric types are supported.
+            l3: Only attempted values are supported. No committed values for metrics.
           - class: apex
             l1: 'No'
             l2: Not implemented in runner.
@@ -758,4 +758,4 @@ categories:
           - class: gearpump
             l1: 'No'
             l2: pending model support
-            l3: ''
\ No newline at end of file
+            l3: ''