(beam) branch users/damccorm/compactDocs created (now 507336b76fe)

[da...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a change to branch users/damccorm/compactDocs
in repository https://gitbox.apache.org/repos/asf/beam.git


      at 507336b76fe Update programming-guide.md

This branch includes the following new commits:

     new 507336b76fe Update programming-guide.md

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.

(beam) 01/01: Update programming-guide.md

[da...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a commit to branch users/damccorm/compactDocs
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 507336b76fe9e061143486b7a32490626df0afd2
Author: Danny McCormick <da...@google.com>
AuthorDate: Fri Nov 17 07:10:58 2023 -0500

    Update programming-guide.md
---
 .../content/en/documentation/programming-guide.md  | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/website/www/site/content/en/documentation/programming-guide.md b/website/www/site/content/en/documentation/programming-guide.md
index 564b01a7146..1f1dbbdafc3 100644
--- a/website/www/site/content/en/documentation/programming-guide.md
+++ b/website/www/site/content/en/documentation/programming-guide.md
@@ -1598,10 +1598,12 @@ You should use a `CombineFn` if the combine function requires a more sophisticat
 accumulator, must perform additional pre- or post-processing, might change the
 output type, or takes the key into account.
 
-A general combining operation consists of four operations. When you create a
+A general combining operation consists of five operations. When you create a
 <span class="language-java language-py">subclass of</span>
-`CombineFn`, you must provide four operations by overriding the
-corresponding methods:
+`CombineFn`, you must provide five operations by overriding the
+corresponding methods. Only `MergeAccumulators` is a required method. The
+others will have a default interpretation based on the accumulator type. The
+lifecycle methods are:
 
 1. **Create Accumulator** creates a new "local" accumulator. In the example
    case, taking a mean average, a local accumulator tracks the running sum of
@@ -1623,6 +1625,14 @@ corresponding methods:
    mean average, this means dividing the combined sum of all the values by the
    number of values summed. It is called once on the final, merged accumulator.
 
+5. **Compact** returns a more compact represenation of the accumulator. This is
+    called before an accumulator is sent across the wire, and can be useful in
+    cases where values are buffered or otherwise lazily kept unprocessed when
+    added to the accumulator.  Compact should return an equivalent, though
+    possibly modified, accumulator. In most cases, Compact is not necessary. For
+    a real world example of using Compact, see
+    https://github.com/apache/beam/blob/5e9ea74c85e2ce6e6fb3418a0abc984fbebaefb1/sdks/python/apache_beam/transforms/combiners.py#L523
+
 The following example code shows how to define a `CombineFn` that computes a
 mean average:
 
@@ -1657,6 +1667,9 @@ public class AverageFn extends CombineFn<Integer, AverageFn.Accum, Double> {
   public Double extractOutput(Accum accum) {
     return ((double) accum.sum) / accum.count;
   }
+
+  @Override
+  public Accum compact(Accum accum) { return accum; }
 }
 {{< /highlight >}}
 
@@ -1675,9 +1688,6 @@ pc = ...
 
 <span class="language-go">
 
-> **Note**: Only `MergeAccumulators` is a required method. The others will have a default interpretation
-> based on the accumulator type.
-
 </span>
 
 ##### 4.2.4.3. Combining a PCollection into a single value {#combining-pcollection}