You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/06/06 13:20:33 UTC

[GitHub] [beam] InigoSJ opened a new pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

InigoSJ opened a new pull request #11943:
URL: https://github.com/apache/beam/pull/11943


   When need Windows and the Mean combiner in a Globally way, we cannot specify `without_defaults()`, forcing us to use the workaround of `CombineGlobally(MeanCombineFn()).without_defaults()`.
   
   This PR fix this, adding the `without_defaults()` to Mean in Python
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   
    - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
    - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
    - [ ] Update `CHANGES.md` with noteworthy changes.
    - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
   
   Post-Commit Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
   Java | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
   Python | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
   XLang | --- | --- | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
   
   Pre-Commit Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   
   --- |Java | Python | Go | Website
   --- | --- | --- | --- | ---
   Non-portable | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/) 
   Portable | --- | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
   
   See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ commented on a change in pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ commented on a change in pull request #11943:
URL: https://github.com/apache/beam/pull/11943#discussion_r438366946



##########
File path: sdks/python/apache_beam/transforms/combiners_test.py
##########
@@ -105,6 +109,16 @@ def test_builtin_combines(self):
       assert_that(result_mean, equal_to([mean]), label='assert:mean')
       assert_that(result_count, equal_to([size]), label='assert:size')
 
+      # Now for global combines without default
+      timestamped = pcoll | Map(lambda x: TimestampedValue(timestamp))

Review comment:
       Thanks for spotting this, I think I missed it when moving from my test env to the PR env!




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ commented on a change in pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ commented on a change in pull request #11943:
URL: https://github.com/apache/beam/pull/11943#discussion_r438368619



##########
File path: sdks/python/apache_beam/transforms/combiners.py
##########
@@ -66,8 +66,21 @@ class Mean(object):
   """Combiners for computing arithmetic means of elements."""
   class Globally(ptransform.PTransform):
     """combiners.Mean.Globally computes the arithmetic mean of the elements."""
+    def __init__(self, has_defaults=True, *args, **kwargs):
+      super(Mean.Globally, self).__init__()
+      self.has_defaults = has_defaults
+      self.args = args
+      self.kwargs = kwargs
+
     def expand(self, pcoll):
-      return pcoll | core.CombineGlobally(MeanCombineFn())
+      if self.has_defaults:
+        return pcoll | core.CombineGlobally(MeanCombineFn())
+      else:
+        return pcoll | core.CombineGlobally(MeanCombineFn()).without_defaults()
+
+    def without_defaults(self):
+      self.has_defaults = False

Review comment:
       I'm not sure if I understand it correctly, but I made a modification to return
   
   ```
   return self | core.CombineGlobally(MeanCombineFn()).without_defaults()
   ```
   
   I personally think it's less intuitive, but you know more :D.
   
   I also tried directly returning without modifying `self.has_defaults`, but it executes the first sequence of the Reduce fine, but the second returns  an error saying that I need the `without_defaults()`. 
   
   If you have suggestions, I'm all ears.
   
   Thanks again




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-644777809


   retest this please
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-645002698






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ commented on a change in pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ commented on a change in pull request #11943:
URL: https://github.com/apache/beam/pull/11943#discussion_r439507508



##########
File path: sdks/python/apache_beam/transforms/combiners.py
##########
@@ -66,8 +66,21 @@ class Mean(object):
   """Combiners for computing arithmetic means of elements."""
   class Globally(ptransform.PTransform):
     """combiners.Mean.Globally computes the arithmetic mean of the elements."""
+    def __init__(self, has_defaults=True, *args, **kwargs):

Review comment:
       I took out the `args` and `kwargs`, but didn't add the Pydoc to keep it consisten [`CombineGlobally`](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/transforms/core.py#L1834) (doesn't have it either). Let me know if you want me to add them.
   
   Thanks again




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] robertwb commented on a change in pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #11943:
URL: https://github.com/apache/beam/pull/11943#discussion_r438291777



##########
File path: sdks/python/apache_beam/transforms/combiners_test.py
##########
@@ -105,6 +109,16 @@ def test_builtin_combines(self):
       assert_that(result_mean, equal_to([mean]), label='assert:mean')
       assert_that(result_count, equal_to([size]), label='assert:size')
 
+      # Now for global combines without default
+      timestamped = pcoll | Map(lambda x: TimestampedValue(timestamp))

Review comment:
       TimestampedValue requires the timestamp and the value in the constructor. 

##########
File path: sdks/python/apache_beam/transforms/combiners_test.py
##########
@@ -97,6 +100,7 @@ def test_builtin_combines(self):
       vals = [6, 3, 1, 1, 9, 1, 5, 2, 0, 6]
       mean = sum(vals) / float(len(vals))
       size = len(vals)
+      timestamp = 1591485720

Review comment:
       Just use a timestamp of 0 or similar. 

##########
File path: sdks/python/apache_beam/transforms/combiners.py
##########
@@ -66,8 +66,21 @@ class Mean(object):
   """Combiners for computing arithmetic means of elements."""
   class Globally(ptransform.PTransform):
     """combiners.Mean.Globally computes the arithmetic mean of the elements."""
+    def __init__(self, has_defaults=True, *args, **kwargs):
+      super(Mean.Globally, self).__init__()
+      self.has_defaults = has_defaults
+      self.args = args
+      self.kwargs = kwargs
+
     def expand(self, pcoll):
-      return pcoll | core.CombineGlobally(MeanCombineFn())
+      if self.has_defaults:
+        return pcoll | core.CombineGlobally(MeanCombineFn())
+      else:
+        return pcoll | core.CombineGlobally(MeanCombineFn()).without_defaults()
+
+    def without_defaults(self):
+      self.has_defaults = False

Review comment:
       Better to return a new copy than mutate self. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-645630046


   thanks @InigoSJ !


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ closed pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ closed pull request #11943:
URL: https://github.com/apache/beam/pull/11943


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-645008448


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on a change in pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on a change in pull request #11943:
URL: https://github.com/apache/beam/pull/11943#discussion_r439077983



##########
File path: sdks/python/apache_beam/transforms/combiners.py
##########
@@ -66,8 +66,21 @@ class Mean(object):
   """Combiners for computing arithmetic means of elements."""
   class Globally(ptransform.PTransform):
     """combiners.Mean.Globally computes the arithmetic mean of the elements."""
+    def __init__(self, has_defaults=True, *args, **kwargs):

Review comment:
       Can you add Pydoc for this? And I don't think we need `args` and `kwargs` to be added. Probably take only has_defaults.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-642854438






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ edited a comment on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ edited a comment on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-644381553


   > this looks fine. @InigoSJ can you fix the Lint issues?
   > See the logs in https://builds.apache.org/job/beam_PreCommit_PythonLint_Commit/4261/console
   > specifically
   > 
   > ```
   > 17:14:30 apache_beam/transforms/combiners_test.py:115:0: C0301: Line too long (84/80) (line-too-long)
   > ```
   
   Done, thanks!


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ commented on a change in pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ commented on a change in pull request #11943:
URL: https://github.com/apache/beam/pull/11943#discussion_r438952009



##########
File path: sdks/python/apache_beam/transforms/combiners.py
##########
@@ -66,8 +66,21 @@ class Mean(object):
   """Combiners for computing arithmetic means of elements."""
   class Globally(ptransform.PTransform):
     """combiners.Mean.Globally computes the arithmetic mean of the elements."""
+    def __init__(self, has_defaults=True, *args, **kwargs):
+      super(Mean.Globally, self).__init__()
+      self.has_defaults = has_defaults
+      self.args = args
+      self.kwargs = kwargs
+
     def expand(self, pcoll):
-      return pcoll | core.CombineGlobally(MeanCombineFn())
+      if self.has_defaults:
+        return pcoll | core.CombineGlobally(MeanCombineFn())
+      else:
+        return pcoll | core.CombineGlobally(MeanCombineFn()).without_defaults()
+
+    def without_defaults(self):
+      self.has_defaults = False

Review comment:
       Another way could be doing `Mean.Globally(without_defaults=True)`,  but I'd say it's against the standard use of `CombineGlobally().without_defaults()`




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-643510251


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ commented on a change in pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ commented on a change in pull request #11943:
URL: https://github.com/apache/beam/pull/11943#discussion_r439507508



##########
File path: sdks/python/apache_beam/transforms/combiners.py
##########
@@ -66,8 +66,21 @@ class Mean(object):
   """Combiners for computing arithmetic means of elements."""
   class Globally(ptransform.PTransform):
     """combiners.Mean.Globally computes the arithmetic mean of the elements."""
+    def __init__(self, has_defaults=True, *args, **kwargs):

Review comment:
       I took out the `args` and `kwargs`, but didn't add the Pydoc to keep it consistent [`CombineGlobally`](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/transforms/core.py#L1834) (doesn't have it either). Let me know if you want me to add them.
   
   Thanks again




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-645006644


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-643446115


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-643510332


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-643510656


   retest this please
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-642863592


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-645145421


   PythonDocker errors are likely unrelated, but there's lint errors: https://builds.apache.org/job/beam_PreCommit_PythonLint_Commit/4293/console
   
   If you look at the console output. the errors are related to the import order in the test file


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on a change in pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on a change in pull request #11943:
URL: https://github.com/apache/beam/pull/11943#discussion_r439075720



##########
File path: sdks/python/apache_beam/transforms/combiners.py
##########
@@ -66,8 +66,21 @@ class Mean(object):
   """Combiners for computing arithmetic means of elements."""
   class Globally(ptransform.PTransform):
     """combiners.Mean.Globally computes the arithmetic mean of the elements."""
+    def __init__(self, has_defaults=True, *args, **kwargs):
+      super(Mean.Globally, self).__init__()
+      self.has_defaults = has_defaults
+      self.args = args
+      self.kwargs = kwargs
+
     def expand(self, pcoll):
-      return pcoll | core.CombineGlobally(MeanCombineFn())
+      if self.has_defaults:
+        return pcoll | core.CombineGlobally(MeanCombineFn())
+      else:
+        return pcoll | core.CombineGlobally(MeanCombineFn()).without_defaults()
+
+    def without_defaults(self):
+      self.has_defaults = False

Review comment:
       what robert means in this case is that rather than mutating and returning self like so:
   ```
   def without_defaults(self):
     self.has_defaults = False
     return self
   ```
   
   It would be more useful to return a new instance with the new property. Somewhat like this:
   ```
   def with_defaults(self, has_defaults=True):
     return Mean.Globally(has_defaults=has_defaults, *self.args, **self.kwargs)
   ```
   It's useful to do this because then instance attributes are immutable, and it's easier to manage instances with immutable attributes.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-645003399


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-645072500


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-644381766






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-644381553


   > this looks fine. @InigoSJ can you fix the Lint issues?
   > See the logs in https://builds.apache.org/job/beam_PreCommit_PythonLint_Commit/4261/console
   > specifically
   > 
   > ```
   > 17:14:30 apache_beam/transforms/combiners_test.py:115:0: C0301: Line too long (84/80) (line-too-long)
   > ```
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ commented on a change in pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ commented on a change in pull request #11943:
URL: https://github.com/apache/beam/pull/11943#discussion_r438368619



##########
File path: sdks/python/apache_beam/transforms/combiners.py
##########
@@ -66,8 +66,21 @@ class Mean(object):
   """Combiners for computing arithmetic means of elements."""
   class Globally(ptransform.PTransform):
     """combiners.Mean.Globally computes the arithmetic mean of the elements."""
+    def __init__(self, has_defaults=True, *args, **kwargs):
+      super(Mean.Globally, self).__init__()
+      self.has_defaults = has_defaults
+      self.args = args
+      self.kwargs = kwargs
+
     def expand(self, pcoll):
-      return pcoll | core.CombineGlobally(MeanCombineFn())
+      if self.has_defaults:
+        return pcoll | core.CombineGlobally(MeanCombineFn())
+      else:
+        return pcoll | core.CombineGlobally(MeanCombineFn()).without_defaults()
+
+    def without_defaults(self):
+      self.has_defaults = False

Review comment:
       I'm not sure if I understand it correctly, but I made a modification to return
   
   ```
   return self | core.CombineGlobally(MeanCombineFn()).without_defaults()
   ```
   
   I personally think it's less intuitive, but you know more :D.
   
   I also tried directly returning without modifying `self.has_defaults`, but it executes the first sequence of the Reduce fine, but the second returns  an error saying that I need the `without_defaults()`, since it directly goes to the `expand()`. 
   
   If you have suggestions, I'm all ears.
   
   Thanks again




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-640087218


   Modification: 
   
   Changed from GlobalWindows to FixedWindows, since GlobalWindows doesn't require .without_defautls()


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-644352330


   this looks fine. @InigoSJ can you fix the Lint issues?
   See the logs in https://builds.apache.org/job/beam_PreCommit_PythonLint_Commit/4261/console
   specifically
   ```
   17:14:30 apache_beam/transforms/combiners_test.py:115:0: C0301: Line too long (84/80) (line-too-long)
   
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-642825460


   R: @pabloem 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-645013448


   okay I'll try later today


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ commented on a change in pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ commented on a change in pull request #11943:
URL: https://github.com/apache/beam/pull/11943#discussion_r438366434



##########
File path: sdks/python/apache_beam/transforms/combiners_test.py
##########
@@ -97,6 +100,7 @@ def test_builtin_combines(self):
       vals = [6, 3, 1, 1, 9, 1, 5, 2, 0, 6]
       mean = sum(vals) / float(len(vals))
       size = len(vals)
+      timestamp = 1591485720

Review comment:
       Modified! Thanks




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-645012054


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ commented on a change in pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ commented on a change in pull request #11943:
URL: https://github.com/apache/beam/pull/11943#discussion_r439505838



##########
File path: sdks/python/apache_beam/transforms/combiners.py
##########
@@ -66,8 +66,21 @@ class Mean(object):
   """Combiners for computing arithmetic means of elements."""
   class Globally(ptransform.PTransform):
     """combiners.Mean.Globally computes the arithmetic mean of the elements."""
+    def __init__(self, has_defaults=True, *args, **kwargs):
+      super(Mean.Globally, self).__init__()
+      self.has_defaults = has_defaults
+      self.args = args
+      self.kwargs = kwargs
+
     def expand(self, pcoll):
-      return pcoll | core.CombineGlobally(MeanCombineFn())
+      if self.has_defaults:
+        return pcoll | core.CombineGlobally(MeanCombineFn())
+      else:
+        return pcoll | core.CombineGlobally(MeanCombineFn()).without_defaults()
+
+    def without_defaults(self):
+      self.has_defaults = False

Review comment:
       Thanks for the suggestions.
   
   I made it look more like the original `without_defaults()` from [`CombineGlobally`](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/transforms/core.py#L1834)




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-642854438






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ edited a comment on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ edited a comment on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-640087218


   Modification: 
   
   Changed from GlobalWindows to FixedWindows in the combiners_test, since GlobalWindows doesn't require .without_defautls()


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-645547638


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ commented on pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ commented on pull request #11943:
URL: https://github.com/apache/beam/pull/11943#issuecomment-640060464


   R: @robertwb 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] pabloem merged pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
pabloem merged pull request #11943:
URL: https://github.com/apache/beam/pull/11943


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] InigoSJ commented on a change in pull request #11943: [BEAM-10209] Add without_defaults to Mean combiner in Python

Posted by GitBox <gi...@apache.org>.
InigoSJ commented on a change in pull request #11943:
URL: https://github.com/apache/beam/pull/11943#discussion_r439505838



##########
File path: sdks/python/apache_beam/transforms/combiners.py
##########
@@ -66,8 +66,21 @@ class Mean(object):
   """Combiners for computing arithmetic means of elements."""
   class Globally(ptransform.PTransform):
     """combiners.Mean.Globally computes the arithmetic mean of the elements."""
+    def __init__(self, has_defaults=True, *args, **kwargs):
+      super(Mean.Globally, self).__init__()
+      self.has_defaults = has_defaults
+      self.args = args
+      self.kwargs = kwargs
+
     def expand(self, pcoll):
-      return pcoll | core.CombineGlobally(MeanCombineFn())
+      if self.has_defaults:
+        return pcoll | core.CombineGlobally(MeanCombineFn())
+      else:
+        return pcoll | core.CombineGlobally(MeanCombineFn()).without_defaults()
+
+    def without_defaults(self):
+      self.has_defaults = False

Review comment:
       Thanks for the suggestions.
   
   I made it look more like the original `without_defaults()` from [`CombineGlobally`](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/transforms/core.py#L1834) using both:
   
   - `with_defaults`: actually making the request to compute `Mean`
   - `without_defaults`: pointing to `with_defaults`




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org