You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/07/21 19:04:34 UTC
[GitHub] [beam] davidcavazos opened a new pull request #12328: [BEAM-7390] Add combinevalues code snippets
davidcavazos opened a new pull request #12328:
URL: https://github.com/apache/beam/pull/12328
Add `CombineValues` code snippets.
R: @aaltay
R: @rosetn
Staged: http://apache-beam-website-pull-requests.storage.googleapis.com/12277/documentation/transforms/python/aggregation/combinevalues/index.html
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2
--- | --- | --- | --- | --- | --- | ---
Go | [](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | ---
Java | [](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/)
Python | [](https://ci-beam.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/) | ---
XLang | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/) | ---
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #12328: [BEAM-7390] Add combinevalues code snippets
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #12328:
URL: https://github.com/apache/beam/pull/12328#issuecomment-662194077
LGTM. I can merge after @rosetn gets a chance to review. Please ping me after that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] rosetn commented on a change in pull request #12328: [BEAM-7390] Add combinevalues code snippets
Posted by GitBox <gi...@apache.org>.
rosetn commented on a change in pull request #12328:
URL: https://github.com/apache/beam/pull/12328#discussion_r458916295
##########
File path: website/www/site/content/en/documentation/transforms/python/aggregation/combinevalues.md
##########
@@ -0,0 +1,230 @@
+---
+title: "CombineValues"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# CombineValues
+
+{{< localstorage language language-py >}}
+
+{{< button-pydoc path="apache_beam.transforms.core" class="CombineValues" >}}
+
+Combines an iterable of values in a keyed collection of elements.
+
+See more information in the [Beam Programming Guide](/documentation/programming-guide/#combine).
+
+## Examples
+
+In the following examples, we create a pipeline with a `PCollection` of produce.
+Then, we apply `CombineValues` in multiple ways to combine the keyed values in the `PCollection`.
+
+`CombineValues` accepts a function that takes an `iterable` of elements as an input, and combines them to return a single element.
+`CombineValues` expects a keyed `PCollection` of elements, where the value is an iterable of elements to be combined.
+
Review comment:
Can you add a sentence here that briefly explains saturation arithmetic?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] rosetn commented on pull request #12328: [BEAM-7390] Add combinevalues code snippets
Posted by GitBox <gi...@apache.org>.
rosetn commented on pull request #12328:
URL: https://github.com/apache/beam/pull/12328#issuecomment-663174268
I think you might have embedded the wrong code sample for example #1--you're using `saturated_sum` instead of just `sum` and the output is the result of `sum`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] davidcavazos commented on pull request #12328: [BEAM-7390] Add combinevalues code snippets
Posted by GitBox <gi...@apache.org>.
davidcavazos commented on pull request #12328:
URL: https://github.com/apache/beam/pull/12328#issuecomment-663183382
> I think you might have embedded the wrong code sample for example 1--you're using `saturated_sum` instead of just `sum` and the output is the result of `sum`
Thank you for catching this!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] davidcavazos commented on a change in pull request #12328: [BEAM-7390] Add combinevalues code snippets
Posted by GitBox <gi...@apache.org>.
davidcavazos commented on a change in pull request #12328:
URL: https://github.com/apache/beam/pull/12328#discussion_r459645020
##########
File path: website/www/site/content/en/documentation/transforms/python/aggregation/combinevalues.md
##########
@@ -0,0 +1,230 @@
+---
+title: "CombineValues"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# CombineValues
+
+{{< localstorage language language-py >}}
+
+{{< button-pydoc path="apache_beam.transforms.core" class="CombineValues" >}}
+
+Combines an iterable of values in a keyed collection of elements.
+
+See more information in the [Beam Programming Guide](/documentation/programming-guide/#combine).
+
+## Examples
+
+In the following examples, we create a pipeline with a `PCollection` of produce.
+Then, we apply `CombineValues` in multiple ways to combine the keyed values in the `PCollection`.
+
+`CombineValues` accepts a function that takes an `iterable` of elements as an input, and combines them to return a single element.
+`CombineValues` expects a keyed `PCollection` of elements, where the value is an iterable of elements to be combined.
+
Review comment:
Sure, I added an explanation in the second example that is where the saturated sum is introduced.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #12328: [BEAM-7390] Add combinevalues code snippets
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #12328:
URL: https://github.com/apache/beam/pull/12328#issuecomment-663212932
Is this ready to be merged now? Seems like the snippet issue is fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay merged pull request #12328: [BEAM-7390] Add combinevalues code snippets
Posted by GitBox <gi...@apache.org>.
aaltay merged pull request #12328:
URL: https://github.com/apache/beam/pull/12328
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] rosetn edited a comment on pull request #12328: [BEAM-7390] Add combinevalues code snippets
Posted by GitBox <gi...@apache.org>.
rosetn edited a comment on pull request #12328:
URL: https://github.com/apache/beam/pull/12328#issuecomment-663174268
I think you might have embedded the wrong code sample for example 1--you're using `saturated_sum` instead of just `sum` and the output is the result of `sum`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org