You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/04/10 00:27:50 UTC
[GitHub] [beam] Hannah-Jiang opened a new pull request #11372: [BEAM-9443]
support direct_num_workers=0
Hannah-Jiang opened a new pull request #11372: [BEAM-9443] support direct_num_workers=0
URL: https://github.com/apache/beam/pull/11372
R: @ibzib
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] Hannah-Jiang commented on issue #11372: [BEAM-9443] support
direct_num_workers=0
Posted by GitBox <gi...@apache.org>.
Hannah-Jiang commented on issue #11372: [BEAM-9443] support direct_num_workers=0
URL: https://github.com/apache/beam/pull/11372#issuecomment-612271655
The failures are not related to the PR, so merge it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] ibzib commented on a change in pull request #11372:
[BEAM-9443] support direct_num_workers=0
Posted by GitBox <gi...@apache.org>.
ibzib commented on a change in pull request #11372: [BEAM-9443] support direct_num_workers=0
URL: https://github.com/apache/beam/pull/11372#discussion_r406884154
##########
File path: CHANGES.md
##########
@@ -47,6 +47,33 @@
* Fixed X (Java/Python) ([BEAM-X](https://issues.apache.org/jira/browse/BEAM-X)).
-->
+# [2.22.0] - Unreleased
+
+## Highlights
+
+* New highly anticipated feature X added to Python SDK ([BEAM-X](https://issues.apache.org/jira/browse/BEAM-X)).
Review comment:
Can you please remove these placeholders? ie keep the section headers but remove "feature X..."
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] ibzib commented on a change in pull request #11372:
[BEAM-9443] support direct_num_workers=0
Posted by GitBox <gi...@apache.org>.
ibzib commented on a change in pull request #11372: [BEAM-9443] support direct_num_workers=0
URL: https://github.com/apache/beam/pull/11372#discussion_r406789872
##########
File path: CHANGES.md
##########
@@ -67,6 +67,7 @@
[Ensuring Python Type Safety](https://beam.apache.org/documentation/sdks/python-type-safety/)
and an upcoming
[blog post](https://beam.apache.org/blog/python/typing/2020/03/06/python-typing.html).
+* --direct_num_workers=0 is supported for FnApi. It will set the number of threads/subprocesses to number of cores of the machine executing the pipeline([BEAM-9443](https://issues.apache.org/jira/browse/BEAM-9443)).
Review comment:
```suggestion
* --direct_num_workers=0 is supported for FnApi runner. It will set the number of threads/subprocesses to number of cores of the machine executing the pipeline([BEAM-9443](https://issues.apache.org/jira/browse/BEAM-9443)).
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] Hannah-Jiang commented on a change in pull request #11372:
[BEAM-9443] support direct_num_workers=0
Posted by GitBox <gi...@apache.org>.
Hannah-Jiang commented on a change in pull request #11372: [BEAM-9443] support direct_num_workers=0
URL: https://github.com/apache/beam/pull/11372#discussion_r406546390
##########
File path: CHANGES.md
##########
@@ -67,6 +67,7 @@
[Ensuring Python Type Safety](https://beam.apache.org/documentation/sdks/python-type-safety/)
and an upcoming
[blog post](https://beam.apache.org/blog/python/typing/2020/03/06/python-typing.html).
+* --direct_num_workers=0 is supported for FnApi. It will set the number of threads/subprocesses to number of cores of the machine executing the pipeline([BEAM-9443](https://issues.apache.org/jira/browse/BEAM-9443)).
Review comment:
It is added to 2.21.0, please let me know if the branch is already cut, I can move it to 2.22.0 if needed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] ibzib commented on a change in pull request #11372:
[BEAM-9443] support direct_num_workers=0
Posted by GitBox <gi...@apache.org>.
ibzib commented on a change in pull request #11372: [BEAM-9443] support direct_num_workers=0
URL: https://github.com/apache/beam/pull/11372#discussion_r406789105
##########
File path: CHANGES.md
##########
@@ -67,6 +67,7 @@
[Ensuring Python Type Safety](https://beam.apache.org/documentation/sdks/python-type-safety/)
and an upcoming
[blog post](https://beam.apache.org/blog/python/typing/2020/03/06/python-typing.html).
+* --direct_num_workers=0 is supported for FnApi. It will set the number of threads/subprocesses to number of cores of the machine executing the pipeline([BEAM-9443](https://issues.apache.org/jira/browse/BEAM-9443)).
Review comment:
The branch is cut, please move to 2.22.0
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] Hannah-Jiang commented on a change in pull request #11372:
[BEAM-9443] support direct_num_workers=0
Posted by GitBox <gi...@apache.org>.
Hannah-Jiang commented on a change in pull request #11372: [BEAM-9443] support direct_num_workers=0
URL: https://github.com/apache/beam/pull/11372#discussion_r406987566
##########
File path: CHANGES.md
##########
@@ -47,6 +47,33 @@
* Fixed X (Java/Python) ([BEAM-X](https://issues.apache.org/jira/browse/BEAM-X)).
-->
+# [2.22.0] - Unreleased
+
+## Highlights
+
+* New highly anticipated feature X added to Python SDK ([BEAM-X](https://issues.apache.org/jira/browse/BEAM-X)).
Review comment:
I don't see problems here. It's good to provide a sample when there is no entry. Contributors will overwrite the placeholder when they add something new.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] Hannah-Jiang merged pull request #11372: [BEAM-9443] support
direct_num_workers=0
Posted by GitBox <gi...@apache.org>.
Hannah-Jiang merged pull request #11372: [BEAM-9443] support direct_num_workers=0
URL: https://github.com/apache/beam/pull/11372
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services