You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/04/28 01:58:44 UTC
[GitHub] [beam] pabloem opened a new pull request #11546: [BEAM-9832] Ensuring tags are not added to ProcessBundleDescriptors o…
pabloem opened a new pull request #11546:
URL: https://github.com/apache/beam/pull/11546
…n the worker-side
**Please** add a meaningful description for your change here
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lukecwik commented on a change in pull request #11546: [BEAM-9832] Ensuring tags are not added to ProcessBundleDescriptors o…
Posted by GitBox <gi...@apache.org>.
lukecwik commented on a change in pull request #11546:
URL: https://github.com/apache/beam/pull/11546#discussion_r416284576
##########
File path: sdks/python/apache_beam/runners/worker/operations.py
##########
@@ -724,6 +724,8 @@ def pcollection_count_monitoring_infos(self, tag_to_pcollection_id):
if self.tagged_receivers:
for tag, receiver in self.tagged_receivers.items():
+ if str(tag) not in tag_to_pcollection_id:
Review comment:
We should be able to drop the if pcollection_id check below
##########
File path: sdks/python/apache_beam/runners/worker/operations.py
##########
@@ -724,6 +724,8 @@ def pcollection_count_monitoring_infos(self, tag_to_pcollection_id):
if self.tagged_receivers:
for tag, receiver in self.tagged_receivers.items():
+ if str(tag) not in tag_to_pcollection_id:
Review comment:
We should be able to drop the `if pcollection_id` check below
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11546: [BEAM-9832] Ensuring tags are not added to ProcessBundleDescriptors o…
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11546:
URL: https://github.com/apache/beam/pull/11546#issuecomment-620359255
Run Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lukecwik commented on a change in pull request #11546: [BEAM-9832] Ensuring tags are not added to ProcessBundleDescriptors o…
Posted by GitBox <gi...@apache.org>.
lukecwik commented on a change in pull request #11546:
URL: https://github.com/apache/beam/pull/11546#discussion_r416274806
##########
File path: sdks/python/apache_beam/runners/worker/operations.py
##########
@@ -724,6 +724,8 @@ def pcollection_count_monitoring_infos(self, tag_to_pcollection_id):
if self.tagged_receivers:
for tag, receiver in self.tagged_receivers.items():
+ if str(tag) not in tag_to_pcollection_id:
Review comment:
It would be weird that accessing the dict adds the entry in this case but if this is the solution then it would be better we created a copy of the dict we pass from here:
https://github.com/apache/beam/blob/2434d496571aa46efda1a18d9ccc319815b712f8/sdks/python/apache_beam/runners/worker/bundle_processor.py#L1038
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on a change in pull request #11546: [BEAM-9832] Ensuring tags are not added to ProcessBundleDescriptors o…
Posted by GitBox <gi...@apache.org>.
pabloem commented on a change in pull request #11546:
URL: https://github.com/apache/beam/pull/11546#discussion_r416275978
##########
File path: sdks/python/apache_beam/runners/worker/operations.py
##########
@@ -724,6 +724,8 @@ def pcollection_count_monitoring_infos(self, tag_to_pcollection_id):
if self.tagged_receivers:
for tag, receiver in self.tagged_receivers.items():
+ if str(tag) not in tag_to_pcollection_id:
Review comment:
you're right, but that causes another error. I might be looking at the wrog thing entirely.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org