You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/03/25 18:35:04 UTC
[GitHub] [beam] robertwb opened a new pull request #11224: [BEAM-9340]
Validate pipeline requirements in PipelineValidator.
robertwb opened a new pull request #11224: [BEAM-9340] Validate pipeline requirements in PipelineValidator.
URL: https://github.com/apache/beam/pull/11224
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] robertwb commented on issue #11224: [BEAM-9340] Validate
pipeline requirements in PipelineValidator.
Posted by GitBox <gi...@apache.org>.
robertwb commented on issue #11224: [BEAM-9340] Validate pipeline requirements in PipelineValidator.
URL: https://github.com/apache/beam/pull/11224#issuecomment-604145091
Run Java PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] robertwb commented on issue #11224: [BEAM-9340] Validate
pipeline requirements in PipelineValidator.
Posted by GitBox <gi...@apache.org>.
robertwb commented on issue #11224: [BEAM-9340] Validate pipeline requirements in PipelineValidator.
URL: https://github.com/apache/beam/pull/11224#issuecomment-604014835
Follow-up to https://github.com/apache/beam/pull/11165
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] robertwb commented on issue #11224: [BEAM-9340] Validate
pipeline requirements in PipelineValidator.
Posted by GitBox <gi...@apache.org>.
robertwb commented on issue #11224: [BEAM-9340] Validate pipeline requirements in PipelineValidator.
URL: https://github.com/apache/beam/pull/11224#issuecomment-604144495
Run Java PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lukecwik commented on a change in pull request #11224:
[BEAM-9340] Validate pipeline requirements in PipelineValidator.
Posted by GitBox <gi...@apache.org>.
lukecwik commented on a change in pull request #11224: [BEAM-9340] Validate pipeline requirements in PipelineValidator.
URL: https://github.com/apache/beam/pull/11224#discussion_r398695607
##########
File path: runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/graph/PipelineValidator.java
##########
@@ -221,23 +229,39 @@ private static void validateParDo(String id, PTransform transform, Components co
id,
sideInputId);
}
- // TODO: Validate state_specs and timer_specs
+ if (payload.getStateSpecsCount() > 0 || payload.getTimerSpecsCount() > 0) {
Review comment:
This should also have timer family specs.
@boyuanzz Once this code is merged, this is another place where TimerSpecs will need to be removed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lukecwik merged pull request #11224: [BEAM-9340] Validate
pipeline requirements in PipelineValidator.
Posted by GitBox <gi...@apache.org>.
lukecwik merged pull request #11224: [BEAM-9340] Validate pipeline requirements in PipelineValidator.
URL: https://github.com/apache/beam/pull/11224
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lukecwik commented on a change in pull request #11224:
[BEAM-9340] Validate pipeline requirements in PipelineValidator.
Posted by GitBox <gi...@apache.org>.
lukecwik commented on a change in pull request #11224: [BEAM-9340] Validate pipeline requirements in PipelineValidator.
URL: https://github.com/apache/beam/pull/11224#discussion_r398771741
##########
File path: runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/graph/PipelineValidator.java
##########
@@ -221,23 +229,39 @@ private static void validateParDo(String id, PTransform transform, Components co
id,
sideInputId);
}
- // TODO: Validate state_specs and timer_specs
+ if (payload.getStateSpecsCount() > 0 || payload.getTimerSpecsCount() > 0) {
Review comment:
We can merge now and you will need to swap from TimerSpecs to TimerFamilySpecs here as part of your PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] boyuanzz commented on a change in pull request #11224:
[BEAM-9340] Validate pipeline requirements in PipelineValidator.
Posted by GitBox <gi...@apache.org>.
boyuanzz commented on a change in pull request #11224: [BEAM-9340] Validate pipeline requirements in PipelineValidator.
URL: https://github.com/apache/beam/pull/11224#discussion_r398741485
##########
File path: runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/graph/PipelineValidator.java
##########
@@ -221,23 +229,39 @@ private static void validateParDo(String id, PTransform transform, Components co
id,
sideInputId);
}
- // TODO: Validate state_specs and timer_specs
+ if (payload.getStateSpecsCount() > 0 || payload.getTimerSpecsCount() > 0) {
Review comment:
Ack. can we merge it as soon as possible? Then I can rebase my changes on top of this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services