You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/09/15 16:42:25 UTC
[GitHub] [beam] timclemons opened a new pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
timclemons opened a new pull request #12850:
URL: https://github.com/apache/beam/pull/12850
Move registration of accumulator so as to include those recovered from checkpoints. Prior to this, if the accumulator was recovered from a checkpoint it would not be registered, resulting in a runtime exception.
R: @jbonofre
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [X] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [X] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2
--- | --- | --- | --- | --- | --- | ---
Go | [](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | ---
Java | [](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/)
Python | [](https://ci-beam.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/) | ---
XLang | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/) | ---
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website | Whitespace | Typescript
--- | --- | --- | --- | --- | --- | ---
Non-portable | [](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/) <br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/)
Portable | --- | [](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | --- | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-698462069
Run Java Spark PortableValidatesRunner Streaming
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-697635483
@ibzib I see that `beam_PostCommit_Java_PVR_Spark_Streaming_PR` fails for this PR. Do you think it could be related to this change or it's just flaky tests?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ibzib commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
ibzib commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-696373462
R: @aromanenko-dev
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev merged pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
aromanenko-dev merged pull request #12850:
URL: https://github.com/apache/beam/pull/12850
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-699052169
@robinyqiu @timclemons Well, I believe it was introduced in 2.13.0 while moving to AccumulatorsV2 API, so I think it can wait till 2.26.0. Wdyt?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] timclemons commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
timclemons commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-698481828
Any chance this will make the 2.25.0 release?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] timclemons commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
timclemons commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-696380555
R: @aromanenko-dev
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-697344156
Run Spark ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ibzib commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
ibzib commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-696373462
R: @aromanenko-dev
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] timclemons removed a comment on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
timclemons removed a comment on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-696380555
R: @aromanenko-dev
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev removed a comment on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
aromanenko-dev removed a comment on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-697346457
Run Java Spark PortableValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-698461942
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-697347001
Run Java Spark PortableValidatesRunner Batch
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ibzib commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
ibzib commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-699030658
> Run Java Spark PortableValidatesRunner Streaming
Spark portable streaming tests aren't stable yet, so don't worry about their results. BEAM-10784
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ibzib commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
ibzib commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-696373462
R: @aromanenko-dev
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] timclemons removed a comment on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
timclemons removed a comment on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-696380555
R: @aromanenko-dev
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-698761445
@robinyqiu Do you think we still can cherry-pick this into 2.25.0 branch or not?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-698461942
Run Spark Runner Nexmark Tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robinyqiu commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
robinyqiu commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-699015305
Hi Alexey, the release branch is already cut. Per the release guide [here](https://beam.apache.org/contribute/release-guide/#review-cherry-picks), we only cherry-pick fixes for regression:
> Is this a regression from a previous release? (If no, fix could go to a newer version.)
Is this the case here? If not could we make it in 2.26.0?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] timclemons commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
timclemons commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-696380555
R: @aromanenko-dev
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-697347826
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] timclemons commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
timclemons commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-698481828
Any chance this will make the 2.25.0 release?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] timclemons removed a comment on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
timclemons removed a comment on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-696380555
R: @aromanenko-dev
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-697346732
Run Java Spark PortableValidatesRunner Streaming
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-698985017
@timclemons Would it be possible to add a regression test for that?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robinyqiu commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
robinyqiu commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-699055117
I agree. Let's wait till 2.26.0. Thanks for letting me know.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-697346457
Run Java Spark PortableValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] timclemons commented on pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.
Posted by GitBox <gi...@apache.org>.
timclemons commented on pull request #12850:
URL: https://github.com/apache/beam/pull/12850#issuecomment-696380555
R: @aromanenko-dev
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org