You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/07/30 07:33:00 UTC
[GitHub] [beam] tysonjh opened a new pull request #12416: Update google-api-services versions.
tysonjh opened a new pull request #12416:
URL: https://github.com/apache/beam/pull/12416
Update google-api-services* versions and google-client version to latest.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2
--- | --- | --- | --- | --- | --- | ---
Go | [](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | ---
Java | [](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/)
Python | [](https://ci-beam.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/) | ---
XLang | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/) | ---
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tysonjh commented on pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
tysonjh commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-666751030
run java postcommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tysonjh commented on pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
tysonjh commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-668733014
> Failed to publish publication 'mavenJava' to repository 'mavenLocal'
>
> > Invalid publication 'mavenJava': supplied version does not match POM file (cannot edit version directly in the POM file).
>
> Not sure what this means, will need to investigate unless you know.
May just need to run a ./gradlew clean (https://issues.apache.org/jira/browse/BEAM-9437). Trying that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tysonjh edited a comment on pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
tysonjh edited a comment on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-669303736
> > Failed to publish publication 'mavenJava' to repository 'mavenLocal'
> > > Invalid publication 'mavenJava': supplied version does not match POM file (cannot edit version directly in the POM file).
> >
> >
> > Not sure what this means, will need to investigate unless you know.
>
> May just need to run a ./gradlew clean (https://issues.apache.org/jira/browse/BEAM-9437). Trying that.
What a mess. I spent a good amount of time, ~3hr or so, trying to get the beam-linkage-check.sh to work. I ran into numerous issues from a problem with the ':sdks:java:bom:publishMavenJavaPublicationToMavenLocal' task, problems building flink tests for 1.8 and 1.9, apex runner tasks executing even though the runner is gone, major/minor version mismatches in Java classes, unsupported errorprone versions. I reached out to a @lukecwik and @apilloud for help as well (thank you both).
What worked in the end:
* ensure using Java 8 (both binary and classpath)
* a fresh new clone of apache/beam
https://gist.github.com/tysonjh/d8d28689fc18b536b32498cb0fd27382
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tysonjh commented on pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
tysonjh commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-666812322
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles commented on pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
kennknowles commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-666573578
Notably, the failing test does a bunch of mocking. So I wouldn't assume anything is actually wrong.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles commented on pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
kennknowles commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-666574292
But it also probably isn't a flake, since it is a pretty simple unit test. Most likely the assumptions of the mock were overconstrained so changes in the Pubsub client library can break it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles commented on pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
kennknowles commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-666572179
https://scans.gradle.com/s/opclpkpolskh6/tests/:sdks:java:io:google-cloud-platform:test/org.apache.beam.sdk.io.gcp.pubsub.PubsubJsonClientTest/pullOneMessage#1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tysonjh commented on pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
tysonjh commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-666539863
Hard to tell what the precommit failure is from the logs or scan. It references a local file on the Jenkins agent. I'll run locally.
https://scans.gradle.com/s/opclpkpolskh6/console-log?anchor=20400
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tysonjh commented on pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
tysonjh commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-668722029
Failed to publish publication 'mavenJava' to repository 'mavenLocal'
> Invalid publication 'mavenJava': supplied version does not match POM file (cannot edit version directly in the POM file).
Not sure what this means, will need to investigate unless you know.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tysonjh commented on pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
tysonjh commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-668212843
R: @apilloud
Could you take a look at this please?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tysonjh commented on pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
tysonjh commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-666808017
More test failures. NPEs in what seems like unrelated code, but notably the pubsub one again which passes locally when I run it for some reason.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] apilloud commented on pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
apilloud commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-668260372
LGTM. There is a script `sdks/java/build-tools/beam-linkage-check.sh` that is suppose to validate version changes don't introduce new compatibility issues. Can you run it and ensure it passes?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] apilloud merged pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
apilloud merged pull request #12416:
URL: https://github.com/apache/beam/pull/12416
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tysonjh commented on pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
tysonjh commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-669303736
> > Failed to publish publication 'mavenJava' to repository 'mavenLocal'
> > > Invalid publication 'mavenJava': supplied version does not match POM file (cannot edit version directly in the POM file).
> >
> >
> > Not sure what this means, will need to investigate unless you know.
>
> May just need to run a ./gradlew clean (https://issues.apache.org/jira/browse/BEAM-9437). Trying that.
What a mess. I spent a good amount of time, ~3hr or so, trying to get the beam-linkage-check.sh to work. I ran into numerous issues from a problem with the ':sdks:java:bom:publishMavenJavaPublicationToMavenLocal' task, problems building flink tests for 1.8 and 1.9, apex runner tasks executing even though the runner is gone, major/minor version mismatches in Java classes, unsupported errorprone versions. I reached out to a @lukecwik and@apilloud for help as well (thank you both).
What worked in the end:
* ensure using Java 8 (both binary and classpath)
* a fresh new clone of apache/beam
https://gist.github.com/tysonjh/d8d28689fc18b536b32498cb0fd27382
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles commented on pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
kennknowles commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-665951261
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tysonjh commented on pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
tysonjh commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-665843586
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tysonjh commented on pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
tysonjh commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-666812699
No, my mistake. It fails locally too, I was running it incorrectly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tysonjh commented on pull request #12416: Update google-api-services versions.
Posted by GitBox <gi...@apache.org>.
tysonjh commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-728201724
> @tysonjh Hey, Seems like this commit breaks my code. Building my code, I get a `NoSuchMethodError` over `setOrderingKey`/`getOrderingKey` for the `pubsubMessage` (the `protoMessage` methods do exist). I will be happy to understand what do I miss, and whether there is a referrer to a solution for my problem. 10x!
What version of the pubsub library are you using?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org