You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/05/06 09:41:42 UTC
[GitHub] [beam] pawelpasterz opened a new pull request #11619: Compile and run tests on java 11 for PreCommit portability api
pawelpasterz opened a new pull request #11619:
URL: https://github.com/apache/beam/pull/11619
This is next step for java 11 test compilation. #11527 enabled this approach.
For task within test should be compiled with java 11 add `-PcompileAndRunTestsWithJava11` [1] and `-Pjava11Home=[path_to_java_11_home]` [2]
[1] adds tasks dependency to run `verifyJavaVersion` before
```
if (project.hasProperty('compileAndRunTestsWithJava11')) {
project.javaPreCommitPortabilityApi.dependsOn ':sdks:java:testing:test-utils:verifyJavaVersion'
}
```
Task contains 3 JUnit tests that verify:
1. if runtime jvm version is 11
2. if beam bytecode version is 8
3. if test bytecode version is 11
If [1] is not present then verification tests are excluded from tasks of type `Test`.
```
} else {
allprojects {
tasks.withType(Test) {
exclude '**/JvmVerification.class'
}
}
}
```
[2] is used in forking options for `compileTestJava` task and as executable for `Test` type tasks.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pawelpasterz edited a comment on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
pawelpasterz edited a comment on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-624546245
R: @Ardagan @iemejia @kennknowles @mwalenia @aaltay @lukecwik
Please, take a look and let me know what do you think
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pawelpasterz edited a comment on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
pawelpasterz edited a comment on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625624564
@iemejia
> @pawelpasterz I am ok with the current version can you please squash and update so we can run the tests and see if everything work as expected before merge.
>
> Also this has probably to be updated once https://issues.apache.org/jira/browse/BEAM-9915 is done.
Yes, definitely!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
iemejia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625160831
@pawelpasterz I am ok with the current version can you please squash and update so we can run the tests and see if everything work as expected before merge.
Also this has probably to be updated once https://issues.apache.org/jira/browse/BEAM-9915 is done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pawelpasterz commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
pawelpasterz commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-624546245
R: @Ardagan @iemejia @kennknowles @mwalenia
Please, take a look and let me know what do you think
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625683788
run seed job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625715466
run seed job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625668101
run seed job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625727950
Run JavaPortabilityApi PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625715547
run seed job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pawelpasterz edited a comment on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
pawelpasterz edited a comment on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625136033
@iemejia
> Can you please squash the commits and rename them to include the valid JIRA name e.g. "[[BEAM-2530](https://issues.apache.org/jira/browse/BEAM-2530)] ..."
Sure, I'll squash commits once PR will have all requested changes implemented
> Just for curiosity, since this only build/run the tests with Java 11. What are the current blockers for the full build with Java 11?
Prior was error prone plugin version. Now, when I try to compile beam with java I get errors like:
```
> Task :model:pipeline:compileJava
/Users/pawel/beam/model/pipeline/build/generated/source/proto/main/grpc/org/apache/beam/model/pipeline/v1/TestStreamServiceGrpc.java:20: error: cannot find symbol
@javax.annotation.Generated(
^
symbol: class Generated
location: package javax.annotation
1 error
```
TBH, I did not spend much time figuring this out since it is not main concern of this PR. I'll definitely try to fix this in the future. Or maybe you have some tips regarding this error?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625719414
run seed job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625715813
run seed job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pawelpasterz commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
pawelpasterz commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625624564
> @pawelpasterz I am ok with the current version can you please squash and update so we can run the tests and see if everything work as expected before merge.
>
> Also this has probably to be updated once https://issues.apache.org/jira/browse/BEAM-9915 is done.
Yes, definitely!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625679517
run seed job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625679351
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625678085
Run JavaPortabilityApiJava11 PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
iemejia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625150830
> Prior was error prone plugin version. Now, when I try to compile beam with java I get errors like:
>
> ```
> > Task :model:pipeline:compileJava
> /Users/pawel/beam/model/pipeline/build/generated/source/proto/main/grpc/org/apache/beam/model/pipeline/v1/TestStreamServiceGrpc.java:20: error: cannot find symbol
> @javax.annotation.Generated(
> ^
> symbol: class Generated
> location: package javax.annotation
> 1 error
> ```
>
> TBH, I did not spend much time figuring this out since it is not main concern of this PR. I'll definitely try to fix this in the future. Or maybe you have some tips regarding this error?
Yes it is not necessary to do this for this PR. Those errors are because some modules are not part of Java by default anymore and should be included explicitly with `--add-module` and the right module.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625715931
run seed job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia removed a comment on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia removed a comment on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625679611
run seed job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625715386
run seed job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625678500
Run JavaPortabilityApi PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia removed a comment on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia removed a comment on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625671644
run seed job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625679611
run seed job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia commented on pull request #11619: [BEAM-2530] Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
iemejia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625859443
Merged manually to add the missing ticket prefix `[BEAM-2530`. Thanks again!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625671876
run seed job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pawelpasterz commented on a change in pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
pawelpasterz commented on a change in pull request #11619:
URL: https://github.com/apache/beam/pull/11619#discussion_r421333577
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -698,6 +700,21 @@ class BeamModulePlugin implements Plugin<Project> {
+ (defaultLintSuppressions + configuration.disableLintWarnings).collect { "-Xlint:-${it}" })
}
+ if (project.hasProperty("compileAndRunTestsWithJava11")) {
+ def java11Home = project.findProperty("java11Home")
+ project.tasks.compileTestJava {
+ options.fork = true
+ options.forkOptions.javaHome = java11Home as File
+ sourceCompatibility = JavaVersion.VERSION_1_8
+ targetCompatibility = JavaVersion.VERSION_11
+ options.compilerArgs += ['-Xlint:-path']
Review comment:
Sure, thanks for pointing this out
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-624862871
/cc @tysonjh
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625715055
run seed job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625668322
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625679748
run seed job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia removed a comment on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia removed a comment on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625715466
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625679405
run seed job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625687880
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625727900
Run JavaPortabilityApiJava11 PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pawelpasterz commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
pawelpasterz commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625136033
> Can you please squash the commits and rename them to include the valid JIRA name e.g. "[[BEAM-2530](https://issues.apache.org/jira/browse/BEAM-2530)] ..."
Sure, I'll squash commits once PR will have all requested changes implemented
> Just for curiosity, since this only build/run the tests with Java 11. What are the current blockers for the full build with Java 11?
Prior was error prone plugin version. Now, when I try to compile beam with java I get errors like:
```
> Task :model:pipeline:compileJava
/Users/pawel/beam/model/pipeline/build/generated/source/proto/main/grpc/org/apache/beam/model/pipeline/v1/TestStreamServiceGrpc.java:20: error: cannot find symbol
@javax.annotation.Generated(
^
symbol: class Generated
location: package javax.annotation
1 error
```
TBH, I did not spend much time figuring this out since it is not main concern of this PR. I'll definitely try to fix this in the future. Or maybe you have some tips regarding this error?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625714904
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625671592
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625727805
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625714163
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia commented on a change in pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
iemejia commented on a change in pull request #11619:
URL: https://github.com/apache/beam/pull/11619#discussion_r421116028
##########
File path: sdks/java/testing/test-utils/src/test/java/org/apache/beam/sdk/testutils/jvmverification/JvmVerification.java
##########
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.testutils.jvmverification;
+
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v10;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v11;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v12;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v13;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v14;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v1_1;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v1_2;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v1_3;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v1_4;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v1_5;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v1_6;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v1_7;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v1_8;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v9;
+import static org.junit.Assert.assertEquals;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.Map;
+import org.apache.beam.repackaged.core.org.apache.commons.compress.utils.IOUtils;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.commons.codec.binary.Hex;
+import org.junit.Test;
+
+public class JvmVerification {
+
+ private static final Map<String, Java> versionMapping = new HashMap<>();
+
+ static {
+ versionMapping.put("002D", v1_1);
+ versionMapping.put("002E", v1_2);
+ versionMapping.put("002F", v1_3);
+ versionMapping.put("0030", v1_4);
+ versionMapping.put("0031", v1_5);
Review comment:
Maybe we can get rid here of all the intermediary releases and let only the ones we care about (aka the LTS ones) so 1_8 and v11 for the moment, no?
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -698,6 +700,21 @@ class BeamModulePlugin implements Plugin<Project> {
+ (defaultLintSuppressions + configuration.disableLintWarnings).collect { "-Xlint:-${it}" })
}
+ if (project.hasProperty("compileAndRunTestsWithJava11")) {
+ def java11Home = project.findProperty("java11Home")
+ project.tasks.compileTestJava {
+ options.fork = true
+ options.forkOptions.javaHome = java11Home as File
+ sourceCompatibility = JavaVersion.VERSION_1_8
+ targetCompatibility = JavaVersion.VERSION_11
+ options.compilerArgs += ['-Xlint:-path']
Review comment:
What about putting here the equivalent of `--release=11` (to ensure we use the bootclasspath of the Java 11 JVM).
https://stackoverflow.com/questions/43102787/what-is-the-release-flag-in-the-java-9-compiler
I suppose the source compatibility with Java 8 is already validated in the default build.
##########
File path: sdks/java/testing/test-utils/src/test/java/org/apache/beam/sdk/testutils/jvmverification/JvmVerification.java
##########
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.testutils.jvmverification;
+
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v10;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v11;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v12;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v13;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v14;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v1_1;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v1_2;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v1_3;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v1_4;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v1_5;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v1_6;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v1_7;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v1_8;
+import static org.apache.beam.sdk.testutils.jvmverification.JvmVerification.Java.v9;
+import static org.junit.Assert.assertEquals;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.Map;
+import org.apache.beam.repackaged.core.org.apache.commons.compress.utils.IOUtils;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.commons.codec.binary.Hex;
+import org.junit.Test;
+
+public class JvmVerification {
+
+ private static final Map<String, Java> versionMapping = new HashMap<>();
+
+ static {
+ versionMapping.put("002D", v1_1);
+ versionMapping.put("002E", v1_2);
+ versionMapping.put("002F", v1_3);
+ versionMapping.put("0030", v1_4);
+ versionMapping.put("0031", v1_5);
+ versionMapping.put("0032", v1_6);
+ versionMapping.put("0033", v1_7);
+ versionMapping.put("0034", v1_8);
+ versionMapping.put("0035", v9);
+ versionMapping.put("0036", v10);
+ versionMapping.put("0037", v11);
+ versionMapping.put("0038", v12);
+ versionMapping.put("0039", v13);
+ versionMapping.put("003A", v14);
+ }
+
+ // bytecode
+ @Test
+ public void verifyCodeIsCompiledWithJava8() throws IOException {
+ assertEquals(v1_8, getByteCodeVersion(DoFn.class));
+ }
+
+ @Test
+ public void verifyTestCodeIsCompiledWithJava8() throws IOException {
+ assertEquals(v1_8, getByteCodeVersion(JvmVerification.class));
+ }
+
+ @Test
+ public void verifyTestCodeIsCompiledWithJava11() throws IOException {
+ assertEquals(v11, getByteCodeVersion(JvmVerification.class));
+ }
+
+ // jvm
+ @Test
+ public void verifyRunningJVMVersionIs11() {
+ final String version = getJavaSpecification();
+ assertEquals(v11.name, version);
+ }
+
+ private static <T> Java getByteCodeVersion(final Class<T> clazz) throws IOException {
+ final InputStream stream =
+ clazz.getClassLoader().getResourceAsStream(clazz.getName().replace(".", "/") + ".class");
+ final byte[] classBytes = IOUtils.toByteArray(stream);
+ final String versionInHexString =
+ Hex.encodeHexString(new byte[] {classBytes[6], classBytes[7]});
+ return versionMapping.get(versionInHexString);
+ }
+
+ private static String getJavaSpecification() {
+ return System.getProperty("java.specification.version");
+ }
+
+ enum Java {
+ v1_1("1.1"),
Review comment:
Same comment as above on removing the non supported versions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11619: Compile and run tests on java 11 for PreCommit portability api
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11619:
URL: https://github.com/apache/beam/pull/11619#issuecomment-625678328
Run JavaPortabilityApi PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org