You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/05/09 03:13:25 UTC
[GitHub] [beam] veblush opened a new pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
veblush opened a new pull request #11651:
URL: https://github.com/apache/beam/pull/11651
This is part of BEAM-8889 to get new GCS connector 2.1.3 after #11315. This is the first version to support gRPC protocol so it has a new dependency against gRPC and it affects the dependency tree of beam SDK transitively. gRPC feature is only available with `use_grpc_for_gcs` experiment flag and the whitelisted GCS buckets.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631695755
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631695524
Having trouble triggering tests at the moment. I think this is good to go when we get following to pass with the latest code.
Run Dataflow ValidatesRunner
Run Java PostCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] veblush commented on a change in pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
veblush commented on a change in pull request #11651:
URL: https://github.com/apache/beam/pull/11651#discussion_r427581972
##########
File path: sdks/java/io/google-cloud-platform/build.gradle
##########
@@ -56,11 +56,13 @@ dependencies {
compile library.java.google_http_client
compile library.java.google_http_client_jackson2
compile library.java.grpc_all
+ compile library.java.grpc_alts
compile library.java.grpc_auth
compile library.java.grpc_core
compile library.java.grpc_context
compile library.java.grpc_grpclb
compile library.java.grpc_netty
+ compile library.java.grpc_netty_shaded
Review comment:
Same as above.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] veblush commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
veblush commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631025815
This is most likely caused by gcsio 2.1.3 dependency to `grpc-netty-shaded` ([ref](https://mvnrepository.com/artifact/com.google.cloud.bigdataoss/gcsio/2.1.3)). Meanwhile, Beam uses `grpc_netty` so it happens to have the same netty library in two different forms.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] veblush commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
veblush commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631065488
It turns out that `grpc-netty` and `grpc-netty-shaded` can coexist. So I'm trying to list those new dependencies in the build file to make gRPC component have the same version.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631695091
Run Dataflow ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631132405
Retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-630351596
Retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #11651:
URL: https://github.com/apache/beam/pull/11651#discussion_r427580442
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -489,6 +490,7 @@ class BeamModulePlugin implements Plugin<Project> {
grpc_protobuf : "io.grpc:grpc-protobuf:$grpc_version",
grpc_protobuf_lite : "io.grpc:grpc-protobuf-lite:$grpc_version",
grpc_netty : "io.grpc:grpc-netty:$grpc_version",
+ grpc_netty_shaded : "io.grpc:grpc-netty-shaded:$grpc_version",
Review comment:
Why do we need both grpc_netty and grpc_netty_shaded ?
##########
File path: sdks/java/io/google-cloud-platform/build.gradle
##########
@@ -56,11 +56,13 @@ dependencies {
compile library.java.google_http_client
compile library.java.google_http_client_jackson2
compile library.java.grpc_all
+ compile library.java.grpc_alts
compile library.java.grpc_auth
compile library.java.grpc_core
compile library.java.grpc_context
compile library.java.grpc_grpclb
compile library.java.grpc_netty
+ compile library.java.grpc_netty_shaded
Review comment:
Ditto.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631748270
whoops
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631252696
Run Python2_PVR_Flink PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631254015
Have you run the linkage checker ?
https://cwiki.apache.org/confluence/display/BEAM/Dependency+Upgrades
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] veblush commented on a change in pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
veblush commented on a change in pull request #11651:
URL: https://github.com/apache/beam/pull/11651#discussion_r427581456
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -489,6 +490,7 @@ class BeamModulePlugin implements Plugin<Project> {
grpc_protobuf : "io.grpc:grpc-protobuf:$grpc_version",
grpc_protobuf_lite : "io.grpc:grpc-protobuf-lite:$grpc_version",
grpc_netty : "io.grpc:grpc-netty:$grpc_version",
+ grpc_netty_shaded : "io.grpc:grpc-netty-shaded:$grpc_version",
Review comment:
This comes from grpc-alts. (grpc-alts is for directpath)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] veblush commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
veblush commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631024519
It appears that it actually has a problem with dependency.
#### Java PostCommit / [testReadEmptySketchFromBigQuery](https://builds.apache.org/job/beam_PostCommit_Java_PR/366/testReport/junit/org.apache.beam.sdk.extensions.zetasketch/BigQueryHllSketchCompatibilityIT/testReadEmptySketchFromBigQuery/)
```
Caused by: java.lang.NoSuchFieldError: NETTY_SHADED
at io.grpc.netty.shaded.io.grpc.netty.NettyClientStream.<clinit>(NettyClientStream.java:59)
at io.grpc.netty.shaded.io.grpc.netty.NettyClientTransport.newStream(NettyClientTransport.java:177)
at io.grpc.internal.MetadataApplierImpl.apply(MetadataApplierImpl.java:70)
at io.grpc.CallCredentials2$1.apply(CallCredentials2.java:61)
at io.grpc.auth.GoogleAuthLibraryCallCredentials$1.onSuccess(GoogleAuthLibraryCallCredentials.java:133)
at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:134)
at io.grpc.auth.GoogleAuthLibraryCallCredentials.applyRequestMetadata(GoogleAuthLibraryCallCredentials.java:110)
at io.grpc.CallCredentials2.applyRequestMetadata(CallCredentials2.java:58)
at io.grpc.internal.CallCredentialsApplyingTransportFactory$CallCredentialsApplyingTransport.newStream(CallCredentialsApplyingTransportFactory.java:108)
at io.grpc.internal.ForwardingConnectionClientTransport.newStream(ForwardingConnectionClientTransport.java:49)
at io.grpc.internal.InternalSubchannel$CallTracingTransport.newStream(InternalSubchannel.java:635)
at io.grpc.internal.DelayedClientTransport$PendingStream.createRealStream(DelayedClientTransport.java:353)
at io.grpc.internal.DelayedClientTransport$PendingStream.access$300(DelayedClientTransport.java:341)
at io.grpc.internal.DelayedClientTransport$5.run(DelayedClientTransport.java:300)
```
#### Java PostCommit / [SpannerWriteIT/testWrite_2](https://builds.apache.org/job/beam_PostCommit_Java_PR/366/testReport/junit/org.apache.beam.sdk.io.gcp.spanner/SpannerWriteIT/testWrite_2/)
```
Caused by: java.lang.NoClassDefFoundError: Could not initialize class io.grpc.netty.shaded.io.grpc.netty.NettyClientStream
at io.grpc.netty.shaded.io.grpc.netty.NettyClientTransport.newStream(NettyClientTransport.java:177)
at io.grpc.internal.MetadataApplierImpl.apply(MetadataApplierImpl.java:70)
at io.grpc.CallCredentials2$1.apply(CallCredentials2.java:61)
at io.grpc.auth.GoogleAuthLibraryCallCredentials$1.onSuccess(GoogleAuthLibraryCallCredentials.java:133)
at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:134)
at io.grpc.auth.GoogleAuthLibraryCallCredentials.applyRequestMetadata(GoogleAuthLibraryCallCredentials.java:110)
at io.grpc.CallCredentials2.applyRequestMetadata(CallCredentials2.java:58)
at io.grpc.internal.CallCredentialsApplyingTransportFactory$CallCredentialsApplyingTransport.newStream(CallCredentialsApplyingTransportFactory.java:108)
at io.grpc.internal.ForwardingConnectionClientTransport.newStream(ForwardingConnectionClientTransport.java:49)
at io.grpc.internal.InternalSubchannel$CallTracingTransport.newStream(InternalSubchannel.java:635)
at io.grpc.internal.DelayedClientTransport$PendingStream.createRealStream(DelayedClientTransport.java:353)
at io.grpc.internal.DelayedClientTransport$PendingStream.access$300(DelayedClientTransport.java:341)
at io.grpc.internal.DelayedClientTransport$5.run(DelayedClientTransport.java:300)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] veblush commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
veblush commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631815316
Thanks for running the test and the result doesn't make sense because this change won't introduce any runtime behavior other than dependency. Is it possible to be caused by flaky environment? Can we rerun the test again?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj merged pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj merged pull request #11651:
URL: https://github.com/apache/beam/pull/11651
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631105532
Retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631105651
Retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-630962985
Run Java PostCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-630351782
Run Dataflow ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631825554
Run Dataflow ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] veblush commented on a change in pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
veblush commented on a change in pull request #11651:
URL: https://github.com/apache/beam/pull/11651#discussion_r427623777
##########
File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##########
@@ -489,6 +490,7 @@ class BeamModulePlugin implements Plugin<Project> {
grpc_protobuf : "io.grpc:grpc-protobuf:$grpc_version",
grpc_protobuf_lite : "io.grpc:grpc-protobuf-lite:$grpc_version",
grpc_netty : "io.grpc:grpc-netty:$grpc_version",
+ grpc_netty_shaded : "io.grpc:grpc-netty-shaded:$grpc_version",
Review comment:
Note that current beam already has it from [gax-grpc](https://mvnrepository.com/artifact/com.google.api/gax-grpc/1.54.0) transitively. This can make sure that all these components are working with the same version.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631695565
LGTM
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] veblush commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
veblush commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-630948467
No performance regression with this change. (Tested with internal tests)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631776883
It looks like Dataflow VR tests are going to fail:
```
15:10:31 org.apache.beam.sdk.transforms.join.CoGroupByKeyTest > testCoGroupByKeyWithWindowing FAILED
15:10:31 java.lang.RuntimeException at CoGroupByKeyTest.java:494
15:10:31 Caused by: java.lang.OutOfMemoryError
15:10:31
15:10:31 org.apache.beam.sdk.transforms.FlattenTest > testFlattenIterablesLists FAILED
15:10:31 java.lang.RuntimeException at FlattenTest.java:270
15:10:31 Caused by: java.lang.OutOfMemoryError
15:10:31
15:10:31 org.apache.beam.sdk.transforms.CreateTest > testCreateWithVoidType FAILED
15:10:31 java.lang.RuntimeException at CreateTest.java:326
15:10:31 Caused by: java.lang.OutOfMemoryError
15:10:31
15:10:31 org.apache.beam.sdk.transforms.ReshuffleTest > testReshuffleAfterSlidingWindows FAILED
15:10:31 java.lang.RuntimeException at ReshuffleTest.java:251
15:10:31 Caused by: java.lang.RuntimeException
15:10:31 Caused by: java.lang.IllegalArgumentException
15:10:31 Caused by: java.lang.OutOfMemoryError
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631252638
Run Python2_PVR_Flink PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] veblush commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
veblush commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631882467
Thanks, Brian and Chamikara!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-630351692
Run Java PostCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631745259
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631694934
Run Dataflow ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-630962850
Run Java PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631748120
Run Java PostCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] veblush commented on pull request #11651: [BEAM-8889] Upgrades gcsio to 2.1.3
Posted by GitBox <gi...@apache.org>.
veblush commented on pull request #11651:
URL: https://github.com/apache/beam/pull/11651#issuecomment-631632732
This is the result of linkage chekcer. (it was challenging because it seems to require Java8 strictly to finish it smoothly)
```
# /bin/bash sdks/java/build-tools/beam-linkage-check.sh
(omitted)
Wed May 20 17:57:11 UTC 2020: Done: 0
No new linkage errors
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org