You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "shashi reddy (JIRA)" <ji...@apache.org> on 2019/03/18 15:23:00 UTC
[jira] [Created] (BEAM-6849) SpannerIO.Write "Part Running" at
schemaView, never completes with Wait.on
shashi reddy created BEAM-6849:
----------------------------------
Summary: SpannerIO.Write "Part Running" at schemaView, never completes with Wait.on
Key: BEAM-6849
URL: https://issues.apache.org/jira/browse/BEAM-6849
Project: Beam
Issue Type: Bug
Components: io-java-gcp
Affects Versions: 2.10.0
Reporter: shashi reddy
Fix For: Not applicable
Here is a gist:
[https://gist.github.com/sgireddy/859261f6335d303ec08049fcb2a7972a]
I am using scio wrapper but I don't think this is scio issue given I am using direct beam call and I see "Schema View" task is blocking on dataflow DAG.
Here is my build.sbt:
name := "dataflow-spanner-template"
version := "0.1"
scalaVersion := "2.11.12"
val beamVersion = "2.10.0"
val scioVersion = "0.7.2"
libraryDependencies ++= Seq(
"org.apache.beam" % "beam-runners-google-cloud-dataflow-java" % beamVersion,
"com.spotify" %% "scio-core" % scioVersion,
"com.spotify" %% "scio-spanner" % scioVersion,
"com.google.cloud" % "google-cloud-spanner" % "1.4.0"
)
I saw the following exception with the above gist (but not in my original code).
java.lang.RuntimeException: org.apache.beam.sdk.util.UserCodeException: java.lang.NoSuchMethodError: com.google.api.gax.grpc.InstantiatingGrpcChannelProvider$Builder.setMaxInboundMetadataSize(Ljava/lang/Integer;)Lcom/google/api/gax/grpc/InstantiatingGrpcChannelProvider$Builder; org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:194) org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:165) org.apache.beam.runners.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:63) org.apache.beam.runners.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:50) org.apache.beam.runners.dataflow.worker.graph.Networks.replaceDirectedNetworkNodes(Networks.java:87) org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory.create(IntrinsicMapTaskExecutorFactory.java:125) org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:1149) org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.access$1000(StreamingDataflowWorker.java:144) org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker$6.run(StreamingDataflowWorker.java:972) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) Caused by: org.apache.beam.sdk.util.UserCodeException: java.lang.NoSuchMethodError: com.google.api.gax.grpc.InstantiatingGrpcChannelProvider$Builder.setMaxInboundMetadataSize(Ljava/lang/Integer;)Lcom/google/api/gax/grpc/InstantiatingGrpcChannelProvider$Builder; org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:34) org.apache.beam.sdk.io.gcp.spanner.ReadSpannerSchema$DoFnInvoker.invokeSetup(Unknown Source) org.apache.beam.runners.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.deserializeCopy(DoFnInstanceManagers.java:80) org.apache.beam.runners.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.peek(DoFnInstanceManagers.java:62) org.apache.beam.runners.dataflow.worker.UserParDoFnFactory.create(UserParDoFnFactory.java:95) org.apache.beam.runners.dataflow.worker.DefaultParDoFnFactory.create(DefaultParDoFnFactory.java:75) org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory.createParDoOperation(IntrinsicMapTaskExecutorFactory.java:264) org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory.access$000(IntrinsicMapTaskExecutorFactory.java:86) org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:183) org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:165) org.apache.beam.runners.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:63) org.apache.beam.runners.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:50) org.apache.beam.runners.dataflow.worker.graph.Networks.replaceDirectedNetworkNodes(Networks.java:87) org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory.create(IntrinsicMapTaskExecutorFactory.java:125) org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:1149) org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.access$1000(StreamingDataflowWorker.java:144) org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker$6.run(StreamingDataflowWorker.java:972) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NoSuchMethodError: com.google.api.gax.grpc.InstantiatingGrpcChannelProvider$Builder.setMaxInboundMetadataSize(Ljava/lang/Integer;)Lcom/google/api/gax/grpc/InstantiatingGrpcChannelProvider$Builder; com.google.cloud.spanner.spi.v1.GapicSpannerRpc.<init>(GapicSpannerRpc.java:181) com.google.cloud.spanner.SpannerOptions$DefaultSpannerRpcFactory.create(SpannerOptions.java:71) com.google.cloud.spanner.SpannerOptions$DefaultSpannerRpcFactory.create(SpannerOptions.java:66) com.google.cloud.ServiceOptions.getRpc(ServiceOptions.java:512) com.google.cloud.spanner.SpannerOptions.getSpannerRpcV1(SpannerOptions.java:270) com.google.cloud.spanner.SpannerImpl.<init>(SpannerImpl.java:178) com.google.cloud.spanner.SpannerOptions$DefaultSpannerFactory.create(SpannerOptions.java:61) com.google.cloud.spanner.SpannerOptions$DefaultSpannerFactory.create(SpannerOptions.java:56) com.google.cloud.ServiceOptions.getService(ServiceOptions.java:499) org.apache.beam.sdk.io.gcp.spanner.SpannerConfig.connectToSpanner(SpannerConfig.java:158) org.apache.beam.sdk.io.gcp.spanner.ReadSpannerSchema.setup(ReadSpannerSchema.java:42)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)