[jira] [Commented] (BEAM-13569) Python postcommit failing portableWordCountSparkRunnerBatch

["Kyle Weaver (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-13569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17466640#comment-17466640 ] 

Kyle Weaver commented on BEAM-13569:
------------------------------------

Python 3.6 postcommit started failing after https://github.com/apache/beam/pull/16308 with this error:

16:02:41 Could not determine the dependencies of task ':runners:google-cloud-dataflow-java:worker:shadowJar'.
16:02:41 > Cannot change dependencies of dependency configuration ':runners:spark:2:compileOnly' after it has been included in dependency resolution.

https://ci-beam.apache.org/job/beam_PostCommit_Python36/4751/

----------

Then after https://github.com/apache/beam/pull/16319, portableWordCountSparkRunnerBatch fails with the error above. 

https://ci-beam.apache.org/job/beam_PostCommit_Python36/4758/



> Python postcommit failing portableWordCountSparkRunnerBatch
> -----------------------------------------------------------
>
>                 Key: BEAM-13569
>                 URL: https://issues.apache.org/jira/browse/BEAM-13569
>             Project: Beam
>          Issue Type: Sub-task
>          Components: runner-spark, test-failures
>            Reporter: Kyle Weaver
>            Assignee: Kyle Weaver
>            Priority: P1
>
> Guessing this was due to recent changes such as for the gradle upgrade (BEAM-13430).
> Execution failed for task ':sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch'.
> 'Exception in thread "grpc-default-executor-0" java.lang.NoClassDefFoundError: org/apache/spark/streaming/api/java/JavaStreamingListener '
> '\tat org.apache.beam.runners.spark.SparkJobInvoker.createJobInvocation(SparkJobInvoker.java:101)'
> '\tat org.apache.beam.runners.spark.SparkJobInvoker.invokeWithExecutor(SparkJobInvoker.java:82)'
> '\tat org.apache.beam.runners.jobsubmission.JobInvoker.invoke(JobInvoker.java:48)'
> '\tat org.apache.beam.runners.jobsubmission.InMemoryJobService.run(InMemoryJobService.java:246)'
> '\tat org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:948)'
> '\tat org.apache.beam.vendor.grpc.v1p36p0.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCall s.java:182)'
> '\tat org.apache.beam.vendor.grpc.v1p36p0.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.ja va:35)'
> '\tat org.apache.beam.vendor.grpc.v1p36p0.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)'
> '\tat org.apache.beam.vendor.grpc.v1p36p0.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(Forwarding ServerCallListener.java:40)'
> '\tat org.apache.beam.vendor.grpc.v1p36p0.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)'
> '\tat org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331)'
> '\tat org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInConte xt(ServerImpl.java:797)'
> '\tat org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)'
> '\tat org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)'
> '\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)'
> '\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)'
> '\tat java.lang.Thread.run(Thread.java:748)'
> 'Caused by: java.lang.ClassNotFoundException: org.apache.spark.streaming.api.java.JavaStreamingListener'
> '\tat java.net.URLClassLoader.findClass(URLClassLoader.java:382)'



--
This message was sent by Atlassian Jira
(v8.20.1#820001)