You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Brian Hulette (Jira)" <ji...@apache.org> on 2021/04/21 23:53:00 UTC
[jira] [Assigned] (BEAM-12207) Python PostCommits failing
portableWordCountSparkRunnerBatch
[ https://issues.apache.org/jira/browse/BEAM-12207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Hulette reassigned BEAM-12207:
------------------------------------
Assignee: Ismaël Mejía
> Python PostCommits failing portableWordCountSparkRunnerBatch
> ------------------------------------------------------------
>
> Key: BEAM-12207
> URL: https://issues.apache.org/jira/browse/BEAM-12207
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
> Reporter: Brian Hulette
> Assignee: Ismaël Mejía
> Priority: P1
> Labels: currently-failing
>
> The first run in the Python 3.8 suite with this particular failure appears to be https://ci-beam.apache.org/job/beam_PostCommit_Python38/1104
> {code}
> 06:26:36 OK (SKIP=6)
> 06:26:36
> 06:26:38 FAILURE: Build failed with an exception.
> 06:26:38
> 06:26:38 * What went wrong:
> 06:26:38 Execution failed for task ':sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch'.
> 06:26:38 > Process 'command 'sh'' finished with non-zero exit value 1
> 06:26:38
> 06:26:38 * Try:
> 06:26:38 Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
> 06:26:38
> 06:26:38 * Get more help at https://help.gradle.org
> 06:26:38
> 06:26:38 Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
> 06:26:38 Use '--warning-mode all' to show the individual deprecation warnings.
> 06:26:38 See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
> 06:26:38
> 06:26:38 BUILD FAILED in 1h 26m 11s
> {code}
> {code}
> 05:25:34 > Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch FAILED
> 05:25:34 INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:42005
> 05:25:34 WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
> 05:25:34 INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.30.0.dev
> 05:25:34 INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f71dcdf5a60> ====================
> 05:25:34 INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f71dcdfa1f0> ====================
> 05:25:34 INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python38/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.30.0-SNAPSHOT.jar' '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempgetxx5hw/artifactsw4djif3r' '--job-port' '42069' '--artifact-port' '0' '--expansion-port' '0']
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:46287'
> 05:25:34 WARNING:root:Waiting for grpc channel to be ready at localhost:42069.
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:30 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:41861'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:30 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:42069'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:30 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
> 05:25:34 WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:31 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_7c9a3a39-f855-455c-8f36-bcd6bfd8152e.'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:31 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_7c9a3a39-f855-455c-8f36-bcd6bfd8152e.ref_Environment_default_environment_1.'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:31 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_7c9a3a39-f855-455c-8f36-bcd6bfd8152e.null.'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:31 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_7c9a3a39-f855-455c-8f36-bcd6bfd8152e.'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:32 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0416122532-da1b901c_d29eb552-6c06-4531-b0bd-fd234973ff23'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:32 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0416122532-da1b901c_d29eb552-6c06-4531-b0bd-fd234973ff23'
> 05:25:34 INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
> 05:25:34 with Pipeline() as p:
> 05:25:34 p.apply(..)
> 05:25:34 This ensures that the pipeline finishes before this program exits.
> 05:25:34 INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
> 05:25:34 INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
> 05:25:34 INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:33 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation BeamApp-jenkins-0416122532-da1b901c_d29eb552-6c06-4531-b0bd-fd234973ff23.'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'java.lang.NullPointerException'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat java.lang.Thread.run(Thread.java:748)'
> 05:25:34 ERROR:root:java.lang.NullPointerException
> 05:25:34 INFO:apache_beam.runners.portability.portable_runner:Job state changed to FAILED
> 05:25:34 Traceback (most recent call last):
> 05:25:34 File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
> 05:25:34 return _run_code(code, main_globals, None,
> 05:25:34 File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
> 05:25:34 exec(code, run_globals)
> 05:25:34 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python38/src/sdks/python/apache_beam/examples/wordcount.py", line 94, in <module>
> 05:25:34 run()
> 05:25:34 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python38/src/sdks/python/apache_beam/examples/wordcount.py", line 89, in run
> 05:25:34 output | 'Write' >> WriteToText(known_args.output)
> 05:25:34 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python38/src/sdks/python/apache_beam/pipeline.py", line 580, in __exit__
> 05:25:34 self.result.wait_until_finish()
> 05:25:34 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python38/src/sdks/python/apache_beam/runners/portability/portable_runner.py", line 602, in wait_until_finish
> 05:25:34 raise self._runtime_exception
> 05:25:34 RuntimeError: Pipeline BeamApp-jenkins-0416122532-da1b901c_d29eb552-6c06-4531-b0bd-fd234973ff23 failed in state FAILED: java.lang.NullPointerException
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)