You are viewing a plain text version of this content. The canonical link for it is here.

Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2020/01/07 16:51:33 UTC

Build failed in Jenkins: beam_PostCommit_Py_VR_Dataflow #5454

See <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/5454/display/redirect?page=changes>

Changes:

[iemejia] [BEAM-8701] Remove unused commons-io_1x dependency

[iemejia] [BEAM-8701] Update commons-io to version 2.6


------------------------------------------
[...truncated 703.67 KB...]
        "serialized_fn": "ref_AppliedPTransform_Start/Map(decode)_13",
        "user_name": "Start/Map(decode)"
      }
    },
    {
      "kind": "ParallelDo",
      "name": "s9",
      "properties": {
        "display_data": [
          {
            "key": "fn",
            "label": "Transform Function",
            "namespace": "apache_beam.transforms.core.ParDo",
            "shortValue": "CallSequenceEnforcingDoFn",
            "type": "STRING",
            "value": "apache_beam.transforms.dofn_lifecycle_test.CallSequenceEnforcingDoFn"
          }
        ],
        "non_parallel_inputs": {},
        "output_info": [
          {
            "encoding": {
              "@type": "kind:windowed_value",
              "component_encodings": [
                {
                  "@type": "FastPrimitivesCoder$eNprYE5OLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqYIfgYGhvi0xJycpMTk7HiwlkJ8pgVkJmfnpEJNYQGawlpbyJZUnKQHACYlLgM=",
                  "component_encodings": [
                    {
                      "@type": "FastPrimitivesCoder$eNprYE5OLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqYIfgYGhvi0xJycpMTk7HiwlkJ8pgVkJmfnpEJNYQGawlpbyJZUnKQHACYlLgM=",
                      "component_encodings": [],
                      "pipeline_proto_coder_id": "ref_Coder_FastPrimitivesCoder_6"
                    },
                    {
                      "@type": "FastPrimitivesCoder$eNprYE5OLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqYIfgYGhvi0xJycpMTk7HiwlkJ8pgVkJmfnpEJNYQGawlpbyJZUnKQHACYlLgM=",
                      "component_encodings": [],
                      "pipeline_proto_coder_id": "ref_Coder_FastPrimitivesCoder_6"
                    }
                  ],
                  "is_pair_like": true,
                  "pipeline_proto_coder_id": "ref_Coder_FastPrimitivesCoder_6"
                },
                {
                  "@type": "kind:global_window"
                }
              ],
              "is_wrapper": true
            },
            "output_name": "out",
            "user_name": "Do.out"
          }
        ],
        "parallel_input": {
          "@type": "OutputReference",
          "output_name": "out",
          "step_name": "s8"
        },
        "serialized_fn": "ref_AppliedPTransform_Do_14",
        "user_name": "Do"
      }
    }
  ],
  "type": "JOB_TYPE_STREAMING"
}
apache_beam.runners.dataflow.internal.apiclient: INFO: Create job: <Job
 createTime: '2020-01-07T16:20:51.234957Z'
 currentStateTime: '1970-01-01T00:00:00Z'
 id: '2020-01-07_08_20_49-16263618190933252273'
 location: 'us-central1'
 name: 'beamapp-jenkins-0107162020-675235'
 projectId: 'apache-beam-testing'
 stageStates: []
 startTime: '2020-01-07T16:20:51.234957Z'
 steps: []
 tempFiles: []
 type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
apache_beam.runners.dataflow.internal.apiclient: INFO: Created job with id: [2020-01-07_08_20_49-16263618190933252273]
apache_beam.runners.dataflow.internal.apiclient: INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_20_49-16263618190933252273?project=apache-beam-testing
apache_beam.runners.dataflow.test_dataflow_runner: WARNING: Waiting indefinitely for streaming job.
apache_beam.runners.dataflow.dataflow_runner: INFO: Job 2020-01-07_08_20_49-16263618190933252273 is in state JOB_STATE_RUNNING
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:53.525Z: JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service Account.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:54.306Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-4 in us-central1-f.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.162Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.165Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.172Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.184Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.186Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.192Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.214Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.217Z: JOB_MESSAGE_DETAILED: Fusing consumer Start/FlatMap(<lambda at core.py:2591>) into Start/Impulse
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.219Z: JOB_MESSAGE_DETAILED: Fusing consumer Start/MaybeReshuffle/Reshuffle/AddRandomKeys into Start/FlatMap(<lambda at core.py:2591>)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.221Z: JOB_MESSAGE_DETAILED: Fusing consumer Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps) into Start/MaybeReshuffle/Reshuffle/AddRandomKeys
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.224Z: JOB_MESSAGE_DETAILED: Fusing consumer Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/WriteStream into Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.227Z: JOB_MESSAGE_DETAILED: Fusing consumer Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/MergeBuckets into Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/ReadStream
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.230Z: JOB_MESSAGE_DETAILED: Fusing consumer Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps) into Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/MergeBuckets
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.232Z: JOB_MESSAGE_DETAILED: Fusing consumer Start/MaybeReshuffle/Reshuffle/RemoveRandomKeys into Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.235Z: JOB_MESSAGE_DETAILED: Fusing consumer Start/Map(decode) into Start/MaybeReshuffle/Reshuffle/RemoveRandomKeys
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.237Z: JOB_MESSAGE_DETAILED: Fusing consumer Do into Start/Map(decode)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.249Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.272Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.285Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.433Z: JOB_MESSAGE_DEBUG: Executing wait step start2
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.475Z: JOB_MESSAGE_DEBUG: Starting worker pool setup.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:55.480Z: JOB_MESSAGE_BASIC: Starting 1 workers...
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:57.806Z: JOB_MESSAGE_BASIC: Executing operation Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/ReadStream+Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/MergeBuckets+Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+Start/MaybeReshuffle/Reshuffle/RemoveRandomKeys+Start/Map(decode)+Do
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:20:57.806Z: JOB_MESSAGE_BASIC: Executing operation Start/Impulse+Start/FlatMap(<lambda at core.py:2591>)+Start/MaybeReshuffle/Reshuffle/AddRandomKeys+Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/WriteStream
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:21:16.999Z: JOB_MESSAGE_WARNING: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:21:26.314Z: JOB_MESSAGE_DEBUG: Executing input step topology_init_attach_disk_input_step
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:21:26.314Z: JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service Account.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:21:27.007Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-4 in us-central1-f.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:21:37.666Z: JOB_MESSAGE_ERROR: Workflow failed.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:21:37.692Z: JOB_MESSAGE_BASIC: Finished operation Start/Impulse+Start/FlatMap(<lambda at core.py:2591>)+Start/MaybeReshuffle/Reshuffle/AddRandomKeys+Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/WriteStream
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:21:37.692Z: JOB_MESSAGE_BASIC: Finished operation Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/ReadStream+Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/MergeBuckets+Start/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+Start/MaybeReshuffle/Reshuffle/RemoveRandomKeys+Start/Map(decode)+Do
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:21:37.861Z: JOB_MESSAGE_DETAILED: Cleaning up.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:21:37.886Z: JOB_MESSAGE_BASIC: Worker pool stopped.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:21:37.890Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:21:37.892Z: JOB_MESSAGE_DEBUG: Starting worker pool teardown.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:21:37.898Z: JOB_MESSAGE_BASIC: Stopping worker pool...
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:21:40.108Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-01-07T16:23:11.357Z: JOB_MESSAGE_BASIC: Worker pool stopped.
apache_beam.runners.dataflow.dataflow_runner: INFO: Job 2020-01-07_08_20_49-16263618190933252273 is in state JOB_STATE_FAILED
--------------------- >> end captured logging << ---------------------
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_20_50-18099685570275384164?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_28_06-14831978760780519172?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_35_20-995801420735531225?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_20_48-15109189360554525070?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_28_48-15590665062021303441?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_36_13-12765219893115240145?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_20_51-502064313294775483?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_28_42-11083980542379184386?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_20_49-16263618190933252273?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_23_50-1000870339621491635?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_31_04-6357429243989867734?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_20_48-1149421288009276600?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_28_36-8642364254578023337?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_36_17-13019833417344582712?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_44_12-10710463749163565756?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_20_48-15948994369707590329?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_28_41-5699669252959694693?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_36_15-5278237671258441514?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_20_49-17033792202151930755?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_29_14-4271285007400884745?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_20_48-4397119725350443555?project=apache-beam-testing
Worker logs: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-01-07_08_28_45-4482720574956939940?project=apache-beam-testing

----------------------------------------------------------------------
XML: nosetests-validatesRunnerStreamingTests-df-py35.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 26 tests in 1873.258s

FAILED (errors=1)

> Task :sdks:python:test-suites:dataflow:py35:validatesRunnerStreamingTests FAILED

FAILURE: Build completed with 6 failures.

1: Task failed with an exception.
-----------
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/py2/build.gradle'> line: 107

* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py2:validatesRunnerBatchTests'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/py37/build.gradle'> line: 111

* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/py36/build.gradle'> line: 78

* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesRunnerBatchTests'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/py2/build.gradle'> line: 130

* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py2:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

5: Task failed with an exception.
-----------
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/py37/build.gradle'> line: 134

* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

6: Task failed with an exception.
-----------
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/py35/build.gradle'> line: 101

* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py35:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 1h 15m 57s
74 actionable tasks: 73 executed, 1 from cache

Publishing build scan...
https://gradle.com/s/4aftkgjndowbu

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org