You are viewing a plain text version of this content. The canonical link for it is here.

Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2020/05/22 18:28:06 UTC

Build failed in Jenkins: beam_PostCommit_Py_ValCont #6368

See <https://builds.apache.org/job/beam_PostCommit_Py_ValCont/6368/display/redirect?page=changes>

Changes:

[github] [BEAM-10051] Move closed reader check after sentinel. (#11768)

[github] [BEAM-10056] Fix validation for struct CoGBKs (#11782)


------------------------------------------
[...truncated 769.29 KB...]
copying apache_beam/utils/timestamp.py -> apache-beam-2.23.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.23.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.23.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.23.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.23.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.23.0.dev0/apache_beam/utils
Writing apache-beam-2.23.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.23.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz

# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
  --attr ValidatesContainer \
  --nologcapture \
  --processes=1 \
  --process-timeout=900 \
  --with-xunitmp \
  --xunitmp-file=$XUNIT_FILE \
  --ignore-files '.*py3\d?\.py$' \
  --test-pipeline-options=" \
    --runner=TestDataflowRunner \
    --project=$PROJECT \
    --region=$REGION \
    --worker_harness_container_image=$CONTAINER:$TAG \
    --staging_location=$GCS_LOCATION/staging-validatesrunner-test \
    --temp_location=$GCS_LOCATION/temp-validatesrunner-test \
    --output=$GCS_LOCATION/output \
    --sdk_location=$SDK_LOCATION \
    --num_workers=1"
setup.py:262: UserWarning: You are using Apache Beam with Python 2. New releases of Apache Beam will soon support Python 3 only.
  'You are using Apache Beam with Python 2. '
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/setuptools/dist.py>:476: UserWarning: Normalizing '2.23.0.dev' to '2.23.0.dev0'
  normalized_version,
running nosetests
running egg_info
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:241: UserWarning: Installing grpcio-tools is recommended for development.
  warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/__init__.py>:82: UserWarning: You are using Apache Beam with Python 2. New releases of Apache Beam will soon support Python 3 only.
  'You are using Apache Beam with Python 2. '
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
WARNING:apache_beam.runners.interactive.interactive_environment:Interactive Beam requires Python 3.5.3+.
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ERROR

======================================================================
ERROR: test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py",> line 812, in run
    test(orig)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",> line 45, in __call__
    return self.run(*arg, **kwarg)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",> line 133, in run
    self.runTest(result)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",> line 151, in runTest
    test(result)
  File "/usr/lib/python2.7/unittest/case.py", line 393, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 54, in test_wordcount_fnapi_it
    self._run_wordcount_it(wordcount.run, experiment='beam_fn_api')
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 91, in _run_wordcount_it
    save_main_session=False)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 121, in run
    result = p.run()
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 512, in run
    self._options).run(False)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 525, in run
    return self.runner.run_pipeline(self, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 76, in run_pipeline
    self.wait_until_in_state(PipelineState.CANCELLED)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 96, in wait_until_in_state
    job_state = self.result.state
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1617, in state
    self._update_job()
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1570, in _update_job
    self._job = self._runner.dataflow_client.get_job(self.job_id())
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/utils/retry.py",> line 236, in wrapper
    return fun(*args, **kwargs)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 838, in get_job
    response = self._client.projects_locations_jobs.Get(request)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py",> line 699, in Get
    config, request, global_params=global_params)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/apitools/base/py/base_api.py",> line 729, in _RunMethod
    http, http_request, **opts)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/apitools/base/py/http_wrapper.py",> line 350, in MakeRequest
    check_response_func=check_response_func)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/apitools/base/py/http_wrapper.py",> line 400, in _MakeRequestNoRetry
    redirections=redirections, connection_type=connection_type)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/oauth2client/transport.py",> line 169, in new_request
    redirections, connection_type)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/oauth2client/transport.py",> line 169, in new_request
    redirections, connection_type)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/httplib2/__init__.py",> line 2189, in request
    cachekey,
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/httplib2/__init__.py",> line 1845, in _request
    conn, request_uri, method, body, headers
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/httplib2/__init__.py",> line 1786, in _conn_request
    response = conn.getresponse()
  File "/usr/lib/python2.7/httplib.py", line 1165, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 463, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 419, in _read_status
    line = self.fp.readline(_MAXLINE + 1)
  File "/usr/lib/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)
  File "/usr/lib/python2.7/ssl.py", line 756, in recv
    return self.read(buflen)
  File "/usr/lib/python2.7/ssl.py", line 643, in read
    v = self._sslobj.read(len)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py",> line 276, in signalhandler
    raise TimedOutException()
TimedOutException: 'test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT)'

======================================================================
ERROR: test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py",> line 812, in run
    test(orig)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",> line 45, in __call__
    return self.run(*arg, **kwarg)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",> line 133, in run
    self.runTest(result)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",> line 151, in runTest
    test(result)
  File "/usr/lib/python2.7/unittest/case.py", line 393, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py",> line 57, in test_metrics_fnapi_it
    result = self.run_pipeline(experiment='beam_fn_api')
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py",> line 45, in run_pipeline
    return dataflow_exercise_metrics_pipeline.apply_and_run(p)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline.py",> line 167, in apply_and_run
    | 'm_out' >> beam.FlatMap(
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 562, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 654, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 153, in apply
    return super(DataflowRunner, self).apply(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/core.py",> line 2625, in expand
    | Map(coder.decode).with_output_types(self.get_output_type()))
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/core.py",> line 1480, in FlatMap
    pardo = ParDo(CallableWrapperDoFn(fn), *args, **kwargs)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/core.py",> line 1246, in __init__
    self._signature = DoFnSignature(self.fn)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/common.py",> line 287, in __init__
    self._validate()
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/common.py",> line 312, in _validate
    self._validate_stateful_dofn()
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/common.py",> line 339, in _validate_stateful_dofn
    userstate.validate_stateful_dofn(self.do_fn)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/userstate.py",> line 272, in validate_stateful_dofn
    all_state_specs, all_timer_specs = get_dofn_specs(dofn)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/userstate.py",> line 243, in get_dofn_specs
    method = MethodWrapper(dofn, method_name)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/common.py",> line 183, in __init__
    method_name)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/core.py",> line 339, in get_function_arguments
    return get_function_args_defaults(f)
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/core.py",> line 369, in get_function_args_defaults
    p.default for p in signature.parameters.values()
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/funcsigs/__init__.py",> line 601, in parameters
    return OrderedDict(self._parameters.items())
  File "/usr/lib/python2.7/collections.py", line 127, in items
    return [(key, self[key]) for key in self]
  File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py",> line 276, in signalhandler
    raise TimedOutException()
TimedOutException: 'test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest)'

----------------------------------------------------------------------
XML: nosetests-beam_python2.7_sdk.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 2 tests in 904.535s

FAILED (errors=2)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk:20200522-180024
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk@sha256:dac25392665b7804c78ed4eb1bc4daeb50a4c75e3d9ed7c03a117df4a2006c7d
Deleted: sha256:1c898ec57b424dfe37e640212a7b45cd2e2ec09cf1ec355cb7167d3fa9f0ed43
Deleted: sha256:cc9c938fbaa9395804059fa17a34c4ec95d52cb4037cea9fa8baaabebad5ba71
Deleted: sha256:4475e39f77f9e7a144b15c99a7464994e13d8c4927533e218f01e81df6dfcab9
Deleted: sha256:68dc6bd2a60a94f374ea7167aa29657f6f8593b8debb69645b3d3133893c1151
Deleted: sha256:b24d293314712c9a4c75360004ddf5731257c523f649d27233d5fd110d03b16b
Deleted: sha256:3003c1360e33302766e9406e6d865b4282c3dff1fed3b039d3a17afdee6ad981
Deleted: sha256:19122e000fe116cb99c04f628071fe1727fb084218fd3ea25f795a8dba005981
Deleted: sha256:a86caf2335d91e681e3ad3a6a3ea6a63a5d81149008cf453cb0667bb3fb89b9a
Deleted: sha256:427010fa355c0ed871620a11e1cb5bf43f155d11c9ff945be9219c8d5a118554
Deleted: sha256:64a2248699a94c6df1baecbbae1482402d9d94334a5b936bbbe7dce23e231ac1
Deleted: sha256:9f0463e2a3df425e79cff3bb310ff417b2962b34069d4566e3d713a7748bff3c
Deleted: sha256:71dc8b30e3dec2e9ba7af0fc9c30defa67ce5d9fa48ff04e840cd89fb78dafb5
Deleted: sha256:83a62233e778103a0c4afbcf692f838a141ea90ddefc5f268ef0ae7694b0d171
Deleted: sha256:c3f50fcd006a97af483a575de41bf35c7d8a80fc82aef9f61b26ed397a2e4bf3
Deleted: sha256:b04cfbcd258ea11f27f33d30d105a62684cbd46917355ec1e631da2759976e1d
Deleted: sha256:1bbd5b430949d5cb89854f26d7dc3e5460b0436b2687030b9f2af03436c7ddcf
Deleted: sha256:9f37a10f6a73623c480372fa17a38b31826c1c26dfbfaf997bac1f53726efe0b
Deleted: sha256:0e7183db3d7957c5f1125a789734b623e9d9db3507ac847cae7979c805934f35
Deleted: sha256:6d841015e1b8625e0fb6df7397a35225adea2d55d3e3c265c1d59d178239aada
Deleted: sha256:b2b88f43c0ba5eba7d456057abccf0b25be8ba7afb6b7101d06af3330c3ead49
Deleted: sha256:a1f079f273e857ae6862f3224a7ffdbe7944223848b406867276c7e40c347ff7
Deleted: sha256:32c5c860560167ab52c37de287d91cffbc0fdb19a42b1fac676e4777e4f04ec4
Deleted: sha256:d714f8649cb4c7c89d55f688c21c96d6b9a6dc0d83cb2a1e3f61630436e9def1
Deleted: sha256:f7dc4fb4a23387b71684896c382d033023ef3c704babd7583ae0449d7393d156
Deleted: sha256:a48b8b488f60395cd3533743ace2eb8e0560683ecd1c9365d30cb3d5ccfa9cdc
Deleted: sha256:8bc49e6c3f1f262b2cf942cd58925609a1ff88865ef239e5d5a7c60cf94b8710
Deleted: sha256:c6040b9decfb242c9017972395696a4d2bd7498f710a917205691789cf2de07f
Deleted: sha256:dd0dc2dab3bb4e7615e87e670073ffa6baeef51eb1b759b1c62e89ec1363ec86
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk@sha256:dac25392665b7804c78ed4eb1bc4daeb50a4c75e3d9ed7c03a117df4a2006c7d
  Associated tags:
 - 20200522-180024
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk:20200522-180024
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk:20200522-180024].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk@sha256:dac25392665b7804c78ed4eb1bc4daeb50a4c75e3d9ed7c03a117df4a2006c7d].
Removed the container
Build step 'Execute shell' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org