You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2017/07/26 21:37:14 UTC
Build failed in Jenkins: beam_PostCommit_Python_Verify #2806
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2806/display/redirect>
------------------------------------------
[...truncated 568.35 KB...]
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
======================================================================
ERROR: test_par_do_with_multiple_outputs_and_using_return (apache_beam.transforms.ptransform_test.PTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/ptransform_test.py",> line 236, in test_par_do_with_multiple_outputs_and_using_return
pipeline.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/testing/test_pipeline.py",> line 96, in run
result = super(TestPipeline, self).run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 319, in run
self.to_runner_api(), self.runner, self._options).run(False)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 272, in run
self.dataflow_client.create_job(self.job), self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/retry.py",> line 168, in wrapper
return fun(*args, **kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 438, in create_job
self.create_job_description(job)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 461, in create_job_description
job.options, file_copy=self._gcs_file_copy)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",> line 327, in stage_job_resources
setup_options.requirements_file, requirements_cache_path)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",> line 261, in _populate_requirements_cache
processes.check_call(cmd_args)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/processes.py",> line 44, in check_call
return subprocess.check_call(*args, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/bin/python',> '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--no-binary', ':all:']' returned non-zero exit status 2
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by Some Numbers/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/FlatMap(some_fn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Unkey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/FlatMap(some_fn) (tag odd): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Unkey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/FlatMap(some_fn) (tag even): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Unkey (tag None): refcount: 1 => 0
root: DEBUG: Connecting using Google Application Default Credentials.
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726211510-354687.1501103710.354912/requirements.txt...
oauth2client.transport: INFO: Attempting refresh to obtain initial access_token
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726211510-354687.1501103710.354912/requirements.txt
root: INFO: Executing command: ['<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/bin/python',> '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--no-binary', ':all:']
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/ptransform_test.py",> line 220, in test_par_do_with_multiple_outputs_and_using_yield
pipeline.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/testing/test_pipeline.py",> line 96, in run
result = super(TestPipeline, self).run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 319, in run
self.to_runner_api(), self.runner, self._options).run(False)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 272, in run
self.dataflow_client.create_job(self.job), self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/retry.py",> line 168, in wrapper
return fun(*args, **kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 438, in create_job
self.create_job_description(job)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 461, in create_job_description
job.options, file_copy=self._gcs_file_copy)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",> line 327, in stage_job_resources
setup_options.requirements_file, requirements_cache_path)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",> line 261, in _populate_requirements_cache
processes.check_call(cmd_args)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/processes.py",> line 44, in check_call
return subprocess.check_call(*args, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/bin/python',> '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--no-binary', ':all:']' returned non-zero exit status 2
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by Some Numbers/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/ParDo(SomeDoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Unkey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/ParDo(SomeDoFn) (tag odd): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Unkey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/ParDo(SomeDoFn) (tag even): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Unkey (tag None): refcount: 1 => 0
root: DEBUG: Connecting using Google Application Default Credentials.
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726211510-383046.1501103710.383246/requirements.txt...
oauth2client.transport: INFO: Attempting refresh to obtain initial access_token
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726211510-383046.1501103710.383246/requirements.txt
root: INFO: Executing command: ['<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/bin/python',> '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--no-binary', ':all:']
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 15 tests in 1324.443s
FAILED (errors=2)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_14_15_30-12209507452459781415?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_14_20_34-768316653688620561?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_14_26_23-7475628260501990158?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_14_31_32-8491151589855664171?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_14_15_42-16693350550771445813?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_14_21_56-1335614021495420801?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_14_27_34-3696300777918439775?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_14_15_37-18273955232876906653?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_14_20_59-16315884215728650345?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_14_26_15-15593042113350978822?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_14_15_33-638823764017045293?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_14_21_13-12736407222899122865?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_14_26_33-1390602188084649174?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PostCommit_Python_Verify #2846
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2846/display/redirect?page=changes>
Changes:
[tgroh] Remove References to CloudObject from the Java Harness
------------------------------------------
[...truncated 559.75 KB...]
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_par_do_with_multiple_outputs_and_using_return (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Downloading setuptools-36.2.7.zip (716kB)
Saved /tmp/dataflow-requirements-cache/setuptools-36.2.7.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.7.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.7.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1373.001s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_15_28_53-1144589655106184741?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_15_34_42-10926770874638307476?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_15_40_30-6559990572926952066?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_15_46_10-3498744298558941624?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_15_28_54-10570843170346952609?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_15_35_03-14905898436547221715?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_15_40_52-2208718843501549822?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_15_28_55-6247790414516817016?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_15_34_00-10964828517766777942?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_15_40_37-15677515414570305286?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_15_45_46-13004561290815715940?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_15_28_53-2836528604540610213?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_15_35_13-3979774015915458204?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_15_40_42-8973584466090970813?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_15_46_05-16879814385177586298?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 379.135s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_15_51_29-13711368581657492868?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2845
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2845/display/redirect>
------------------------------------------
[...truncated 559.36 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1436.372s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_14_29_15-4843074810104019916?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_14_36_06-571126882816171312?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_14_41_35-3162920655677601528?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_14_47_23-6805549513905699609?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_14_29_15-1544638928175605957?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_14_34_50-5785410033054298809?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_14_41_19-10127004682939670016?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_14_47_13-4713292984277616015?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_14_29_16-1544638928175606113?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_14_40_29-6402771611754469008?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_14_45_22-15530741581027858804?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_14_29_17-14894608594298276801?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_14_35_37-4367778732568271324?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_14_41_17-11355039775104657222?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_14_46_20-7928500092363826008?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 369.835s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_14_52_52-14030828291283216018?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2844
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2844/display/redirect>
------------------------------------------
[...truncated 559.28 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1365.410s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_08_23_14-2122125597925486528?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_08_29_01-6089669724013349618?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_08_34_24-17397747572354823697?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_08_39_53-18139040272902910268?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_08_23_15-17688310164066020646?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_08_31_33-6655483665915488372?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_08_37_04-7856065164727990501?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_08_23_14-15070627786627794388?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_08_28_59-13812302397416329453?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_08_35_02-7125956839719340012?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_08_40_09-2860196016128904415?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_08_23_14-5752148931575751201?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_08_29_18-11977703353969784711?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_08_34_41-2882394622351079542?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_08_40_03-5498830239349840468?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 369.246s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_08_45_42-3847960063529742553?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2843
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2843/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-2690] HCatalogIO needs provided scope dependencies on Hadoop, Hive
------------------------------------------
[...truncated 559.68 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1358.455s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_05_13_12-9337856414783242779?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_05_19_05-11451099752342330420?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_05_24_52-6551239770747510602?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_05_29_53-5646440937464924998?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_05_13_14-6362166894354830064?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_05_18_16-8909711864487991949?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_05_24_05-14472123073777380226?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_05_30_01-9047030204096625887?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_05_13_13-2680816219984143303?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_05_18_55-12308596496245366065?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_05_24_37-14915561342749063859?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_05_13_13-15852809906734069743?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_05_18_59-6540980115836362390?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_05_24_21-16819593232275772113?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_05_30_03-10587363853869860884?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 404.011s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_05_35_35-14851614067755131059?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2842
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2842/display/redirect>
------------------------------------------
[...truncated 558.54 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_par_do_with_multiple_outputs_and_using_return (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1346.896s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_02_15_10-12698463959487282820?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_02_21_28-3693832329131653663?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_02_26_54-7053139949431833617?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_02_15_10-978304491251176559?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_02_20_37-6054598978764278408?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_02_26_35-8303081708489932589?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_02_32_12-4261078406414058974?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_02_15_10-8146991178629056356?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_02_21_09-16242299637625842408?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_02_26_13-17476693937553427070?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_02_31_46-13125181941976055441?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_02_15_33-15079181611005878529?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_02_21_01-13110043536000771184?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_02_26_40-10595565219668300406?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_02_31_59-7427320364524963258?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 374.012s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_02_37_21-9128284690524406274?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2841
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2841/display/redirect?page=changes>
Changes:
[jbonofre] [BEAM-2530] Upgrade maven plugins to the latest versions
------------------------------------------
[...truncated 559.70 KB...]
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_par_do_with_multiple_outputs_and_using_return (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1401.277s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_23_51_13-5749367975902696097?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_23_57_14-17616831433463224538?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_00_03_09-11310159116156651855?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_00_08_45-6200051075594970679?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_23_51_12-4934127488534412491?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_23_57_06-18165491502893033295?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_00_02_52-10452644101828983603?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_00_08_12-14233532192579656088?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_23_51_12-6952833674419764468?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_23_57_06-9445272553097576617?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_00_02_32-1458243789954074775?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_23_51_11-12074030064355914637?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_23_56_29-2392769710046755643?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_00_02_30-10658899967859867239?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_00_08_06-14131285496285322712?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 368.977s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-08-01_00_14_14-6588400832308529894?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2840
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2840/display/redirect>
------------------------------------------
[...truncated 558.55 KB...]
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1355.204s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_20_15_21-17112700137618939925?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_20_21_45-3726731373617056811?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_20_27_15-2109144297571302007?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_20_32_37-617492034941283514?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_20_15_20-9499018594661815393?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_20_21_35-18369675677568395015?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_20_26_36-14917812744956900283?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_20_31_42-2524209649707912115?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_20_15_21-489305835672273798?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_20_21_41-13355961119162957903?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_20_27_02-8701190313592978399?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_20_15_20-13662684987703461265?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_20_21_01-16163207274163759990?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_20_26_46-15705093256082511412?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_20_32_23-11541706266184190589?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 369.091s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_20_37_41-2080855408915902983?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2839
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2839/display/redirect?page=changes>
Changes:
[tgroh] Programmatically Create Beam Jenkins View.
------------------------------------------
[...truncated 576.29 KB...]
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.6.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
======================================================================
ERROR: test_par_do_with_multiple_outputs_and_using_return (apache_beam.transforms.ptransform_test.PTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/ptransform_test.py",> line 236, in test_par_do_with_multiple_outputs_and_using_return
pipeline.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/testing/test_pipeline.py",> line 96, in run
result = super(TestPipeline, self).run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 319, in run
self.to_runner_api(), self.runner, self._options).run(False)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 272, in run
self.dataflow_client.create_job(self.job), self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/retry.py",> line 168, in wrapper
return fun(*args, **kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 438, in create_job
self.create_job_description(job)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 461, in create_job_description
job.options, file_copy=self._gcs_file_copy)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",> line 327, in stage_job_resources
setup_options.requirements_file, requirements_cache_path)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",> line 261, in _populate_requirements_cache
processes.check_call(cmd_args)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/processes.py",> line 44, in check_call
return subprocess.check_call(*args, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/bin/python',> '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--no-binary', ':all:']' returned non-zero exit status 2
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by Some Numbers/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/FlatMap(some_fn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Unkey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/FlatMap(some_fn) (tag odd): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Unkey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/FlatMap(some_fn) (tag even): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Unkey (tag None): refcount: 1 => 0
root: DEBUG: Connecting using Google Application Default Credentials.
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0801010505-385909.1501549505.386211/requirements.txt...
oauth2client.transport: INFO: Attempting refresh to obtain initial access_token
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0801010505-385909.1501549505.386211/requirements.txt
root: INFO: Executing command: ['<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/bin/python',> '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--no-binary', ':all:']
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/ptransform_test.py",> line 262, in test_multiple_empty_outputs
pipeline.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/testing/test_pipeline.py",> line 96, in run
result = super(TestPipeline, self).run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 319, in run
self.to_runner_api(), self.runner, self._options).run(False)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 272, in run
self.dataflow_client.create_job(self.job), self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/retry.py",> line 168, in wrapper
return fun(*args, **kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 438, in create_job
self.create_job_description(job)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 461, in create_job_description
job.options, file_copy=self._gcs_file_copy)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",> line 327, in stage_job_resources
setup_options.requirements_file, requirements_cache_path)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",> line 261, in _populate_requirements_cache
processes.check_call(cmd_args)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/processes.py",> line 44, in check_call
return subprocess.check_call(*args, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/bin/python',> '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--no-binary', ':all:']' returned non-zero exit status 2
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by Some Numbers/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/FlatMap(<lambda at ptransform_test.py:256>) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Unkey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/FlatMap(<lambda at ptransform_test.py:256>) (tag odd): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Unkey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/FlatMap(<lambda at ptransform_test.py:256>) (tag even): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Unkey (tag None): refcount: 1 => 0
root: DEBUG: Connecting using Google Application Default Credentials.
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0801010505-446652.1501549505.446956/requirements.txt...
oauth2client.transport: INFO: Attempting refresh to obtain initial access_token
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0801010505-446652.1501549505.446956/requirements.txt
root: INFO: Executing command: ['<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/bin/python',> '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--no-binary', ':all:']
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/ptransform_test.py",> line 220, in test_par_do_with_multiple_outputs_and_using_yield
pipeline.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/testing/test_pipeline.py",> line 96, in run
result = super(TestPipeline, self).run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 319, in run
self.to_runner_api(), self.runner, self._options).run(False)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 272, in run
self.dataflow_client.create_job(self.job), self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/retry.py",> line 168, in wrapper
return fun(*args, **kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 438, in create_job
self.create_job_description(job)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 461, in create_job_description
job.options, file_copy=self._gcs_file_copy)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",> line 327, in stage_job_resources
setup_options.requirements_file, requirements_cache_path)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",> line 261, in _populate_requirements_cache
processes.check_call(cmd_args)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/processes.py",> line 44, in check_call
return subprocess.check_call(*args, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/bin/python',> '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--no-binary', ':all:']' returned non-zero exit status 2
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by Some Numbers/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/ParDo(SomeDoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Unkey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/ParDo(SomeDoFn) (tag odd): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Unkey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/ParDo(SomeDoFn) (tag even): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Unkey (tag None): refcount: 1 => 0
root: DEBUG: Connecting using Google Application Default Credentials.
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0801010505-660177.1501549505.660489/requirements.txt...
oauth2client.transport: INFO: Attempting refresh to obtain initial access_token
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0801010505-660177.1501549505.660489/requirements.txt
root: INFO: Executing command: ['<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/bin/python',> '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--no-binary', ':all:']
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 15 tests in 1035.903s
FAILED (errors=3)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_18_05_32-5275270917741467293?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_18_11_10-11280716714667579685?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_18_16_39-13158276645247547183?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_18_05_33-11046520378820619768?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_18_11_21-3860513990837218362?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_18_16_59-16387184309955258247?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_18_05_32-4612729877220309270?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_18_10_57-16977854447028125978?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_18_16_26-13262601761728075143?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_18_05_28-7962425454474060676?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_18_10_38-9310571958505075853?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_18_16_27-1471366206163666902?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2838
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2838/display/redirect>
------------------------------------------
[...truncated 558.63 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1495.664s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_14_15_12-2330131343116753601?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_14_22_39-875854324976650368?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_14_28_22-12781690820671926836?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_14_34_04-12542471370924525?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_14_15_13-15719959153451060105?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_14_22_37-6982778719441077899?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_14_28_52-18133677505202264853?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_14_33_49-9798518604696497865?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_14_15_12-17070806621524584625?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_14_22_30-14870017677573937315?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_14_28_21-6991937767273157943?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_14_15_13-7051973667077987202?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_14_21_11-11626455465552118900?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_14_28_00-14549739713811958592?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_14_33_24-5168820457666412960?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 369.033s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_14_39_52-7284168804415968063?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2837
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2837/display/redirect>
------------------------------------------
[...truncated 558.63 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1336.390s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_08_15_42-10025094601891883466?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_08_21_43-12286489549075203446?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_08_26_54-2101925558967836714?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_08_32_45-11840974421376393236?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_08_15_42-7689823128501497174?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_08_21_22-17865273955750638922?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_08_27_01-1814982152776506767?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_08_15_41-7499066813794682915?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_08_20_23-411620466706692270?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_08_26_29-4979092106972795445?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_08_31_34-1560323529585653387?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_08_15_42-14694222175898644591?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_08_21_27-11856974789004818690?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_08_26_43-3264692184864446962?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_08_31_48-4162974511705694317?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 374.203s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_08_37_43-3955143813501745952?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2836
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2836/display/redirect>
------------------------------------------
[...truncated 558.16 KB...]
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1361.115s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_02_15_02-2971319010835791115?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_02_20_52-8025253927167535195?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_02_25_47-11334362705891066379?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_02_31_24-8706297238098350428?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_02_15_03-18121236868225544754?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_02_20_45-1796974352324896559?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_02_26_06-17219716015139671221?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_02_15_03-2575095922157952810?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_02_20_30-4276807485753644440?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_02_26_15-15911316750769614704?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_02_31_35-4289908648205539843?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_02_15_03-4242175787041835650?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_02_20_43-1363457207956640695?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_02_26_18-17959397657041955430?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_02_31_39-6783484994016676705?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 389.546s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-31_02_37_28-453724328188699997?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2835
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2835/display/redirect>
------------------------------------------
[...truncated 559.83 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Union[Tuple[str, NoneType], Tuple[str, int]].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_par_do_with_multiple_outputs_and_using_return (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.5.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
======================================================================
ERROR: test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/ptransform_test.py",> line 220, in test_par_do_with_multiple_outputs_and_using_yield
pipeline.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/testing/test_pipeline.py",> line 96, in run
result = super(TestPipeline, self).run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 319, in run
self.to_runner_api(), self.runner, self._options).run(False)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 272, in run
self.dataflow_client.create_job(self.job), self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/retry.py",> line 168, in wrapper
return fun(*args, **kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 438, in create_job
self.create_job_description(job)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 461, in create_job_description
job.options, file_copy=self._gcs_file_copy)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",> line 327, in stage_job_resources
setup_options.requirements_file, requirements_cache_path)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",> line 261, in _populate_requirements_cache
processes.check_call(cmd_args)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/processes.py",> line 44, in check_call
return subprocess.check_call(*args, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/bin/python',> '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--no-binary', ':all:']' returned non-zero exit status 2
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by Some Numbers/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/ParDo(SomeDoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Unkey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/ParDo(SomeDoFn) (tag odd): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:odd/Unkey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by ClassifyNumbers/ParDo(SomeDoFn) (tag even): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert:even/Unkey (tag None): refcount: 1 => 0
root: DEBUG: Connecting using Google Application Default Credentials.
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0731031709-480299.1501471029.480494/requirements.txt...
oauth2client.transport: INFO: Attempting refresh to obtain initial access_token
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0731031709-480299.1501471029.480494/requirements.txt
root: INFO: Executing command: ['<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/bin/python',> '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--no-binary', ':all:']
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 15 tests in 1371.367s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_20_17_32-3986241740865846000?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_20_23_34-2671533721445312179?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_20_29_16-7828043317176011463?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_20_34_39-7843403513354886589?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_20_17_30-232910442735190395?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_20_23_34-1928886647723057725?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_20_29_32-54101911725900582?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_20_17_30-10786723820586405577?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_20_23_47-14612708435626741256?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_20_29_24-9335791485243174586?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_20_17_29-161804892624112853?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_20_22_47-6611768695408329427?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_20_28_05-18179462131038729295?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_20_33_38-16424965961146897456?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2834
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2834/display/redirect?page=changes>
Changes:
[kirpichov] [BEAM-2623] Introduces Watch transform
------------------------------------------
[...truncated 562.20 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1279.876s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_16_24_17-5300723415916246102?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_16_29_46-16233111818785517930?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_16_35_10-9545947355705935560?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_16_40_24-18122853604218989226?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_16_24_18-15311777293511471441?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_16_30_09-2126340701631354453?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_16_35_29-4718814657648889652?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_16_24_18-12584344951252363336?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_16_29_57-7626744227027416062?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_16_34_42-11187569947546564728?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_16_39_56-8826369885252602636?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_16_24_17-15110965956978619448?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_16_29_18-4503168334645843544?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_16_35_12-9086474498156335819?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_16_40_06-6435212759570164462?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 348.849s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_16_45_22-12118212954571304457?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2833
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2833/display/redirect>
------------------------------------------
[...truncated 562.14 KB...]
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1313.217s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_14_30_48-395333883518660841?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_14_36_53-13369134680087183323?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_14_42_13-4252463314885301143?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_14_47_19-8492614309573956398?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_14_30_48-18402236031717048652?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_14_36_09-15282445608099133690?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_14_42_24-17347884769354538377?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_14_30_50-8276044193352658175?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_14_36_44-8354928884953102890?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_14_41_43-2441623869426618573?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_14_47_28-16485004877964716872?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_14_30_49-3384285686231786906?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_14_36_29-13817723487908513122?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_14_41_58-3924762488141789645?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_14_47_18-4599567335630605185?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 349.120s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_14_52_27-2354157512238268141?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2832
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2832/display/redirect>
------------------------------------------
[...truncated 562.28 KB...]
test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1348.011s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_08_30_26-13704672365046902869?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_08_35_58-17604385536326090725?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_08_41_57-9810622064419776712?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_08_47_28-10692007848870795022?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_08_30_27-13586445360407425338?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_08_36_39-1931117003105832656?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_08_42_06-11480448657271533273?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_08_30_27-216753165670855589?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_08_35_24-4425104009263285691?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_08_41_16-16693947152916227040?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_08_46_48-5950805751670575781?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_08_30_26-11450487114394006373?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_08_36_22-7001925838635283193?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_08_41_35-17203917946598937370?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_08_47_12-4012448416882566498?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 363.635s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_08_52_36-12202668556843684396?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2831
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2831/display/redirect>
------------------------------------------
[...truncated 561.98 KB...]
test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1391.230s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_02_30_25-14690608256442262495?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_02_36_46-6260398198743793718?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_02_42_28-3640288967988305353?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_02_48_01-13729641253227218421?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_02_30_23-3299506412762963434?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_02_36_39-16599370627950956194?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_02_41_40-17408527346970254751?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_02_46_31-5263399091216330775?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_02_30_24-9907077760573384787?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_02_35_57-2260810350904804691?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_02_41_49-10989912505544319025?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_02_47_05-10084448921458209307?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_02_30_24-3585225824790928931?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_02_36_40-3826793833127550964?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_02_42_17-16110135269598609900?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 334.007s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-30_02_53_19-1305548451474676984?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2830
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2830/display/redirect>
------------------------------------------
[...truncated 561.02 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1340.290s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_20_24_01-2701792664809468889?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_20_29_51-189225829158084520?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_20_34_58-5218976451896374986?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_20_40_38-12497621573614298018?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_20_24_05-17275487695202355839?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_20_29_16-18230035289671687928?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_20_35_32-4645396406380302529?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_20_40_41-2238612478239026736?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_20_24_01-10644611910878234140?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_20_30_05-9442080640884501402?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_20_36_00-10097779171630625629?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_20_24_06-14028708628769217981?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_20_30_00-11819018440301347501?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_20_35_30-9852208584145807299?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_20_40_37-8443602991366810612?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 379.048s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_20_46_07-4052739590882762353?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2829
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2829/display/redirect>
------------------------------------------
[...truncated 599.54 KB...]
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 623, in run_Read
names.SERIALIZED_SOURCE_KEY: pickler.dumps(transform.source)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/internal/pickler.py",> line 193, in dumps
s = dill.dumps(o)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/dill/dill.py",> line 259, in dumps
dump(obj, file, protocol, byref, fmode, recurse)#, strictio)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/dill/dill.py",> line 252, in dump
pik.dump(obj)
File "/usr/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
File "/usr/lib/python2.7/pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "/usr/lib/python2.7/pickle.py", line 396, in save_reduce
save(cls)
File "/usr/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/internal/pickler.py",> line 94, in wrapper
obj=obj)
File "/usr/lib/python2.7/pickle.py", line 400, in save_reduce
save(func)
File "/usr/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/dill/dill.py",> line 1316, in save_function
log.info("# F2")
File "/usr/lib/python2.7/logging/__init__.py", line 1151, in info
if self.isEnabledFor(INFO):
File "/usr/lib/python2.7/logging/__init__.py", line 1349, in isEnabledFor
if self.manager.disable >= level:
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest)'
======================================================================
ERROR: test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 812, in run
test(orig)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 45, in __call__
return self.run(*arg, **kwarg)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 133, in run
self.runTest(result)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 151, in runTest
test(result)
File "/usr/lib/python2.7/unittest/case.py", line 395, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python2.7/unittest/case.py", line 331, in run
testMethod()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/sideinputs_test.py",> line 177, in test_iterable_side_input
pipeline.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/testing/test_pipeline.py",> line 96, in run
result = super(TestPipeline, self).run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 268, in run
pipeline._options)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 390, in __init__
get_credentials=(not self.google_cloud_options.no_auth))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py",> line 63, in __init__
self.projects_templates = self.ProjectsTemplatesService(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py",> line 638, in __init__
supports_download=False,
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py",> line 761, in __init__
def __init__(self, **kwargs):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest)'
======================================================================
ERROR: test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 812, in run
test(orig)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 45, in __call__
return self.run(*arg, **kwarg)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 133, in run
self.runTest(result)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 151, in runTest
test(result)
File "/usr/lib/python2.7/unittest/case.py", line 395, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python2.7/unittest/case.py", line 331, in run
testMethod()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/sideinputs_test.py",> line 156, in test_multi_valued_singleton_side_input
pipeline.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/testing/test_pipeline.py",> line 96, in run
result = super(TestPipeline, self).run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 272, in run
self.dataflow_client.create_job(self.job), self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/retry.py",> line 168, in wrapper
return fun(*args, **kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 438, in create_job
self.create_job_description(job)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 461, in create_job_description
job.options, file_copy=self._gcs_file_copy)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",> line 316, in stage_job_resources
file_copy(setup_options.requirements_file, staged_path)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/retry.py",> line 168, in wrapper
return fun(*args, **kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 401, in _gcs_file_copy
self.stage_file(to_folder, to_name, f)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 420, in stage_file
response = self._storage_client.objects.Insert(request, upload=upload)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/internal/clients/storage/storage_v1_client.py",> line 971, in Insert
download=download)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/apitools/base/py/base_api.py",> line 721, in _RunMethod
http, http_request, **opts)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/apitools/base/py/http_wrapper.py",> line 341, in MakeRequest
check_response_func=check_response_func)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/apitools/base/py/http_wrapper.py",> line 391, in _MakeRequestNoRetry
redirections=redirections, connection_type=connection_type)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/oauth2client/transport.py",> line 169, in new_request
redirections, connection_type)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/httplib2/__init__.py",> line 1609, in request
(response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/httplib2/__init__.py",> line 1351, in _request
(response, content) = self._conn_request(conn, request_uri, method, body, headers)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/httplib2/__init__.py",> line 1307, in _conn_request
response = conn.getresponse()
File "/usr/lib/python2.7/httplib.py", line 1089, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 444, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 400, in _read_status
line = self.fp.readline(_MAXLINE + 1)
File "/usr/lib/python2.7/socket.py", line 476, in readline
data = self._sock.recv(self._rbufsize)
File "/usr/lib/python2.7/ssl.py", line 341, in recv
return self.read(buflen)
File "/usr/lib/python2.7/ssl.py", line 260, in read
return self._sslobj.read(len)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest)'
======================================================================
ERROR: test_par_do_with_multiple_outputs_and_using_return (apache_beam.transforms.ptransform_test.PTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 812, in run
test(orig)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 45, in __call__
return self.run(*arg, **kwarg)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 133, in run
self.runTest(result)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 151, in runTest
test(result)
File "/usr/lib/python2.7/unittest/case.py", line 395, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python2.7/unittest/case.py", line 331, in run
testMethod()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/ptransform_test.py",> line 236, in test_par_do_with_multiple_outputs_and_using_return
pipeline.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/testing/test_pipeline.py",> line 96, in run
result = super(TestPipeline, self).run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 319, in run
self.to_runner_api(), self.runner, self._options).run(False)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 48, in run
self.result.wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 893, in wait_until_finish
time.sleep(5.0)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_par_do_with_multiple_outputs_and_using_return (apache_beam.transforms.ptransform_test.PTransformTest)'
======================================================================
ERROR: test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 812, in run
test(orig)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 45, in __call__
return self.run(*arg, **kwarg)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 133, in run
self.runTest(result)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 151, in runTest
test(result)
File "/usr/lib/python2.7/unittest/case.py", line 395, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python2.7/unittest/case.py", line 331, in run
testMethod()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/ptransform_test.py",> line 220, in test_par_do_with_multiple_outputs_and_using_yield
pipeline.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/testing/test_pipeline.py",> line 96, in run
result = super(TestPipeline, self).run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 319, in run
self.to_runner_api(), self.runner, self._options).run(False)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 48, in run
self.result.wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 893, in wait_until_finish
time.sleep(5.0)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest)'
======================================================================
ERROR: test_read_from_text_metrics (apache_beam.transforms.ptransform_test.PTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 812, in run
test(orig)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 45, in __call__
return self.run(*arg, **kwarg)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 133, in run
self.runTest(result)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 151, in runTest
test(result)
File "/usr/lib/python2.7/unittest/case.py", line 395, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python2.7/unittest/case.py", line 331, in run
testMethod()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/ptransform_test.py",> line 191, in test_read_from_text_metrics
res = pipeline.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/testing/test_pipeline.py",> line 96, in run
result = super(TestPipeline, self).run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 319, in run
self.to_runner_api(), self.runner, self._options).run(False)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 48, in run
self.result.wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 893, in wait_until_finish
time.sleep(5.0)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_read_from_text_metrics (apache_beam.transforms.ptransform_test.PTransformTest)'
----------------------------------------------------------------------
Ran 15 tests in 902.770s
FAILED (errors=15)
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2828
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2828/display/redirect>
------------------------------------------
[...truncated 558.57 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1624.670s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_08_15_45-8644841776034507063?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_08_21_49-5691795866585216390?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_08_27_16-5992859251266451712?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_08_34_54-17593821928065005462?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_08_15_46-1781000890168469021?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_08_22_08-4944215850432485839?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_08_27_35-12553497832144954271?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_08_33_13-7965303998617261055?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_08_15_46-2848456811581367437?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_08_23_34-10169562144212525917?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_08_28_45-673042729822649107?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_08_15_46-16608342959109679968?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_08_22_04-9023021442540871272?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_08_27_37-1139304797351380011?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_08_33_19-16626860284575028604?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 364.025s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_08_42_33-14720669366786102041?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2827
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2827/display/redirect>
------------------------------------------
[...truncated 584.74 KB...]
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 133, in run
self.runTest(result)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 151, in runTest
test(result)
File "/usr/lib/python2.7/unittest/case.py", line 395, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python2.7/unittest/case.py", line 331, in run
testMethod()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/sideinputs_test.py",> line 146, in test_empty_singleton_side_input
pipeline.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/testing/test_pipeline.py",> line 96, in run
result = super(TestPipeline, self).run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 527, in run_ParDo
step.add_property(PropertyNames.SERIALIZED_FN, pickler.dumps(fn_data))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/internal/pickler.py",> line 193, in dumps
s = dill.dumps(o)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/dill/dill.py",> line 259, in dumps
dump(obj, file, protocol, byref, fmode, recurse)#, strictio)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/dill/dill.py",> line 252, in dump
pik.dump(obj)
File "/usr/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
File "/usr/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/lib/python2.7/pickle.py", line 562, in save_tuple
save(element)
File "/usr/lib/python2.7/pickle.py", line 304, in save
reduce = getattr(obj, "__reduce_ex__", None)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest)'
======================================================================
ERROR: test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 812, in run
test(orig)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 45, in __call__
return self.run(*arg, **kwarg)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 133, in run
self.runTest(result)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 151, in runTest
test(result)
File "/usr/lib/python2.7/unittest/case.py", line 395, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python2.7/unittest/case.py", line 331, in run
testMethod()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/sideinputs_test.py",> line 305, in test_flattened_side_input
pipeline.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/testing/test_pipeline.py",> line 96, in run
result = super(TestPipeline, self).run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 525, in run_ParDo
transform_node.transform.output_tags)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 356, in _add_step
DisplayData.create_from(transform_node.transform).items])
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/display.py",> line 154, in create_from
return cls(has_display_data._namespace(), has_display_data.display_data())
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/display.py",> line 85, in __init__
self._populate_items(display_data_dict)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/display.py",> line 93, in _populate_items
element.display_data())
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/display.py",> line 85, in __init__
self._populate_items(display_data_dict)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/display.py",> line 98, in _populate_items
if element.should_drop():
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/display.py",> line 203, in should_drop
def should_drop(self):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest)'
======================================================================
ERROR: test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 812, in run
test(orig)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 45, in __call__
return self.run(*arg, **kwarg)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 133, in run
self.runTest(result)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 151, in runTest
test(result)
File "/usr/lib/python2.7/unittest/case.py", line 395, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python2.7/unittest/case.py", line 331, in run
testMethod()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/sideinputs_test.py",> line 177, in test_iterable_side_input
pipeline.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/testing/test_pipeline.py",> line 96, in run
result = super(TestPipeline, self).run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 272, in run
self.dataflow_client.create_job(self.job), self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/retry.py",> line 168, in wrapper
return fun(*args, **kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 438, in create_job
self.create_job_description(job)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 461, in create_job_description
job.options, file_copy=self._gcs_file_copy)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",> line 298, in stage_job_resources
setup_options = options.view_as(SetupOptions)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/options/pipeline_options.py",> line 227, in view_as
view = cls(self._flags)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/options/pipeline_options.py",> line 156, in __init__
cls._add_argparse_args(parser)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/options/pipeline_options.py",> line 571, in _add_argparse_args
('Save the main session state so that pickled functions and classes '
File "/usr/lib/python2.7/argparse.py", line 1293, in add_argument
self._get_formatter()._format_args(action, None)
File "/usr/lib/python2.7/argparse.py", line 2316, in _get_formatter
return self.formatter_class(prog=self.prog)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest)'
======================================================================
ERROR: test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 812, in run
test(orig)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 45, in __call__
return self.run(*arg, **kwarg)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 133, in run
self.runTest(result)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/case.py",> line 151, in runTest
test(result)
File "/usr/lib/python2.7/unittest/case.py", line 395, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python2.7/unittest/case.py", line 331, in run
testMethod()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/sideinputs_test.py",> line 156, in test_multi_valued_singleton_side_input
pipeline.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/testing/test_pipeline.py",> line 96, in run
result = super(TestPipeline, self).run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 272, in run
self.dataflow_client.create_job(self.job), self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/retry.py",> line 168, in wrapper
return fun(*args, **kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 438, in create_job
self.create_job_description(job)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 461, in create_job_description
job.options, file_copy=self._gcs_file_copy)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",> line 316, in stage_job_resources
file_copy(setup_options.requirements_file, staged_path)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/retry.py",> line 168, in wrapper
return fun(*args, **kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 401, in _gcs_file_copy
self.stage_file(to_folder, to_name, f)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 420, in stage_file
response = self._storage_client.objects.Insert(request, upload=upload)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/internal/clients/storage/storage_v1_client.py",> line 971, in Insert
download=download)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/apitools/base/py/base_api.py",> line 721, in _RunMethod
http, http_request, **opts)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/apitools/base/py/http_wrapper.py",> line 341, in MakeRequest
check_response_func=check_response_func)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/apitools/base/py/http_wrapper.py",> line 391, in _MakeRequestNoRetry
redirections=redirections, connection_type=connection_type)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/oauth2client/transport.py",> line 169, in new_request
redirections, connection_type)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/httplib2/__init__.py",> line 1609, in request
(response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/httplib2/__init__.py",> line 1351, in _request
(response, content) = self._conn_request(conn, request_uri, method, body, headers)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/httplib2/__init__.py",> line 1307, in _conn_request
response = conn.getresponse()
File "/usr/lib/python2.7/httplib.py", line 1089, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 444, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 400, in _read_status
line = self.fp.readline(_MAXLINE + 1)
File "/usr/lib/python2.7/socket.py", line 476, in readline
data = self._sock.recv(self._rbufsize)
File "/usr/lib/python2.7/ssl.py", line 341, in recv
return self.read(buflen)
File "/usr/lib/python2.7/ssl.py", line 260, in read
return self._sslobj.read(len)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/multiprocess.py",> line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest)'
----------------------------------------------------------------------
Ran 15 tests in 1105.727s
FAILED (errors=9)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_02_24_34-8959890274008165452?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_02_33_22-15513007839403960297?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_02_24_32-10761554173760053380?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_02_29_45-3695958426678901429?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_02_24_34-12382739999895314576?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-29_02_31_03-16074337642115074660?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2826
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2826/display/redirect>
------------------------------------------
[...truncated 558.03 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1520.823s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_20_15_50-6291836152573019729?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_20_22_34-17147147730920495589?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_20_27_23-11451986467372894708?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_20_34_06-11053656841226096132?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_20_15_51-1127153804711708632?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_20_24_28-12964520878550201758?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_20_30_12-8822040202822594612?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_20_15_52-3453677652070985664?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_20_21_41-12255848405673254373?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_20_27_29-10612120054890144165?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_20_32_52-9220873369597772052?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_20_15_51-13497438803271285180?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_20_23_09-7287324204157790047?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_20_28_36-18100296516845872359?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_20_35_09-12929502455391931645?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 369.221s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_20_40_53-6238999672331598585?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2825
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2825/display/redirect?page=changes>
Changes:
[kirpichov] [BEAM-92] Supports DynamicDestinations in AvroIO.
------------------------------------------
[...truncated 559.42 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1441.128s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_18_13_33-6836613119183755111?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_18_20_02-201534147600269790?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_18_25_50-9197939228800473426?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_18_31_32-6656239308042080729?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_18_13_56-5795105353810845603?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_18_20_20-6346752538305180412?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_18_25_43-2166155923356819751?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_18_14_01-17337561756406208118?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_18_20_33-9991440536264959009?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_18_25_56-8938875906014348066?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_18_31_33-10203575397250344337?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_18_14_10-12871190347805221270?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_18_19_24-9840778576920168648?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_18_25_02-12555947188303031826?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_18_30_10-6814891774945156021?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 369.139s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_18_37_11-16807968026194738993?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2824
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2824/display/redirect?page=changes>
Changes:
[tgroh] Perform a Multi-step combine in the DirectRunner
------------------------------------------
[...truncated 558.43 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1381.277s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_15_46_48-2032788334199652889?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_15_52_30-5444991024022049985?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_15_58_38-5435392826425162944?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_16_03_49-11946605851627795227?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_15_46_50-13640020767690415913?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_15_53_02-11705607513049382762?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_15_58_50-13181335240658187466?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_16_04_21-10742624357918442311?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_15_46_52-17758788480803121709?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_15_52_40-12116725735450675124?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_15_58_22-16668525130133403576?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_16_03_44-3028863174494139274?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_15_46_47-3177593784505332167?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_15_52_11-12782479518385435685?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_15_58_43-13011141975392550937?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 319.278s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_16_09_30-11025275290541383343?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2823
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2823/display/redirect>
------------------------------------------
[...truncated 558.30 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1317.515s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_14_16_41-13882353087724203184?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_14_22_09-24860452282648959?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_14_28_07-7837600847613048712?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_14_33_14-3270247985670599870?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_14_16_39-2642870745308691879?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_14_22_27-2014854053775512578?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_14_27_21-2647403519211946060?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_14_32_40-8153370373669453733?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_14_16_48-10912603337434817149?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_14_22_02-546088897683806499?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_14_28_53-7862676563420732483?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_14_16_38-10792429532657964568?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_14_22_17-3365261586195620752?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_14_28_24-6087138075790959866?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_14_33_12-10990883291608865584?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 394.251s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_14_38_18-12506201488309905754?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2822
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2822/display/redirect?page=changes>
Changes:
[kirpichov] Introduces TypeDescriptors.extractFromTypeParameters
[kirpichov] [BEAM-2677] AvroIO.parseGenericRecords - schemaless AvroIO.read
[kirpichov] Removes TransformingSource that is now unused
------------------------------------------
[...truncated 40.17 KB...]
msg: installpkg
cmdargs: [local('<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27cython/bin/pip'),> 'install', '<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/dist/apache-beam-2.2.0.dev.zip']>
env: {'JENKINS_HOME': '/x1/jenkins/jenkins-home', 'BUILD_CAUSE': 'SCMTRIGGER', 'GIT_COMMIT': 'a94d680eabd47a69be628028799ac4d41bd110d5', 'HUDSON_URL': 'https://builds.apache.org/', 'BUILD_URL': 'https://builds.apache.org/job/beam_PostCommit_Python_Verify/2822/', 'GIT_PREVIOUS_COMMIT': '7ab8954b9f9fc835750172163a01bdc05d6dfa04', 'BUILD_TAG': 'jenkins-beam_PostCommit_Python_Verify-2822', 'SSH_CLIENT': '62.210.60.235 47232 22', 'JENKINS_URL': 'https://builds.apache.org/', 'LOGNAME': 'jenkins', 'USER': 'jenkins', 'WORKSPACE': '<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/',> 'HOME': '/home/jenkins', 'PATH': '<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27cython/bin>:/home/jenkins/tools/java/latest1.8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games', 'JOB_NAME': 'beam_PostCommit_Python_Verify', 'LANG': 'en_US.UTF-8', 'RUN_DISPLAY_URL': 'https://builds.apache.org/job/beam_PostCommit_Python_Verify/2822/display/redirect', 'VIRTUAL_ENV': '<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27cython',> 'SHELL': '/bin/bash', 'GIT_PREVIOUS_SUCCESSFUL_COMMIT': 'b67a30bf1586990c56fc19e1510417bdab71f280', 'SHLVL': '3', 'NLSPATH': '/usr/dt/lib/nls/msg/%L/%N.cat', 'GIT_AUTHOR_EMAIL': 'builds@apache.org', 'HUDSON_HOME': '/x1/jenkins/jenkins-home', 'NODE_LABELS': 'beam beam3', 'GIT_COMMITTER_EMAIL': 'builds@apache.org', 'PYTHONHASHSEED': '3941944114', 'JAVA_HOME': '/home/jenkins/tools/java/latest1.8', 'ROOT_BUILD_CAUSE': 'SCMTRIGGER', 'BUILD_ID': '2822', 'BUILD_NUMBER': '2822', 'XDG_RUNTIME_DIR': '/run/user/9997', 'HUDSON_COOKIE': '69343948-c914-43b5-8c65-c6524676d482', 'JOB_URL': 'https://builds.apache.org/job/beam_PostCommit_Python_Verify/', 'TEST_HOST': 'beam3', 'GIT_BRANCH': 'origin/master', 'JOB_BASE_NAME': 'beam_PostCommit_Python_Verify', 'GIT_AUTHOR_NAME': 'jenkins', 'GIT_COMMITTER_NAME': 'jenkins', 'XDG_SESSION_ID': '109', '_': '/home/jenkins/.local/bin//tox', 'sha1': 'master', 'COVERALLS_REPO_TOKEN': '****', 'JOB_DISPLAY_URL': 'https://builds.apache.org/job/beam_PostCommit_Python_Verify/display/redirect', 'BUILD_CAUSE_SCMTRIGGER': 'true', 'HUDSON_SERVER_COOKIE': 'f4ebd1e6b0d976e8', 'EXECUTOR_NUMBER': '0', 'NODE_NAME': 'beam3', 'PWD': '<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/',> 'SPARK_LOCAL_IP': '127.0.0.1', 'JENKINS_SERVER_COOKIE': 'f4ebd1e6b0d976e8', 'BUILD_DISPLAY_NAME': '#2822', 'ROOT_BUILD_CAUSE_SCMTRIGGER': 'true', 'XFILESEARCHPATH': '/usr/dt/app-defaults/%L/Dt', 'MAIL': '/var/mail/jenkins', 'SSH_CONNECTION': '62.210.60.235 47232 10.128.0.4 22', 'RUN_CHANGES_DISPLAY_URL': 'https://builds.apache.org/job/beam_PostCommit_Python_Verify/2822/display/redirect?page=changes', 'GIT_URL': 'https://github.com/apache/beam.git'}
Processing ./target/.tox/dist/apache-beam-2.2.0.dev.zip
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.2.0.dev0)
Collecting crcmod<2.0,>=1.7 (from apache-beam==2.2.0.dev0)
Collecting dill==0.2.6 (from apache-beam==2.2.0.dev0)
Requirement already satisfied: grpcio<2.0,>=1.0 in ./target/.tox/py27cython/lib/python2.7/site-packages (from apache-beam==2.2.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.2.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.2.0.dev0)
Using cached mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<4.0.0,>=2.0.1 (from apache-beam==2.2.0.dev0)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in ./target/.tox/py27cython/lib/python2.7/site-packages (from apache-beam==2.2.0.dev0)
Collecting pyyaml<4.0.0,>=3.12 (from apache-beam==2.2.0.dev0)
Requirement already satisfied: six>=1.5.2 in ./target/.tox/py27cython/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.2.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in ./target/.tox/py27cython/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.2.0.dev0)
Requirement already satisfied: futures>=2.2.0 in ./target/.tox/py27cython/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.2.0.dev0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.2.0.dev0)
Using cached funcsigs-1.0.2-py2.py3-none-any.whl
Collecting pbr>=0.11 (from mock<3.0.0,>=1.0.1->apache-beam==2.2.0.dev0)
Using cached pbr-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5 (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.2.0.dev0)
Using cached pyasn1_modules-0.0.10-py2.py3-none-any.whl
Collecting pyasn1>=0.1.7 (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.2.0.dev0)
Using cached pyasn1-0.3.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.2.0.dev0)
Using cached rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./target/.tox/py27cython/lib/python2.7/site-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.2.0.dev0)
Building wheels for collected packages: apache-beam
Running setup.py bdist_wheel for apache-beam: started
Running setup.py bdist_wheel for apache-beam: finished with status 'error'
Complete output from command <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27cython/bin/python> -u -c "import setuptools, tokenize;__file__='/tmp/pip-lUY46p-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpQJe0mMpip-wheel- --python-tag cp27:
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27cython/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running bdist_wheel
running build
running build_py
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-lUY46p-build/setup.py", line 197, in <module>
'test': generate_protos_first(test),
File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27cython/local/lib/python2.7/site-packages/wheel/bdist_wheel.py",> line 199, in run
self.run_command('build')
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/lib/python2.7/distutils/command/build.py", line 128, in run
self.run_command(cmd_name)
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/tmp/pip-lUY46p-build/setup.py", line 137, in run
gen_protos.generate_proto_files()
File "gen_protos.py", line 65, in generate_proto_files
'Not in apache git tree; unable to find proto definitions.')
RuntimeError: Not in apache git tree; unable to find proto definitions.
----------------------------------------
Failed building wheel for apache-beam
Running setup.py clean for apache-beam
Failed to build apache-beam
Installing collected packages: avro, crcmod, dill, httplib2, funcsigs, pbr, mock, pyasn1, pyasn1-modules, rsa, oauth2client, pyyaml, apache-beam
Running setup.py install for apache-beam: started
Running setup.py install for apache-beam: finished with status 'error'
Complete output from command <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27cython/bin/python> -u -c "import setuptools, tokenize;__file__='/tmp/pip-lUY46p-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-AZOaTc-record/install-record.txt --single-version-externally-managed --compile --install-headers <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27cython/include/site/python2.7/apache-beam>:
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27cython/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running install
running build
running build_py
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-lUY46p-build/setup.py", line 197, in <module>
'test': generate_protos_first(test),
File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27cython/local/lib/python2.7/site-packages/setuptools/command/install.py",> line 61, in run
return orig.install.run(self)
File "/usr/lib/python2.7/distutils/command/install.py", line 601, in run
self.run_command('build')
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/lib/python2.7/distutils/command/build.py", line 128, in run
self.run_command(cmd_name)
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/tmp/pip-lUY46p-build/setup.py", line 137, in run
gen_protos.generate_proto_files()
File "gen_protos.py", line 65, in generate_proto_files
'Not in apache git tree; unable to find proto definitions.')
RuntimeError: Not in apache git tree; unable to find proto definitions.
----------------------------------------
Command "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27cython/bin/python> -u -c "import setuptools, tokenize;__file__='/tmp/pip-lUY46p-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-AZOaTc-record/install-record.txt --single-version-externally-managed --compile --install-headers <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27cython/include/site/python2.7/apache-beam"> failed with error code 1 in /tmp/pip-lUY46p-build/
py27cython installed: avro==1.8.2,crcmod==1.7,Cython==0.25.2,dill==0.2.6,enum34==1.1.6,funcsigs==1.0.2,futures==3.1.1,grpcio==1.4.0,grpcio-tools==1.3.5,httplib2==0.9.2,mock==2.0.0,nose==1.3.7,oauth2client==3.0.0,pbr==3.1.1,protobuf==3.3.0,pyasn1==0.3.1,pyasn1-modules==0.0.10,PyYAML==3.12,rsa==3.4.2,six==1.10.0
py27gcp create: <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27gcp>
py27gcp installdeps: nose==1.3.7
py27gcp inst: <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/dist/apache-beam-2.2.0.dev.zip>
ERROR: invocation failed (exit code 1), logfile: <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27gcp/log/py27gcp-2.log>
ERROR: actionid: py27gcp
msg: installpkg
cmdargs: [local('<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27gcp/bin/pip'),> 'install', '<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/dist/apache-beam-2.2.0.dev.zip']>
env: {'JENKINS_HOME': '/x1/jenkins/jenkins-home', 'BUILD_CAUSE': 'SCMTRIGGER', 'GIT_COMMIT': 'a94d680eabd47a69be628028799ac4d41bd110d5', 'HUDSON_URL': 'https://builds.apache.org/', 'BUILD_URL': 'https://builds.apache.org/job/beam_PostCommit_Python_Verify/2822/', 'GIT_PREVIOUS_COMMIT': '7ab8954b9f9fc835750172163a01bdc05d6dfa04', 'BUILD_TAG': 'jenkins-beam_PostCommit_Python_Verify-2822', 'SSH_CLIENT': '62.210.60.235 47232 22', 'JENKINS_URL': 'https://builds.apache.org/', 'LOGNAME': 'jenkins', 'USER': 'jenkins', 'WORKSPACE': '<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/',> 'HOME': '/home/jenkins', 'PATH': '<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27gcp/bin>:/home/jenkins/tools/java/latest1.8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games', 'JOB_NAME': 'beam_PostCommit_Python_Verify', 'LANG': 'en_US.UTF-8', 'RUN_DISPLAY_URL': 'https://builds.apache.org/job/beam_PostCommit_Python_Verify/2822/display/redirect', 'VIRTUAL_ENV': '<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27gcp',> 'SHELL': '/bin/bash', 'GIT_PREVIOUS_SUCCESSFUL_COMMIT': 'b67a30bf1586990c56fc19e1510417bdab71f280', 'SHLVL': '3', 'NLSPATH': '/usr/dt/lib/nls/msg/%L/%N.cat', 'GIT_AUTHOR_EMAIL': 'builds@apache.org', 'HUDSON_HOME': '/x1/jenkins/jenkins-home', 'NODE_LABELS': 'beam beam3', 'GIT_COMMITTER_EMAIL': 'builds@apache.org', 'PYTHONHASHSEED': '3941944114', 'JAVA_HOME': '/home/jenkins/tools/java/latest1.8', 'ROOT_BUILD_CAUSE': 'SCMTRIGGER', 'BUILD_ID': '2822', 'BUILD_NUMBER': '2822', 'XDG_RUNTIME_DIR': '/run/user/9997', 'HUDSON_COOKIE': '69343948-c914-43b5-8c65-c6524676d482', 'JOB_URL': 'https://builds.apache.org/job/beam_PostCommit_Python_Verify/', 'TEST_HOST': 'beam3', 'GIT_BRANCH': 'origin/master', 'JOB_BASE_NAME': 'beam_PostCommit_Python_Verify', 'GIT_AUTHOR_NAME': 'jenkins', 'GIT_COMMITTER_NAME': 'jenkins', 'XDG_SESSION_ID': '109', '_': '/home/jenkins/.local/bin//tox', 'sha1': 'master', 'COVERALLS_REPO_TOKEN': '****', 'JOB_DISPLAY_URL': 'https://builds.apache.org/job/beam_PostCommit_Python_Verify/display/redirect', 'BUILD_CAUSE_SCMTRIGGER': 'true', 'HUDSON_SERVER_COOKIE': 'f4ebd1e6b0d976e8', 'EXECUTOR_NUMBER': '0', 'NODE_NAME': 'beam3', 'PWD': '<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/',> 'SPARK_LOCAL_IP': '127.0.0.1', 'JENKINS_SERVER_COOKIE': 'f4ebd1e6b0d976e8', 'BUILD_DISPLAY_NAME': '#2822', 'ROOT_BUILD_CAUSE_SCMTRIGGER': 'true', 'XFILESEARCHPATH': '/usr/dt/app-defaults/%L/Dt', 'MAIL': '/var/mail/jenkins', 'SSH_CONNECTION': '62.210.60.235 47232 10.128.0.4 22', 'RUN_CHANGES_DISPLAY_URL': 'https://builds.apache.org/job/beam_PostCommit_Python_Verify/2822/display/redirect?page=changes', 'GIT_URL': 'https://github.com/apache/beam.git'}
Processing ./target/.tox/dist/apache-beam-2.2.0.dev.zip
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.2.0.dev0)
Collecting crcmod<2.0,>=1.7 (from apache-beam==2.2.0.dev0)
Collecting dill==0.2.6 (from apache-beam==2.2.0.dev0)
Collecting grpcio<2.0,>=1.0 (from apache-beam==2.2.0.dev0)
Using cached grpcio-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.2.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.2.0.dev0)
Using cached mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<4.0.0,>=2.0.1 (from apache-beam==2.2.0.dev0)
Collecting protobuf<=3.3.0,>=3.2.0 (from apache-beam==2.2.0.dev0)
Using cached protobuf-3.3.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pyyaml<4.0.0,>=3.12 (from apache-beam==2.2.0.dev0)
Collecting six>=1.5.2 (from grpcio<2.0,>=1.0->apache-beam==2.2.0.dev0)
Using cached six-1.10.0-py2.py3-none-any.whl
Collecting enum34>=1.0.4 (from grpcio<2.0,>=1.0->apache-beam==2.2.0.dev0)
Using cached enum34-1.1.6-py2-none-any.whl
Collecting futures>=2.2.0 (from grpcio<2.0,>=1.0->apache-beam==2.2.0.dev0)
Using cached futures-3.1.1-py2-none-any.whl
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.2.0.dev0)
Using cached funcsigs-1.0.2-py2.py3-none-any.whl
Collecting pbr>=0.11 (from mock<3.0.0,>=1.0.1->apache-beam==2.2.0.dev0)
Using cached pbr-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5 (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.2.0.dev0)
Using cached pyasn1_modules-0.0.10-py2.py3-none-any.whl
Collecting pyasn1>=0.1.7 (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.2.0.dev0)
Using cached pyasn1-0.3.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.2.0.dev0)
Using cached rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./target/.tox/py27gcp/lib/python2.7/site-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.2.0.dev0)
Building wheels for collected packages: apache-beam
Running setup.py bdist_wheel for apache-beam: started
Running setup.py bdist_wheel for apache-beam: finished with status 'error'
Complete output from command <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27gcp/bin/python> -u -c "import setuptools, tokenize;__file__='/tmp/pip-10dckv-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpA99zvTpip-wheel- --python-tag cp27:
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27gcp/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running bdist_wheel
running build
running build_py
gen_protos.py:46: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-10dckv-build/setup.py", line 197, in <module>
'test': generate_protos_first(test),
File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27gcp/local/lib/python2.7/site-packages/wheel/bdist_wheel.py",> line 199, in run
self.run_command('build')
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/lib/python2.7/distutils/command/build.py", line 128, in run
self.run_command(cmd_name)
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/tmp/pip-10dckv-build/setup.py", line 137, in run
gen_protos.generate_proto_files()
File "gen_protos.py", line 65, in generate_proto_files
'Not in apache git tree; unable to find proto definitions.')
RuntimeError: Not in apache git tree; unable to find proto definitions.
----------------------------------------
Failed building wheel for apache-beam
Running setup.py clean for apache-beam
Failed to build apache-beam
Installing collected packages: avro, crcmod, dill, six, protobuf, enum34, futures, grpcio, httplib2, funcsigs, pbr, mock, pyasn1, pyasn1-modules, rsa, oauth2client, pyyaml, apache-beam
Running setup.py install for apache-beam: started
Running setup.py install for apache-beam: finished with status 'error'
Complete output from command <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27gcp/bin/python> -u -c "import setuptools, tokenize;__file__='/tmp/pip-10dckv-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-BL71n2-record/install-record.txt --single-version-externally-managed --compile --install-headers <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27gcp/include/site/python2.7/apache-beam>:
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27gcp/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running install
running build
running build_py
gen_protos.py:46: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-10dckv-build/setup.py", line 197, in <module>
'test': generate_protos_first(test),
File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27gcp/local/lib/python2.7/site-packages/setuptools/command/install.py",> line 61, in run
return orig.install.run(self)
File "/usr/lib/python2.7/distutils/command/install.py", line 601, in run
self.run_command('build')
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/lib/python2.7/distutils/command/build.py", line 128, in run
self.run_command(cmd_name)
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/tmp/pip-10dckv-build/setup.py", line 137, in run
gen_protos.generate_proto_files()
File "gen_protos.py", line 65, in generate_proto_files
'Not in apache git tree; unable to find proto definitions.')
RuntimeError: Not in apache git tree; unable to find proto definitions.
----------------------------------------
Command "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27gcp/bin/python> -u -c "import setuptools, tokenize;__file__='/tmp/pip-10dckv-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-BL71n2-record/install-record.txt --single-version-externally-managed --compile --install-headers <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27gcp/include/site/python2.7/apache-beam"> failed with error code 1 in /tmp/pip-10dckv-build/
py27gcp installed: avro==1.8.2,crcmod==1.7,dill==0.2.6,enum34==1.1.6,funcsigs==1.0.2,futures==3.1.1,grpcio==1.4.0,httplib2==0.9.2,mock==2.0.0,nose==1.3.7,oauth2client==3.0.0,pbr==3.1.1,protobuf==3.3.0,pyasn1==0.3.1,pyasn1-modules==0.0.10,PyYAML==3.12,rsa==3.4.2,six==1.10.0
___________________________________ summary ____________________________________
ERROR: docs: InvocationError: <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/docs/bin/pip> install <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/dist/apache-beam-2.2.0.dev.zip> (see <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/docs/log/docs-2.log)>
ERROR: lint: InvocationError: <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/lint/bin/pip> install <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/dist/apache-beam-2.2.0.dev.zip> (see <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/lint/log/lint-2.log)>
ERROR: py27: InvocationError: <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27/bin/pip> install <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/dist/apache-beam-2.2.0.dev.zip> (see <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27/log/py27-2.log)>
ERROR: py27cython: InvocationError: <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27cython/bin/pip> install <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/dist/apache-beam-2.2.0.dev.zip> (see <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27cython/log/py27cython-2.log)>
ERROR: py27gcp: InvocationError: <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27gcp/bin/pip> install <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/dist/apache-beam-2.2.0.dev.zip> (see <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/target/.tox/py27gcp/log/py27gcp-2.log)>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2821
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2821/display/redirect?page=changes>
Changes:
[alex-filatov] Fix View.asMultimap javadoc.
------------------------------------------
[...truncated 558.53 KB...]
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1376.365s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_09_42_02-17689585953362976786?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_09_48_10-15062423298350353262?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_09_53_38-9189340722484223314?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_09_58_52-15839387606439034247?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_09_42_00-16228554520421637741?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_09_47_39-4795830113815213651?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_09_53_46-9320661692549435802?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_09_42_01-4268297565183657324?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_09_48_04-8663242572863342116?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_09_53_46-6209265847391487403?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_09_59_33-5931625191440236249?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_09_42_01-10851297453491344606?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_09_48_09-17597392609314660201?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_09_53_21-4128565168209766145?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_09_59_03-16288867488288301297?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 344.519s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_10_04_41-9073834642608901898?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user alex-filatov@users.noreply.github.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2820
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2820/display/redirect>
------------------------------------------
[...truncated 558.48 KB...]
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1758.377s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_28_51-17260964717694745383?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_34_49-13903636937577295362?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_40_36-6149287203070245400?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_47_43-2230146897183365769?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_28_51-15412384525646914637?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_35_19-16957570223813386317?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_41_01-12196288579570321593?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_28_50-10588226291504094471?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_34_48-15159797454756694831?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_40_30-12388383038904932243?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_47_32-15123794253958166082?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_28_49-10013835890604947079?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_34_38-14322949870942475998?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_40_02-16094385843641808235?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_47_29-4496827073761519402?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 364.346s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_57_53-11332588397052234983?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2819
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2819/display/redirect?page=changes>
Changes:
[iemejia] Add missing displayData to ElasticsearchIO.Read
------------------------------------------
[...truncated 558.17 KB...]
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1339.807s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_07_44_34-5478524511630972013?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_07_50_36-9190249511899974379?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_07_56_04-18307322780891417902?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_01_11-10492486565866949549?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_07_44_33-2479521614489011326?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_07_50_07-13863130445050447351?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_07_55_44-13172043450463005859?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_01_01-17098523649378973248?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_07_44_32-14680919384113093249?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_07_50_22-11560962193873598967?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_07_56_25-15246228037961782571?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_07_44_32-10414449764404726396?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_07_49_49-7476303757373707463?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_07_55_12-12699212994715184650?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_00_45-9613573820507938621?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 344.107s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_08_06_35-12064337029860284115?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2818
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2818/display/redirect>
------------------------------------------
[...truncated 558.46 KB...]
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1327.019s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_02_20_00-2887083163367217784?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_02_26_08-17166716486297454658?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_02_31_29-9934147230824515335?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_02_36_41-3095108913190063062?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_02_20_00-15841120043045857349?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_02_24_53-6616172911684267957?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_02_30_52-4557379596799278220?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_02_36_19-3865785906318296174?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_02_20_00-14255554814559465909?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_02_25_37-9218194395050699775?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_02_30_49-10703232717754693030?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_02_36_21-11976792694604000604?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_02_20_00-697429178899159148?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_02_25_49-14135812850471353295?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_02_31_16-3447693718980947910?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 368.807s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_02_41_49-12068639316632849425?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2817
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2817/display/redirect?page=changes>
Changes:
[jbonofre] [BEAM-2541] Check Elasticsearch backend version when the pipeline is run
------------------------------------------
[...truncated 559.02 KB...]
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1360.587s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_01_35_29-10059374265054961950?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_01_40_37-18410070321236918952?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_01_46_38-13198260447655701277?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_01_52_34-14644421857265201441?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_01_35_29-17274393108159329607?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_01_41_32-8122014632066804632?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_01_46_44-2562770875211963424?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_01_52_20-6934876127821873805?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_01_35_32-714928996869944287?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_01_41_49-5956978689668894124?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_01_47_16-892194186809304875?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_01_35_30-5350824959428136326?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_01_41_36-18129174828978083735?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_01_46_41-7400862461158666277?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_01_52_12-2851888418392830469?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 348.755s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-28_01_57_54-10719789416390139472?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2816
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2816/display/redirect>
------------------------------------------
[...truncated 558.55 KB...]
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1390.143s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_20_16_12-12060416348450276401?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_20_21_57-6264030358668214947?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_20_27_51-6992109198733896683?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_20_33_26-9509689251858290074?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_20_16_12-10206772721611792615?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_20_21_53-922962048316117326?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_20_27_57-8882907079021653728?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_20_33_16-17586282309999091335?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_20_16_12-5266804305014474012?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_20_21_43-501265953902848089?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_20_27_43-5698923025379869345?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_20_16_12-86097573309018328?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_20_21_18-4034434479179115064?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_20_27_44-2433587711831761898?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_20_33_19-15078390983925058289?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 364.173s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_20_39_08-7900266422405022476?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2815
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2815/display/redirect?page=changes>
Changes:
[kirpichov] [BEAM-2640] Introduces Create.ofProvider(ValueProvider)
[kirpichov] [BEAM-2641] Introduces TextIO.read().withHintMatchesManyFiles()
[kirpichov] Adds AvroIO.read().withHintMatchesManyFiles()
------------------------------------------
[...truncated 561.05 KB...]
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_par_do_with_multiple_outputs_and_using_return (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1394.708s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_50_21-9744134576684388448?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_56_47-18430804722926880030?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_16_02_54-5254037256850822201?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_50_14-5700591490336256844?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_55_52-17830419559893386189?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_16_01_55-14090468208712626612?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_16_07_57-17333734091438586141?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_50_16-13345470443431752534?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_56_28-8922901712006244094?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_16_01_35-15202072348114514432?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_16_08_05-2293543757192827021?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_50_15-11684843510292661517?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_57_18-13159276744313139036?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_16_02_59-7225511038406546304?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_16_08_05-227603892796672012?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 354.421s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_16_13_11-2737993407315156080?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2814
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2814/display/redirect?page=changes>
Changes:
[kirpichov] Fixes BigQueryIO.Write javadoc for to(ValueInSingleWindow)
------------------------------------------
[...truncated 561.26 KB...]
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_par_do_with_multiple_outputs_and_using_return (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1422.915s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_02_43-10838772118455365800?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_08_35-1056961323110897333?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_14_38-14134048547336591730?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_20_35-4453570814015089444?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_02_43-5439355140386664891?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_08_28-680745370870074968?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_14_28-14380683643048741767?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_20_21-14730902203946276521?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_02_43-12917813365525749834?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_09_26-659482421988882617?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_15_40-4263759585716074987?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_02_43-7631512750047800008?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_09_08-13607297272876860143?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_15_27-1975442437232466017?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_20_49-16183460783451477172?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 399.155s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_15_26_09-9057711753240986554?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user kirpichov@google.com
Build failed in Jenkins: beam_PostCommit_Python_Verify #2813
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2813/display/redirect>
------------------------------------------
[...truncated 561.86 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1350.161s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_14_17_24-2060102436850538283?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_14_22_54-11463930554350031766?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_14_28_58-2029720124001094751?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_14_34_22-3218228209275970845?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_14_17_23-17221619393525651135?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_14_23_16-9280324600234647765?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_14_29_30-11131172655203928492?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_14_17_24-8565257415163932715?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_14_23_09-18195403814995039877?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_14_28_37-11624670584618898312?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_14_34_15-5703959350160651418?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_14_17_25-10309471686479129103?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_14_23_28-9601813685468937415?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_14_28_55-18377614573183728286?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_14_34_41-1943217452744851968?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 324.619s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_14_39_36-10738744195231154715?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PostCommit_Python_Verify #2812
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2812/display/redirect>
------------------------------------------
[...truncated 559.79 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_par_do_with_multiple_outputs_and_using_return (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1409.631s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_08_17_33-16273517845281068485?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_08_24_07-11927564092854442262?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_08_30_09-5435219896234791885?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_08_17_34-12363981789286481831?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_08_24_18-7203616696186092886?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_08_29_50-14841254300247152489?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_08_35_26-14473653342157254275?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_08_17_35-993007544312126554?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_08_23_47-9378086953138079095?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_08_29_30-13637090567064271521?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_08_34_44-5434408064401108724?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_08_17_33-9544281188702617534?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_08_23_03-10004445097055206552?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_08_29_13-3000538205042720156?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_08_35_11-4190195428822318291?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 344.479s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_08_40_45-5086903724344385119?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PostCommit_Python_Verify #2811
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2811/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-1984] Fix scope for dependencies needed only for test purposes
------------------------------------------
[...truncated 558.06 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_read_from_text_metrics (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1409.119s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_07_07_07-7855336051707081362?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_07_13_15-11589171958521276284?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_07_19_19-8834591398575572458?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_07_24_37-3535090208904061461?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_07_07_05-4346952285986144212?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_07_12_55-15968363135669917546?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_07_18_28-17056091250605356259?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_07_24_41-99390319213993977?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_07_07_28-659195854277658328?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_07_13_22-1836157188856909876?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_07_19_05-9160599700109799243?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_07_07_07-15752721949386064431?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_07_12_42-7967819655023135390?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_07_18_26-10669549171824918129?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_07_23_43-16336096688309822734?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 348.921s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_07_30_16-6182229229687953720?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PostCommit_Python_Verify #2810
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2810/display/redirect>
------------------------------------------
[...truncated 559.48 KB...]
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1377.085s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_02_30_09-10133176361038970752?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_02_36_07-12891196772320102468?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_02_41_16-15819065318551668002?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_02_46_35-18020857767965938060?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_02_30_17-2124784123852428223?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_02_36_25-662866585191017185?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_02_42_08-14765417496697336853?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_02_30_18-2704336966219737395?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_02_36_27-2352913827313023620?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_02_41_56-15134743726405126875?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_02_47_33-7431613116033832882?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_02_30_17-7862997545574273966?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_02_35_57-7358816056042930445?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_02_42_10-11360545247327483000?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_02_47_22-10287291448234777961?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 389.610s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-27_02_52_47-10297042496143809707?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PostCommit_Python_Verify #2809
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2809/display/redirect>
------------------------------------------
[...truncated 558.38 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting mock (from -r postcommit_requirements.txt (line 2))
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1357.229s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_20_14_58-18143932068551616157?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_20_25_56-9364321588114270356?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_20_31_51-6415514012078556474?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_20_14_57-14778737912843219521?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_20_20_09-1284136696895126184?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_20_25_52-3196712979754582454?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_20_31_38-16885929475565758353?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_20_14_56-339978665257506944?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_20_20_30-12165161527235977370?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_20_26_02-13681575985952648675?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_20_31_34-7492400558311349583?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_20_14_59-15380282847153524441?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_20_21_07-7275690223748102061?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_20_26_23-2029927968698204491?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_20_31_34-16881260912660614121?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 369.123s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_20_37_19-14529325430319607154?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PostCommit_Python_Verify #2808
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2808/display/redirect?page=changes>
Changes:
[robertwb] Migrate checkCombineFn in TestUtils to CombineFnTester
------------------------------------------
[...truncated 562.27 KB...]
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
======================================================================
ERROR: test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/transforms/sideinputs_test.py",> line 291, in test_as_dict_twice
pipeline.run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/testing/test_pipeline.py",> line 96, in run
result = super(TestPipeline, self).run()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 272, in run
self.dataflow_client.create_job(self.job), self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/utils/retry.py",> line 168, in wrapper
return fun(*args, **kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 438, in create_job
self.create_job_description(job)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 461, in create_job_description
job.options, file_copy=self._gcs_file_copy)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",> line 432, in stage_job_resources
shutil.rmtree(temp_dir)
File "/usr/lib/python2.7/shutil.py", line 239, in rmtree
onerror(os.listdir, path, sys.exc_info())
File "/usr/lib/python2.7/shutil.py", line 237, in rmtree
names = os.listdir(path)
OSError: [Errno 2] No such file or directory: '/tmp/tmpJcOFC3'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by main input/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by side kvs/Read (tag None): refcount: 2 => 1
root: DEBUG: PValue computed by side kvs/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by Map(<lambda at sideinputs_test.py:278>) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/WindowInto(WindowIntoFn) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Create/Read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/ToVoidKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/pair_with_0 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/pair_with_1 (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/Flatten (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/GroupByKey (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Group/Map(_merge_tagged_vals_under_key) (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by assert_that/Unkey (tag None): refcount: 1 => 0
root: DEBUG: Connecting using Google Application Default Credentials.
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/requirements.txt...
oauth2client.transport: INFO: Attempting refresh to obtain initial access_token
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/requirements.txt
root: INFO: Executing command: ['<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/bin/python',> '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--no-binary', ':all:']
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-35.0.2.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-35.0.2.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.4.1.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.4.1.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.1.1.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.1.1.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/def.txt...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/def.txt
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.3.0.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.3.0.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.2.4.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.2.4.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/appdirs-1.4.0.tar.gz...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/appdirs-1.4.0.tar.gz
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pbr-3.0.1.tar.gz...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pbr-3.0.1.tar.gz
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/abc.txt...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/abc.txt
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/appdirs-1.4.1.tar.gz...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/appdirs-1.4.1.tar.gz
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.0.3.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.0.3.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/appdirs-1.4.2.tar.gz...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/appdirs-1.4.2.tar.gz
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.2.2.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.2.2.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-35.0.1.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-35.0.1.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/appdirs-1.4.3.tar.gz...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/appdirs-1.4.3.tar.gz
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.1.0.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.1.0.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.0.0.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.0.0.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/six-1.10.0.tar.gz...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/six-1.10.0.tar.gz
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.4.0.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.4.0.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pyparsing-2.2.0.tar.gz...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pyparsing-2.2.0.tar.gz
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.1.0.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.1.0.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.2.0.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.2.0.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pbr-2.0.0.tar.gz...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pbr-2.0.0.tar.gz
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/packaging-16.8.tar.gz...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/packaging-16.8.tar.gz
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/mock-2.0.0.tar.gz...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/mock-2.0.0.tar.gz
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.2.3.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.2.3.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pbr-3.0.0.tar.gz...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pbr-3.0.0.tar.gz
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pbr-2.1.0.tar.gz...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pbr-2.1.0.tar.gz
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pbr-3.1.1.tar.gz...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pbr-3.1.1.tar.gz
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/funcsigs-1.0.2.tar.gz...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/funcsigs-1.0.2.tar.gz
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/PyHamcrest-1.9.0.tar.gz...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/PyHamcrest-1.9.0.tar.gz
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.2.0.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.2.0.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pbr-1.10.0.tar.gz...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pbr-1.10.0.tar.gz
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.2.1.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.2.1.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.0.1.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-36.0.1.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-35.0.0.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-35.0.0.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pbr-3.1.0.tar.gz...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pbr-3.1.0.tar.gz
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.3.1.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.3.1.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.3.3.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.3.3.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.3.2.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.3.2.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.1.1.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.1.1.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pyparsing-2.1.10.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/pyparsing-2.1.10.zip
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.0.2.zip...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/setuptools-34.0.2.zip
root: INFO: Copying Beam SDK "dist/apache-beam-2.2.0.dev0.tar.gz" to staging location.
root: INFO: Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/dataflow_python_sdk.tar...
root: INFO: Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0726235431-285504.1501113271.285637/dataflow_python_sdk.tar
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 15 tests in 1390.974s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_16_49_00-1250256746298732206?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_16_55_08-14081788439321722365?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_17_01_02-16145038235086866686?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_16_48_59-8080200817421834761?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_16_55_22-11238930816478400398?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_17_01_07-17981210335103942848?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_16_49_01-14608094058020454511?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_16_55_05-8559592977226316159?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_17_00_49-12732115660480383174?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_17_06_15-12476673418120037578?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_16_49_00-3937756840714372387?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_16_54_24-9567440326510366700?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_17_00_12-18246776311720495675?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_17_05_57-3435715775572654337?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PostCommit_Python_Verify #2807
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/2807/display/redirect?page=changes>
Changes:
[robertwb] Translate combining operations through the Runner API.
------------------------------------------
[...truncated 558.89 KB...]
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: List[Any].
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz
Collecting mock (from -r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz
Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/setuptools-36.2.4.zip
Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1))
File was already downloaded /tmp/dataflow-requirements-cache/six-1.10.0.tar.gz
Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz
Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2))
File was already downloaded /tmp/dataflow-requirements-cache/pbr-3.1.1.tar.gz
Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr
test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
test_multi_valued_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok
----------------------------------------------------------------------
Ran 15 tests in 1456.484s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_15_25_36-4093475978902711818?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_15_32_39-5220112990310286504?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_15_38_22-2680888367332438380?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_15_43_53-3544375150124456267?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_15_26_10-1529298416654813600?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_15_31_54-14420483834190134559?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_15_38_03-15005549802228598894?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_15_25_36-1371814998183242732?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_15_31_16-16979118291223631195?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_15_37_56-6317233632212267761?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_15_43_37-15901449735983041198?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_15_25_36-10033574684154807419?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_15_31_44-11278648976022809804?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_15_37_17-5219846323289996108?project=apache-beam-testing
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_15_42_53-12224567181451285590?project=apache-beam-testing
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/io/gcp/gcsio.py>:121: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ERROR
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/coders/typecoders.py>:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
======================================================================
ERROR: test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py",> line 61, in test_bigquery_tornadoes_it
test_pipeline.get_full_options_as_args(**extra_opts))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes.py",> line 90, in run
write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 335, in __exit__
self.run().wait_until_finish()
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 328, in run
return self.runner.run(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 38, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 259, in run
super(DataflowRunner, self).run(pipeline)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 133, in run
pipeline.visit(RunVisitor(self))
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 353, in visit
self._root_transform().visit(visitor, self, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 685, in visit
part.visit(visitor, pipeline, visited)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/pipeline.py",> line 688, in visit
visitor.visit_transform(self)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 128, in visit_transform
self.runner.run_transform(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/runner.py",> line 171, in run_transform
return m(transform_node)
File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 595, in run_CombineValues
transform_node.fn.get_accumulator_coder())
AttributeError: 'AppliedPTransform' object has no attribute 'fn'
-------------------- >> begin captured logging << --------------------
root: DEBUG: PValue computed by read (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by months with tornadoes (tag None): refcount: 1 => 0
root: DEBUG: PValue computed by monthly count/GroupByKey (tag None): refcount: 1 => 0
root: ERROR: Error while visiting monthly count/Combine
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 354.689s
FAILED (errors=1)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2017-07-26_15_49_27-4293755119393878824?project=apache-beam-testing
Build step 'Execute shell' marked build as failure