You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/04/14 18:04:57 UTC
Build failed in Jenkins: beam_PerformanceTests_TextIOIT_HDFS #55
See <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/55/display/redirect>
------------------------------------------
[...truncated 19.40 KB...]
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins250952856257442004.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:339: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting pbr>=0.11 (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/e1/ba/f95e3ec83f93919b1437028e989cf3fa5ff4f5cae4a1f62255f71deddb5b/pbr-4.0.2-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.1.18)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
cheetah 2.4.4 requires Markdown>=2.0.1, which is not installed.
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, pbr, mock, rsa, pyasn1-modules, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, cachetools, google-auth, google-auth-httplib2, google-cloud-core, future, ply, google-gax, grpc-google-iam-v1, proto-google-cloud-pubsub-v1, gapic-google-cloud-pubsub-v1, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 pbr-4.0.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins2196976937120644908.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.textioit_hdfs_pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/> --official=true --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.text.TextIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000]' '--beam_extra_mvn_properties=[filesystem=hdfs]' --beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/pkb-config.yml> --beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml,/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT_HDFS/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-14 18:01:40,577 4d4e35e4 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/4d4e35e4/pkb.log>
2018-04-14 18:01:40,577 4d4e35e4 MainThread INFO PerfKitBenchmarker version: v1.12.0-489-gf0acf31
2018-04-14 18:01:40,578 4d4e35e4 MainThread INFO Flag values:
--beam_extra_mvn_properties=[filesystem=hdfs]
--beam_it_class=org.apache.beam.sdk.io.text.TextIOIT
--beam_it_timeout=1200
--beam_sdk=java
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/>
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000]
--beam_prebuilt
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836>
--project=apache-beam-testing
--beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml,/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT_HDFS/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
--bigquery_table=beam_performance.textioit_hdfs_pkb_results
--official
--beam_it_module=sdks/java/io/file-based-io-tests
--dpb_log_level=INFO
--beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/pkb-config.yml>
--beam_it_profile=io-it
--benchmarks=beam_integration_benchmark
2018-04-14 18:01:40,864 4d4e35e4 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-14 18:01:40,864 4d4e35e4 MainThread INFO Initializing the edw service decoder
2018-04-14 18:01:41,019 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-14 18:01:41,021 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-14 18:01:41,022 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-14 18:01:51,531 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> create -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml>
2018-04-14 18:01:52,208 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> create -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-14 18:01:53,093 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-14 18:01:53,101 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:02:03,375 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:02:13,623 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:02:23,893 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:02:34,087 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:02:44,247 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:02:54,391 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:03:04,548 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:03:14,711 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:03:24,859 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:03:35,019 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:03:45,160 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:03:55,315 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:04:05,472 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:04:15,612 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:04:25,813 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:04:35,951 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:04:46,079 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:04:56,279 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-14 18:04:56,452 4d4e35e4 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 623, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 148, in Run
dynamic_pipeline_options)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 100, in GenerateAllPipelineOptions
EvaluateDynamicPipelineOptions(dynamic_pipeline_options))
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 67, in EvaluateDynamicPipelineOptions
argValue = RetrieveLoadBalancerIp(optionDescriptor)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 153, in RetrieveLoadBalancerIp
raise "Could not retrieve LoadBalancer IP address"
TypeError: exceptions must be old-style classes or derived from BaseException, not str
2018-04-14 18:04:56,453 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-14 18:04:56,453 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> delete -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml>
2018-04-14 18:04:56,868 4d4e35e4 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523718071836> delete -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-14 18:04:57,057 4d4e35e4 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 757, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 623, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 148, in Run
dynamic_pipeline_options)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 100, in GenerateAllPipelineOptions
EvaluateDynamicPipelineOptions(dynamic_pipeline_options))
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 67, in EvaluateDynamicPipelineOptions
argValue = RetrieveLoadBalancerIp(optionDescriptor)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 153, in RetrieveLoadBalancerIp
raise "Could not retrieve LoadBalancer IP address"
TypeError: exceptions must be old-style classes or derived from BaseException, not str
2018-04-14 18:04:57,057 4d4e35e4 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-14 18:04:57,096 4d4e35e4 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-14 18:04:57,097 4d4e35e4 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/4d4e35e4/pkb.log>
2018-04-14 18:04:57,097 4d4e35e4 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/4d4e35e4/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Jenkins build is back to normal :
beam_PerformanceTests_TextIOIT_HDFS #60
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/60/display/redirect>
Build failed in Jenkins: beam_PerformanceTests_TextIOIT_HDFS #59
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/59/display/redirect>
------------------------------------------
[...truncated 2.68 KB...]
++ kubectl config current-context
+ kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> config set-context gke_apache-beam-testing_us-central1-a_io-datastores --namespace=filebasedioithdfs-1523804455779
Context "gke_apache-beam-testing_us-central1-a_io-datastores" modified.
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins6878310545128830911.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins2372242007048551942.sh
+ rm -rf .env
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins2119092009487470249.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins3089463502605911202.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/20/d7/04a0b689d3035143e2ff288f4b9ee4bf6ed80585cc121c90bfd85a1a8c2e/setuptools-39.0.1-py2.py3-none-any.whl#md5=ca299c7acd13a72e1171a3697f2b99bc
Downloading/unpacking pip from https://pypi.python.org/packages/62/a1/0d452b6901b0157a0134fd27ba89bf95a857fbda64ba52e1ca2cf61d8412/pip-10.0.0-py2.py3-none-any.whl#md5=be3e30acf78a44cd750bf2db0912c701
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins5239769411081540346.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins5066052051091997274.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Collecting absl-py (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (39.0.1)
Collecting colorlog[windows]==2.6.0 (from -r PerfKitBenchmarker/requirements.txt (line 17))
Using cached colorlog-2.6.0-py2.py3-none-any.whl
Collecting blinker>=1.3 (from -r PerfKitBenchmarker/requirements.txt (line 18))
Collecting futures>=3.0.3 (from -r PerfKitBenchmarker/requirements.txt (line 19))
Using cached futures-3.2.0-py2-none-any.whl
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Collecting pint>=0.7 (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.2.5)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached xmltodict-0.11.0-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached ntlm_auth-1.1.0-py2.py3-none-any.whl
Collecting cryptography>=1.3 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached cryptography-2.2.2-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.1.18)
Collecting cffi>=1.7; platform_python_implementation != "PyPy" (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached cffi-1.11.5-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Collecting asn1crypto>=0.21.0 (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached asn1crypto-0.24.0-py2.py3-none-any.whl
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.19)
Collecting pycparser (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
cheetah 2.4.4 requires Markdown>=2.0.1, which is not installed.
Installing collected packages: absl-py, colorlog, blinker, futures, pint, numpy, contextlib2, xmltodict, ntlm-auth, pycparser, cffi, asn1crypto, cryptography, requests-ntlm, pywinrm
Successfully installed absl-py-0.1.13 asn1crypto-0.24.0 blinker-1.4 cffi-1.11.5 colorlog-2.6.0 contextlib2-0.5.5 cryptography-2.2.2 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pycparser-2.18 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins6871003763716056046.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached funcsigs-1.0.2-py2.py3-none-any.whl
Collecting pbr>=0.11 (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached pbr-4.0.2-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached rsa-3.4.2-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached pyasn1_modules-0.2.1-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached google_cloud_core-0.25.0-py2.py3-none-any.whl
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.1.18)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached monotonic-1.4-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached google_auth-1.4.1-py2.py3-none-any.whl
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached cachetools-2.0.1-py2.py3-none-any.whl
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
cheetah 2.4.4 requires Markdown>=2.0.1, which is not installed.
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, pbr, mock, rsa, pyasn1-modules, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, cachetools, google-auth, google-auth-httplib2, google-cloud-core, future, ply, google-gax, grpc-google-iam-v1, proto-google-cloud-pubsub-v1, gapic-google-cloud-pubsub-v1, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 pbr-4.0.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins6574483059793151988.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.textioit_hdfs_pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/> --official=true --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.text.TextIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000]' '--beam_extra_mvn_properties=[filesystem=hdfs]' --beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/pkb-config.yml> --beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml,/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT_HDFS/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-15 18:00:53,331 9a1269a9 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/9a1269a9/pkb.log>
2018-04-15 18:00:53,332 9a1269a9 MainThread INFO PerfKitBenchmarker version: v1.12.0-489-gf0acf31
2018-04-15 18:00:53,333 9a1269a9 MainThread INFO Flag values:
--beam_extra_mvn_properties=[filesystem=hdfs]
--beam_it_class=org.apache.beam.sdk.io.text.TextIOIT
--beam_it_timeout=1200
--beam_sdk=java
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/>
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000]
--beam_prebuilt
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779>
--project=apache-beam-testing
--beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml,/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT_HDFS/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
--bigquery_table=beam_performance.textioit_hdfs_pkb_results
--official
--beam_it_module=sdks/java/io/file-based-io-tests
--dpb_log_level=INFO
--beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/pkb-config.yml>
--beam_it_profile=io-it
--benchmarks=beam_integration_benchmark
2018-04-15 18:00:53,799 9a1269a9 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-15 18:00:53,800 9a1269a9 MainThread INFO Initializing the edw service decoder
2018-04-15 18:00:54,035 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-15 18:00:54,039 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-15 18:00:54,040 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-15 18:01:05,989 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> create -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml>
2018-04-15 18:01:06,628 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> create -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-15 18:01:07,038 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-15 18:01:07,046 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:01:17,387 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:01:27,879 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:01:38,023 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:01:48,155 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:01:58,311 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:02:08,487 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:02:18,627 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:02:28,768 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:02:38,956 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:02:49,096 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:02:59,231 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:03:09,403 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:03:19,547 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:03:29,683 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:03:39,847 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:03:49,996 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:04:00,155 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:04:10,307 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 18:04:10,454 9a1269a9 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 623, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 148, in Run
dynamic_pipeline_options)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 100, in GenerateAllPipelineOptions
EvaluateDynamicPipelineOptions(dynamic_pipeline_options))
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 67, in EvaluateDynamicPipelineOptions
argValue = RetrieveLoadBalancerIp(optionDescriptor)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 153, in RetrieveLoadBalancerIp
raise "Could not retrieve LoadBalancer IP address"
TypeError: exceptions must be old-style classes or derived from BaseException, not str
2018-04-15 18:04:10,455 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-15 18:04:10,455 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> delete -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml>
2018-04-15 18:04:10,888 9a1269a9 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523804455779> delete -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-15 18:04:11,079 9a1269a9 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 757, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 623, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 148, in Run
dynamic_pipeline_options)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 100, in GenerateAllPipelineOptions
EvaluateDynamicPipelineOptions(dynamic_pipeline_options))
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 67, in EvaluateDynamicPipelineOptions
argValue = RetrieveLoadBalancerIp(optionDescriptor)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 153, in RetrieveLoadBalancerIp
raise "Could not retrieve LoadBalancer IP address"
TypeError: exceptions must be old-style classes or derived from BaseException, not str
2018-04-15 18:04:11,079 9a1269a9 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-15 18:04:11,131 9a1269a9 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-15 18:04:11,132 9a1269a9 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/9a1269a9/pkb.log>
2018-04-15 18:04:11,132 9a1269a9 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/9a1269a9/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_TextIOIT_HDFS #58
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/58/display/redirect>
------------------------------------------
[...truncated 2.68 KB...]
++ kubectl config current-context
+ kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> config set-context gke_apache-beam-testing_us-central1-a_io-datastores --namespace=filebasedioithdfs-1523786457633
Context "gke_apache-beam-testing_us-central1-a_io-datastores" modified.
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins7983171130588259476.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins8965244339551324314.sh
+ rm -rf .env
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins299824119634525188.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins2538520178098981919.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/20/d7/04a0b689d3035143e2ff288f4b9ee4bf6ed80585cc121c90bfd85a1a8c2e/setuptools-39.0.1-py2.py3-none-any.whl#md5=ca299c7acd13a72e1171a3697f2b99bc
Downloading/unpacking pip from https://pypi.python.org/packages/62/a1/0d452b6901b0157a0134fd27ba89bf95a857fbda64ba52e1ca2cf61d8412/pip-10.0.0-py2.py3-none-any.whl#md5=be3e30acf78a44cd750bf2db0912c701
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins304372835887973553.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins3491174878926685977.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Collecting absl-py (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (39.0.1)
Collecting colorlog[windows]==2.6.0 (from -r PerfKitBenchmarker/requirements.txt (line 17))
Using cached colorlog-2.6.0-py2.py3-none-any.whl
Collecting blinker>=1.3 (from -r PerfKitBenchmarker/requirements.txt (line 18))
Collecting futures>=3.0.3 (from -r PerfKitBenchmarker/requirements.txt (line 19))
Using cached futures-3.2.0-py2-none-any.whl
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Collecting pint>=0.7 (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.2.5)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached xmltodict-0.11.0-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached ntlm_auth-1.1.0-py2.py3-none-any.whl
Collecting cryptography>=1.3 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached cryptography-2.2.2-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.1.18)
Collecting cffi>=1.7; platform_python_implementation != "PyPy" (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached cffi-1.11.5-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Collecting asn1crypto>=0.21.0 (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached asn1crypto-0.24.0-py2.py3-none-any.whl
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.19)
Collecting pycparser (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
cheetah 2.4.4 requires Markdown>=2.0.1, which is not installed.
Installing collected packages: absl-py, colorlog, blinker, futures, pint, numpy, contextlib2, xmltodict, ntlm-auth, pycparser, cffi, asn1crypto, cryptography, requests-ntlm, pywinrm
Successfully installed absl-py-0.1.13 asn1crypto-0.24.0 blinker-1.4 cffi-1.11.5 colorlog-2.6.0 contextlib2-0.5.5 cryptography-2.2.2 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pycparser-2.18 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins4581065846866352503.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached funcsigs-1.0.2-py2.py3-none-any.whl
Collecting pbr>=0.11 (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached pbr-4.0.2-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached rsa-3.4.2-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached pyasn1_modules-0.2.1-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached google_cloud_core-0.25.0-py2.py3-none-any.whl
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.1.18)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached monotonic-1.4-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached google_auth-1.4.1-py2.py3-none-any.whl
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached cachetools-2.0.1-py2.py3-none-any.whl
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
cheetah 2.4.4 requires Markdown>=2.0.1, which is not installed.
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, pbr, mock, rsa, pyasn1-modules, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, cachetools, google-auth, google-auth-httplib2, google-cloud-core, future, ply, google-gax, grpc-google-iam-v1, proto-google-cloud-pubsub-v1, gapic-google-cloud-pubsub-v1, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 pbr-4.0.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins287748508646749734.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.textioit_hdfs_pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/> --official=true --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.text.TextIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000]' '--beam_extra_mvn_properties=[filesystem=hdfs]' --beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/pkb-config.yml> --beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml,/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT_HDFS/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-15 12:01:30,745 95451a40 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/95451a40/pkb.log>
2018-04-15 12:01:30,745 95451a40 MainThread INFO PerfKitBenchmarker version: v1.12.0-489-gf0acf31
2018-04-15 12:01:30,746 95451a40 MainThread INFO Flag values:
--beam_extra_mvn_properties=[filesystem=hdfs]
--beam_it_class=org.apache.beam.sdk.io.text.TextIOIT
--beam_it_timeout=1200
--beam_sdk=java
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/>
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000]
--beam_prebuilt
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633>
--project=apache-beam-testing
--beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml,/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT_HDFS/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
--bigquery_table=beam_performance.textioit_hdfs_pkb_results
--official
--beam_it_module=sdks/java/io/file-based-io-tests
--dpb_log_level=INFO
--beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/pkb-config.yml>
--beam_it_profile=io-it
--benchmarks=beam_integration_benchmark
2018-04-15 12:01:30,959 95451a40 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-15 12:01:30,960 95451a40 MainThread INFO Initializing the edw service decoder
2018-04-15 12:01:31,120 95451a40 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-15 12:01:31,124 95451a40 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-15 12:01:31,125 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-15 12:01:45,742 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> create -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml>
2018-04-15 12:01:46,336 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> create -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-15 12:01:46,753 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-15 12:01:46,762 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:01:57,057 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:02:07,183 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:02:17,303 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:02:27,423 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:02:37,543 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:02:47,667 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:02:57,791 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:03:07,947 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:03:18,075 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:03:28,199 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:03:38,332 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:03:48,487 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:03:58,619 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:04:08,745 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:04:18,873 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:04:28,995 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:04:39,119 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:04:49,247 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 12:04:49,365 95451a40 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 623, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 148, in Run
dynamic_pipeline_options)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 100, in GenerateAllPipelineOptions
EvaluateDynamicPipelineOptions(dynamic_pipeline_options))
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 67, in EvaluateDynamicPipelineOptions
argValue = RetrieveLoadBalancerIp(optionDescriptor)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 153, in RetrieveLoadBalancerIp
raise "Could not retrieve LoadBalancer IP address"
TypeError: exceptions must be old-style classes or derived from BaseException, not str
2018-04-15 12:04:49,366 95451a40 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-15 12:04:49,367 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> delete -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml>
2018-04-15 12:04:49,805 95451a40 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523786457633> delete -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-15 12:04:49,974 95451a40 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 757, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 623, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 148, in Run
dynamic_pipeline_options)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 100, in GenerateAllPipelineOptions
EvaluateDynamicPipelineOptions(dynamic_pipeline_options))
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 67, in EvaluateDynamicPipelineOptions
argValue = RetrieveLoadBalancerIp(optionDescriptor)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 153, in RetrieveLoadBalancerIp
raise "Could not retrieve LoadBalancer IP address"
TypeError: exceptions must be old-style classes or derived from BaseException, not str
2018-04-15 12:04:49,974 95451a40 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-15 12:04:50,024 95451a40 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-15 12:04:50,025 95451a40 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/95451a40/pkb.log>
2018-04-15 12:04:50,025 95451a40 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/95451a40/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_TextIOIT_HDFS #57
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/57/display/redirect>
------------------------------------------
[...truncated 2.68 KB...]
++ kubectl config current-context
+ kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> config set-context gke_apache-beam-testing_us-central1-a_io-datastores --namespace=filebasedioithdfs-1523768474283
Context "gke_apache-beam-testing_us-central1-a_io-datastores" modified.
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins5583167697139045339.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins4114323711429810826.sh
+ rm -rf .env
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins3138658966907924222.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins8514072739236970466.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/20/d7/04a0b689d3035143e2ff288f4b9ee4bf6ed80585cc121c90bfd85a1a8c2e/setuptools-39.0.1-py2.py3-none-any.whl#md5=ca299c7acd13a72e1171a3697f2b99bc
Downloading/unpacking pip from https://pypi.python.org/packages/62/a1/0d452b6901b0157a0134fd27ba89bf95a857fbda64ba52e1ca2cf61d8412/pip-10.0.0-py2.py3-none-any.whl#md5=be3e30acf78a44cd750bf2db0912c701
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins7100373285173802804.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins7924670090050057243.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Collecting absl-py (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (39.0.1)
Collecting colorlog[windows]==2.6.0 (from -r PerfKitBenchmarker/requirements.txt (line 17))
Using cached colorlog-2.6.0-py2.py3-none-any.whl
Collecting blinker>=1.3 (from -r PerfKitBenchmarker/requirements.txt (line 18))
Collecting futures>=3.0.3 (from -r PerfKitBenchmarker/requirements.txt (line 19))
Using cached futures-3.2.0-py2-none-any.whl
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Collecting pint>=0.7 (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.2.5)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached xmltodict-0.11.0-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached ntlm_auth-1.1.0-py2.py3-none-any.whl
Collecting cryptography>=1.3 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached cryptography-2.2.2-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.1.18)
Collecting cffi>=1.7; platform_python_implementation != "PyPy" (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached cffi-1.11.5-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Collecting asn1crypto>=0.21.0 (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached asn1crypto-0.24.0-py2.py3-none-any.whl
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.19)
Collecting pycparser (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
cheetah 2.4.4 requires Markdown>=2.0.1, which is not installed.
Installing collected packages: absl-py, colorlog, blinker, futures, pint, numpy, contextlib2, xmltodict, ntlm-auth, pycparser, cffi, asn1crypto, cryptography, requests-ntlm, pywinrm
Successfully installed absl-py-0.1.13 asn1crypto-0.24.0 blinker-1.4 cffi-1.11.5 colorlog-2.6.0 contextlib2-0.5.5 cryptography-2.2.2 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pycparser-2.18 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins3313430394493620420.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached funcsigs-1.0.2-py2.py3-none-any.whl
Collecting pbr>=0.11 (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached pbr-4.0.2-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached rsa-3.4.2-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached pyasn1_modules-0.2.1-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached google_cloud_core-0.25.0-py2.py3-none-any.whl
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.1.18)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached monotonic-1.4-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached google_auth-1.4.1-py2.py3-none-any.whl
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached cachetools-2.0.1-py2.py3-none-any.whl
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
cheetah 2.4.4 requires Markdown>=2.0.1, which is not installed.
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, pbr, mock, rsa, pyasn1-modules, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, cachetools, google-auth, google-auth-httplib2, google-cloud-core, future, ply, google-gax, grpc-google-iam-v1, proto-google-cloud-pubsub-v1, gapic-google-cloud-pubsub-v1, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 pbr-4.0.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins3109755306277138530.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.textioit_hdfs_pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/> --official=true --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.text.TextIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000]' '--beam_extra_mvn_properties=[filesystem=hdfs]' --beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/pkb-config.yml> --beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml,/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT_HDFS/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-15 06:01:26,740 fcc7b4ba MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/fcc7b4ba/pkb.log>
2018-04-15 06:01:26,740 fcc7b4ba MainThread INFO PerfKitBenchmarker version: v1.12.0-489-gf0acf31
2018-04-15 06:01:26,741 fcc7b4ba MainThread INFO Flag values:
--beam_extra_mvn_properties=[filesystem=hdfs]
--beam_it_class=org.apache.beam.sdk.io.text.TextIOIT
--beam_it_timeout=1200
--beam_sdk=java
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/>
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000]
--beam_prebuilt
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283>
--project=apache-beam-testing
--beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml,/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT_HDFS/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
--bigquery_table=beam_performance.textioit_hdfs_pkb_results
--official
--beam_it_module=sdks/java/io/file-based-io-tests
--dpb_log_level=INFO
--beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/pkb-config.yml>
--beam_it_profile=io-it
--benchmarks=beam_integration_benchmark
2018-04-15 06:01:26,977 fcc7b4ba MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-15 06:01:26,977 fcc7b4ba MainThread INFO Initializing the edw service decoder
2018-04-15 06:01:27,118 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-15 06:01:27,121 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-15 06:01:27,121 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-15 06:01:35,635 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> create -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml>
2018-04-15 06:01:35,945 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> create -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-15 06:01:36,147 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-15 06:01:36,151 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:01:46,315 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:01:56,460 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:02:06,605 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:02:16,775 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:02:26,903 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:02:37,035 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:02:47,279 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:02:57,411 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:03:07,543 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:03:17,675 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:03:27,811 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:03:37,952 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:03:48,084 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:03:58,219 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:04:08,347 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:04:18,483 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:04:28,623 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:04:38,748 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 06:04:38,956 fcc7b4ba MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 623, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 148, in Run
dynamic_pipeline_options)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 100, in GenerateAllPipelineOptions
EvaluateDynamicPipelineOptions(dynamic_pipeline_options))
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 67, in EvaluateDynamicPipelineOptions
argValue = RetrieveLoadBalancerIp(optionDescriptor)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 153, in RetrieveLoadBalancerIp
raise "Could not retrieve LoadBalancer IP address"
TypeError: exceptions must be old-style classes or derived from BaseException, not str
2018-04-15 06:04:38,957 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-15 06:04:38,957 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> delete -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml>
2018-04-15 06:04:39,518 fcc7b4ba MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523768474283> delete -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-15 06:04:39,713 fcc7b4ba MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 757, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 623, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 148, in Run
dynamic_pipeline_options)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 100, in GenerateAllPipelineOptions
EvaluateDynamicPipelineOptions(dynamic_pipeline_options))
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 67, in EvaluateDynamicPipelineOptions
argValue = RetrieveLoadBalancerIp(optionDescriptor)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 153, in RetrieveLoadBalancerIp
raise "Could not retrieve LoadBalancer IP address"
TypeError: exceptions must be old-style classes or derived from BaseException, not str
2018-04-15 06:04:39,713 fcc7b4ba MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-15 06:04:39,758 fcc7b4ba MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-15 06:04:39,759 fcc7b4ba MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/fcc7b4ba/pkb.log>
2018-04-15 06:04:39,759 fcc7b4ba MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/fcc7b4ba/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_TextIOIT_HDFS #56
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/56/display/redirect>
------------------------------------------
[...truncated 2.68 KB...]
++ kubectl config current-context
+ kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> config set-context gke_apache-beam-testing_us-central1-a_io-datastores --namespace=filebasedioithdfs-1523736118453
Context "gke_apache-beam-testing_us-central1-a_io-datastores" modified.
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins4916775981211956120.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins3730215378066656185.sh
+ rm -rf .env
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins7633027985717570783.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins4762508037952693798.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/20/d7/04a0b689d3035143e2ff288f4b9ee4bf6ed80585cc121c90bfd85a1a8c2e/setuptools-39.0.1-py2.py3-none-any.whl#md5=ca299c7acd13a72e1171a3697f2b99bc
Downloading/unpacking pip from https://pypi.python.org/packages/62/a1/0d452b6901b0157a0134fd27ba89bf95a857fbda64ba52e1ca2cf61d8412/pip-10.0.0-py2.py3-none-any.whl#md5=be3e30acf78a44cd750bf2db0912c701
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins2268875462006793921.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins3352635416933982408.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Collecting absl-py (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (39.0.1)
Collecting colorlog[windows]==2.6.0 (from -r PerfKitBenchmarker/requirements.txt (line 17))
Using cached colorlog-2.6.0-py2.py3-none-any.whl
Collecting blinker>=1.3 (from -r PerfKitBenchmarker/requirements.txt (line 18))
Collecting futures>=3.0.3 (from -r PerfKitBenchmarker/requirements.txt (line 19))
Using cached futures-3.2.0-py2-none-any.whl
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Collecting pint>=0.7 (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.2.5)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached xmltodict-0.11.0-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached ntlm_auth-1.1.0-py2.py3-none-any.whl
Collecting cryptography>=1.3 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached cryptography-2.2.2-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.1.18)
Collecting cffi>=1.7; platform_python_implementation != "PyPy" (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached cffi-1.11.5-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Collecting asn1crypto>=0.21.0 (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached asn1crypto-0.24.0-py2.py3-none-any.whl
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.19)
Collecting pycparser (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
cheetah 2.4.4 requires Markdown>=2.0.1, which is not installed.
Installing collected packages: absl-py, colorlog, blinker, futures, pint, numpy, contextlib2, xmltodict, ntlm-auth, pycparser, cffi, asn1crypto, cryptography, requests-ntlm, pywinrm
Successfully installed absl-py-0.1.13 asn1crypto-0.24.0 blinker-1.4 cffi-1.11.5 colorlog-2.6.0 contextlib2-0.5.5 cryptography-2.2.2 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pycparser-2.18 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins6269677040348777374.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached funcsigs-1.0.2-py2.py3-none-any.whl
Collecting pbr>=0.11 (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached pbr-4.0.2-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached rsa-3.4.2-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached pyasn1_modules-0.2.1-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached google_cloud_core-0.25.0-py2.py3-none-any.whl
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.1.18)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached monotonic-1.4-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached google_auth-1.4.1-py2.py3-none-any.whl
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached cachetools-2.0.1-py2.py3-none-any.whl
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
cheetah 2.4.4 requires Markdown>=2.0.1, which is not installed.
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, pbr, mock, rsa, pyasn1-modules, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, cachetools, google-auth, google-auth-httplib2, google-cloud-core, future, ply, google-gax, grpc-google-iam-v1, proto-google-cloud-pubsub-v1, gapic-google-cloud-pubsub-v1, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 pbr-4.0.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins7274516565113629463.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.textioit_hdfs_pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/> --official=true --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.text.TextIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000]' '--beam_extra_mvn_properties=[filesystem=hdfs]' --beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/pkb-config.yml> --beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml,/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT_HDFS/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-15 00:01:40,343 fd756e26 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/fd756e26/pkb.log>
2018-04-15 00:01:40,343 fd756e26 MainThread INFO PerfKitBenchmarker version: v1.12.0-489-gf0acf31
2018-04-15 00:01:40,344 fd756e26 MainThread INFO Flag values:
--beam_extra_mvn_properties=[filesystem=hdfs]
--beam_it_class=org.apache.beam.sdk.io.text.TextIOIT
--beam_it_timeout=1200
--beam_sdk=java
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/>
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000]
--beam_prebuilt
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453>
--project=apache-beam-testing
--beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml,/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT_HDFS/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
--bigquery_table=beam_performance.textioit_hdfs_pkb_results
--official
--beam_it_module=sdks/java/io/file-based-io-tests
--dpb_log_level=INFO
--beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/pkb-config.yml>
--beam_it_profile=io-it
--benchmarks=beam_integration_benchmark
2018-04-15 00:01:40,665 fd756e26 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-15 00:01:40,666 fd756e26 MainThread INFO Initializing the edw service decoder
2018-04-15 00:01:40,834 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-15 00:01:40,838 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-15 00:01:40,838 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-15 00:01:49,204 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> create -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml>
2018-04-15 00:01:49,501 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> create -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-15 00:01:49,705 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-15 00:01:49,710 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:01:59,931 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:02:10,075 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:02:20,211 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:02:30,383 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:02:40,539 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:02:50,684 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:03:00,859 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:03:11,043 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:03:21,167 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:03:31,357 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:03:41,507 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:03:51,643 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:04:01,805 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:04:11,939 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:04:22,079 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:04:32,230 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:04:42,379 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:04:52,532 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-15 00:04:52,671 fd756e26 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 623, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 148, in Run
dynamic_pipeline_options)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 100, in GenerateAllPipelineOptions
EvaluateDynamicPipelineOptions(dynamic_pipeline_options))
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 67, in EvaluateDynamicPipelineOptions
argValue = RetrieveLoadBalancerIp(optionDescriptor)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 153, in RetrieveLoadBalancerIp
raise "Could not retrieve LoadBalancer IP address"
TypeError: exceptions must be old-style classes or derived from BaseException, not str
2018-04-15 00:04:52,672 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-15 00:04:52,672 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> delete -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml>
2018-04-15 00:04:53,105 fd756e26 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-filebasedioithdfs-1523736118453> delete -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-15 00:04:53,287 fd756e26 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 757, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 623, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 148, in Run
dynamic_pipeline_options)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 100, in GenerateAllPipelineOptions
EvaluateDynamicPipelineOptions(dynamic_pipeline_options))
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 67, in EvaluateDynamicPipelineOptions
argValue = RetrieveLoadBalancerIp(optionDescriptor)
File "<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",> line 153, in RetrieveLoadBalancerIp
raise "Could not retrieve LoadBalancer IP address"
TypeError: exceptions must be old-style classes or derived from BaseException, not str
2018-04-15 00:04:53,288 fd756e26 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-15 00:04:53,343 fd756e26 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-15 00:04:53,343 fd756e26 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/fd756e26/pkb.log>
2018-04-15 00:04:53,344 fd756e26 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/fd756e26/completion_statuses.json>
Build step 'Execute shell' marked build as failure