You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/10/01 12:06:51 UTC
Build failed in Jenkins: beam_Dependency_Check #153
See <https://builds.apache.org/job/beam_Dependency_Check/153/display/redirect?page=changes>
Changes:
[pablo] Adding display data to BQ write transform
[qinyeli] Interactive Beam -- read_cache_ids and write_cache_ids
[qinyeli] Interactive Beam -- renaming variables and functions
[afulton] Fix unstable test by ensuring acks are processed synchronously
[andrew] Ensuring Spotless Java compliance
[qinyeli] Interactive Beam -- fixing PTransform # display issue
[jeff] Composite transform lambdas via PTransform::compose
[pablo] Updating BigQuerySink pydoc
[huangry] Refactor CombineGroupValues by using existing function in CombineFn.
[github] typo Python -> Go
[mxm] Add tests for port supplier methods in ServerFactory
[ptomasroos] Fix combine.go to run on dataflow
[github] Update TODO to reference Jira
[github] Fix sliding time windows example documentation
[github] Update session windows Python to match description
[aaltay] [BEAM-5209] Precommit timings notebook: fix illegal characters in URL,
[shaun] added avroio package
[pablo] [BEAM-5240] Add metrics dashboard deployment script and logic (#6277)
[aaltay] Remove standard_library.import_aliases() calls (#6471)
[github] Fix sliding time windows example documentation
[pablo] Revert "[BEAM-5240] Add metrics dashboard deployment script and logic
[mergebot] Fix a few typos in beam site docs
[shaun] updated read emits to support both string and custom type reflects
[apilloud] [BEAM-5491] Update NightlySnapshot job name
[github] [BEAM-5378] Remove go wordcap example Dataflow runner (#6395)
[ehudm] GCSIO: Allow empty object prefix in list_prefix().
[shaun] added avro write support
[github] Trigger pre-commit checks
[swegner] Upgrade assertj dependecny to latest (3.11.1)
[github] Improved Flatten input error message
[mergebot] [BEAM-5436] Improve docs for Go SDK
[github] Undo changes to website programming guide :rewind:
[migryz] Add metrics dashboard deployment script and logic
[migryz] Fix rat issues
[aaltay] [BEAM-1251] Upgrade pylint version for py27-lint3 (#6489)
[amaliujia] [BEAM-5506] Add reference link.
[aaltay] [BEAM-5319] Partially port runners (#6451)
[valentyn] Fix futurize-based checker in py27-lint3 compatibility tox test suite.
[ehudm] Pass --pubsubRooturl option to Dataflow runner.
[pablo] Fixing Py2-3 lint issue
[github] fix typo
[ccy] Revert "[BEAM-4747] mkdirs if they don't exist in localfilesystem
[ankurgoenka] Add PortablePipelineOption for environment
[ankurgoenka] Lazy initializing JobBundleFactory supporting multiple type of
[shaun] updated to be in-line with beam project specifications
[shaun] update package log prints
[shaun] added readavro example
[shaun] updated example package header
[shaun] removed output.avro file
[kedin] Fix Java11 Jira link
[github] Adding clarification to use WriteToBigQuery
[amyrvold] Website publiish task and Jenkins job
[yifanzou] [BEAM-5339] Modify the dependency tool based on new JIRA policy.
[ankurgoenka] Adding environment to python portable options
[ankurgoenka] Go Changes for using configurable environment. Only docker is supported
[github] Fix trailing whitespace
[daniel.o.programmer] [BEAM-5304] Adding ReferenceRunner Job Server Gradle subproject.
[robertwb] [BEAM-5270] Fix ToString coder to return bytes objects in Python 3.
[kevinsi] When getting display data from a runtime parameter, don't call get().
[kevinsi] Randomize the reduced splits in BigtableIO so that multiple workers may
[scott] [BEAM-5518] Ignore failing ssl validation of globenewswire (#6502)
[pablo] Updating Dataflow API protocol buffers
[altay] update dataflow container name
[thw] [BEAM-5509] Fix int/long pipeline option serialization in
[robertwb] [BEAM-5500] Fix memory leak in pickler.
[thw] [BEAM-5443] Pipeline option defaults for portable runner.
[scott] [BEAM-4496] Fix branch fetch failure on job_PostCommit_Website_Publish
------------------------------------------
[...truncated 487.05 KB...]
if ! [ -z "${PYTHONHOME+_}" ] ; then
_OLD_VIRTUAL_PYTHONHOME="$PYTHONHOME"
unset PYTHONHOME
fi
if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT-}" ] ; then
_OLD_VIRTUAL_PS1="$PS1"
if [ "x" != x ] ; then
PS1="$PS1"
else
PS1="(`basename \"$VIRTUAL_ENV\"`) $PS1"
fi
export PS1
fi
basename "$VIRTUAL_ENV"
# Make sure to unalias pydoc if it's already there
alias pydoc 2>/dev/null >/dev/null && unalias pydoc
pydoc () {
python -m pydoc "$@"
}
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
hash -r 2>/dev/null
fi
pip install --upgrade google-cloud-bigquery
Collecting google-cloud-bigquery
Using cached https://files.pythonhosted.org/packages/27/0b/4b8d48d602e12766187203767ebcf213f9e8297e1e1a38b714d5f921d5f6/google_cloud_bigquery-1.5.1-py2.py3-none-any.whl
Collecting google-api-core<2.0.0dev,>=1.0.0 (from google-cloud-bigquery)
Downloading https://files.pythonhosted.org/packages/a5/be/de30100034c391f4c56e2543f1507eb1b30b3030bd9a6764dd6cfe7a954e/google_api_core-1.4.1-py2.py3-none-any.whl (53kB)
Collecting google-cloud-core<0.29dev,>=0.28.0 (from google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/0f/41/ae2418b4003a14cf21c1c46d61d1b044bf02cf0f8f91598af572b9216515/google_cloud_core-0.28.1-py2.py3-none-any.whl
Collecting google-resumable-media>=0.2.1 (from google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/77/95/2e4020a54366423ddba715f89fb7ca456c8f048b15cada6cd6a54cf10e8c/google_resumable_media-0.3.1-py2.py3-none-any.whl
Collecting protobuf>=3.4.0 (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/b8/c2/b7f587c0aaf8bf2201405e8162323037fe8d17aa21d3c7dda811b8d01469/protobuf-3.6.1-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied, skipping upgrade: setuptools>=34.0.0 in ./dependency/check/lib/python2.7/site-packages (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery) (40.4.3)
Collecting six>=1.10.0 (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/58/cb/96dbb4e50e7a9d856e89cc9c8e36ab1055f9774f7d85f37e2156c1d79d9f/google_auth-1.5.1-py2.py3-none-any.whl
Collecting futures>=3.2.0; python_version < "3.2" (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/2d/99/b2c4e9d5a30f6471e410a146232b4118e697fa3ffc06d6a65efde84debd0/futures-3.2.0-py2-none-any.whl
Collecting pytz (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/30/4e/27c34b62430286c6d59177a0842ed90dc789ce5d1ed740887653b898779a/pytz-2018.5-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.3 (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Collecting requests<3.0.0dev,>=2.18.0 (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/65/47/7e02164a2a3db50ed6d8a6ab1d6d60b69c4c3fdf57a284257925dfc12bda/requests-2.19.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from google-auth<2.0.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/0a/58/cbee863250b31d80f47401d04f34038db6766f95dea1cc909ea099c7e571/cachetools-2.1.0-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1 (from google-auth<2.0.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/19/02/fa63f7ba30a0d7b925ca29d034510fc1ffde53264b71b4155022ddf3ab5d/pyasn1_modules-0.2.2-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/df/f7/04fee6ac349e915b82171f8e23cee63644d83663b34c539f7a09aed18f9e/certifi-2018.8.24-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting urllib3<1.24,>=1.21.1 (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/bd/c9/6fdd990019071a4a32a5e7cb78a1d92c53851ef4f56f62a3486e6a7d8ffb/urllib3-1.23-py2.py3-none-any.whl
Collecting idna<2.8,>=2.5 (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3 (from rsa>=3.1.4->google-auth<2.0.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/d1/a1/7790cc85db38daa874f6a2e6308131b9953feb1367f2ae2d1123bb93a9f5/pyasn1-0.4.4-py2.py3-none-any.whl
Installing collected packages: six, protobuf, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, futures, pytz, googleapis-common-protos, certifi, chardet, urllib3, idna, requests, google-api-core, google-cloud-core, google-resumable-media, google-cloud-bigquery
Successfully installed cachetools-2.1.0 certifi-2018.8.24 chardet-3.0.4 futures-3.2.0 google-api-core-1.4.1 google-auth-1.5.1 google-cloud-bigquery-1.5.1 google-cloud-core-0.28.1 google-resumable-media-0.3.1 googleapis-common-protos-1.5.3 idna-2.7 protobuf-3.6.1 pyasn1-0.4.4 pyasn1-modules-0.2.2 pytz-2018.5 requests-2.19.1 rsa-4.0 six-1.11.0 urllib3-1.23
rm -f build/dependencyUpdates/beam-dependency-check-report.txt
# Insall packages and run the unit tests of the report generator and the jira manager
pip install mock jira pyyaml
Collecting mock
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting jira
Using cached https://files.pythonhosted.org/packages/cb/c2/4203b6cad72d115b0ad62ae7028567b538c8933745f665d0eee5082baf67/jira-2.0.0-py2.py3-none-any.whl
Collecting pyyaml
Collecting pbr>=0.11 (from mock)
Using cached https://files.pythonhosted.org/packages/69/1c/98cba002ed975a91a0294863d9c774cc0ebe38e05bbb65e83314550b1677/pbr-4.2.0-py2.py3-none-any.whl
Collecting funcsigs>=1; python_version < "3.3" (from mock)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: six>=1.9 in ./dependency/check/lib/python2.7/site-packages (from mock) (1.11.0)
Requirement already satisfied: requests>=2.10.0 in ./dependency/check/lib/python2.7/site-packages (from jira) (2.19.1)
Collecting oauthlib[signedtoken]>=1.0.0 (from jira)
Using cached https://files.pythonhosted.org/packages/e6/d1/ddd9cfea3e736399b97ded5c2dd62d1322adef4a72d816f1ed1049d6a179/oauthlib-2.1.0-py2.py3-none-any.whl
Collecting defusedxml (from jira)
Using cached https://files.pythonhosted.org/packages/87/1c/17f3e3935a913dfe2a5ca85fa5ccbef366bfd82eb318b1f75dadbf0affca/defusedxml-0.5.0-py2.py3-none-any.whl
Collecting requests-toolbelt (from jira)
Using cached https://files.pythonhosted.org/packages/97/8a/d710f792d6f6ecc089c5e55b66e66c3f2f35516a1ede5a8f54c13350ffb0/requests_toolbelt-0.8.0-py2.py3-none-any.whl
Collecting requests-oauthlib>=0.6.1 (from jira)
Using cached https://files.pythonhosted.org/packages/94/e7/c250d122992e1561690d9c0f7856dadb79d61fd4bdd0e598087dce607f6c/requests_oauthlib-1.0.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=20.10.1 in ./dependency/check/lib/python2.7/site-packages (from jira) (40.4.3)
Requirement already satisfied: certifi>=2017.4.17 in ./dependency/check/lib/python2.7/site-packages (from requests>=2.10.0->jira) (2018.8.24)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./dependency/check/lib/python2.7/site-packages (from requests>=2.10.0->jira) (3.0.4)
Requirement already satisfied: urllib3<1.24,>=1.21.1 in ./dependency/check/lib/python2.7/site-packages (from requests>=2.10.0->jira) (1.23)
Requirement already satisfied: idna<2.8,>=2.5 in ./dependency/check/lib/python2.7/site-packages (from requests>=2.10.0->jira) (2.7)
Collecting cryptography; extra == "signedtoken" (from oauthlib[signedtoken]>=1.0.0->jira)
Using cached https://files.pythonhosted.org/packages/87/e6/915a482dbfef98bbdce6be1e31825f591fc67038d4ee09864c1d2c3db371/cryptography-2.3.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pyjwt>=1.0.0; extra == "signedtoken" (from oauthlib[signedtoken]>=1.0.0->jira)
Using cached https://files.pythonhosted.org/packages/93/d1/3378cc8184a6524dc92993090ee8b4c03847c567e298305d6cf86987e005/PyJWT-1.6.4-py2.py3-none-any.whl
Collecting enum34; python_version < "3" (from cryptography; extra == "signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting asn1crypto>=0.21.0 (from cryptography; extra == "signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
Using cached https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl
Collecting cffi!=1.11.3,>=1.7 (from cryptography; extra == "signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
Using cached https://files.pythonhosted.org/packages/14/dd/3e7a1e1280e7d767bd3fa15791759c91ec19058ebe31217fe66f3e9a8c49/cffi-1.11.5-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3" (from cryptography; extra == "signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
Using cached https://files.pythonhosted.org/packages/fc/d0/7fc3a811e011d4b388be48a0e381db8d990042df54aa4ef4599a31d39853/ipaddress-1.0.22-py2.py3-none-any.whl
Collecting pycparser (from cffi!=1.11.3,>=1.7->cryptography; extra == "signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
Installing collected packages: pbr, funcsigs, mock, enum34, asn1crypto, pycparser, cffi, ipaddress, cryptography, pyjwt, oauthlib, defusedxml, requests-toolbelt, requests-oauthlib, jira, pyyaml
Successfully installed asn1crypto-0.24.0 cffi-1.11.5 cryptography-2.3.1 defusedxml-0.5.0 enum34-1.1.6 funcsigs-1.0.2 ipaddress-1.0.22 jira-2.0.0 mock-2.0.0 oauthlib-2.1.0 pbr-4.2.0 pycparser-2.19 pyjwt-1.6.4 pyyaml-3.13 requests-oauthlib-1.0.0 requests-toolbelt-0.8.0
cd $WORKSPACE/src/.test-infra/jenkins
python -m dependency_check.dependency_check_report_generator_test
INFO:root:
Start processing: - group1:artifact1 [Release1-123 -> Release2-456]
.INFO:root:
Start processing: - group1:artifact1 [0.rc1.0 -> 0.rc2.0]
.INFO:root:Clean Up Starts
.INFO:root:
Start processing: - group1:artifact1 (1.0.0, 2.0.0)
Traceback (most recent call last):
File "dependency_check/dependency_check_report_generator.py", line 107, in prioritize_dependencies
File "dependency_check/dependency_check_report_generator.py", line 76, in extract_single_dep
InvalidFormatError: Failed to extract the dependency information: - group1:artifact1 (1.0.0, 2.0.0)
INFO:root:
Start processing: - group2:artifact2 [1.0.0 -> 2.0.0]
.INFO:root:
Start processing: - group1:artifact1 [1.0.0 -> 3.0.0]
INFO:root:
Start processing: - group2:artifact2 [1.0.0 -> 1.3.0]
INFO:root:
Start processing: - group3:artifact3 [1.0.0 -> 1.1.0]
INFO:root:
Start processing: - group4:artifact4 [1.0.0 -> 1.1.0]
.
----------------------------------------------------------------------
Ran 5 tests in 0.028s
OK
Test : test_dep_with_nondigit_major_versions
Test : test_dep_with_nondigit_minor_versions
Test : test_empty_dep_input
Test : test_invalid_dep_input
Test : test_normal_dep_input
python -m jira_utils.jira_manager_test
INFO:root:Start handling the JIRA issues for Python dependency: dep0 1.0.0
INFO:root:Creating a new JIRA issue to track dep0 upgrade process
INFO:root:Found owners of dep0: owner0, owner1 , owner2,
INFO:root:Created a new issue <MagicMock name='JiraClient().create_issue().key' id='139931617705552'> of dep0 1.0.0
.INFO:root:Start handling the JIRA issues for Java dependency: group0:artifact0 1.0.0
INFO:root:Found the parent issue BEAM-1000. Continuous to create or update the sub-task for group0:artifact0
INFO:root:Creating a new JIRA issue to track group0:artifact0 upgrade process
INFO:root:Found owners of group0:artifact0: owner0
INFO:root:Created a new issue <MagicMock name='JiraClient().create_issue().key' id='139931617634832'> of group0:artifact0 1.0.0
.INFO:root:Start handling the JIRA issues for Java dependency: group0:artifact0 1.0.0
INFO:root:The parent issue BEAM-1000 is not opening. Attempt reopening the issue
INFO:root:Found the parent issue BEAM-1000. Continuous to create or update the sub-task for group0:artifact0
INFO:root:Creating a new JIRA issue to track group0:artifact0 upgrade process
INFO:root:Found owners of group0:artifact0: owner0
INFO:root:Created a new issue <MagicMock name='JiraClient().create_issue().key' id='139931617128720'> of group0:artifact0 1.0.0
.INFO:root:Start handling the JIRA issues for Python dependency: dep0 1.0.0
INFO:root:Updating JIRA issue BEAM-1000 to track dep0 upgrade process
WARNING:root:Could not find owners for dep0
INFO:root:Reopened the issue BEAM-1000 for dep0 1.0.0
.INFO:root:Start handling the JIRA issues for Python dependency: dep0 1.3
INFO:root:Updating JIRA issue BEAM-1000 to track dep0 upgrade process
WARNING:root:Could not find owners for dep0
INFO:root:Reopened the issue BEAM-1000 for dep0 1.3
.INFO:root:Start handling the JIRA issues for Python dependency: dep0 1.0.0
INFO:root:Updating JIRA issue BEAM-1000 to track dep0 upgrade process
WARNING:root:Could not find owners for dep0
INFO:root:Updated the existing issue BEAM-1000 of dep0 1.0.0
.
----------------------------------------------------------------------
Ran 6 tests in 0.028s
OK
Test : test_run_with_creating_new_issue
Test : test_run_with_creating_new_subtask
Test : test_run_with_reopening_existing_parent_issue
Test : test_run_with_reopening_issue_with_fixversions
Test : test_run_with_reopening_issue_with_new_release_available
Test : test_run_with_updating_existing_task
python -m dependency_check.version_comparer_test
.....
----------------------------------------------------------------------
Ran 5 tests in 0.000s
OK
Test : test_compare_major_verison_true
Test : test_compare_minor_version_false
Test : test_compare_minor_version_true
Test : test_compare_non_semantic_version_true
Test : test_compare_same_version_false
echo "<html><body>" > $WORKSPACE/src/build/dependencyUpdates/beam-dependency-check-report.html
python -m dependency_check/dependency_check_report_generator Python
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "<https://builds.apache.org/job/beam_Dependency_Check/ws/src/.test-infra/jenkins/dependency_check/dependency_check_report_generator.py",> line 25, in <module>
import version_comparer
ImportError: No module named version_comparer
Build step 'Execute shell' marked build as failure
Jenkins build is back to normal : beam_Dependency_Check #155
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_Dependency_Check/155/display/redirect>
Build failed in Jenkins: beam_Dependency_Check #154
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_Dependency_Check/154/display/redirect>
------------------------------------------
[...truncated 925.29 KB...]
echo $line | while IFS=' ' read dep curr_ver new_ver type
do
echo $line
echo " - $dep [$curr_ver -> $new_ver]" >> $WORKSPACE/src/build/dependencyUpdates/python_dependency_report.txt
done
done
google-apitools 0.5.20 0.5.24 wheel
google-cloud-bigquery 0.25.0 1.5.1 wheel
google-cloud-core 0.25.0 0.28.1 wheel
google-cloud-pubsub 0.26.0 0.38.0 wheel
google-gax 0.15.16 0.16.0 wheel
ply 3.8 3.11 wheel
pytz 2018.4 2018.5 wheel
:beam-sdks-python:dependencyUpdates (Thread[Task worker for ':',5,main]) completed. Took 35.97 secs.
:runBeamDependencyCheck (Thread[Task worker for ':',5,main]) started.
> Task :runBeamDependencyCheck
Skipping task ':runBeamDependencyCheck' as it has no actions.
:runBeamDependencyCheck (Thread[Task worker for ':',5,main]) completed. Took 0.0 secs.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 17m 52s
2 actionable tasks: 2 executed
Publishing build scan...
https://gradle.com/s/dcgp6ndej56mk
Closing Git repo: <https://builds.apache.org/job/beam_Dependency_Check/ws/src/.git>
Build step 'Invoke Gradle script' changed build result to SUCCESS
[beam_Dependency_Check] $ /bin/bash -xe /tmp/jenkins4808729908222905742.sh
+ cd src
+ bash .test-infra/jenkins/dependency_check/generate_report.sh
REPORT_DESCRIPTION="
<h4> A dependency update is high priority if it satisfies one of following criteria: </h4>
<ul>
<li> It has major versions update available, e.g. org.assertj:assertj-core 2.5.0 -> 3.10.0; </li>
</ul>
<ul>
<li> It is over 3 minor versions behind the latest version, e.g. org.tukaani:xz 1.5 -> 1.8; </li>
</ul>
<ul>
<li> The current version is behind the later version for over 180 days, e.g. com.google.auto.service:auto-service 2014-10-24 -> 2017-12-11. </li>
</ul>
<h4> In Beam, we make a best-effort attempt at keeping all dependencies up-to-date.
In the future, issues will be filed and tracked for these automatically,
but in the meantime you can search for existing issues or open a new one.
</h4>
<h4> For more information: <a href=\"https://beam.apache.org/contribute/dependencies/\"> Beam Dependency Guide </a></h4>"
# Virtualenv for the rest of the script to run setup
/usr/bin/virtualenv dependency/check
New python executable in <https://builds.apache.org/job/beam_Dependency_Check/ws/src/dependency/check/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_Dependency_Check/ws/src/dependency/check/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
. dependency/check/bin/activate
# This file must be used with "source bin/activate" *from bash*
# you cannot run it directly
deactivate () {
unset -f pydoc >/dev/null 2>&1
# reset old environment variables
# ! [ -z ${VAR+_} ] returns true if VAR is declared at all
if ! [ -z "${_OLD_VIRTUAL_PATH+_}" ] ; then
PATH="$_OLD_VIRTUAL_PATH"
export PATH
unset _OLD_VIRTUAL_PATH
fi
if ! [ -z "${_OLD_VIRTUAL_PYTHONHOME+_}" ] ; then
PYTHONHOME="$_OLD_VIRTUAL_PYTHONHOME"
export PYTHONHOME
unset _OLD_VIRTUAL_PYTHONHOME
fi
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
hash -r 2>/dev/null
fi
if ! [ -z "${_OLD_VIRTUAL_PS1+_}" ] ; then
PS1="$_OLD_VIRTUAL_PS1"
export PS1
unset _OLD_VIRTUAL_PS1
fi
unset VIRTUAL_ENV
if [ ! "${1-}" = "nondestructive" ] ; then
# Self destruct!
unset -f deactivate
fi
}
# unset irrelevant variables
deactivate nondestructive
VIRTUAL_ENV="<https://builds.apache.org/job/beam_Dependency_Check/ws/src/dependency/check">
export VIRTUAL_ENV
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
export PATH
# unset PYTHONHOME if set
if ! [ -z "${PYTHONHOME+_}" ] ; then
_OLD_VIRTUAL_PYTHONHOME="$PYTHONHOME"
unset PYTHONHOME
fi
if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT-}" ] ; then
_OLD_VIRTUAL_PS1="$PS1"
if [ "x" != x ] ; then
PS1="$PS1"
else
PS1="(`basename \"$VIRTUAL_ENV\"`) $PS1"
fi
export PS1
fi
basename "$VIRTUAL_ENV"
# Make sure to unalias pydoc if it's already there
alias pydoc 2>/dev/null >/dev/null && unalias pydoc
pydoc () {
python -m pydoc "$@"
}
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
hash -r 2>/dev/null
fi
pip install --upgrade google-cloud-bigquery
Collecting google-cloud-bigquery
Downloading https://files.pythonhosted.org/packages/27/0b/4b8d48d602e12766187203767ebcf213f9e8297e1e1a38b714d5f921d5f6/google_cloud_bigquery-1.5.1-py2.py3-none-any.whl (78kB)
Collecting google-api-core<2.0.0dev,>=1.0.0 (from google-cloud-bigquery)
Downloading https://files.pythonhosted.org/packages/a5/be/de30100034c391f4c56e2543f1507eb1b30b3030bd9a6764dd6cfe7a954e/google_api_core-1.4.1-py2.py3-none-any.whl (53kB)
Collecting google-cloud-core<0.29dev,>=0.28.0 (from google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/0f/41/ae2418b4003a14cf21c1c46d61d1b044bf02cf0f8f91598af572b9216515/google_cloud_core-0.28.1-py2.py3-none-any.whl
Collecting google-resumable-media>=0.2.1 (from google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/77/95/2e4020a54366423ddba715f89fb7ca456c8f048b15cada6cd6a54cf10e8c/google_resumable_media-0.3.1-py2.py3-none-any.whl
Collecting protobuf>=3.4.0 (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/b8/c2/b7f587c0aaf8bf2201405e8162323037fe8d17aa21d3c7dda811b8d01469/protobuf-3.6.1-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied, skipping upgrade: setuptools>=34.0.0 in ./dependency/check/lib/python2.7/site-packages (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery) (40.4.3)
Collecting six>=1.10.0 (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/58/cb/96dbb4e50e7a9d856e89cc9c8e36ab1055f9774f7d85f37e2156c1d79d9f/google_auth-1.5.1-py2.py3-none-any.whl
Collecting futures>=3.2.0; python_version < "3.2" (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/2d/99/b2c4e9d5a30f6471e410a146232b4118e697fa3ffc06d6a65efde84debd0/futures-3.2.0-py2-none-any.whl
Collecting pytz (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/30/4e/27c34b62430286c6d59177a0842ed90dc789ce5d1ed740887653b898779a/pytz-2018.5-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.3 (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Collecting requests<3.0.0dev,>=2.18.0 (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/65/47/7e02164a2a3db50ed6d8a6ab1d6d60b69c4c3fdf57a284257925dfc12bda/requests-2.19.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from google-auth<2.0.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/0a/58/cbee863250b31d80f47401d04f34038db6766f95dea1cc909ea099c7e571/cachetools-2.1.0-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1 (from google-auth<2.0.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/19/02/fa63f7ba30a0d7b925ca29d034510fc1ffde53264b71b4155022ddf3ab5d/pyasn1_modules-0.2.2-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/df/f7/04fee6ac349e915b82171f8e23cee63644d83663b34c539f7a09aed18f9e/certifi-2018.8.24-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting urllib3<1.24,>=1.21.1 (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/bd/c9/6fdd990019071a4a32a5e7cb78a1d92c53851ef4f56f62a3486e6a7d8ffb/urllib3-1.23-py2.py3-none-any.whl
Collecting idna<2.8,>=2.5 (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3 (from rsa>=3.1.4->google-auth<2.0.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Using cached https://files.pythonhosted.org/packages/d1/a1/7790cc85db38daa874f6a2e6308131b9953feb1367f2ae2d1123bb93a9f5/pyasn1-0.4.4-py2.py3-none-any.whl
Installing collected packages: six, protobuf, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, futures, pytz, googleapis-common-protos, certifi, chardet, urllib3, idna, requests, google-api-core, google-cloud-core, google-resumable-media, google-cloud-bigquery
Successfully installed cachetools-2.1.0 certifi-2018.8.24 chardet-3.0.4 futures-3.2.0 google-api-core-1.4.1 google-auth-1.5.1 google-cloud-bigquery-1.5.1 google-cloud-core-0.28.1 google-resumable-media-0.3.1 googleapis-common-protos-1.5.3 idna-2.7 protobuf-3.6.1 pyasn1-0.4.4 pyasn1-modules-0.2.2 pytz-2018.5 requests-2.19.1 rsa-4.0 six-1.11.0 urllib3-1.23
rm -f build/dependencyUpdates/beam-dependency-check-report.txt
# Insall packages and run the unit tests of the report generator and the jira manager
pip install mock jira pyyaml
Collecting mock
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting jira
Downloading https://files.pythonhosted.org/packages/cb/c2/4203b6cad72d115b0ad62ae7028567b538c8933745f665d0eee5082baf67/jira-2.0.0-py2.py3-none-any.whl (57kB)
Collecting pyyaml
Collecting pbr>=0.11 (from mock)
Using cached https://files.pythonhosted.org/packages/69/1c/98cba002ed975a91a0294863d9c774cc0ebe38e05bbb65e83314550b1677/pbr-4.2.0-py2.py3-none-any.whl
Collecting funcsigs>=1; python_version < "3.3" (from mock)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: six>=1.9 in ./dependency/check/lib/python2.7/site-packages (from mock) (1.11.0)
Requirement already satisfied: requests>=2.10.0 in ./dependency/check/lib/python2.7/site-packages (from jira) (2.19.1)
Collecting oauthlib[signedtoken]>=1.0.0 (from jira)
Downloading https://files.pythonhosted.org/packages/e6/d1/ddd9cfea3e736399b97ded5c2dd62d1322adef4a72d816f1ed1049d6a179/oauthlib-2.1.0-py2.py3-none-any.whl (121kB)
Collecting defusedxml (from jira)
Downloading https://files.pythonhosted.org/packages/87/1c/17f3e3935a913dfe2a5ca85fa5ccbef366bfd82eb318b1f75dadbf0affca/defusedxml-0.5.0-py2.py3-none-any.whl
Collecting requests-toolbelt (from jira)
Downloading https://files.pythonhosted.org/packages/97/8a/d710f792d6f6ecc089c5e55b66e66c3f2f35516a1ede5a8f54c13350ffb0/requests_toolbelt-0.8.0-py2.py3-none-any.whl (54kB)
Collecting requests-oauthlib>=0.6.1 (from jira)
Downloading https://files.pythonhosted.org/packages/94/e7/c250d122992e1561690d9c0f7856dadb79d61fd4bdd0e598087dce607f6c/requests_oauthlib-1.0.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=20.10.1 in ./dependency/check/lib/python2.7/site-packages (from jira) (40.4.3)
Requirement already satisfied: certifi>=2017.4.17 in ./dependency/check/lib/python2.7/site-packages (from requests>=2.10.0->jira) (2018.8.24)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./dependency/check/lib/python2.7/site-packages (from requests>=2.10.0->jira) (3.0.4)
Requirement already satisfied: urllib3<1.24,>=1.21.1 in ./dependency/check/lib/python2.7/site-packages (from requests>=2.10.0->jira) (1.23)
Requirement already satisfied: idna<2.8,>=2.5 in ./dependency/check/lib/python2.7/site-packages (from requests>=2.10.0->jira) (2.7)
Collecting cryptography; extra == "signedtoken" (from oauthlib[signedtoken]>=1.0.0->jira)
Using cached https://files.pythonhosted.org/packages/87/e6/915a482dbfef98bbdce6be1e31825f591fc67038d4ee09864c1d2c3db371/cryptography-2.3.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pyjwt>=1.0.0; extra == "signedtoken" (from oauthlib[signedtoken]>=1.0.0->jira)
Downloading https://files.pythonhosted.org/packages/93/d1/3378cc8184a6524dc92993090ee8b4c03847c567e298305d6cf86987e005/PyJWT-1.6.4-py2.py3-none-any.whl
Collecting enum34; python_version < "3" (from cryptography; extra == "signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting asn1crypto>=0.21.0 (from cryptography; extra == "signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
Using cached https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl
Collecting cffi!=1.11.3,>=1.7 (from cryptography; extra == "signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
Using cached https://files.pythonhosted.org/packages/14/dd/3e7a1e1280e7d767bd3fa15791759c91ec19058ebe31217fe66f3e9a8c49/cffi-1.11.5-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3" (from cryptography; extra == "signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
Using cached https://files.pythonhosted.org/packages/fc/d0/7fc3a811e011d4b388be48a0e381db8d990042df54aa4ef4599a31d39853/ipaddress-1.0.22-py2.py3-none-any.whl
Collecting pycparser (from cffi!=1.11.3,>=1.7->cryptography; extra == "signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
Installing collected packages: pbr, funcsigs, mock, enum34, asn1crypto, pycparser, cffi, ipaddress, cryptography, pyjwt, oauthlib, defusedxml, requests-toolbelt, requests-oauthlib, jira, pyyaml
Successfully installed asn1crypto-0.24.0 cffi-1.11.5 cryptography-2.3.1 defusedxml-0.5.0 enum34-1.1.6 funcsigs-1.0.2 ipaddress-1.0.22 jira-2.0.0 mock-2.0.0 oauthlib-2.1.0 pbr-4.2.0 pycparser-2.19 pyjwt-1.6.4 pyyaml-3.13 requests-oauthlib-1.0.0 requests-toolbelt-0.8.0
cd $WORKSPACE/src/.test-infra/jenkins
#python -m dependency_check.dependency_check_report_generator_test
#python -m jira_utils.jira_manager_test
#python -m dependency_check.version_comparer_test
echo "<html><body>" > $WORKSPACE/src/build/dependencyUpdates/beam-dependency-check-report.html
python -m dependency_check/dependency_check_report_generator Python
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "<https://builds.apache.org/job/beam_Dependency_Check/ws/src/.test-infra/jenkins/dependency_check/dependency_check_report_generator.py",> line 19, in <module>
import dependency_check.version_comparer as version_comparer
File "dependency_check/version_comparer.py", line 19, in <module>
from dependency_check.report_generator_config import ReportGeneratorConfig
File "dependency_check/report_generator_config.py", line 21, in <module>
class ReportGeneratorConfig:
File "dependency_check/report_generator_config.py", line 41, in ReportGeneratorConfig
BEAM_JIRA_BOT_USRENAME= os.environ['BEAM_JIRA_BOT_USERNAME']
File "<https://builds.apache.org/job/beam_Dependency_Check/ws/src/dependency/check/lib/python2.7/UserDict.py",> line 40, in __getitem__
raise KeyError(key)
KeyError: 'BEAM_JIRA_BOT_USERNAME'
Build step 'Execute shell' marked build as failure