You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2022/03/19 18:50:35 UTC
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8980
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8980/display/redirect>
Changes:
------------------------------------------
[...truncated 106.02 KB...]
tensorflow==2.6.2
tensorflow-estimator==2.6.0
termcolor==1.1.0
testcontainers==3.4.2
threadpoolctl==3.1.0
tqdm==4.63.0
typing-extensions==3.7.4.3
typing-inspect==0.7.1
typing-utils==0.1.0
uritemplate==4.1.1
urllib3==1.26.8
wcwidth==0.2.5
websocket-client==1.3.1
Werkzeug==2.0.3
wheel==0.37.0
wrapt==1.12.1
zipp==3.6.0
Removing intermediate container f360ca6e834c
---> c93dcfbcc698
Step 16/31 : COPY target/LICENSE /opt/apache/beam/
---> 414e795169bd
Step 17/31 : COPY target/LICENSE.python /opt/apache/beam/
---> 6a632763e2b5
Step 18/31 : COPY target/NOTICE /opt/apache/beam/
---> b572f0dcb4ea
Step 19/31 : COPY target/launcher/linux_amd64/boot /opt/apache/beam/
---> db0595a5cb88
Step 20/31 : ENTRYPOINT ["/opt/apache/beam/boot"]
---> Running in 80affe4f06fa
Removing intermediate container 80affe4f06fa
---> c27e24400590
Step 21/31 : FROM beam as third_party_licenses
---> c27e24400590
Step 22/31 : ARG pull_licenses
---> Running in a7e4d9bf3c4d
Removing intermediate container a7e4d9bf3c4d
---> 0433d0532916
Step 23/31 : COPY target/license_scripts /tmp/license_scripts/
---> c46e7b192563
Step 24/31 : COPY target/LICENSE target/go-licenses/* /opt/apache/beam/third_party_licenses/golang/
---> c1000ed2f0ee
Step 25/31 : RUN rm /opt/apache/beam/third_party_licenses/golang/LICENSE
---> Running in 58f131648bb7
Removing intermediate container 58f131648bb7
---> 7cbee61ef10d
Step 26/31 : COPY target/license_scripts /tmp/license_scripts/
---> 51730b68fb44
Step 27/31 : RUN if [ "$pull_licenses" = "true" ] ; then pip install 'pip-licenses<3.0.0' pyyaml tenacity && python /tmp/license_scripts/pull_licenses_py.py ; fi
---> Running in 1f9629dc3693
Collecting pip-licenses<3.0.0
Downloading pip_licenses-2.3.0-py3-none-any.whl (14 kB)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.6/site-packages (6.0)
Requirement already satisfied: tenacity in /usr/local/lib/python3.6/site-packages (5.1.5)
Collecting PTable
Downloading PTable-0.9.2.tar.gz (31 kB)
Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.6/site-packages (from tenacity) (1.15.0)
Building wheels for collected packages: PTable
Building wheel for PTable (setup.py): started
Building wheel for PTable (setup.py): finished with status 'done'
Created wheel for PTable: filename=PTable-0.9.2-py3-none-any.whl size=22924 sha256=c8fdd8082353211f345d5833871aafa3fbf9d721f8783ea3476797964e52fbb2
Stored in directory: /root/.cache/pip/wheels/f3/65/67/71f473ec87ea4554d1bdfb1b5128cfe1414c2b113b72a1ee8e
Successfully built PTable
Installing collected packages: PTable, pip-licenses
Successfully installed PTable-0.9.2 pip-licenses-2.3.0
[91mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[0m[91mWARNING: You are using pip version 21.2.4; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
[0m[91mINFO:root:Successfully pulled licenses for 139 dependencies
[0mSkip pulling license for bs4
Removing intermediate container 1f9629dc3693
---> 2b448aea1f84
Step 28/31 : FROM beam
---> c27e24400590
Step 29/31 : ARG pull_licenses
---> Running in 83acab188fe6
Removing intermediate container 83acab188fe6
---> b57e793bd100
Step 30/31 : COPY --from=third_party_licenses /opt/apache/beam/third_party_licenses /opt/apache/beam/third_party_licenses
---> a91c959d81e3
Step 31/31 : RUN if [ "$pull_licenses" != "true" ] ; then rm -rf /opt/apache/beam/third_party_licenses ; fi
---> Running in 61a3842ad4bf
Removing intermediate container 61a3842ad4bf
---> ab72b4725527
Successfully built ab72b4725527
Successfully tagged apache/beam_python3.6_sdk:2.38.0.dev
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer
# Where to store integration test outputs.
GCS_LOCATION=${GCS_LOCATION:-gs://temp-storage-for-end-to-end-tests}
# Project for the container and integration test
PROJECT=${PROJECT:-apache-beam-testing}
REGION=${REGION:-us-central1}
IMAGE_PREFIX="$(grep 'docker_image_default_repo_prefix' gradle.properties | cut -d'=' -f2)"
SDK_VERSION="$(grep 'sdk_version' gradle.properties | cut -d'=' -f2)"
PY_VERSION=$1
IMAGE_NAME="${IMAGE_PREFIX}python${PY_VERSION}_sdk"
CONTAINER_PROJECT="sdks:python:container:py${PY_VERSION//.}" # Note: we substitute away the dot in the version.
PY_INTERPRETER="python${PY_VERSION}"
XUNIT_FILE="pytest-$IMAGE_NAME.xml"
# Verify in the root of the repository
test -d sdks/python/container
# Verify docker and gcloud commands exist
command -v docker
command -v gcloud
docker -v
gcloud -v
# Verify docker image has been built.
docker images | grep "apache/$IMAGE_NAME" | grep "$SDK_VERSION"
TAG=$(date +%Y%m%d-%H%M%S%N)
CONTAINER=us.gcr.io/$PROJECT/$USER/$IMAGE_NAME
PREBUILD_SDK_CONTAINER_REGISTRY_PATH=us.gcr.io/$PROJECT/$USER/prebuild_python${PY_VERSION//.}_sdk
echo "Using container $CONTAINER"
# Tag the docker container.
docker tag "apache/$IMAGE_NAME:$SDK_VERSION" "$CONTAINER:$TAG"
# Push the container.
gcloud docker -- push $CONTAINER:$TAG
WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03.
As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.
See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker
function cleanup_container {
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container image"
for image in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH)
do docker rmi $image || echo "Failed to remove prebuilt sdk container image"
done
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
for digest in $(gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)")
do gcloud container images delete $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk@$digest --force-delete-tags --quiet || echo "Failed to remove prebuilt sdk container image"
done
echo "Removed the container"
}
trap cleanup_container EXIT
echo ">>> Successfully built and push container $CONTAINER"
cd sdks/python
SDK_LOCATION=$2
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
pytest -o junit_suite_name=$IMAGE_NAME \
-m="it_validatescontainer" \
--show-capture=no \
--numprocesses=1 \
--timeout=900 \
--junitxml=$XUNIT_FILE \
--ignore-glob '.*py3\d?\.py$' \
--log-cli-level=INFO \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--sdk_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH"
echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST"
cleanup_container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:aad882efe0d2d1cf1aa5cc8c6280de0f579c3238bddf03bddb6d5d84c28de179
Associated tags:
- 20220319-181529483132772
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20220319-181529483132772
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20220319-181529483132772].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:aad882efe0d2d1cf1aa5cc8c6280de0f579c3238bddf03bddb6d5d84c28de179].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:15d0373ff4dfc2dc668efefe8aa3d782ad4a854160e273f04e4ed1e04b751656
Associated tags:
- 2a398d0e-25f5-465c-b444-e96a4e4246e5
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:2a398d0e-25f5-465c-b444-e96a4e4246e5].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:15d0373ff4dfc2dc668efefe8aa3d782ad4a854160e273f04e4ed1e04b751656].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:0f437cb1531a53cc0106d8e4efcc2c48571b59f5b230de45240b3381d6926170
Associated tags:
- 8f2d725e-06e5-41fa-b785-48e708c9a1a7
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:8f2d725e-06e5-41fa-b785-48e708c9a1a7].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:0f437cb1531a53cc0106d8e4efcc2c48571b59f5b230de45240b3381d6926170].
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:container:py39:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:container:py38:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:container:py37:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
BUILD FAILED in 50m
59 actionable tasks: 48 executed, 4 from cache, 7 up-to-date
Publishing build scan...
https://gradle.com/s/yswwmflnknrkq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Py_ValCont #8981
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8981/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org