You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Laszlo Gaal (JIRA)" <ji...@apache.org> on 2019/02/26 11:50:00 UTC
[jira] [Resolved] (IMPALA-8244) Toolchain build fails to publish
binaries even if asked to do so
[ https://issues.apache.org/jira/browse/IMPALA-8244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Laszlo Gaal resolved IMPALA-8244.
---------------------------------
Resolution: Fixed
Fix Version/s: Impala 3.2.0
[https://github.com/cloudera/native-toolchain/commit/4e28c6c4ebf032a480549d73c4f93e733a65d075]
IMPALA-8244: Fix publishing for non-Docker builds
A previous commit introduced two additional environment variables to control
publishing: PUBLISH_DEPENDENCIES_S3 and PUBLISH_DEPENDENCIES_ARTIFACTORY.
These were set both from init.sh and from functions.sh. However, both init.sh
and functions.sh are sourced, not called, so by the time control reached
functions.sh, the variables were always defined, so the initial value of 0 could
never be overridden. This prevented the artifacts from being published even
if PUBLISH_DEPENDENCIES was set to 1 to request this.
This patch fixes the problem by removing the initialization and definition of the
dependent variables from init.sh.
A seemingly benign typo is also removed from functions.sh
Tested by running a publishing build with the change using the regular
internal publishing job and observing the published results in the S3
bucket (and verifying build logs).
Change-Id: I4c8c6f4eeb7164e29ebc7c9b80c53da81537f63a
Reviewed-on: http://gerrit.cloudera.org:8080/12573
Reviewed-by: hector.acosta@cloudera.com <he...@cloudera.com>
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Laszlo Gaal <la...@cloudera.com>
master
> Toolchain build fails to publish binaries even if asked to do so
> ----------------------------------------------------------------
>
> Key: IMPALA-8244
> URL: https://issues.apache.org/jira/browse/IMPALA-8244
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 3.2.0
> Reporter: Laszlo Gaal
> Assignee: Laszlo Gaal
> Priority: Critical
> Labels: native-toolchain, toolchain
> Fix For: Impala 3.2.0
>
>
> The recent commit to native-toolchain that enabled the publishing of binary artifacts from Docker-based toolchain builds ([https://github.com/cloudera/native-toolchain/commit/eeccbcab224fca5c25a7b78898444037b94d88b7)] changed the logic that gates the publishing calls.
> Internal builds on Jenkins (using the same logic as before the change) now don't even attempt publishing even if parameterized to do so. Excerpt from the build log:
> {code:java}
> 02:19:44.605 install -d /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/build/cctz-2.2/lib
> 02:19:44.608 install -m 644 -p libcctz.a /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/build/cctz-2.2/lib
> 02:19:44.860 install -d /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/build/cctz-2.2/lib
> 02:19:44.863 install -m 644 -p libcctz.so.2.2 /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/build/cctz-2.2/lib
> 02:19:44.880 make: Leaving directory `/data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/source/cctz/cctz-2.2/build'
> 02:19:45.399 Cleaning /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/source/cctz ...
> 02:19:45.400 Entering directory /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/source/cctz
> 02:19:45.457 Removing cctz-2.2.tar.gz
> 02:19:45.457 Removing cctz-2.2/
> 02:19:45.459 Returning to directory /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain
> 02:19:45.459 # Build Complete cctz-2.2
> 02:19:45.459 #######################################################################
> 02:19:45.549 + popd
> 02:19:45.549 /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem
> 02:19:45.579 Archiving artifacts
> 02:19:45.781 [description-setter] Description set: *******************redacted******************
> 02:19:46.010 [WS-CLEANUP] Deleting project workspace...
> 02:19:46.010 [WS-CLEANUP] Deferred wipeout is used...
> 02:19:46.057 [WS-CLEANUP] done
> 02:19:46.057 Email was triggered for: Success
> 02:19:46.057 Sending email for trigger: Success
> 02:19:46.523 Sending email to: laszlo.gaal@cloudera.com
> 02:19:46.551 Finished: SUCCESS{code}
> The happy case with the missing steps (copied from another job that published the bits successfully):
> {code:java}
> 02:26:22.083 install -d /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/build/cctz-2.2/lib
> 02:26:22.085 install -m 644 -p libcctz.a /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/build/cctz-2.2/lib
> 02:26:22.338 install -d /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/build/cctz-2.2/lib
> 02:26:22.340 install -m 644 -p libcctz.so.2.2 /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/build/cctz-2.2/lib
> 02:26:22.358 make: Leaving directory `/data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/source/cctz/cctz-2.2/build'
> 02:26:24.297 [INFO] Scanning for projects...
> 02:26:24.545 [INFO]
> 02:26:24.545 [INFO] ------------------------------------------------------------------------
> 02:26:24.545 [INFO] Building Maven Stub Project (No POM) 1
> 02:26:24.546 [INFO] ------------------------------------------------------------------------
> 02:26:24.552 [INFO]
> 02:26:24.552 [INFO] --- maven-deploy-plugin:2.7:deploy-file (default-cli) @ standalone-pom ---
> 02:26:25.079 Uploading: http://maven.****redacted****/toolchain/cctz/2.2-gcc-4.9.2-253-1ffcf70e3f/cctz-2.2-gcc-4.9.2-253-1ffcf70e3f-ec2-package-centos-5.tar.gz
> 02:26:25.400 Uploaded: http://maven.****redacted****/toolchain/cctz/2.2-gcc-4.9.2-253-1ffcf70e3f/cctz-2.2-gcc-4.9.2-253-1ffcf70e3f-ec2-package-centos-5.tar.gz (2681 KB at 8325.9 KB/sec)
> 02:26:25.401 Uploading: http://maven.****redacted****/toolchain/cctz/2.2-gcc-4.9.2-253-1ffcf70e3f/cctz-2.2-gcc-4.9.2-253-1ffcf70e3f.pom
> 02:26:25.421 Uploaded: http://maven.****redacted****/toolchain/cctz/2.2-gcc-4.9.2-253-1ffcf70e3f/cctz-2.2-gcc-4.9.2-253-1ffcf70e3f.pom (454 B at 21.1 KB/sec)
> 02:26:25.462 Downloading: http://maven.****redacted****/toolchain/cctz/maven-metadata.xml
> 02:26:25.462 [WARNING] Checksum validation failed, expected 760fe8b0820efb0518f69c6c3acc208ebc072663 but is 66cdea48553fd311f97552bd5e4e08c5330e468a for http://maven.****redacted****/toolchain/cctz/maven-metadata.xml
> 02:26:25.476 Downloaded: http://maven.****redacted****/toolchain/cctz/maven-metadata.xml (4 KB at 65.2 KB/sec)
> 02:26:25.492 Uploading: http://maven.****redacted****/toolchain/cctz/maven-metadata.xml
> 02:26:25.515 Uploaded: http://maven.****redacted****/toolchain/cctz/maven-metadata.xml (4 KB at 128.3 KB/sec)
> 02:26:25.516 [INFO] ------------------------------------------------------------------------
> 02:26:25.516 [INFO] BUILD SUCCESS
> 02:26:25.517 [INFO] ------------------------------------------------------------------------
> 02:26:25.518 [INFO] Total time: 1.249s
> 02:26:25.519 [INFO] Finished at: Sun Feb 24 17:27:53 PST 2019
> 02:26:25.652 [INFO] Final Memory: 15M/453M
> 02:26:25.652 [INFO] ------------------------------------------------------------------------
> 02:26:25.673 Uploading /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/build/cctz-2.2-gcc-4.9.2.tar.gz to s3://native-toolchain/build/253-1ffcf70e3f/cctz/2.2-gcc-4.9.2/cctz-2.2-gcc-4.9.2-ec2-package-centos-5.tar.gz
> 02:26:26.487 Cleaning /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/source/cctz ...
> 02:26:26.488 Entering directory /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/source/cctz
> 02:26:26.513 Removing cctz-2.2.tar.gz
> 02:26:26.514 Removing cctz-2.2/
> 02:26:26.554 Returning to directory /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain
> 02:26:26.554 # Build Complete cctz-2.2
> 02:26:26.554 #######################################################################
> 02:26:26.602 + popd
> 02:26:26.602 /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem
> 02:26:27.660 Archiving artifacts
> 02:26:28.316 [description-setter] Description set: ***************redacted*****************
> 02:26:29.285 [WS-CLEANUP] Deleting project workspace...
> 02:26:29.286 [WS-CLEANUP] Deferred wipeout is used...
> 02:26:29.294 [WS-CLEANUP] done
> 02:26:29.294 Email was triggered for: Success
> 02:26:29.294 Sending email for trigger: Success
> 02:26:29.707 Sending email to: laszlo.gaal@cloudera.com
> 02:26:29.736 Finished: SUCCESS
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)