You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "shane knapp (JIRA)" <ji...@apache.org> on 2019/01/07 23:27:00 UTC
[jira] [Comment Edited] (SPARK-26565) modify
dev/create-release/release-build.sh to let jenkins build packages w/o
publishing
[ https://issues.apache.org/jira/browse/SPARK-26565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16736472#comment-16736472 ]
shane knapp edited comment on SPARK-26565 at 1/7/19 11:26 PM:
--------------------------------------------------------------
[~srowen] – herein lies my confusion... after reading the release process page ([https://spark.apache.org/release-process.html)] i don't see any mention of the release manager manually running this script.
i checked for anything in the spark repo that calls this script, and the only one i found was `do-release.sh`... which also doesn't appear to be run (according to the release docs).
{noformat}
➜ spark git:(remove-package-signing) grep -r "release-build.sh" *
dev/create-release/do-release.sh: "$SELF/release-build.sh" package
dev/create-release/do-release.sh: "$SELF/release-build.sh" docs
dev/create-release/do-release.sh: "$SELF/release-build.sh" publish-release
dev/create-release/release-build.sh:usage: release-build.sh <package|docs|publish-snapshot|publish-release>{noformat}
so, if release-build.sh is actually still in use during the release process (for 'package' *only*), then i will nuke my changes and add in a jenkins-specific flag to skip the GPG and svn parts, rather than deleting them outright.
a quick image to sum up my feelings about how all of the release tooling works:
!no-idea.jpg!
was (Author: shaneknapp):
[~srowen] – herein lies my confusion... after reading the release process page ([https://spark.apache.org/release-process.html)] i don't see any mention of the release manager manually running this script.
i checked for anything in the spark repo that calls this script, and the only one i found was `do-release.sh`... which also doesn't appear to be run (according to the release docs).
{noformat}
➜ spark git:(remove-package-signing) grep -r "release-build.sh" *
dev/create-release/do-release.sh: "$SELF/release-build.sh" package
dev/create-release/do-release.sh: "$SELF/release-build.sh" docs
dev/create-release/do-release.sh: "$SELF/release-build.sh" publish-release
dev/create-release/release-build.sh:usage: release-build.sh <package|docs|publish-snapshot|publish-release>{noformat}
so, if release-build.sh is actually still in use during the release process (package), then i will nuke my changes and add in a jenkins-specific flag to skip the GPG and svn parts, rather than deleting them outright.
a quick image to sum up my feelings about how all of the release tooling works:
!no-idea.jpg!
> modify dev/create-release/release-build.sh to let jenkins build packages w/o publishing
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-26565
> URL: https://issues.apache.org/jira/browse/SPARK-26565
> Project: Spark
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.2.3, 2.3.3, 2.4.1, 3.0.0
> Reporter: shane knapp
> Assignee: shane knapp
> Priority: Major
> Attachments: fine.png, no-idea.jpg
>
>
> about a year+ ago, we stopped publishing releases directly from jenkins...
> this means that the spark-\{branch}-packaging builds are failing due to gpg signing failures, and i would like to update these builds to *just* perform packaging.
> example:
> [https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/job/spark-master-package/2183/console]
> i propose to change dev/create-release/release-build.sh...
> when the script is called w/the 'package' option, remove ALL of the following bits:
> 1) gpg signing of the source tarball (lines 184-187)
> 2) gpg signing of the sparkR dist (lines 243-248)
> 3) gpg signing of the python dist (lines 256-261)
> 4) gpg signing of the regular binary dist (lines 264-271)
> 5) the svn push of the signed dists (lines 317-332)
>
> another, and probably much better option, is to nuke the spark-\{branch}-packaging builds and create new ones that just build things w/o touching this incredible fragile shell scripting nightmare.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org