You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2017/11/13 23:28:17 UTC
spark git commit: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof
does not exists in release-build.sh
Repository: spark
Updated Branches:
refs/heads/master f7534b37e -> c8b7f97b8
[SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not exists in release-build.sh
## What changes were proposed in this pull request?
This PR proposes to use `/usr/sbin/lsof` if `lsof` is missing in the path to fix nightly snapshot jenkins jobs. Please refer https://github.com/apache/spark/pull/19359#issuecomment-340139557:
> Looks like some of the snapshot builds are having lsof issues:
>
> https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/job/spark-branch-2.1-maven-snapshots/182/console
>
>https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/job/spark-branch-2.2-maven-snapshots/134/console
>
>spark-build/dev/create-release/release-build.sh: line 344: lsof: command not found
>usage: kill [ -s signal | -p ] [ -a ] pid ...
>kill -l [ signal ]
Up to my knowledge, the full path of `lsof` is required for non-root user in few OSs.
## How was this patch tested?
Manually tested as below:
```bash
#!/usr/bin/env bash
LSOF=lsof
if ! hash $LSOF 2>/dev/null; then
echo "a"
LSOF=/usr/sbin/lsof
fi
$LSOF -P | grep "a"
```
Author: hyukjinkwon <gu...@gmail.com>
Closes #19695 from HyukjinKwon/SPARK-22377.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c8b7f97b
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c8b7f97b
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/c8b7f97b
Branch: refs/heads/master
Commit: c8b7f97b8a58bf4a9f6e3a07dd6e5b0f646d8d99
Parents: f7534b3
Author: hyukjinkwon <gu...@gmail.com>
Authored: Tue Nov 14 08:28:13 2017 +0900
Committer: hyukjinkwon <gu...@gmail.com>
Committed: Tue Nov 14 08:28:13 2017 +0900
----------------------------------------------------------------------
dev/create-release/release-build.sh | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/c8b7f97b/dev/create-release/release-build.sh
----------------------------------------------------------------------
diff --git a/dev/create-release/release-build.sh b/dev/create-release/release-build.sh
index 7e8d5c7..5b43f9b 100755
--- a/dev/create-release/release-build.sh
+++ b/dev/create-release/release-build.sh
@@ -130,6 +130,13 @@ else
fi
fi
+# This is a band-aid fix to avoid the failure of Maven nightly snapshot in some Jenkins
+# machines by explicitly calling /usr/sbin/lsof. Please see SPARK-22377 and the discussion
+# in its pull request.
+LSOF=lsof
+if ! hash $LSOF 2>/dev/null; then
+ LSOF=/usr/sbin/lsof
+fi
if [ -z "$SPARK_PACKAGE_VERSION" ]; then
SPARK_PACKAGE_VERSION="${SPARK_VERSION}-$(date +%Y_%m_%d_%H_%M)-${git_hash}"
@@ -345,7 +352,7 @@ if [[ "$1" == "publish-snapshot" ]]; then
# -DskipTests $SCALA_2_12_PROFILES $PUBLISH_PROFILES clean deploy
# Clean-up Zinc nailgun process
- lsof -P |grep $ZINC_PORT | grep LISTEN | awk '{ print $2; }' | xargs kill
+ $LSOF -P |grep $ZINC_PORT | grep LISTEN | awk '{ print $2; }' | xargs kill
rm $tmp_settings
cd ..
@@ -382,7 +389,7 @@ if [[ "$1" == "publish-release" ]]; then
# -DskipTests $SCALA_2_12_PROFILES ยง$PUBLISH_PROFILES clean install
# Clean-up Zinc nailgun process
- lsof -P |grep $ZINC_PORT | grep LISTEN | awk '{ print $2; }' | xargs kill
+ $LSOF -P |grep $ZINC_PORT | grep LISTEN | awk '{ print $2; }' | xargs kill
#./dev/change-scala-version.sh 2.11
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org