You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2015/10/24 19:21:40 UTC
spark git commit: [SPARK-11264] bin/spark-class can't find assembly
jars with certain GREP_OPTIONS set
Repository: spark
Updated Branches:
refs/heads/master e5bc8c275 -> 28132ceb1
[SPARK-11264] bin/spark-class can't find assembly jars with certain GREP_OPTIONS set
Temporarily remove GREP_OPTIONS if set in bin/spark-class.
Some GREP_OPTIONS will modify the output of the grep commands that are looking for the assembly jars.
For example, if the -n option is specified, the grep output will look like:
5:spark-assembly-1.5.1-hadoop2.4.0.jar
This will not match the regular expressions, and so the jar files will not be found. We could improve the regular expression to handle this case and trim off extra characters, but it is difficult to know which options may or may not be set. Unsetting GREP_OPTIONS within the script handles all the cases and gives the desired output.
Author: Jeffrey Naisbitt <jn...@familysearch.org>
Closes #9231 from naisbitt/unset-GREP_OPTIONS.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/28132ceb
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/28132ceb
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/28132ceb
Branch: refs/heads/master
Commit: 28132ceb10d0c127495ce8cb36135e1cb54164d7
Parents: e5bc8c2
Author: Jeffrey Naisbitt <jn...@familysearch.org>
Authored: Sat Oct 24 18:21:36 2015 +0100
Committer: Sean Owen <so...@cloudera.com>
Committed: Sat Oct 24 18:21:36 2015 +0100
----------------------------------------------------------------------
bin/spark-class | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/28132ceb/bin/spark-class
----------------------------------------------------------------------
diff --git a/bin/spark-class b/bin/spark-class
index e38e08d..8cae6cc 100755
--- a/bin/spark-class
+++ b/bin/spark-class
@@ -42,6 +42,7 @@ else
ASSEMBLY_DIR="$SPARK_HOME/assembly/target/scala-$SPARK_SCALA_VERSION"
fi
+GREP_OPTIONS=
num_jars="$(ls -1 "$ASSEMBLY_DIR" | grep "^spark-assembly.*hadoop.*\.jar$" | wc -l)"
if [ "$num_jars" -eq "0" -a -z "$SPARK_ASSEMBLY_JAR" -a "$SPARK_PREPEND_CLASSES" != "1" ]; then
echo "Failed to find Spark assembly in $ASSEMBLY_DIR." 1>&2
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org