You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by gauravsehgal <ga...@gmail.com> on 2015/08/18 10:34:45 UTC

Re: Running Spark on user-provided Hadoop installation

Refer: http://spark.apache.org/docs/latest/hadoop-provided.html

Specifically if you want to refer s3a paths. Please edit spark-env.sh and
add following lines at end:
SPARK_DIST_CLASSPATH=$(/path/to/hadoop/hadoop-2.7.1/bin/hadoop classpath)
export
SPARK_DIST_CLASSPATH="$SPARK_DIST_CLASSPATH:/path/to/hadoop/hadoop-2.7.1/share/hadoop/tools/lib/*"



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Running-Spark-on-user-provided-Hadoop-installation-tp24076p24310.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org