You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by jkukul <ja...@gmail.com> on 2016/03/15 16:09:48 UTC
Re: how to set log level of spark executor on YARN(using
yarn-cluster mode)
Hi Eric (or rather: anyone who's experiencing similar situation),
I think your problem was, that the /--files/ parameter was provided after
the application jar. Your command should have looked like this, instead:
./bin/spark-submit --class edu.bjut.spark.SparkPageRank --master
yarn-cluster --num-executors 5 --executor-memory 2g
--executor-cores 1 *--files log4j.properties*
/data/hadoopspark/MySparkTest.jar hdfs://master:8000/srcdata/searchengine/*
5 5 hdfs://master:8000/resultdata/searchengine/2014102001/
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/how-to-set-log-level-of-spark-executor-on-YARN-using-yarn-cluster-mode-tp16528p26505.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org