Configuring log4j for Spark 0.8.0 examples

[Markus Losoi <ma...@gmail.com>]

Hi

In the first screencast [1], the property "log4j.rootCategory" of the file
"conf/log4j.properties" is set to "ERROR, console" (originally: "INFO,
console") to avoid a large number of log messages in the console when
spark-shell is run. While this works nicely in spark-shell on Spark 0.8.0,
the setting is somehow ignored when the example SparkPi is run. That is,
when I run SparkPi on Spark 0.8.0, I receive a notable amount of INFO log
messages on the console. This is in contrast with Spark 0.7.3 that doesn't
display these messages when SparkPi is run and the property
"log4j.rootCategory" is set to "ERROR, console" in the file
"conf/log4j.properties". What should be done to make the examples in Spark
0.8.0 to take the settings in the file "conf/log4j.properties" into
consideration?

[1]
http://spark.incubator.apache.org/screencasts/1-first-steps-with-spark.html

Best regards,
Markus Losoi (markus.losoi@gmail.com)