You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Shay Seng <sh...@1618labs.com> on 2014/01/09 19:49:34 UTC
log4j question
Hey,
In "
https://spark.incubator.apache.org/docs/0.8.1/configuration.html#configuring-logging
"
It states:
*"Spark uses log4j <http://logging.apache.org/log4j/> for logging. You can
configure it by adding a log4j.properties file in the conf directory. One
way to start is to copy the existing log4j.properties.template located
there."*
I did as specified and added a log4j.properties file in the spark conf
directory. However this did not suppress any logging.
However if I launch spark from a directory that contains a log4j.properties
file, it does get read in.
Does the docs need re-writing, or is this a bug?
tks,
Shay
Re: log4j question
Posted by Archit Thakur <ar...@gmail.com>.
Did you change
log4j.rootCategory=INFO, console
line in your file. Or you only moved the file removing template extension?
Coz if you didn't by this log4j configuration, it'll output the log to
console and not supress it.
Like Khanderao said do,
log4j.rootCategory=OFF
If you don't want any logs.
On Fri, Jan 10, 2014 at 12:19 AM, Shay Seng <sh...@1618labs.com> wrote:
> Hey,
>
> In "
> https://spark.incubator.apache.org/docs/0.8.1/configuration.html#configuring-logging
> "
> It states:
> *"Spark uses log4j <http://logging.apache.org/log4j/> for logging. You can
> configure it by adding a log4j.properties file in the conf directory. One
> way to start is to copy the existing log4j.properties.template located
> there."*
>
> I did as specified and added a log4j.properties file in the spark conf
> directory. However this did not suppress any logging.
> However if I launch spark from a directory that contains a
> log4j.properties file, it does get read in.
>
> Does the docs need re-writing, or is this a bug?
>
> tks,
> Shay
>
>
>
Re: log4j question
Posted by Andrew Or <an...@databricks.com>.
What do you mean it cannot work? Did you copy the log4j.properties.template
to a new file called log4j.properties? If you're running standalone
cluster, the logs should be in the $SPARK_HOME/logs directory.
On Tue, May 6, 2014 at 8:10 PM, Sophia <sl...@163.com> wrote:
> I have tryed to see the log,but the log4j.properties cannot work,how to do
> to
> see the running logs?
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/log4j-question-tp412p5472.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
Re: log4j question
Posted by Sophia <sl...@163.com>.
I have tryed to see the log,but the log4j.properties cannot work,how to do to
see the running logs?
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/log4j-question-tp412p5472.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: log4j question
Posted by Sophia <sl...@163.com>.
I have tryed to see the log,but the log4j.properties cannot work,how to do?
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/log4j-question-tp412p5471.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: log4j question
Posted by Mike Percy <mp...@apache.org>.
Khanderao, I think Shay is talking about the details of how to deploy /
where to put the file, not the directives to put inside the log4j file.
I also believe that the docs are wrong w.r.t. log4j, particularly for the
YARN standalone instructions @
https://spark.incubator.apache.org/docs/latest/running-on-yarn.html where
it says:
# Configure logging
$ cp conf/log4j.properties.template conf/log4j.properties
Maybe this works / worked for some versions of Spark and/or YARN, just not
the ones I am using. I'd guess there is some classpath / set-up script
related bug somewhere, or maybe some issue with deploying the file to the
cluster via the distributed cache mechanism. Or maybe I just missed a step
somewhere, but I tried a bunch of things.
Mike
On Fri, Jan 10, 2014 at 1:36 PM, Khanderao kand <kh...@gmail.com>wrote:
> Shay,
>
> This is much more log4j question than a Spark question.
>
>
> Anyway, Copying the templates would enable root logger at INFO level.
>
> Are you interested in suppressing logs? "However this did not suppress
> any logging. " If you want to suppress logging, trying setting to OFF
>
> try
>
> log4j.rootCategory=OFF
>
> Similarly make jetty logger off too.
>
>
>
> Khanderao
>
>
> On Thu, Jan 9, 2014 at 10:49 AM, Shay Seng <sh...@1618labs.com> wrote:
>
>> Hey,
>>
>> In "
>> https://spark.incubator.apache.org/docs/0.8.1/configuration.html#configuring-logging
>> "
>> It states:
>> *"Spark uses log4j <http://logging.apache.org/log4j/> for logging. You
>> can configure it by adding a log4j.properties file in the conf directory.
>> One way to start is to copy the existing log4j.properties.template located
>> there."*
>>
>> I did as specified and added a log4j.properties file in the spark conf
>> directory. However this did not suppress any logging.
>> However if I launch spark from a directory that contains a
>> log4j.properties file, it does get read in.
>>
>> Does the docs need re-writing, or is this a bug?
>>
>> tks,
>> Shay
>>
>>
>>
>
Re: log4j question
Posted by Khanderao kand <kh...@gmail.com>.
Shay,
This is much more log4j question than a Spark question.
Anyway, Copying the templates would enable root logger at INFO level.
Are you interested in suppressing logs? "However this did not suppress any
logging. " If you want to suppress logging, trying setting to OFF
try
log4j.rootCategory=OFF
Similarly make jetty logger off too.
Khanderao
On Thu, Jan 9, 2014 at 10:49 AM, Shay Seng <sh...@1618labs.com> wrote:
> Hey,
>
> In "
> https://spark.incubator.apache.org/docs/0.8.1/configuration.html#configuring-logging
> "
> It states:
> *"Spark uses log4j <http://logging.apache.org/log4j/> for logging. You can
> configure it by adding a log4j.properties file in the conf directory. One
> way to start is to copy the existing log4j.properties.template located
> there."*
>
> I did as specified and added a log4j.properties file in the spark conf
> directory. However this did not suppress any logging.
> However if I launch spark from a directory that contains a
> log4j.properties file, it does get read in.
>
> Does the docs need re-writing, or is this a bug?
>
> tks,
> Shay
>
>
>