[jira] [Commented] (SPARK-43782) Support log level configuration with static Spark conf

["Snoot.io (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-43782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17727784#comment-17727784 ] 

Snoot.io commented on SPARK-43782:
----------------------------------

User 'jwang0306' has created a pull request for this issue:
https://github.com/apache/spark/pull/41302

> Support log level configuration with static Spark conf
> ------------------------------------------------------
>
>                 Key: SPARK-43782
>                 URL: https://issues.apache.org/jira/browse/SPARK-43782
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 3.4.0
>            Reporter: Yan-Lin (Jared) Wang
>            Priority: Major
>
> Today users are using {{sc.setLogLevel}} to configure the current log level using {{SparkContext}} accessible from within the {{{}SparkSession{}}}. It is a convenient alternative to directly depending on the Log4j mechanism like statically changing the {{log4j.xml}} or dynamically invoke the Log4j API.
> It would be great to allow the configuration of the log level using a static Spark conf that has the same kind of simplicity, which comes in handy when we don't want to manually change the {{{}log4j.xml{}}}. This is particularly helpful in the Spark Connect scenario where there’s no way for the client to call {{setLogLevel}} because {{SparkContext}} is not yet supported in its API, and when it connects to a platform where it can't easily change the Log4j config file.
> ./bin/pyspark --conf spark.log.level ERROR



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org