[jira] [Commented] (SAMZA-350) Allow dynamic log level toggling

["Chris Riccomini (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081658#comment-14081658 ] 

Chris Riccomini commented on SAMZA-350:
---------------------------------------

I had an idea for this. We can write a new Log4J appender that is a NoOp. It will simply start the JMX MBean for the logLevel setting. Those who wish to use Log4J with JMX can turn on this appender in the log4j.xml file. This is still no "on by default", but it doesn't introduce a log4j dependency to samza-core, and somehow seems more appealing than writing/configuring TaskLifecycleListener.

Maybe there's a more standard pattern for doing this with Log4J than just hacking the appender API. I'll have to dig into this.

> Allow dynamic log level toggling
> --------------------------------
>
>                 Key: SAMZA-350
>                 URL: https://issues.apache.org/jira/browse/SAMZA-350
>             Project: Samza
>          Issue Type: Bug
>          Components: container
>    Affects Versions: 0.7.0
>            Reporter: Chris Riccomini
>
> Samza uses SLF4J for logging. We do not expose any mechanism to dynamically change the log level at runtime. I'm not even sure if SLF4J can do this because the log level settings are usually implementation-specific.
> Log4J does allow this via the setLevel API. By default in Log4J 2, JMX is already enabled. The hello-samza app shows how to use Log4J 1.2.*, and we use Log4J 1.2.* at LinkedIn.
> We should figure out how to implement this. Ideally, we'd do it at the SLF4J level so we don't break the log-implementation-independence that we get with SLF4J. Unfortunately, it doesn't seem that SLF4J supports this. This leaves us with log-implementation-specific support.
> I propose in this ticket that we just write Log4J 1.2.* specific implementation that calls:
> {code}
> Logger.getRootLogger().setLevel(...);
> {code}
> The second question is how to implement the RPC. There are many ways. We could use the ConfigLog (SAMZA-348), some other control stream, an HTTP endpoint in the containers, or JMX.
> My preference is to use JMX for this. It's nice and simple. The ConfigLog approach seems appealing, but it would either require containers to restart (not ideal if you're trying to debug an issue as it's happening) or would require all containers to listen to the ConfigLog topic (seems a big commitment if we want to keep containers light-weight).
> The third question is where this code should live. This code should be run in all containers and the YARN AM. I am favor having this in its own module (samza-log4j) just so we don't have to introduce a Log4J dependency to samza-core, which we've thus far been able to avoid. A reasonable location for this seems to be in a TaskLifecycleListener, which could setup the MBean in the beforeInit method. Unfortunately, the YARN AM does not use TaskLifecycleListener. We could write a SamzaAppMasterLifecycleListener interface, and implement the Log4J MBean for both.
> There might also be a better place for this to live. An argument could be made that this should be part of Samza-proper, and that we should just switch to Log4J. Another argument could be made that we need a better lifecycle interface, or some generic plugin thingy to let us run this kind of logic.



--
This message was sent by Atlassian JIRA
(v6.2#6252)