You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by "Patni, Ankush" <An...@altisource.com> on 2015/10/21 15:38:34 UTC

Need help in log4j.xml externalization

Hello Team,

I am trying to externalize log4j from my task.

So at present I run all the task from one tar.gz. And inside that I have log4j.xml.

But now I want to externalize the log4j.xml so that I can have more control over logs. So before running my task I tried to set the JAVA_OPTS:


export JAVA_OPTS="-Dlog4j.configuration=file://$PWD/deploy/alice/config/log4j.xml "



And I get the following error when I run my task :



Like : deploy/alice/bin/run-job.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/alice/config/EntryTask.properties



log4j:ERROR Could not create an Appender. Reported error follows.

org.apache.samza.SamzaException: Got null container name from system property: samza.container.name. This is used as the key for the log appender, so can't proceed.

        at org.apache.samza.logging.log4j.StreamAppender.activateOptions(StreamAppender.java:89)

        at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)

        at org.apache.log4j.xml.DOMConfigurator.parseAppender(DOMConfigurator.java:295)

        at org.apache.log4j.xml.DOMConfigurator.findAppenderByName(DOMConfigurator.java:176)



Then I tried with following command :


export JAVA_OPTS="-Dlog4j.configuration=file://$PWD/deploy/alice/config/log4j.xml -Dsamza.container.name=samza-application-master"


log4j:ERROR Could not create an Appender. Reported error follows.
java.lang.NullPointerException
        at java.io.StringReader.<init>(StringReader.java:50)
        at org.codehaus.jackson.JsonFactory.createJsonParser(JsonFactory.java:636)
        at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1863)
        at org.apache.samza.logging.log4j.StreamAppender.getConfig(StreamAppender.java:180)
        at org.apache.samza.logging.log4j.StreamAppender.activateOptions(StreamAppender.java:92)
        at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
        at org.apache.log4j.xml.DOMConfigurator.parseAppender(DOMConfigurator.java:295)



So my doubt is : why we need to give first of all -Dsamza.container.name=samza-application-master as it is already present in run-am.sh

Second thing could anyone please help me in externilaztion of log4j.xml.

  Regards,
Ankush
***********************************************************************************************************************

This email message and any attachments are intended solely for the use of the addressee. If you are not the intended recipient, you are prohibited from reading, disclosing, reproducing, distributing, disseminating or otherwise using this transmission. If you have received this message in error, please promptly notify the sender by reply email and immediately delete this message from your system. This message and any attachments may contain information that is confidential, privileged or exempt from disclosure. Delivery of this message to any person other than the intended recipient is not intended to waive any right or privilege. Message transmission is not guaranteed to be secure or free of software viruses.
***********************************************************************************************************************

Re:Need help in log4j.xml externalization

Posted by Yan Fang <ya...@163.com>.

Hi Ankush,


1. why we need to give first of all -Dsamza.container.name=samza-application-master as it is already present in run-am.sh


I think there is a confusion here. When you set export JAVA_OPTS="-Dlog4j.configuration=file://$PWD/deploy/alice/config/log4j.xml ", you only set it in the machine which you submit the job, not the machines that you run the job. 


To be more clear:


1. there are two parts of one Samza job: Yarn client and Yarn Application. Yarn client by default uses the log4j-console.xml, which can not use any StreamAppender. See here https://github.com/apache/samza/blob/master/samza-shell/src/main/bash/run-job.sh . Yarn Application uses the log4j.xml file that you want to externalize. See here https://github.com/apache/samza/blob/master/samza-shell/src/main/bash/run-am.sh .


2. If you use different machine to submit the Samza job and run the Samza job, the solution is simple: export JAVA_OPTS="-Dlog4j.configuration=file://$PWD/deploy/alice/config/log4j.xml " in all the machines which will run the Samza job.


3. If you use one of the machines for running Samza to submit the Samza job, there maybe a bug. Because run-job.sh and run-am.sh scripts can not differenciate which log4j.xml to use. So when you submit the job, it will use the samza log4j.xml, which may have the StreamAppender. You can open a JIRA for this. It is an easy fix. If you do not use the StreamAppender, there should be no issue.


Hope this late reply still helps.


Thanks,
Yan







At 2015-10-21 21:38:34, "Patni, Ankush" <An...@altisource.com> wrote:
>Hello Team,
>
>I am trying to externalize log4j from my task.
>
>So at present I run all the task from one tar.gz. And inside that I have log4j.xml.
>
>But now I want to externalize the log4j.xml so that I can have more control over logs. So before running my task I tried to set the JAVA_OPTS:
>
>
>export JAVA_OPTS="-Dlog4j.configuration=file://$PWD/deploy/alice/config/log4j.xml "
>
>
>
>And I get the following error when I run my task :
>
>
>
>Like : deploy/alice/bin/run-job.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/alice/config/EntryTask.properties
>
>
>
>log4j:ERROR Could not create an Appender. Reported error follows.
>
>org.apache.samza.SamzaException: Got null container name from system property: samza.container.name. This is used as the key for the log appender, so can't proceed.
>
>        at org.apache.samza.logging.log4j.StreamAppender.activateOptions(StreamAppender.java:89)
>
>        at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
>
>        at org.apache.log4j.xml.DOMConfigurator.parseAppender(DOMConfigurator.java:295)
>
>        at org.apache.log4j.xml.DOMConfigurator.findAppenderByName(DOMConfigurator.java:176)
>
>
>
>Then I tried with following command :
>
>
>export JAVA_OPTS="-Dlog4j.configuration=file://$PWD/deploy/alice/config/log4j.xml -Dsamza.container.name=samza-application-master"
>
>
>log4j:ERROR Could not create an Appender. Reported error follows.
>java.lang.NullPointerException
>        at java.io.StringReader.<init>(StringReader.java:50)
>        at org.codehaus.jackson.JsonFactory.createJsonParser(JsonFactory.java:636)
>        at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1863)
>        at org.apache.samza.logging.log4j.StreamAppender.getConfig(StreamAppender.java:180)
>        at org.apache.samza.logging.log4j.StreamAppender.activateOptions(StreamAppender.java:92)
>        at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
>        at org.apache.log4j.xml.DOMConfigurator.parseAppender(DOMConfigurator.java:295)
>
>
>
>So my doubt is : why we need to give first of all -Dsamza.container.name=samza-application-master as it is already present in run-am.sh
>
>Second thing could anyone please help me in externilaztion of log4j.xml.
>
>  Regards,
>Ankush
>***********************************************************************************************************************
>
>This email message and any attachments are intended solely for the use of the addressee. If you are not the intended recipient, you are prohibited from reading, disclosing, reproducing, distributing, disseminating or otherwise using this transmission. If you have received this message in error, please promptly notify the sender by reply email and immediately delete this message from your system. This message and any attachments may contain information that is confidential, privileged or exempt from disclosure. Delivery of this message to any person other than the intended recipient is not intended to waive any right or privilege. Message transmission is not guaranteed to be secure or free of software viruses.
>***********************************************************************************************************************