[jira] [Commented] (FLINK-5339) Ensure Flink works with Log4j version 2

["peidianli (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-5339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17029480#comment-17029480 ] 

peidianli commented on FLINK-5339:
----------------------------------

We use Flink on yarn in per-job mode,Here are the general steps. I hope they can help you.

1.We should ship log4j2.xml file to Blob serer,in AbstractYarnClusterDescriptor.java:

 
{code:java}
public static final String CONFIG_FILE_LOG4J2_NAME = "log4j2.xml";

File log4j2File = new File(configurationDirectory + File.separator + CONFIG_FILE_LOG4J2_NAME);
systemShipFiles.add(log4j2File);{code}
 

2. We should add log4j2.xml to jvm opts,so jobmanager and taskmanager can load the log4j2.xml file. In BootstrapTools.java and AbstractYarnClusterDescriptor.java,add:

 
{code:java}
if (hasLog4j2) { 
logging += " -Dlog4j.configurationFile=file:" + configDirectory + "/log4j2.xml";
 }
{code}
 

In org/apache/flink/yarn/Utils.java:
{code:java}
boolean hasLog4j2 = new File(workingDirectory, "log4j2.xml").exists();boolean hasLog4j2 = new File(workingDirectory, "log4j2.xml").exists();

 String launchCommand = BootstrapTools.getTaskManagerShellCommand( flinkConfig, tmParams, ".", ApplicationConstants.LOG_DIR_EXPANSION_VAR, hasLogback, hasLog4j, hasLog4j2, hasKrb5, taskManagerMainClass);{code}
3. We should also modify flink.sh flink-cli.sh to support log4j2.xml on the client.
{code:java}
log_setting=(-Dlog.file="$log" -Dlog4j.configuration=file:"$FLINK_CONF_DIR"/log4j-cli.properties -Dlogback.configurationFile=file:"$FLINK_CONF_DIR"/logback.xml -Dlog4j.configurationFile=file:"$FLINK_CONF_DIR"/log4j2-cli.xml )
{code}
4. add log4j2 dependencies,our log4j2 version is 2.11.1.
{code:java}
 <!--log4j2 dependencies--> 
<dependency> 
<groupId>org.apache.logging.log4j</groupId>
 <artifactId>log4j-1.2-api</artifactId> 
 <scope>compile</scope>
</dependency> 
<dependency>
 <groupId>org.apache.logging.log4j</groupId>
 <artifactId>log4j-core</artifactId>
 <scope>compile</scope>
 </dependency> 
<dependency>
 <groupId>org.apache.logging.log4j</groupId>
 <artifactId>log4j-slf4j-impl</artifactId>
 <scope>compile</scope>
</dependency> 
<!--support log4j2 async logger--> 
<dependency>
 <groupId>com.lmax</groupId>
 <artifactId>disruptor</artifactId>
 <scope>compile</scope>
</dependency>
{code}
5. add your log4j2.xml.

 

> Ensure Flink works with Log4j version 2
> ---------------------------------------
>
>                 Key: FLINK-5339
>                 URL: https://issues.apache.org/jira/browse/FLINK-5339
>             Project: Flink
>          Issue Type: Improvement
>          Components: Build System, Documentation
>            Reporter: Robert Metzger
>            Assignee: Chesnay Schepler
>            Priority: Major
>
> Flink supports using different logging backends.
> We currently use Log4j 1.2 by default and there is a documentation page for using Flink with Logback.
> With this JIRA, I would like to make all required changes to support Log4j 2 and document how users can set up Flink to use that logging backend.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)