[jira] [Assigned] (OOZIE-3135) Configure log4j2 in SqoopMain

["Attila Sasvari (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/OOZIE-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Attila Sasvari reassigned OOZIE-3135:
-------------------------------------

    Assignee: Attila Sasvari

> Configure log4j2 in SqoopMain
> -----------------------------
>
>                 Key: OOZIE-3135
>                 URL: https://issues.apache.org/jira/browse/OOZIE-3135
>             Project: Oozie
>          Issue Type: Bug
>    Affects Versions: 5.0.0b1
>            Reporter: Attila Sasvari
>            Assignee: Attila Sasvari
>         Attachments: OOZIE-3135-001.patch
>
>
> In Hadoop 3, MAPREDUCE-6983 switched to use slfj4 & log4j2 in {{org.apache.hadoop.mapreduce.Job}} (that prints out MR job id-s needed for Oozie). We need to setup log4j accordingly (it is also related to HADOOP-12956).
> Without proper configuration in the Sqoop action, we won't be able to get external job id-s (SqoopActionExecutor unit tests and real action would be also affected).
>    
> [The API for Log4j 2 is not compatible with Log4j 1.x|https://logging.apache.org/log4j/2.x/], but we will need to support both hadoop 2 and hadoop 3 profiles for a while. 
> We could use reflection to determine the type of the logger object in {{org.apache.hadoop.mapreduce.Job}} and configure log4j settings based on it, but there might be a better way.
> For example we could do something like this:
> - add a new method for configuring log4j2:
> {code}
>     private String setUpSqoopLog4J2(final String rootLogLevel) throws IOException {
>         System.out.println("Setting up log4j2");
>         final String logFile = getSqoopLogFile();
>         final File log4j2Xml = new File(SQOOP_LOG4J2_XML);
>         try (Writer writer = new FileWriter(log4j2Xml))
>         {
>             final String logj2SettingsXml = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n" +
>                     "<Configuration status=\"WARN\">\n" +
>                     "    <Appenders>\n" +
>                     "        <Console name=\"Console\" target=\"SYSTEM_OUT\">\n" +
>                     "            <PatternLayout pattern=\"%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n\"/>\n" +
>                     "        </Console>\n" +
>                     "        <File name=\"File\" fileName=\"" + logFile + "\">  \n" +
>                     "            <PatternLayout pattern=\"%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n\"/>\n" +
>                     "        </File> \n" +
>                     "    </Appenders>\n" +
>                     "    <Loggers>\n" +
>                     "        <Root level=\"" + rootLogLevel.toLowerCase() + "\">\n" +
>                     "            <AppenderRef ref=\"Console\"/>\n" +
>                     "            <AppenderRef ref=\"File\"/>\n" +
>                     "        </Root>\n" +
>                     "    </Loggers>\n" +
>                     "</Configuration>";
>             writer.write(logj2SettingsXml);
>         }
>         System.out.printf("log4j2 configuration file created at %s%n", log4j2Xml.getAbsolutePath());
>         final   LoggerContext context = (LoggerContext) LogManager.getContext(false);
>         context.setConfigLocation(log4j2Xml.toURI()); // forces log4j2 reconfiguration
>         return logFile;
>     }
> {code}
> and call it in the {{run()}} method if the mapreduce client is using slf4j for logging:
> {code}
>         String logFile;
>         // MAPREDUCE-6983 switches to slfj4 & log4j2. Need to setup log4j accordingly
>         if (org.apache.hadoop.mapreduce.Job.class.getDeclaredField("LOG").getType().
>                 isAssignableFrom(org.slf4j.Logger.class)) {
>             logFile = setUpSqoopLog4J2(rootLogLevel);
>         }
>         else {
>             logFile = setUpSqoopLog4J(rootLogLevel, logLevel);
>         }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)