Review Request: FLUME-743 - Log4J Avro Appender Ignores Logging Pattern

[Chris Jansen <ch...@cognitomobile.com>]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1825/
-----------------------------------------------------------

Review request for Flume.


Summary
-------

Fix to allow the flume log4j avro appender to include the log4j pattern layout. 

Updated the Log4JEventAdaptor constructor to require a layout object. As any class that uses the Log4JEventAdaptor extends the log4j AppenderSkeleton the layout object is always present. When a user wishes to configure a layout they use the setLayout method when setting up the log4j appender, if unset the layout will be defaulted to just displaying the event message body. 

Updated the getBody() method of the Log4JEventAdaptor class to apply the layout to the log4j event and output stacktrace.

Updated FlumeLog4jAvroAppender to set up a default layout to display the message body if no user defined layout is specified.

Added a new test to TestFlumeLog4jAvroAppender to test a logging pattern.


Note that when using text sink only raw output will display stack traces on multiple lines, other formats display the newline escape characters.


This addresses bug FLUME-743.
    https://issues.apache.org/jira/browse/FLUME-743


Diffs
-----

  /trunk/flume-core/src/main/java/com/cloudera/flume/handlers/log4j/FlumeSeqfileAppender.java 1170098 
  /trunk/flume-core/src/main/java/com/cloudera/flume/handlers/log4j/Log4JAppenderEventSink.java 1170098 
  /trunk/flume-core/src/main/java/com/cloudera/flume/handlers/log4j/Log4JEventAdaptor.java 1170098 
  /trunk/flume-log4j-appender/src/main/java/com/cloudera/flume/log4j/appender/FlumeLog4jAvroAppender.java 1170098 
  /trunk/flume-log4j-appender/src/test/java/com/cloudera/flume/log4j/appender/TestFlumeLog4jAvroAppender.java 1170098 

Diff: https://reviews.apache.org/r/1825/diff


Testing
-------

Set up two physical nodes, one as the source and one as the collector. Set up the source with an avro logical node listening on port 12345 with the sink being the collector. Set up a text sink on the collector to log to a file with the output format set to raw: text("/tmp/test.log","raw").

Use the test app code below to generate log events:


import com.cloudera.flume.log4j.appender.FlumeLog4jAvroAppender;
import org.apache.log4j.Logger;
import org.apache.log4j.MDC;
import org.apache.log4j.PatternLayout;

import java.util.UUID;


public class Main {
    static Logger log = Logger.getLogger(Main.class);

    public static void main(String[] args) {
        try {
            FlumeLog4jAvroAppender appender = new FlumeLog4jAvroAppender();
            appender.setHostname("localhost");
            appender.setPort(12345);
            appender.setLayout(new PatternLayout("%d [%c] (%t) <%X{user} %X{field}> %m"));
            appender.setReconnectAttempts(100);
            appender.activateOptions();

            log.addAppender(appender);

            MDC.put("user", "chris");
            while (true) {
                MDC.put("field", UUID.randomUUID().toString());
                log.info("Hello World");
                try {
                    throw new Exception("Testing");
                } catch (Exception e) {
                    log.error("Gone wrong ", e);
                }
            }
        }
        catch (Exception e) {
            e.printStackTrace();
        }
    }
}


Thanks,

Chris