[jira] [Created] (STORM-1215) Use Async Loggers to avoid locking and logging overhead

["Kishor Patil (JIRA)" <ji...@apache.org>]

Kishor Patil created STORM-1215:
-----------------------------------

             Summary: Use Async Loggers to avoid locking  and logging overhead
                 Key: STORM-1215
                 URL: https://issues.apache.org/jira/browse/STORM-1215
             Project: Apache Storm
          Issue Type: Improvement
          Components: storm-core
            Reporter: Kishor Patil
            Assignee: Kishor Patil


The loggers are synchronous with immediateFlush to disk, making some of the daemons slow down. In  some other cases, nimbus is slow too with submit-lock.
Making loggers asynchronous with no necessity to write to disk on every logger event would improve cpu resource usage for logging.

{code}
"pool-7-thread-986" #1025 prio=5 os_prio=0 tid=0x00007f0f9628c800 nid=0x1b84 runnable [0x00007f0f0fa2a000]
   java.lang.Thread.State: RUNNABLE
	at java.io.FileOutputStream.writeBytes(Native Method)
	at java.io.FileOutputStream.write(FileOutputStream.java:326)
	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
	at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
	- locked <0x00000003c00ae520> (a java.io.BufferedOutputStream)
	at java.io.PrintStream.write(PrintStream.java:482)
	- locked <0x00000003c00ae500> (a java.io.PrintStream)
	at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
	at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
	at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)
	- locked <0x00000003c00ae640> (a java.io.OutputStreamWriter)
	at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185)
	at java.io.PrintStream.write(PrintStream.java:527)
	- locked <0x00000003c00ae500> (a java.io.PrintStream)
	at java.io.PrintStream.print(PrintStream.java:669)
	at java.io.PrintStream.println(PrintStream.java:806)
	- locked <0x00000003c00ae500> (a java.io.PrintStream)
	at org.apache.logging.log4j.status.StatusConsoleListener.log(StatusConsoleListener.java:81)
	at org.apache.logging.log4j.status.StatusLogger.logMessage(StatusLogger.java:218)
	at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:727)
	at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:716)
	at org.apache.logging.log4j.spi.AbstractLogger.error(AbstractLogger.java:344)
	at org.apache.logging.log4j.core.appender.DefaultErrorHandler.error(DefaultErrorHandler.java:59)
	at org.apache.logging.log4j.core.appender.AbstractAppender.error(AbstractAppender.java:86)
	at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:116)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:99)
	at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:430)
	at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:409)
	at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:367)
	at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:112)
	at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:727)
	at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:716)
	at org.apache.logging.slf4j.Log4jLogger.info(Log4jLogger.java:198)
	at clojure.tools.logging$eval1$fn__7.invoke(NO_SOURCE_FILE:0)
	at clojure.tools.logging.impl$fn__28$G__8__39.invoke(impl.clj:16)
	at clojure.tools.logging$log_STAR_.invoke(logging.clj:59)
	at backtype.storm.daemon.nimbus$mk_assignments.doInvoke(nimbus.clj:781)
	at clojure.lang.RestFn.invoke(RestFn.java:410)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)