You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4j-dev@logging.apache.org by "Frissaer, Jeroen" <je...@be.unisys.com> on 2002/05/07 11:38:57 UTC
JDBCAppender performance
Hi everybody,
I started using Log4j last week and find it a really great product. We will
be using Log4J in a performance critical application. As appenders we will
use the FileAppender, SocketAppender and JDBCAppender. After some tests
however, I noticed that the performance of log4j is severly degraded when
there is a blockage on the Database level.
The execution time of the program was more than 76 times higher.
When I stop the database (to simulate a crash) the execution time is even
much worse (more than 2790 times higher) which is unacceptable off course.
Does anyone encountered the same problem, or knows how to deal with it? We
could rewrite the JDBCAppender but prefer not to, due to time limitations.
Thanks in advance for your comments and suggestions
Best regards
Jeroen Frissaer
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: JDBCAppender performance
Posted by Ceki Gülcü <ce...@qos.ch>.
At 11:17 08.05.2002 -0700, you wrote:
>Ceki, You should look more carefully at getConnection() and
>closeConnection(). execute(String sql) calls getConnection() which simply
>returns an already open connection. Only when a connection is not yet
>open is a new one created. This is an improvement in the 1.2 version over
>the 1.1.3 contrib version. The connection is closed when the appender is
>closed -- usually at the end of the application run.
>Thus, it does cache the connection, and it does have the desirable behavior.
Indeed.
--
Ceki
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: JDBCAppender performance
Posted by Kevin Steppe <ks...@pacbell.net>.
Ceki,
You should look more carefully at getConnection() and
closeConnection(). execute(String sql) calls getConnection() which
simply returns an already open connection. Only when a connection is
not yet open is a new one created. This is an improvement in the 1.2
version over the 1.1.3 contrib version. The connection is closed when
the appender is closed -- usually at the end of the application run.
Thus, it does cache the connection, and it does have the desirable behavior.
Kevin
Ceki Gülcü wrote:
> At 10:27 07.05.2002 -0700, you wrote:
>
>> Performance questions---
>> First off, databases are pathetically slow compared to file or
>> memory writes. If you have a performance critical application avoid
>> a database where-ever possible. Ironicly, this is true even for
>> database driven applications -- you'll get better performance by
>> reducing database calls.
>> Second, the JDBCAppender falls victim because of this. The
>> JDBCAppender itself is not slow (I tried writting sql statements to a
>> file instead of call a database and it's plenty fast), but inserts to
>> a db are going to be slow. The JDBCAppender has a log buffer, which
>> can be used to clump your performance costs (very fast until the
>> buffer is filled, then slow for the inserts, then fast again).
>
>
> Hmm, looking at the code it seems to me that the execute(String sql)
> method is called
> for every LoggingEvent. In turn, execute(String sql) method opens and
> closes a new
> connection each time. Given that opening a new connection is slow, this
> can't be the desired behavior, can it?
>
>
> --
> Ceki
>
>
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>
>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: JDBCAppender performance
Posted by Ceki Gülcü <ce...@qos.ch>.
At 10:27 07.05.2002 -0700, you wrote:
>Performance questions---
> First off, databases are pathetically slow compared to file or memory
> writes. If you have a performance critical application avoid a database
> where-ever possible. Ironicly, this is true even for database driven
> applications -- you'll get better performance by reducing database calls.
> Second, the JDBCAppender falls victim because of this. The
> JDBCAppender itself is not slow (I tried writting sql statements to a
> file instead of call a database and it's plenty fast), but inserts to a
> db are going to be slow. The JDBCAppender has a log buffer, which can be
> used to clump your performance costs (very fast until the buffer is
> filled, then slow for the inserts, then fast again).
Hmm, looking at the code it seems to me that the execute(String sql) method
is called
for every LoggingEvent. In turn, execute(String sql) method opens and
closes a new
connection each time. Given that opening a new connection is slow, this
can't be the desired behavior, can it?
--
Ceki
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: JDBCAppender performance
Posted by Kevin Steppe <ks...@pacbell.net>.
Performance questions---
First off, databases are pathetically slow compared to file or memory
writes. If you have a performance critical application avoid a database
where-ever possible. Ironicly, this is true even for database driven
applications -- you'll get better performance by reducing database calls.
Second, the JDBCAppender falls victim because of this. The
JDBCAppender itself is not slow (I tried writting sql statements to a
file instead of call a database and it's plenty fast), but inserts to a
db are going to be slow. The JDBCAppender has a log buffer, which can
be used to clump your performance costs (very fast until the buffer is
filled, then slow for the inserts, then fast again).
Concerning an unavailable database -- the driver calls the database
and waits for a response. It will wait for it's entire time-out period
(often multiple seconds) before failing. If you want it to fail more
quickly, call DriverManager.setLoginTimeout(int seconds) in your own
code. If the database crashes after a connection is opened, it will
again wait for some time-out period before failure. I'm not sure if or
how that time-out can be set. Welcome to dealing with databases.
About connection pooling -- by default the JDBCAppender creates one
connection and holds that open for everything it does -- so there is no
performance penalty. If you have a J2EE server and want to use it's
connection pooling, it is very easy to override the getConnection()
method to do this. Several people have reported doing so.
If performance is critical, but you need long term log storage in a
database, I recommend putting the JDBCAppender behind an AsyncAppender,
or JMS, or override flushBuffer() to generate a new thread, or anything
else you think of to make it non-blocking -- using an AsyncAppender is
probably the easiest.
Kevin
Frissaer, Jeroen wrote:
> Hi everybody,
>
> I started using Log4j last week and find it a really great product. We will
> be using Log4J in a performance critical application. As appenders we will
> use the FileAppender, SocketAppender and JDBCAppender. After some tests
> however, I noticed that the performance of log4j is severly degraded when
> there is a blockage on the Database level.
>
> The execution time of the program was more than 76 times higher.
>
> When I stop the database (to simulate a crash) the execution time is even
> much worse (more than 2790 times higher) which is unacceptable off course.
>
> Does anyone encountered the same problem, or knows how to deal with it? We
> could rewrite the JDBCAppender but prefer not to, due to time limitations.
>
> Thanks in advance for your comments and suggestions
>
> Best regards
> Jeroen Frissaer
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: JDBCAppender performance
Posted by Kevin Steppe <ks...@pacbell.net>.
Performance questions---
First off, databases are pathetically slow compared to file or memory
writes. If you have a performance critical application avoid a database
where-ever possible. Ironicly, this is true even for database driven
applications -- you'll get better performance by reducing database calls.
Second, the JDBCAppender falls victim because of this. The
JDBCAppender itself is not slow (I tried writting sql statements to a
file instead of call a database and it's plenty fast), but inserts to a
db are going to be slow. The JDBCAppender has a log buffer, which can
be used to clump your performance costs (very fast until the buffer is
filled, then slow for the inserts, then fast again).
Concerning an unavailable database -- the driver calls the database
and waits for a response. It will wait for it's entire time-out period
(often multiple seconds) before failing. If you want it to fail more
quickly, call DriverManager.setLoginTimeout(int seconds) in your own
code. If the database crashes after a connection is opened, it will
again wait for some time-out period before failure. I'm not sure if or
how that time-out can be set. Welcome to dealing with databases.
About connection pooling -- by default the JDBCAppender creates one
connection and holds that open for everything it does -- so there is no
performance penalty. If you have a J2EE server and want to use it's
connection pooling, it is very easy to override the getConnection()
method to do this. Several people have reported doing so.
If performance is critical, but you need long term log storage in a
database, I recommend putting the JDBCAppender behind an AsyncAppender,
or JMS, or override flushBuffer() to generate a new thread, or anything
else you think of to make it non-blocking -- using an AsyncAppender is
probably the easiest.
Kevin
Frissaer, Jeroen wrote:
> Hi everybody,
>
> I started using Log4j last week and find it a really great product. We will
> be using Log4J in a performance critical application. As appenders we will
> use the FileAppender, SocketAppender and JDBCAppender. After some tests
> however, I noticed that the performance of log4j is severly degraded when
> there is a blockage on the Database level.
>
> The execution time of the program was more than 76 times higher.
>
> When I stop the database (to simulate a crash) the execution time is even
> much worse (more than 2790 times higher) which is unacceptable off course.
>
> Does anyone encountered the same problem, or knows how to deal with it? We
> could rewrite the JDBCAppender but prefer not to, due to time limitations.
>
> Thanks in advance for your comments and suggestions
>
> Best regards
> Jeroen Frissaer
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>