Stat gathering

[Ramiro Aparicio <ra...@prot-on.com>]

Hi,

I am trying to figure some way to get the number or queries and time 
elapsed in a DataContext, or something similar, I want to get agregate 
data about DB performance and be able to find where we should try to 
reduce the number of DB queries.
I thought I could use some kind of ServletFilter to get the data and 
post back as headers or something like that, but I don't se anywhere how 
I can get this information, the best I can see is that all that 
information is logged but not agregated, so maybe I could create a 
custom QueryLogger, but I can not see any way to inject a custom 
QueryLogger either so I am lost.
Maybe this is a marginal use case but I think this is a interesting 
feature for everyone using Cayenne in production enviroments to gather 
information without having to parse the logs.

Ramiro Aparicio

Re: Stat gathering

[Aristedes Maniatis <ar...@maniatis.org>]

On 10/05/12 3:28am, Ramiro Aparicio wrote:
> Hi,
>
> I am trying to figure some way to get the number or queries and time elapsed in a DataContext, or something similar, I want to get agregate data about DB performance and be able to find where we should try to reduce the number of DB queries.
> I thought I could use some kind of ServletFilter to get the data and post back as headers or something like that, but I don't se anywhere how I can get this information, the best I can see is that all that information is logged but not agregated, so maybe I could create a custom QueryLogger, but I can not see any way to inject a custom QueryLogger either so I am lost.
> Maybe this is a marginal use case but I think this is a interesting feature for everyone using Cayenne in production enviroments to gather information without having to parse the logs.
>
> Ramiro Aparicio


I don't have an answer for you for how to do it, but I think it is a fine idea. The latest Rails even has a mode you can enable which detects any query taking longer than a certain number of seconds, and then automatically issues an EXPLAIN to the database with the same query and puts the result in the logs.

And perhaps statistical data could be exposed with a JMX interface from Cayenne? I've never had to write such a thing so I don't know what is involved, but it certainly would be useful.

Ari



-- 
-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A

Re: Stat gathering

[Andrus Adamchik <an...@objectstyle.org>]

On May 9, 2012, at 8:28 PM, Ramiro Aparicio wrote:

> Hi,
> 
> I am trying to figure some way to get the number or queries and time elapsed in a DataContext, or something similar, I want to get agregate data about DB performance and be able to find where we should try to reduce the number of DB queries.
> I thought I could use some kind of ServletFilter to get the data and post back as headers or something like that, but I don't se anywhere how I can get this information, the best I can see is that all that information is logged but not agregated, so maybe I could create a custom QueryLogger, but I can not see any way to inject a custom QueryLogger either so I am lost.
> Maybe this is a marginal use case but I think this is a interesting feature for everyone using Cayenne in production enviroments to gather information without having to parse the logs.
> 
> Ramiro Aparicio

Hi,

I wanted to create some JMX hooks to Cayenne for a long time. Unfortunately never got around to actually doing it. For now, as Mike have mentioned you can plug custom appenders to Log4J. If you are on Cayenne 3.1, you also have a few more good extension points to monitor Cayenne stats:

1. QueryLogger is called JdbcEventLogger in 3.1. It is defined in ServerModule:

 binder.bind(JdbcEventLogger.class).to(CommonsJdbcEventLogger.class);

A custom implementation class can be defined in your own module.

2. There's also DataChannelFilter which is a concept very close to ServletFilter, except that it filters all traffic between a DataContext and its parent DataDomain. So you can analyze every select and commit. Registering a custom filter is done either via DI, or directly in the code:


DataChannelFilter filter = new MyDataChannelFilter();
ServerRuntime runtime = ...
DataDomain domain = runtime.getDataDomain();
domain.addFilter(filter);

Andrus

Re: Stat gathering

[Mike Kienenberger <mk...@gmail.com>]

Instead of injecting a custom QueryLogger, it would probably be easier
to create a custom log4j.appender class and assign it to log data for
"log4j.logger.org.apache.cayenne.access.QueryLogger = INFO" and parse
the data which is logged.

http://cayenne.apache.org/doc/configuring-logging.html

Write custom appenders for log4j
http://www.javaworld.com/javaworld/jw-12-2004/jw-1220-toolbox.html

On Wed, May 9, 2012 at 1:28 PM, Ramiro Aparicio
<ra...@prot-on.com> wrote:
> Hi,
>
> I am trying to figure some way to get the number or queries and time elapsed
> in a DataContext, or something similar, I want to get agregate data about DB
> performance and be able to find where we should try to reduce the number of
> DB queries.
> I thought I could use some kind of ServletFilter to get the data and post
> back as headers or something like that, but I don't se anywhere how I can
> get this information, the best I can see is that all that information is
> logged but not agregated, so maybe I could create a custom QueryLogger, but
> I can not see any way to inject a custom QueryLogger either so I am lost.
> Maybe this is a marginal use case but I think this is a interesting feature
> for everyone using Cayenne in production enviroments to gather information
> without having to parse the logs.
>
> Ramiro Aparicio