Logging strategy -- help!!

[Karan Malhi <ka...@gmail.com>]

I would like to bring your attention to one thing here. We really are
not always going through log4j configuration , we are infact "creating
and caching" log4j logger instances on our own. There are two
instances where we "might" be using the configuration defined in
default.logging.conf files. This is when we call the
Logger.initialize() method which then uses Log4jConfigUtils, which in
turn tries to configure logging based on the default.logging.conf.
There are two instances in our code where we make use of
Logger.initialize() before we get to the Logger instance (through
Logger.getInstance()).
1. OpenEJB.java
2. EjbValidator.java

So there could be a scenario where I could run the tests which willl
then use our own "cached" log4j Loggers which will have their own log
levels, but whichever tests use OpenEJB or EjbValidator might then use
loggers whose settings are defined in default.logging.conf. This can
give a feeling of "the logging levels being overridden by something".
So, I am pretty close to thinking that it may not be surefire, but the
way our code uses the Logger instance.

We need to debate on this issue and first arrive at an agreeable
logging strategy. Once, we clearly define what and how we need in
terms of logging in OpenEJB, I will be able to work towards that goal.

The above discussion can also lead to a general logging documentation
for OpenEJB

-- 
Karan Malhi

Re: Logging strategy -- help!!

[Mohammad Nour El-Din <no...@gmail.com>]

Done, the manual is in ur personal gmail

On 6/24/07, Mohammad Nour El-Din <no...@gmail.com> wrote:
>
> What ever u decide man I only give alternatives so we can make a good
> decision :-)
>
> On 6/24/07, Karan Malhi <karan.malhi@gmail.com > wrote:
> >
> > "but if you redefine an entity it will be deleted from the current
> > configuration and
> > then the new one is added"
> >
> > Thats cool, we keep the control :).
> >
> > Please do send the log4j manual, however, I am also tilting towards
> > java.util.logging so that we can remove the dep on log4j .
> >
> > On 6/24/07, Mohammad Nour El-Din <no...@gmail.com> wrote:
> > > Well I may not following you guys here well, but we still can use
> > log4j to
> > > load configuration files, which is called reloading or
> > reconfiguration, and
> > > it works incrementally, I am as long as you defined new log4j
> > entities,
> > > these new entities will be added to the current log4j configuration,
> > but if
> > > you redefine an entity it will be deleted from the current
> > configuration and
> > > then the new one is added, and we can make sure that we define our own
> > > entities without affecting any other entities by prefixing our
> > entities with
> > > our package name org.apache.openejb for example. And BTW log4j now
> > supports
> > > xml configuration files which has more features than the normal
> > properties
> > > configuration files. Karan I will send you the log4j complete manual
> > so you
> > > can get a better idea how we can make our logging better.
> > >
> > > On 6/22/07, Karan Malhi <ka...@gmail.com> wrote:
> > > >
> > > > Lovely!!
> > > >
> > > > This is definitely going to the wiki tomorrow. I will start one on
> > > > logging so that we build up stuff like this and then can extract a
> > > > nice documentation on logging.
> > > >
> > > > > As far as the caching in Logger.getInstance, I'm not sure we
> > really
> > > > > need it.  Presumably, log4j or java.util.Logging would do that
> > sort
> > > > > of thing -- but that's a hopeful guess.
> > > >
> > > > Yes, if we ask log4j or java.util.logging for a named logger, its
> > > > gonna gives us the same one if we had asked for it previously
> > > >
> > > >
> > > > > I created the ConfUtils.getConfResource utility to do that pattern
> > > > This one should defintely be used. We just need to ask it for a
> > > > resource and it does the magic, I saw how it was used for groups and
> > > > users.properties files. Nice!!
> > > >
> > > > >the "default" part in our file names and the
> > > > > lingering usage of the "conf" file extension is something that
> > needs
> > > > > to go bye-bye -- we should use properties for properties files and
> > > > > xml for xml files, etc.
> > > > Agree. There should be a pattern and it should be intuitive
> > > >
> > > > >It's fine to have such a thing, but I think it's
> > > > > being used in situations where a logging file *should* be
> > available.
> > > > This is where I think surefire broke it. Surefire was setting the
> > > > log4j.configuration property and I dont think it was setting it to
> > the
> > > > correct url.
> > > >
> > > > > -David
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Karan Malhi
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks
> > > - Mohammad Nour
> > >
> >
> >
> > --
> > Karan Malhi
> >
>
>
>
> --
> Thanks
> - Mohammad Nour




-- 
Thanks
- Mohammad Nour

Re: Logging strategy -- help!!

[Mohammad Nour El-Din <no...@gmail.com>]

What ever u decide man I only give alternatives so we can make a good
decision :-)

On 6/24/07, Karan Malhi <ka...@gmail.com> wrote:
>
> "but if you redefine an entity it will be deleted from the current
> configuration and
> then the new one is added"
>
> Thats cool, we keep the control :).
>
> Please do send the log4j manual, however, I am also tilting towards
> java.util.logging so that we can remove the dep on log4j .
>
> On 6/24/07, Mohammad Nour El-Din <no...@gmail.com> wrote:
> > Well I may not following you guys here well, but we still can use log4j
> to
> > load configuration files, which is called reloading or reconfiguration,
> and
> > it works incrementally, I am as long as you defined new log4j entities,
> > these new entities will be added to the current log4j configuration, but
> if
> > you redefine an entity it will be deleted from the current configuration
> and
> > then the new one is added, and we can make sure that we define our own
> > entities without affecting any other entities by prefixing our entities
> with
> > our package name org.apache.openejb for example. And BTW log4j now
> supports
> > xml configuration files which has more features than the normal
> properties
> > configuration files. Karan I will send you the log4j complete manual so
> you
> > can get a better idea how we can make our logging better.
> >
> > On 6/22/07, Karan Malhi <ka...@gmail.com> wrote:
> > >
> > > Lovely!!
> > >
> > > This is definitely going to the wiki tomorrow. I will start one on
> > > logging so that we build up stuff like this and then can extract a
> > > nice documentation on logging.
> > >
> > > > As far as the caching in Logger.getInstance, I'm not sure we really
> > > > need it.  Presumably, log4j or java.util.Logging would do that sort
> > > > of thing -- but that's a hopeful guess.
> > >
> > > Yes, if we ask log4j or java.util.logging for a named logger, its
> > > gonna gives us the same one if we had asked for it previously
> > >
> > >
> > > > I created the ConfUtils.getConfResource utility to do that pattern
> > > This one should defintely be used. We just need to ask it for a
> > > resource and it does the magic, I saw how it was used for groups and
> > > users.properties files. Nice!!
> > >
> > > >the "default" part in our file names and the
> > > > lingering usage of the "conf" file extension is something that needs
> > > > to go bye-bye -- we should use properties for properties files and
> > > > xml for xml files, etc.
> > > Agree. There should be a pattern and it should be intuitive
> > >
> > > >It's fine to have such a thing, but I think it's
> > > > being used in situations where a logging file *should* be available.
> > > This is where I think surefire broke it. Surefire was setting the
> > > log4j.configuration property and I dont think it was setting it to the
> > > correct url.
> > >
> > > > -David
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Karan Malhi
> > >
> >
> >
> >
> > --
> > Thanks
> > - Mohammad Nour
> >
>
>
> --
> Karan Malhi
>



-- 
Thanks
- Mohammad Nour

Re: Logging strategy -- help!!

[Karan Malhi <ka...@gmail.com>]

"but if you redefine an entity it will be deleted from the current
configuration and
then the new one is added"

Thats cool, we keep the control :).

Please do send the log4j manual, however, I am also tilting towards
java.util.logging so that we can remove the dep on log4j .

On 6/24/07, Mohammad Nour El-Din <no...@gmail.com> wrote:
> Well I may not following you guys here well, but we still can use log4j to
> load configuration files, which is called reloading or reconfiguration, and
> it works incrementally, I am as long as you defined new log4j entities,
> these new entities will be added to the current log4j configuration, but if
> you redefine an entity it will be deleted from the current configuration and
> then the new one is added, and we can make sure that we define our own
> entities without affecting any other entities by prefixing our entities with
> our package name org.apache.openejb for example. And BTW log4j now supports
> xml configuration files which has more features than the normal properties
> configuration files. Karan I will send you the log4j complete manual so you
> can get a better idea how we can make our logging better.
>
> On 6/22/07, Karan Malhi <ka...@gmail.com> wrote:
> >
> > Lovely!!
> >
> > This is definitely going to the wiki tomorrow. I will start one on
> > logging so that we build up stuff like this and then can extract a
> > nice documentation on logging.
> >
> > > As far as the caching in Logger.getInstance, I'm not sure we really
> > > need it.  Presumably, log4j or java.util.Logging would do that sort
> > > of thing -- but that's a hopeful guess.
> >
> > Yes, if we ask log4j or java.util.logging for a named logger, its
> > gonna gives us the same one if we had asked for it previously
> >
> >
> > > I created the ConfUtils.getConfResource utility to do that pattern
> > This one should defintely be used. We just need to ask it for a
> > resource and it does the magic, I saw how it was used for groups and
> > users.properties files. Nice!!
> >
> > >the "default" part in our file names and the
> > > lingering usage of the "conf" file extension is something that needs
> > > to go bye-bye -- we should use properties for properties files and
> > > xml for xml files, etc.
> > Agree. There should be a pattern and it should be intuitive
> >
> > >It's fine to have such a thing, but I think it's
> > > being used in situations where a logging file *should* be available.
> > This is where I think surefire broke it. Surefire was setting the
> > log4j.configuration property and I dont think it was setting it to the
> > correct url.
> >
> > > -David
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Karan Malhi
> >
>
>
>
> --
> Thanks
> - Mohammad Nour
>


-- 
Karan Malhi

Re: Logging strategy -- help!!

[Mohammad Nour El-Din <no...@gmail.com>]

Well I may not following you guys here well, but we still can use log4j to
load configuration files, which is called reloading or reconfiguration, and
it works incrementally, I am as long as you defined new log4j entities,
these new entities will be added to the current log4j configuration, but if
you redefine an entity it will be deleted from the current configuration and
then the new one is added, and we can make sure that we define our own
entities without affecting any other entities by prefixing our entities with
our package name org.apache.openejb for example. And BTW log4j now supports
xml configuration files which has more features than the normal properties
configuration files. Karan I will send you the log4j complete manual so you
can get a better idea how we can make our logging better.

On 6/22/07, Karan Malhi <ka...@gmail.com> wrote:
>
> Lovely!!
>
> This is definitely going to the wiki tomorrow. I will start one on
> logging so that we build up stuff like this and then can extract a
> nice documentation on logging.
>
> > As far as the caching in Logger.getInstance, I'm not sure we really
> > need it.  Presumably, log4j or java.util.Logging would do that sort
> > of thing -- but that's a hopeful guess.
>
> Yes, if we ask log4j or java.util.logging for a named logger, its
> gonna gives us the same one if we had asked for it previously
>
>
> > I created the ConfUtils.getConfResource utility to do that pattern
> This one should defintely be used. We just need to ask it for a
> resource and it does the magic, I saw how it was used for groups and
> users.properties files. Nice!!
>
> >the "default" part in our file names and the
> > lingering usage of the "conf" file extension is something that needs
> > to go bye-bye -- we should use properties for properties files and
> > xml for xml files, etc.
> Agree. There should be a pattern and it should be intuitive
>
> >It's fine to have such a thing, but I think it's
> > being used in situations where a logging file *should* be available.
> This is where I think surefire broke it. Surefire was setting the
> log4j.configuration property and I dont think it was setting it to the
> correct url.
>
> > -David
> >
> >
> >
> >
>
>
> --
> Karan Malhi
>



-- 
Thanks
- Mohammad Nour

Re: Logging strategy -- help!!

[Karan Malhi <ka...@gmail.com>]

Lovely!!

This is definitely going to the wiki tomorrow. I will start one on
logging so that we build up stuff like this and then can extract a
nice documentation on logging.

> As far as the caching in Logger.getInstance, I'm not sure we really
> need it.  Presumably, log4j or java.util.Logging would do that sort
> of thing -- but that's a hopeful guess.

Yes, if we ask log4j or java.util.logging for a named logger, its
gonna gives us the same one if we had asked for it previously


> I created the ConfUtils.getConfResource utility to do that pattern
This one should defintely be used. We just need to ask it for a
resource and it does the magic, I saw how it was used for groups and
users.properties files. Nice!!

>the "default" part in our file names and the
> lingering usage of the "conf" file extension is something that needs
> to go bye-bye -- we should use properties for properties files and
> xml for xml files, etc.
Agree. There should be a pattern and it should be intuitive

>It's fine to have such a thing, but I think it's
> being used in situations where a logging file *should* be available.
This is where I think surefire broke it. Surefire was setting the
log4j.configuration property and I dont think it was setting it to the
correct url.

> -David
>
>
>
>


-- 
Karan Malhi

Re: Logging strategy -- help!!

[David Blevins <da...@visi.com>]

I think there's plenty of room for rethinking just about every aspect  
of this code.

On Jun 21, 2007, at 9:02 AM, Karan Malhi wrote:

> I would like to bring your attention to one thing here. We really are
> not always going through log4j configuration , we are infact "creating
> and caching" log4j logger instances on our own. There are two
> instances where we "might" be using the configuration defined in
> default.logging.conf files. This is when we call the
> Logger.initialize() method which then uses Log4jConfigUtils, which in
> turn tries to configure logging based on the default.logging.conf.
> There are two instances in our code where we make use of
> Logger.initialize() before we get to the Logger instance (through
> Logger.getInstance()).
> 1. OpenEJB.java
> 2. EjbValidator.java

The EjbValidator is a command line tool that runs in it's own VM but  
doesn't involve a running container system (i.e. it doesn't use the  
OpenEJB.java) so it wound up needing to initialize logging in order  
to print out the validation information.  I can't think of a  
situation where the EjbValidator.main method would be called *and*  
the OpenEJB.init() method called.

As far as the caching in Logger.getInstance, I'm not sure we really  
need it.  Presumably, log4j or java.util.Logging would do that sort  
of thing -- but that's a hopeful guess.

As far as default.logging.conf, we have this pattern for searching  
for config files which I recently coded up in a utility class as it's  
popped up in a few places.  The basic idea is that first we look for  
say conf/logging.conf in the openejb.base directory.  If we don't  
find it there, we look for default.logging.conf in the classpath.  If  
we did find default.logging.conf (which we should) and there is an  
openejb.base/conf/ directory then expand the default.logging.conf to  
openejb.base/conf/logging.conf where we expected to find the file in  
the first place.  If there was no openejb.base/conf/ directory, then  
it's safe to assume we're running embedded (in a test case perhaps)  
and just use the default.logging.conf and do no extra work.

We have default.logging.conf which we use this way as well as  
default.openejb.conf and now more recently users.properties and  
groups.properties.  We search on disk for the resource in  
openejb.base/conf/ if we don't find them we unpack the default one we  
stuffed in openejb-core jar and extract it to disk in the  
openejb.base/conf directory if there is one -- if there isn't one we  
just use the default file.

The basic ideas behind the pattern are that:
  1. If you've messed up your configuration, just delete or rename  
the respective files in your conf/ directory and new (working) ones  
will magically appear.
  2. When upgrading its nice that our zip file won't overwrite any  
existing files in conf/
  3. If you're running embedded you don't have to setup any  
directories or have any config files, we can run on defaults.

I created the ConfUtils.getConfResource utility to do that pattern  
generically when I did the security work, but so far we're only using  
it for the users.properties and groups.properties files.  We should  
be using it everywhere.  Having the code in multiple places has lead  
to some inconsistencies such as we expand the default.openejb.conf  
file to conf/openejb.xml (not even the same file extension).  We  
really don't need the "default" part in our file names and the  
lingering usage of the "conf" file extension is something that needs  
to go bye-bye -- we should use properties for properties files and  
xml for xml files, etc.

Back on the logging thing we added one extra complication,  
doFallbackConfiguration.  At some point the regular measures stopped  
working in some situations and we added the "doFallbackConfiguration"  
which attempts to force at least a minimal configuration in the event  
of total failure.  It's fine to have such a thing, but I think it's  
being used in situations where a logging file *should* be available.

I've got a could different ideas four our i18n stuff which doesn't  
necessarily tie into the logging stuff, but could.  I'll post that in  
another email.

-David