Status of our AnnotatedType integration + Class Scanning

[Mark Struberg <st...@yahoo.de>]

Hi!

While fixing OWB-593 and readong through OWB-569, I came around a few issues in the way we do our AnnotatedType handling. It is not basically wrong what we are currently doing, but it is not perfect neither.

Historically all of the information needed to be scanned from the classes. There have been no interface which holds some meta information in the specification. The AnnotatedType stuff only got introduced with the Extension mechanism. 

We added this part by introducing a 2nd set of 'MetaData' (see MetaDataProvider#THIRDPARTY) and switch between those 2 when creation our Bean<T> instances.


PROPOSAL:

Imo we should get rid of the legacy manual scanning of the metadata directly from the classes since this has a few cons:

1.) with every Class.getDeclaredXxxx we blow up the PermGenSpace since this space cannot get freed in the ClassLoader. Not in Java <= 1.6 and also not in JDK-1.7. So this should get reduced.

2.) we have 2 completely independent ways to configure beans. This blows up the code and makes the internal flow unclear. 

3.) We have problems with 'manually' (per Extensions) registered Interceptors, Decorators, etc because those can invote addAnnotatedType which partly gets ignored later. There are still a few issues hidden in the source which are hard to track down. Thankfully those kind of things get used very seldom.


INSTEAD we should do the following:

A.) Extend our ClassScanner SPI to deliver the filled AnnotatedType tree. By moving to xbean-finder (there is currently some work going on to improve this further) we can get all the infos from there instead of needing to get any info directly from Class.
 
B.) Create the Beans solely based on the AnnotatedType and ditch the 'default' bean creation (see AbstractBeanCreator#isDefaultMetaDataProvider())

C.) cleanup sources. We could reorganize the packages from our historically grown (non-)structure to a clean separation into the infrastructure and steps necessary in the container e.g. 
 utils
 scanner
 events
 deployment
 definition
 beans
 interceptor
 decorator
 proxy
 ...

and remove unused classes and utilities like the AnnotationUtils.

Any ideas are highly welcome!

LieGrue,
strub

PS: yea, that's a lot of work and we need to do some homework upfront. So this is more like a 'vision' than a quick issue. We don't need to hurry, but we should have the same view on our goals.
 

Re: Status of our AnnotatedType integration + Class Scanning

[Mark Struberg <st...@yahoo.de>]

The ScannerService SPI I have in mind exposes no xbean-finder class at all. There are ideas around having this discussed with David Blevins, Matt Benson and Gerhard Petracek brainstorming a bit on IRC. 

The idea is basically to take the stuff I did as proposal for a JSR on github which is about unifying the n classscans multiple frameworks do on a server (MyFaces scans, OpenEJB scans, Tomcat scans, OWB scans, ...) and do all that in only one go. Plus add the xbean-finder implementation + cleanup + new features + parser SPI (needed e.g. for WebLogic, WebSphere on Solaris, etc) as default impl. 
We thought about starting this in a sandbox SVN over at commons, because it's really a commons thingy. And if this works out, then we eventually propose this as JSR one day...

How does it work: 
a.) David did a great job on xbean-finder. It doesnt use Class.getDeclaredXxx but instead just takes the JAR/Classes as InputStream and scans the structure directly from the bytecode. This way we dont consume PermGenSpace. 
    
b.) How does the 'all-in-1-go' work: This is what I did at github [1]. When the first 'client' requests some scanning information, the classscan-server will resolve all classscan-clients via java.util.ServiceLoader (most likely self written to be Java5 compatible) and ask them to register their needs. E.g. OWB will register: "I need all classes +infos from classpath entries which have a META-INF/beans.xml + the WEB-INF/classes if a WEB-INF/beans.xml exists". Other clients might register other needs/filters.
After all clients have registered what they need, the classscan-server will calculate what to do and will start with the scan. At the end, each classscan-client can just get the info they need and finally de-register themself to free up the caches held by the classscan-server.

c.) The collected information from xbean-finder in a.) contains all infos we also have in AnnotatedType, but only in a different format. So our ScannerService must go through them and create AnnotatadTypes from it at the end. It will _not_ expose any xbean internas to OWB!

Of course, those are just ideas and nothing is pinned down yet...

LieGrue,
strub

[1] https://github.com/struberg/Apache-commons-classscanner
--- On Thu, 7/21/11, Joseph Bergmark <be...@apache.org> wrote:

> From: Joseph Bergmark <be...@apache.org>
> Subject: Re: Status of our AnnotatedType integration + Class Scanning
> To: dev@openwebbeans.apache.org
> Date: Thursday, July 21, 2011, 8:22 PM
> I agree with pretty much everything
> you have spelled out.  #2 in your
> con list is an item has been a pain point for me as
> well.  #1 and #3
> look like real concerns, but I don't have as much
> experience in those
> areas.
> 
> As far as a proposed solution, I hope we can do this
> without tying the
> API too closely to xbean-finder.  I think there are
> some general EE
> server issues when too many different components are trying
> to do
> their own scans.  We should make sure the interface is
> as pluggable as
> possible.  I'm also not sure what you are proposing
> with C, is that
> just package and code cleanup, or a refactoring of the
> modules as
> well?
> 
> Sincerely,
> 
> Joe
> 
> On Thu, Jul 21, 2011 at 4:00 PM, Mark Struberg <st...@yahoo.de>
> wrote:
> > Hi!
> >
> > While fixing OWB-593 and readong through OWB-569, I
> came around a few issues in the way we do our AnnotatedType
> handling. It is not basically wrong what we are currently
> doing, but it is not perfect neither.
> >
> > Historically all of the information needed to be
> scanned from the classes. There have been no interface which
> holds some meta information in the specification. The
> AnnotatedType stuff only got introduced with the Extension
> mechanism.
> >
> > We added this part by introducing a 2nd set of
> 'MetaData' (see MetaDataProvider#THIRDPARTY) and switch
> between those 2 when creation our Bean<T> instances.
> >
> >
> > PROPOSAL:
> >
> > Imo we should get rid of the legacy manual scanning of
> the metadata directly from the classes since this has a few
> cons:
> >
> > 1.) with every Class.getDeclaredXxxx we blow up the
> PermGenSpace since this space cannot get freed in the
> ClassLoader. Not in Java <= 1.6 and also not in JDK-1.7.
> So this should get reduced.
> >
> > 2.) we have 2 completely independent ways to configure
> beans. This blows up the code and makes the internal flow
> unclear.
> >
> > 3.) We have problems with 'manually' (per Extensions)
> registered Interceptors, Decorators, etc because those can
> invote addAnnotatedType which partly gets ignored later.
> There are still a few issues hidden in the source which are
> hard to track down. Thankfully those kind of things get used
> very seldom.
> >
> >
> > INSTEAD we should do the following:
> >
> > A.) Extend our ClassScanner SPI to deliver the filled
> AnnotatedType tree. By moving to xbean-finder (there is
> currently some work going on to improve this further) we can
> get all the infos from there instead of needing to get any
> info directly from Class.
> >
> > B.) Create the Beans solely based on the AnnotatedType
> and ditch the 'default' bean creation (see
> AbstractBeanCreator#isDefaultMetaDataProvider())
> >
> > C.) cleanup sources. We could reorganize the packages
> from our historically grown (non-)structure to a clean
> separation into the infrastructure and steps necessary in
> the container e.g.
> >  utils
> >  scanner
> >  events
> >  deployment
> >  definition
> >  beans
> >  interceptor
> >  decorator
> >  proxy
> >  ...
> >
> > and remove unused classes and utilities like the
> AnnotationUtils.
> >
> > Any ideas are highly welcome!
> >
> > LieGrue,
> > strub
> >
> > PS: yea, that's a lot of work and we need to do some
> homework upfront. So this is more like a 'vision' than a
> quick issue. We don't need to hurry, but we should have the
> same view on our goals.
> >
> >
> 

Re: Status of our AnnotatedType integration + Class Scanning

[Joseph Bergmark <be...@apache.org>]

I agree with pretty much everything you have spelled out.  #2 in your
con list is an item has been a pain point for me as well.  #1 and #3
look like real concerns, but I don't have as much experience in those
areas.

As far as a proposed solution, I hope we can do this without tying the
API too closely to xbean-finder.  I think there are some general EE
server issues when too many different components are trying to do
their own scans.  We should make sure the interface is as pluggable as
possible.  I'm also not sure what you are proposing with C, is that
just package and code cleanup, or a refactoring of the modules as
well?

Sincerely,

Joe

On Thu, Jul 21, 2011 at 4:00 PM, Mark Struberg <st...@yahoo.de> wrote:
> Hi!
>
> While fixing OWB-593 and readong through OWB-569, I came around a few issues in the way we do our AnnotatedType handling. It is not basically wrong what we are currently doing, but it is not perfect neither.
>
> Historically all of the information needed to be scanned from the classes. There have been no interface which holds some meta information in the specification. The AnnotatedType stuff only got introduced with the Extension mechanism.
>
> We added this part by introducing a 2nd set of 'MetaData' (see MetaDataProvider#THIRDPARTY) and switch between those 2 when creation our Bean<T> instances.
>
>
> PROPOSAL:
>
> Imo we should get rid of the legacy manual scanning of the metadata directly from the classes since this has a few cons:
>
> 1.) with every Class.getDeclaredXxxx we blow up the PermGenSpace since this space cannot get freed in the ClassLoader. Not in Java <= 1.6 and also not in JDK-1.7. So this should get reduced.
>
> 2.) we have 2 completely independent ways to configure beans. This blows up the code and makes the internal flow unclear.
>
> 3.) We have problems with 'manually' (per Extensions) registered Interceptors, Decorators, etc because those can invote addAnnotatedType which partly gets ignored later. There are still a few issues hidden in the source which are hard to track down. Thankfully those kind of things get used very seldom.
>
>
> INSTEAD we should do the following:
>
> A.) Extend our ClassScanner SPI to deliver the filled AnnotatedType tree. By moving to xbean-finder (there is currently some work going on to improve this further) we can get all the infos from there instead of needing to get any info directly from Class.
>
> B.) Create the Beans solely based on the AnnotatedType and ditch the 'default' bean creation (see AbstractBeanCreator#isDefaultMetaDataProvider())
>
> C.) cleanup sources. We could reorganize the packages from our historically grown (non-)structure to a clean separation into the infrastructure and steps necessary in the container e.g.
>  utils
>  scanner
>  events
>  deployment
>  definition
>  beans
>  interceptor
>  decorator
>  proxy
>  ...
>
> and remove unused classes and utilities like the AnnotationUtils.
>
> Any ideas are highly welcome!
>
> LieGrue,
> strub
>
> PS: yea, that's a lot of work and we need to do some homework upfront. So this is more like a 'vision' than a quick issue. We don't need to hurry, but we should have the same view on our goals.
>
>

Re: Status of our AnnotatedType integration + Class Scanning

[Eric Covener <co...@gmail.com>]

> B.) Create the Beans solely based on the AnnotatedType and ditch the 'default' bean creation (see AbstractBeanCreator#isDefaultMetaDataProvider())

+1 here IIUC, we provide scanner-service produced AnnotatedType's and
just allow them to be replaced by extensions.