You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Matt Benson <gu...@gmail.com> on 2011/07/22 06:30:13 UTC
[sandbox] class scanning + karma requests
First, Simo added [meiyo] to the sandbox. Now, some of the guys from
the various JEE-related communities have expressed interest in working
with class scanning here at Commons. What we would propose to do is
start with the code of Geronimo's xbean-finder, which scans classes by
reading bytecode to populate meta-structures, thereby sparing permgen
space whenever possible. We plan to carefully prune and polish that
code, pull in the best ideas from [meiyo] (Simo and/or any of the rest
of you Commons developers are of course welcome to participate as
always!), and add a single-scan SPI strategy to allow the majority of
consumers to share the impact of scanning. For my own part, I hope to
be able to contribute such that the final API can accommodate most if
not all conceivable use-cases for classpath scanning, while not
becoming too unwieldy. We'd like to get started ASAP (and obviously
nothing stops me from going ahead and taking the first steps). I
would also request our new chair grant sandbox karma to:
struberg (Mark Struberg)
dblevins (David Blevins)
gpetracek (Gerhard Petracek}
Thanks,
Matt
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [sandbox] class scanning + karma requests
Posted by Phil Steitz <ph...@gmail.com>.
On 7/21/11 9:30 PM, Matt Benson wrote:
> First, Simo added [meiyo] to the sandbox. Now, some of the guys from
> the various JEE-related communities have expressed interest in working
> with class scanning here at Commons.
Great!
> What we would propose to do is
> start with the code of Geronimo's xbean-finder, which scans classes by
> reading bytecode to populate meta-structures, thereby sparing permgen
> space whenever possible. We plan to carefully prune and polish that
> code, pull in the best ideas from [meiyo] (Simo and/or any of the rest
> of you Commons developers are of course welcome to participate as
> always!), and add a single-scan SPI strategy to allow the majority of
> consumers to share the impact of scanning. For my own part, I hope to
> be able to contribute such that the final API can accommodate most if
> not all conceivable use-cases for classpath scanning, while not
> becoming too unwieldy. We'd like to get started ASAP (and obviously
> nothing stops me from going ahead and taking the first steps). I
> would also request our new chair grant sandbox karma to:
>
> struberg (Mark Struberg)
> dblevins (David Blevins)
> gpetracek (Gerhard Petracek}
No VOTE or other formalities required at this point.
Luc - let me know if you need help with this or anything else.
Phil
>
> Thanks,
> Matt
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [sandbox] class scanning + karma requests
Posted by Simone Tripodi <si...@apache.org>.
Hallo Mark!
just for he record, there is also the any() filter available that
doesn't perform any check at all, it should fit better for your use
case:
+------------------------------------------------------------------------+
createClassPathFromJVM().withConfiguration( new
AbstractHandlerConfiguration()
{
@Override
public void configure()
{
ifMatches( any() ).handleWith( new ClassPathEntryHandler()
{
public void doHandle( String path, Class<?> classPathEntry )
{
// handle class
}
} );
}
} ).usingContextClassLoader().scan();
+------------------------------------------------------------------------+
Alles gute!
Simo
http://people.apache.org/~simonetripodi/
http://www.99soft.org/
On Fri, Jul 22, 2011 at 7:04 PM, Matt Benson <gu...@gmail.com> wrote:
> On Fri, Jul 22, 2011 at 11:38 AM, Mark Struberg <st...@yahoo.de> wrote:
>> Hoi Simo!
>>
>> Yup, looks pretty fine from a users perspective. I'm just not sure if this can be done in a performance intense scenario. We would need to do a few performance tests upfront.
>> Remember we often have 50.000++ classes to scan through. And every Class has ~30 methods and fields w Annotations + method params + Annotations, ...
>> If you would really invoke this filter method for each and every structure element, then you would get performance problems pretty soon.
>>
>> Of course we need some kind of filtering, but we must be careful and do lots of performance tests upfront to not propagate an non-performing pattern.
>
> Maybe once the basic scan is complete, the client SPIs should be
> called back using a concurrent Executor, where most of the real
> filtering can take place.
>
> Matt
>
>>
>> LieGrue,
>> strub
>>
>>
>> --- On Fri, 7/22/11, Simone Tripodi <si...@apache.org> wrote:
>>
>>> From: Simone Tripodi <si...@apache.org>
>>> Subject: Re: [sandbox] class scanning + karma requests
>>> To: "Commons Developers List" <de...@commons.apache.org>
>>> Date: Friday, July 22, 2011, 3:54 PM
>>> Hi Mark!!!
>>> For meiyo APIs I'd like to keep I mean end-user APIs; as
>>> Matt
>>> suggested you, since you didn't hear about the already
>>> existing
>>> sandbox component before, the most interesting parts of
>>> meiyo are the
>>> filtering/callback APIs, expressed via EDSL. Have a look at
>>> the simple
>>> included testcase[1] to see how they make clear its usage
>>> to end user
>>> - of course they can be improved!!! - and I bet should not
>>> a big issue
>>> integrating them in your more efficient engine :P
>>> Looking forward to hear from you soon, all the best!
>>> Simo
>>>
>>> [1] http://s.apache.org/YIF
>>> [2] http://s.apache.org/5Gt
>>>
>>> http://people.apache.org/~simonetripodi/
>>> http://www.99soft.org/
>>>
>>>
>>>
>>> On Fri, Jul 22, 2011 at 4:59 PM, Matt Benson <gu...@gmail.com>
>>> wrote:
>>> > On Fri, Jul 22, 2011 at 9:51 AM, Mark Struberg <st...@yahoo.de>
>>> wrote:
>>> >> Hi Simo!
>>> >>
>>> >> Yea, taking the best ideas of both projects (meiyo
>>> and xbean-finder) would definitlely be a big bonus.
>>> >> But to be honest: I think we should rather stay on
>>> the xbean-finder API. The reason is that it's already
>>> heavily used in Geronimo, OpenEJB, OpenWebBeans and a few
>>> other projects here at the ASF and abroad. So we should aim
>>> to make the transition as easy as possible. Of course
>>> tweaking the API to some degree is surely possible.
>>> >> I also have to admit that I haven't heard of meiyo
>>> before yesterday, so I don't know it's capabilities.
>>> >>
>>> >
>>> > IMO the most interesting parts of meiyo to take away
>>> are the
>>> > filtering/callback APIs. I would think that this is
>>> mostly part of
>>> > (or at least an option of) the client API e.g. for the
>>> single-scan
>>> > feature, that we have to rewrite anyway. So I don't
>>> think these goals
>>> > are incompatible.
>>> >
>>> > Matt
>>> >
>>> >> In any case you are highly welcome to join forces
>>> - any idea/opinion/help is welcome!
>>> >>
>>> >> LieGrue,
>>> >> strub
>>> >>
>>> >> --- On Fri, 7/22/11, Simone Tripodi <si...@apache.org>
>>> wrote:
>>> >>
>>> >>> From: Simone Tripodi <si...@apache.org>
>>> >>> Subject: Re: [sandbox] class scanning + karma
>>> requests
>>> >>> To: "Commons Developers List" <de...@commons.apache.org>
>>> >>> Date: Friday, July 22, 2011, 2:20 PM
>>> >>> Hi all guys!
>>> >>> That sounds really interesting!!!
>>> >>> I think that bringing the new ideas in
>>> existing [meiyo]
>>> >>> APIs would
>>> >>> allow us releasing a new component soon!
>>> >>> Thanks in advance for your help and
>>> interesting, looking
>>> >>> forward to
>>> >>> hear from you soon!!!
>>> >>> Simo
>>> >>>
>>> >>> http://people.apache.org/~simonetripodi/
>>> >>> http://www.99soft.org/
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Fri, Jul 22, 2011 at 11:41 AM, Mark
>>> Struberg <st...@yahoo.de>
>>> >>> wrote:
>>> >>> > Hi!
>>> >>> >
>>> >>> > Jakob Korherr (apacheId jakobk) is also
>>> interested and
>>> >>> did some work in this area in MyFaces and OWB
>>> already.
>>> >>> >
>>> >>> > There are 2 main parts in this project
>>> >>> >
>>> >>> > a.) A classpath scanner which does _not_
>>> use
>>> >>> Class.getDeclaredXxx etc, since this allocates
>>> memory in the
>>> >>> ClassLoader which cannot get freed up later.
>>> In a mid sized
>>> >>> project this leads to getting 100MB of
>>> PermGenSpace easily!
>>> >>> This is what Davids xbean-finder fixes already
>>> by just
>>> >>> scanning the structure from the bytecode
>>> directly.
>>> >>> >
>>> >>> > b.) Lots of EE apps need to scan the
>>> classpath for
>>> >>> annotations (e.g. MyFaces, OpenWebBeans,
>>> Tomcat, OpenEJB,
>>> >>> etc). So this part is done redundantly a few
>>> times when
>>> >>> starting the server. I hacked a quick API +
>>> SPI to do all
>>> >>> this in one go. The trick is to introduce a
>>> register
>>> >>> mechanism where classscan-clients can register
>>> their needs
>>> >>> before the scan gets performed.
>>> >>> >
>>> >>> >
>>> >>> > We like to do this at commons because
>>> it's really
>>> >>> decoupled from all the single projects and
>>> could be useful
>>> >>> for a lot more projects which added some kind
>>> of annotation
>>> >>> processing lately.
>>> >>> >
>>> >>> > LieGrue,
>>> >>> > strub
>>> >>> >
>>> >>> > --- On Fri, 7/22/11, Matt Benson <gu...@gmail.com>
>>> >>> wrote:
>>> >>> >
>>> >>> >> From: Matt Benson <gu...@gmail.com>
>>> >>> >> Subject: [sandbox] class scanning +
>>> karma
>>> >>> requests
>>> >>> >> To: dev@commons.apache.org
>>> >>> >> Cc: dblevins@apache.org,
>>> >>> "Gerhard Petracek" <ge...@gmail.com>
>>> >>> >> Date: Friday, July 22, 2011, 4:30 AM
>>> >>> >> First, Simo added [meiyo] to the
>>> >>> >> sandbox. Now, some of the guys
>>> from
>>> >>> >> the various JEE-related communities
>>> have expressed
>>> >>> interest
>>> >>> >> in working
>>> >>> >> with class scanning here at
>>> Commons. What we
>>> >>> would
>>> >>> >> propose to do is
>>> >>> >> start with the code of Geronimo's
>>> xbean-finder,
>>> >>> which scans
>>> >>> >> classes by
>>> >>> >> reading bytecode to populate
>>> meta-structures,
>>> >>> thereby
>>> >>> >> sparing permgen
>>> >>> >> space whenever possible. We plan to
>>> carefully
>>> >>> prune
>>> >>> >> and polish that
>>> >>> >> code, pull in the best ideas from
>>> [meiyo] (Simo
>>> >>> and/or any
>>> >>> >> of the rest
>>> >>> >> of you Commons developers are of
>>> course welcome
>>> >>> to
>>> >>> >> participate as
>>> >>> >> always!), and add a single-scan SPI
>>> strategy to
>>> >>> allow the
>>> >>> >> majority of
>>> >>> >> consumers to share the impact of
>>> scanning. For
>>> >>> my own
>>> >>> >> part, I hope to
>>> >>> >> be able to contribute such that the
>>> final API can
>>> >>> >> accommodate most if
>>> >>> >> not all conceivable use-cases for
>>> classpath
>>> >>> scanning, while
>>> >>> >> not
>>> >>> >> becoming too unwieldy. We'd like to
>>> get started
>>> >>> ASAP
>>> >>> >> (and obviously
>>> >>> >> nothing stops me from going ahead and
>>> taking the
>>> >>> first
>>> >>> >> steps). I
>>> >>> >> would also request our new chair
>>> grant sandbox
>>> >>> karma to:
>>> >>> >>
>>> >>> >> struberg (Mark Struberg)
>>> >>> >> dblevins (David Blevins)
>>> >>> >> gpetracek (Gerhard Petracek}
>>> >>> >>
>>> >>> >> Thanks,
>>> >>> >> Matt
>>> >>> >>
>>> >>> >>
>>> >>>
>>> ---------------------------------------------------------------------
>>> >>> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> >>> >> For additional commands, e-mail: dev-help@commons.apache.org
>>> >>> >>
>>> >>> >>
>>> >>> >
>>> >>> >
>>> >>>
>>> ---------------------------------------------------------------------
>>> >>> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> >>> > For additional commands, e-mail: dev-help@commons.apache.org
>>> >>> >
>>> >>> >
>>> >>>
>>> >>>
>>> ---------------------------------------------------------------------
>>> >>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> >>> For additional commands, e-mail: dev-help@commons.apache.org
>>> >>>
>>> >>>
>>> >>
>>> >>
>>> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> >> For additional commands, e-mail: dev-help@commons.apache.org
>>> >>
>>> >>
>>> >
>>> >
>>> ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> > For additional commands, e-mail: dev-help@commons.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [sandbox] class scanning + karma requests
Posted by Matt Benson <gu...@gmail.com>.
On Fri, Jul 22, 2011 at 11:38 AM, Mark Struberg <st...@yahoo.de> wrote:
> Hoi Simo!
>
> Yup, looks pretty fine from a users perspective. I'm just not sure if this can be done in a performance intense scenario. We would need to do a few performance tests upfront.
> Remember we often have 50.000++ classes to scan through. And every Class has ~30 methods and fields w Annotations + method params + Annotations, ...
> If you would really invoke this filter method for each and every structure element, then you would get performance problems pretty soon.
>
> Of course we need some kind of filtering, but we must be careful and do lots of performance tests upfront to not propagate an non-performing pattern.
Maybe once the basic scan is complete, the client SPIs should be
called back using a concurrent Executor, where most of the real
filtering can take place.
Matt
>
> LieGrue,
> strub
>
>
> --- On Fri, 7/22/11, Simone Tripodi <si...@apache.org> wrote:
>
>> From: Simone Tripodi <si...@apache.org>
>> Subject: Re: [sandbox] class scanning + karma requests
>> To: "Commons Developers List" <de...@commons.apache.org>
>> Date: Friday, July 22, 2011, 3:54 PM
>> Hi Mark!!!
>> For meiyo APIs I'd like to keep I mean end-user APIs; as
>> Matt
>> suggested you, since you didn't hear about the already
>> existing
>> sandbox component before, the most interesting parts of
>> meiyo are the
>> filtering/callback APIs, expressed via EDSL. Have a look at
>> the simple
>> included testcase[1] to see how they make clear its usage
>> to end user
>> - of course they can be improved!!! - and I bet should not
>> a big issue
>> integrating them in your more efficient engine :P
>> Looking forward to hear from you soon, all the best!
>> Simo
>>
>> [1] http://s.apache.org/YIF
>> [2] http://s.apache.org/5Gt
>>
>> http://people.apache.org/~simonetripodi/
>> http://www.99soft.org/
>>
>>
>>
>> On Fri, Jul 22, 2011 at 4:59 PM, Matt Benson <gu...@gmail.com>
>> wrote:
>> > On Fri, Jul 22, 2011 at 9:51 AM, Mark Struberg <st...@yahoo.de>
>> wrote:
>> >> Hi Simo!
>> >>
>> >> Yea, taking the best ideas of both projects (meiyo
>> and xbean-finder) would definitlely be a big bonus.
>> >> But to be honest: I think we should rather stay on
>> the xbean-finder API. The reason is that it's already
>> heavily used in Geronimo, OpenEJB, OpenWebBeans and a few
>> other projects here at the ASF and abroad. So we should aim
>> to make the transition as easy as possible. Of course
>> tweaking the API to some degree is surely possible.
>> >> I also have to admit that I haven't heard of meiyo
>> before yesterday, so I don't know it's capabilities.
>> >>
>> >
>> > IMO the most interesting parts of meiyo to take away
>> are the
>> > filtering/callback APIs. I would think that this is
>> mostly part of
>> > (or at least an option of) the client API e.g. for the
>> single-scan
>> > feature, that we have to rewrite anyway. So I don't
>> think these goals
>> > are incompatible.
>> >
>> > Matt
>> >
>> >> In any case you are highly welcome to join forces
>> - any idea/opinion/help is welcome!
>> >>
>> >> LieGrue,
>> >> strub
>> >>
>> >> --- On Fri, 7/22/11, Simone Tripodi <si...@apache.org>
>> wrote:
>> >>
>> >>> From: Simone Tripodi <si...@apache.org>
>> >>> Subject: Re: [sandbox] class scanning + karma
>> requests
>> >>> To: "Commons Developers List" <de...@commons.apache.org>
>> >>> Date: Friday, July 22, 2011, 2:20 PM
>> >>> Hi all guys!
>> >>> That sounds really interesting!!!
>> >>> I think that bringing the new ideas in
>> existing [meiyo]
>> >>> APIs would
>> >>> allow us releasing a new component soon!
>> >>> Thanks in advance for your help and
>> interesting, looking
>> >>> forward to
>> >>> hear from you soon!!!
>> >>> Simo
>> >>>
>> >>> http://people.apache.org/~simonetripodi/
>> >>> http://www.99soft.org/
>> >>>
>> >>>
>> >>>
>> >>> On Fri, Jul 22, 2011 at 11:41 AM, Mark
>> Struberg <st...@yahoo.de>
>> >>> wrote:
>> >>> > Hi!
>> >>> >
>> >>> > Jakob Korherr (apacheId jakobk) is also
>> interested and
>> >>> did some work in this area in MyFaces and OWB
>> already.
>> >>> >
>> >>> > There are 2 main parts in this project
>> >>> >
>> >>> > a.) A classpath scanner which does _not_
>> use
>> >>> Class.getDeclaredXxx etc, since this allocates
>> memory in the
>> >>> ClassLoader which cannot get freed up later.
>> In a mid sized
>> >>> project this leads to getting 100MB of
>> PermGenSpace easily!
>> >>> This is what Davids xbean-finder fixes already
>> by just
>> >>> scanning the structure from the bytecode
>> directly.
>> >>> >
>> >>> > b.) Lots of EE apps need to scan the
>> classpath for
>> >>> annotations (e.g. MyFaces, OpenWebBeans,
>> Tomcat, OpenEJB,
>> >>> etc). So this part is done redundantly a few
>> times when
>> >>> starting the server. I hacked a quick API +
>> SPI to do all
>> >>> this in one go. The trick is to introduce a
>> register
>> >>> mechanism where classscan-clients can register
>> their needs
>> >>> before the scan gets performed.
>> >>> >
>> >>> >
>> >>> > We like to do this at commons because
>> it's really
>> >>> decoupled from all the single projects and
>> could be useful
>> >>> for a lot more projects which added some kind
>> of annotation
>> >>> processing lately.
>> >>> >
>> >>> > LieGrue,
>> >>> > strub
>> >>> >
>> >>> > --- On Fri, 7/22/11, Matt Benson <gu...@gmail.com>
>> >>> wrote:
>> >>> >
>> >>> >> From: Matt Benson <gu...@gmail.com>
>> >>> >> Subject: [sandbox] class scanning +
>> karma
>> >>> requests
>> >>> >> To: dev@commons.apache.org
>> >>> >> Cc: dblevins@apache.org,
>> >>> "Gerhard Petracek" <ge...@gmail.com>
>> >>> >> Date: Friday, July 22, 2011, 4:30 AM
>> >>> >> First, Simo added [meiyo] to the
>> >>> >> sandbox. Now, some of the guys
>> from
>> >>> >> the various JEE-related communities
>> have expressed
>> >>> interest
>> >>> >> in working
>> >>> >> with class scanning here at
>> Commons. What we
>> >>> would
>> >>> >> propose to do is
>> >>> >> start with the code of Geronimo's
>> xbean-finder,
>> >>> which scans
>> >>> >> classes by
>> >>> >> reading bytecode to populate
>> meta-structures,
>> >>> thereby
>> >>> >> sparing permgen
>> >>> >> space whenever possible. We plan to
>> carefully
>> >>> prune
>> >>> >> and polish that
>> >>> >> code, pull in the best ideas from
>> [meiyo] (Simo
>> >>> and/or any
>> >>> >> of the rest
>> >>> >> of you Commons developers are of
>> course welcome
>> >>> to
>> >>> >> participate as
>> >>> >> always!), and add a single-scan SPI
>> strategy to
>> >>> allow the
>> >>> >> majority of
>> >>> >> consumers to share the impact of
>> scanning. For
>> >>> my own
>> >>> >> part, I hope to
>> >>> >> be able to contribute such that the
>> final API can
>> >>> >> accommodate most if
>> >>> >> not all conceivable use-cases for
>> classpath
>> >>> scanning, while
>> >>> >> not
>> >>> >> becoming too unwieldy. We'd like to
>> get started
>> >>> ASAP
>> >>> >> (and obviously
>> >>> >> nothing stops me from going ahead and
>> taking the
>> >>> first
>> >>> >> steps). I
>> >>> >> would also request our new chair
>> grant sandbox
>> >>> karma to:
>> >>> >>
>> >>> >> struberg (Mark Struberg)
>> >>> >> dblevins (David Blevins)
>> >>> >> gpetracek (Gerhard Petracek}
>> >>> >>
>> >>> >> Thanks,
>> >>> >> Matt
>> >>> >>
>> >>> >>
>> >>>
>> ---------------------------------------------------------------------
>> >>> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> >>> >> For additional commands, e-mail: dev-help@commons.apache.org
>> >>> >>
>> >>> >>
>> >>> >
>> >>> >
>> >>>
>> ---------------------------------------------------------------------
>> >>> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> >>> > For additional commands, e-mail: dev-help@commons.apache.org
>> >>> >
>> >>> >
>> >>>
>> >>>
>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> >>> For additional commands, e-mail: dev-help@commons.apache.org
>> >>>
>> >>>
>> >>
>> >>
>> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> >> For additional commands, e-mail: dev-help@commons.apache.org
>> >>
>> >>
>> >
>> >
>> ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> > For additional commands, e-mail: dev-help@commons.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [sandbox] class scanning + karma requests
Posted by Mark Struberg <st...@yahoo.de>.
Hoi Simo!
Yup, looks pretty fine from a users perspective. I'm just not sure if this can be done in a performance intense scenario. We would need to do a few performance tests upfront.
Remember we often have 50.000++ classes to scan through. And every Class has ~30 methods and fields w Annotations + method params + Annotations, ...
If you would really invoke this filter method for each and every structure element, then you would get performance problems pretty soon.
Of course we need some kind of filtering, but we must be careful and do lots of performance tests upfront to not propagate an non-performing pattern.
LieGrue,
strub
--- On Fri, 7/22/11, Simone Tripodi <si...@apache.org> wrote:
> From: Simone Tripodi <si...@apache.org>
> Subject: Re: [sandbox] class scanning + karma requests
> To: "Commons Developers List" <de...@commons.apache.org>
> Date: Friday, July 22, 2011, 3:54 PM
> Hi Mark!!!
> For meiyo APIs I'd like to keep I mean end-user APIs; as
> Matt
> suggested you, since you didn't hear about the already
> existing
> sandbox component before, the most interesting parts of
> meiyo are the
> filtering/callback APIs, expressed via EDSL. Have a look at
> the simple
> included testcase[1] to see how they make clear its usage
> to end user
> - of course they can be improved!!! - and I bet should not
> a big issue
> integrating them in your more efficient engine :P
> Looking forward to hear from you soon, all the best!
> Simo
>
> [1] http://s.apache.org/YIF
> [2] http://s.apache.org/5Gt
>
> http://people.apache.org/~simonetripodi/
> http://www.99soft.org/
>
>
>
> On Fri, Jul 22, 2011 at 4:59 PM, Matt Benson <gu...@gmail.com>
> wrote:
> > On Fri, Jul 22, 2011 at 9:51 AM, Mark Struberg <st...@yahoo.de>
> wrote:
> >> Hi Simo!
> >>
> >> Yea, taking the best ideas of both projects (meiyo
> and xbean-finder) would definitlely be a big bonus.
> >> But to be honest: I think we should rather stay on
> the xbean-finder API. The reason is that it's already
> heavily used in Geronimo, OpenEJB, OpenWebBeans and a few
> other projects here at the ASF and abroad. So we should aim
> to make the transition as easy as possible. Of course
> tweaking the API to some degree is surely possible.
> >> I also have to admit that I haven't heard of meiyo
> before yesterday, so I don't know it's capabilities.
> >>
> >
> > IMO the most interesting parts of meiyo to take away
> are the
> > filtering/callback APIs. I would think that this is
> mostly part of
> > (or at least an option of) the client API e.g. for the
> single-scan
> > feature, that we have to rewrite anyway. So I don't
> think these goals
> > are incompatible.
> >
> > Matt
> >
> >> In any case you are highly welcome to join forces
> - any idea/opinion/help is welcome!
> >>
> >> LieGrue,
> >> strub
> >>
> >> --- On Fri, 7/22/11, Simone Tripodi <si...@apache.org>
> wrote:
> >>
> >>> From: Simone Tripodi <si...@apache.org>
> >>> Subject: Re: [sandbox] class scanning + karma
> requests
> >>> To: "Commons Developers List" <de...@commons.apache.org>
> >>> Date: Friday, July 22, 2011, 2:20 PM
> >>> Hi all guys!
> >>> That sounds really interesting!!!
> >>> I think that bringing the new ideas in
> existing [meiyo]
> >>> APIs would
> >>> allow us releasing a new component soon!
> >>> Thanks in advance for your help and
> interesting, looking
> >>> forward to
> >>> hear from you soon!!!
> >>> Simo
> >>>
> >>> http://people.apache.org/~simonetripodi/
> >>> http://www.99soft.org/
> >>>
> >>>
> >>>
> >>> On Fri, Jul 22, 2011 at 11:41 AM, Mark
> Struberg <st...@yahoo.de>
> >>> wrote:
> >>> > Hi!
> >>> >
> >>> > Jakob Korherr (apacheId jakobk) is also
> interested and
> >>> did some work in this area in MyFaces and OWB
> already.
> >>> >
> >>> > There are 2 main parts in this project
> >>> >
> >>> > a.) A classpath scanner which does _not_
> use
> >>> Class.getDeclaredXxx etc, since this allocates
> memory in the
> >>> ClassLoader which cannot get freed up later.
> In a mid sized
> >>> project this leads to getting 100MB of
> PermGenSpace easily!
> >>> This is what Davids xbean-finder fixes already
> by just
> >>> scanning the structure from the bytecode
> directly.
> >>> >
> >>> > b.) Lots of EE apps need to scan the
> classpath for
> >>> annotations (e.g. MyFaces, OpenWebBeans,
> Tomcat, OpenEJB,
> >>> etc). So this part is done redundantly a few
> times when
> >>> starting the server. I hacked a quick API +
> SPI to do all
> >>> this in one go. The trick is to introduce a
> register
> >>> mechanism where classscan-clients can register
> their needs
> >>> before the scan gets performed.
> >>> >
> >>> >
> >>> > We like to do this at commons because
> it's really
> >>> decoupled from all the single projects and
> could be useful
> >>> for a lot more projects which added some kind
> of annotation
> >>> processing lately.
> >>> >
> >>> > LieGrue,
> >>> > strub
> >>> >
> >>> > --- On Fri, 7/22/11, Matt Benson <gu...@gmail.com>
> >>> wrote:
> >>> >
> >>> >> From: Matt Benson <gu...@gmail.com>
> >>> >> Subject: [sandbox] class scanning +
> karma
> >>> requests
> >>> >> To: dev@commons.apache.org
> >>> >> Cc: dblevins@apache.org,
> >>> "Gerhard Petracek" <ge...@gmail.com>
> >>> >> Date: Friday, July 22, 2011, 4:30 AM
> >>> >> First, Simo added [meiyo] to the
> >>> >> sandbox. Now, some of the guys
> from
> >>> >> the various JEE-related communities
> have expressed
> >>> interest
> >>> >> in working
> >>> >> with class scanning here at
> Commons. What we
> >>> would
> >>> >> propose to do is
> >>> >> start with the code of Geronimo's
> xbean-finder,
> >>> which scans
> >>> >> classes by
> >>> >> reading bytecode to populate
> meta-structures,
> >>> thereby
> >>> >> sparing permgen
> >>> >> space whenever possible. We plan to
> carefully
> >>> prune
> >>> >> and polish that
> >>> >> code, pull in the best ideas from
> [meiyo] (Simo
> >>> and/or any
> >>> >> of the rest
> >>> >> of you Commons developers are of
> course welcome
> >>> to
> >>> >> participate as
> >>> >> always!), and add a single-scan SPI
> strategy to
> >>> allow the
> >>> >> majority of
> >>> >> consumers to share the impact of
> scanning. For
> >>> my own
> >>> >> part, I hope to
> >>> >> be able to contribute such that the
> final API can
> >>> >> accommodate most if
> >>> >> not all conceivable use-cases for
> classpath
> >>> scanning, while
> >>> >> not
> >>> >> becoming too unwieldy. We'd like to
> get started
> >>> ASAP
> >>> >> (and obviously
> >>> >> nothing stops me from going ahead and
> taking the
> >>> first
> >>> >> steps). I
> >>> >> would also request our new chair
> grant sandbox
> >>> karma to:
> >>> >>
> >>> >> struberg (Mark Struberg)
> >>> >> dblevins (David Blevins)
> >>> >> gpetracek (Gerhard Petracek}
> >>> >>
> >>> >> Thanks,
> >>> >> Matt
> >>> >>
> >>> >>
> >>>
> ---------------------------------------------------------------------
> >>> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>> >> For additional commands, e-mail: dev-help@commons.apache.org
> >>> >>
> >>> >>
> >>> >
> >>> >
> >>>
> ---------------------------------------------------------------------
> >>> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>> > For additional commands, e-mail: dev-help@commons.apache.org
> >>> >
> >>> >
> >>>
> >>>
> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>
> >>>
> >>
> >>
> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >> For additional commands, e-mail: dev-help@commons.apache.org
> >>
> >>
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [sandbox] class scanning + karma requests
Posted by Simone Tripodi <si...@apache.org>.
Hi Mark!!!
For meiyo APIs I'd like to keep I mean end-user APIs; as Matt
suggested you, since you didn't hear about the already existing
sandbox component before, the most interesting parts of meiyo are the
filtering/callback APIs, expressed via EDSL. Have a look at the simple
included testcase[1] to see how they make clear its usage to end user
- of course they can be improved!!! - and I bet should not a big issue
integrating them in your more efficient engine :P
Looking forward to hear from you soon, all the best!
Simo
[1] http://s.apache.org/YIF
[2] http://s.apache.org/5Gt
http://people.apache.org/~simonetripodi/
http://www.99soft.org/
On Fri, Jul 22, 2011 at 4:59 PM, Matt Benson <gu...@gmail.com> wrote:
> On Fri, Jul 22, 2011 at 9:51 AM, Mark Struberg <st...@yahoo.de> wrote:
>> Hi Simo!
>>
>> Yea, taking the best ideas of both projects (meiyo and xbean-finder) would definitlely be a big bonus.
>> But to be honest: I think we should rather stay on the xbean-finder API. The reason is that it's already heavily used in Geronimo, OpenEJB, OpenWebBeans and a few other projects here at the ASF and abroad. So we should aim to make the transition as easy as possible. Of course tweaking the API to some degree is surely possible.
>> I also have to admit that I haven't heard of meiyo before yesterday, so I don't know it's capabilities.
>>
>
> IMO the most interesting parts of meiyo to take away are the
> filtering/callback APIs. I would think that this is mostly part of
> (or at least an option of) the client API e.g. for the single-scan
> feature, that we have to rewrite anyway. So I don't think these goals
> are incompatible.
>
> Matt
>
>> In any case you are highly welcome to join forces - any idea/opinion/help is welcome!
>>
>> LieGrue,
>> strub
>>
>> --- On Fri, 7/22/11, Simone Tripodi <si...@apache.org> wrote:
>>
>>> From: Simone Tripodi <si...@apache.org>
>>> Subject: Re: [sandbox] class scanning + karma requests
>>> To: "Commons Developers List" <de...@commons.apache.org>
>>> Date: Friday, July 22, 2011, 2:20 PM
>>> Hi all guys!
>>> That sounds really interesting!!!
>>> I think that bringing the new ideas in existing [meiyo]
>>> APIs would
>>> allow us releasing a new component soon!
>>> Thanks in advance for your help and interesting, looking
>>> forward to
>>> hear from you soon!!!
>>> Simo
>>>
>>> http://people.apache.org/~simonetripodi/
>>> http://www.99soft.org/
>>>
>>>
>>>
>>> On Fri, Jul 22, 2011 at 11:41 AM, Mark Struberg <st...@yahoo.de>
>>> wrote:
>>> > Hi!
>>> >
>>> > Jakob Korherr (apacheId jakobk) is also interested and
>>> did some work in this area in MyFaces and OWB already.
>>> >
>>> > There are 2 main parts in this project
>>> >
>>> > a.) A classpath scanner which does _not_ use
>>> Class.getDeclaredXxx etc, since this allocates memory in the
>>> ClassLoader which cannot get freed up later. In a mid sized
>>> project this leads to getting 100MB of PermGenSpace easily!
>>> This is what Davids xbean-finder fixes already by just
>>> scanning the structure from the bytecode directly.
>>> >
>>> > b.) Lots of EE apps need to scan the classpath for
>>> annotations (e.g. MyFaces, OpenWebBeans, Tomcat, OpenEJB,
>>> etc). So this part is done redundantly a few times when
>>> starting the server. I hacked a quick API + SPI to do all
>>> this in one go. The trick is to introduce a register
>>> mechanism where classscan-clients can register their needs
>>> before the scan gets performed.
>>> >
>>> >
>>> > We like to do this at commons because it's really
>>> decoupled from all the single projects and could be useful
>>> for a lot more projects which added some kind of annotation
>>> processing lately.
>>> >
>>> > LieGrue,
>>> > strub
>>> >
>>> > --- On Fri, 7/22/11, Matt Benson <gu...@gmail.com>
>>> wrote:
>>> >
>>> >> From: Matt Benson <gu...@gmail.com>
>>> >> Subject: [sandbox] class scanning + karma
>>> requests
>>> >> To: dev@commons.apache.org
>>> >> Cc: dblevins@apache.org,
>>> "Gerhard Petracek" <ge...@gmail.com>
>>> >> Date: Friday, July 22, 2011, 4:30 AM
>>> >> First, Simo added [meiyo] to the
>>> >> sandbox. Now, some of the guys from
>>> >> the various JEE-related communities have expressed
>>> interest
>>> >> in working
>>> >> with class scanning here at Commons. What we
>>> would
>>> >> propose to do is
>>> >> start with the code of Geronimo's xbean-finder,
>>> which scans
>>> >> classes by
>>> >> reading bytecode to populate meta-structures,
>>> thereby
>>> >> sparing permgen
>>> >> space whenever possible. We plan to carefully
>>> prune
>>> >> and polish that
>>> >> code, pull in the best ideas from [meiyo] (Simo
>>> and/or any
>>> >> of the rest
>>> >> of you Commons developers are of course welcome
>>> to
>>> >> participate as
>>> >> always!), and add a single-scan SPI strategy to
>>> allow the
>>> >> majority of
>>> >> consumers to share the impact of scanning. For
>>> my own
>>> >> part, I hope to
>>> >> be able to contribute such that the final API can
>>> >> accommodate most if
>>> >> not all conceivable use-cases for classpath
>>> scanning, while
>>> >> not
>>> >> becoming too unwieldy. We'd like to get started
>>> ASAP
>>> >> (and obviously
>>> >> nothing stops me from going ahead and taking the
>>> first
>>> >> steps). I
>>> >> would also request our new chair grant sandbox
>>> karma to:
>>> >>
>>> >> struberg (Mark Struberg)
>>> >> dblevins (David Blevins)
>>> >> gpetracek (Gerhard Petracek}
>>> >>
>>> >> Thanks,
>>> >> Matt
>>> >>
>>> >>
>>> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> >> For additional commands, e-mail: dev-help@commons.apache.org
>>> >>
>>> >>
>>> >
>>> >
>>> ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> > For additional commands, e-mail: dev-help@commons.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [sandbox] class scanning + karma requests
Posted by Matt Benson <gu...@gmail.com>.
On Fri, Jul 22, 2011 at 9:51 AM, Mark Struberg <st...@yahoo.de> wrote:
> Hi Simo!
>
> Yea, taking the best ideas of both projects (meiyo and xbean-finder) would definitlely be a big bonus.
> But to be honest: I think we should rather stay on the xbean-finder API. The reason is that it's already heavily used in Geronimo, OpenEJB, OpenWebBeans and a few other projects here at the ASF and abroad. So we should aim to make the transition as easy as possible. Of course tweaking the API to some degree is surely possible.
> I also have to admit that I haven't heard of meiyo before yesterday, so I don't know it's capabilities.
>
IMO the most interesting parts of meiyo to take away are the
filtering/callback APIs. I would think that this is mostly part of
(or at least an option of) the client API e.g. for the single-scan
feature, that we have to rewrite anyway. So I don't think these goals
are incompatible.
Matt
> In any case you are highly welcome to join forces - any idea/opinion/help is welcome!
>
> LieGrue,
> strub
>
> --- On Fri, 7/22/11, Simone Tripodi <si...@apache.org> wrote:
>
>> From: Simone Tripodi <si...@apache.org>
>> Subject: Re: [sandbox] class scanning + karma requests
>> To: "Commons Developers List" <de...@commons.apache.org>
>> Date: Friday, July 22, 2011, 2:20 PM
>> Hi all guys!
>> That sounds really interesting!!!
>> I think that bringing the new ideas in existing [meiyo]
>> APIs would
>> allow us releasing a new component soon!
>> Thanks in advance for your help and interesting, looking
>> forward to
>> hear from you soon!!!
>> Simo
>>
>> http://people.apache.org/~simonetripodi/
>> http://www.99soft.org/
>>
>>
>>
>> On Fri, Jul 22, 2011 at 11:41 AM, Mark Struberg <st...@yahoo.de>
>> wrote:
>> > Hi!
>> >
>> > Jakob Korherr (apacheId jakobk) is also interested and
>> did some work in this area in MyFaces and OWB already.
>> >
>> > There are 2 main parts in this project
>> >
>> > a.) A classpath scanner which does _not_ use
>> Class.getDeclaredXxx etc, since this allocates memory in the
>> ClassLoader which cannot get freed up later. In a mid sized
>> project this leads to getting 100MB of PermGenSpace easily!
>> This is what Davids xbean-finder fixes already by just
>> scanning the structure from the bytecode directly.
>> >
>> > b.) Lots of EE apps need to scan the classpath for
>> annotations (e.g. MyFaces, OpenWebBeans, Tomcat, OpenEJB,
>> etc). So this part is done redundantly a few times when
>> starting the server. I hacked a quick API + SPI to do all
>> this in one go. The trick is to introduce a register
>> mechanism where classscan-clients can register their needs
>> before the scan gets performed.
>> >
>> >
>> > We like to do this at commons because it's really
>> decoupled from all the single projects and could be useful
>> for a lot more projects which added some kind of annotation
>> processing lately.
>> >
>> > LieGrue,
>> > strub
>> >
>> > --- On Fri, 7/22/11, Matt Benson <gu...@gmail.com>
>> wrote:
>> >
>> >> From: Matt Benson <gu...@gmail.com>
>> >> Subject: [sandbox] class scanning + karma
>> requests
>> >> To: dev@commons.apache.org
>> >> Cc: dblevins@apache.org,
>> "Gerhard Petracek" <ge...@gmail.com>
>> >> Date: Friday, July 22, 2011, 4:30 AM
>> >> First, Simo added [meiyo] to the
>> >> sandbox. Now, some of the guys from
>> >> the various JEE-related communities have expressed
>> interest
>> >> in working
>> >> with class scanning here at Commons. What we
>> would
>> >> propose to do is
>> >> start with the code of Geronimo's xbean-finder,
>> which scans
>> >> classes by
>> >> reading bytecode to populate meta-structures,
>> thereby
>> >> sparing permgen
>> >> space whenever possible. We plan to carefully
>> prune
>> >> and polish that
>> >> code, pull in the best ideas from [meiyo] (Simo
>> and/or any
>> >> of the rest
>> >> of you Commons developers are of course welcome
>> to
>> >> participate as
>> >> always!), and add a single-scan SPI strategy to
>> allow the
>> >> majority of
>> >> consumers to share the impact of scanning. For
>> my own
>> >> part, I hope to
>> >> be able to contribute such that the final API can
>> >> accommodate most if
>> >> not all conceivable use-cases for classpath
>> scanning, while
>> >> not
>> >> becoming too unwieldy. We'd like to get started
>> ASAP
>> >> (and obviously
>> >> nothing stops me from going ahead and taking the
>> first
>> >> steps). I
>> >> would also request our new chair grant sandbox
>> karma to:
>> >>
>> >> struberg (Mark Struberg)
>> >> dblevins (David Blevins)
>> >> gpetracek (Gerhard Petracek}
>> >>
>> >> Thanks,
>> >> Matt
>> >>
>> >>
>> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> >> For additional commands, e-mail: dev-help@commons.apache.org
>> >>
>> >>
>> >
>> >
>> ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> > For additional commands, e-mail: dev-help@commons.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [sandbox] class scanning + karma requests
Posted by Mark Struberg <st...@yahoo.de>.
Hi Simo!
Yea, taking the best ideas of both projects (meiyo and xbean-finder) would definitlely be a big bonus.
But to be honest: I think we should rather stay on the xbean-finder API. The reason is that it's already heavily used in Geronimo, OpenEJB, OpenWebBeans and a few other projects here at the ASF and abroad. So we should aim to make the transition as easy as possible. Of course tweaking the API to some degree is surely possible.
I also have to admit that I haven't heard of meiyo before yesterday, so I don't know it's capabilities.
In any case you are highly welcome to join forces - any idea/opinion/help is welcome!
LieGrue,
strub
--- On Fri, 7/22/11, Simone Tripodi <si...@apache.org> wrote:
> From: Simone Tripodi <si...@apache.org>
> Subject: Re: [sandbox] class scanning + karma requests
> To: "Commons Developers List" <de...@commons.apache.org>
> Date: Friday, July 22, 2011, 2:20 PM
> Hi all guys!
> That sounds really interesting!!!
> I think that bringing the new ideas in existing [meiyo]
> APIs would
> allow us releasing a new component soon!
> Thanks in advance for your help and interesting, looking
> forward to
> hear from you soon!!!
> Simo
>
> http://people.apache.org/~simonetripodi/
> http://www.99soft.org/
>
>
>
> On Fri, Jul 22, 2011 at 11:41 AM, Mark Struberg <st...@yahoo.de>
> wrote:
> > Hi!
> >
> > Jakob Korherr (apacheId jakobk) is also interested and
> did some work in this area in MyFaces and OWB already.
> >
> > There are 2 main parts in this project
> >
> > a.) A classpath scanner which does _not_ use
> Class.getDeclaredXxx etc, since this allocates memory in the
> ClassLoader which cannot get freed up later. In a mid sized
> project this leads to getting 100MB of PermGenSpace easily!
> This is what Davids xbean-finder fixes already by just
> scanning the structure from the bytecode directly.
> >
> > b.) Lots of EE apps need to scan the classpath for
> annotations (e.g. MyFaces, OpenWebBeans, Tomcat, OpenEJB,
> etc). So this part is done redundantly a few times when
> starting the server. I hacked a quick API + SPI to do all
> this in one go. The trick is to introduce a register
> mechanism where classscan-clients can register their needs
> before the scan gets performed.
> >
> >
> > We like to do this at commons because it's really
> decoupled from all the single projects and could be useful
> for a lot more projects which added some kind of annotation
> processing lately.
> >
> > LieGrue,
> > strub
> >
> > --- On Fri, 7/22/11, Matt Benson <gu...@gmail.com>
> wrote:
> >
> >> From: Matt Benson <gu...@gmail.com>
> >> Subject: [sandbox] class scanning + karma
> requests
> >> To: dev@commons.apache.org
> >> Cc: dblevins@apache.org,
> "Gerhard Petracek" <ge...@gmail.com>
> >> Date: Friday, July 22, 2011, 4:30 AM
> >> First, Simo added [meiyo] to the
> >> sandbox. Now, some of the guys from
> >> the various JEE-related communities have expressed
> interest
> >> in working
> >> with class scanning here at Commons. What we
> would
> >> propose to do is
> >> start with the code of Geronimo's xbean-finder,
> which scans
> >> classes by
> >> reading bytecode to populate meta-structures,
> thereby
> >> sparing permgen
> >> space whenever possible. We plan to carefully
> prune
> >> and polish that
> >> code, pull in the best ideas from [meiyo] (Simo
> and/or any
> >> of the rest
> >> of you Commons developers are of course welcome
> to
> >> participate as
> >> always!), and add a single-scan SPI strategy to
> allow the
> >> majority of
> >> consumers to share the impact of scanning. For
> my own
> >> part, I hope to
> >> be able to contribute such that the final API can
> >> accommodate most if
> >> not all conceivable use-cases for classpath
> scanning, while
> >> not
> >> becoming too unwieldy. We'd like to get started
> ASAP
> >> (and obviously
> >> nothing stops me from going ahead and taking the
> first
> >> steps). I
> >> would also request our new chair grant sandbox
> karma to:
> >>
> >> struberg (Mark Struberg)
> >> dblevins (David Blevins)
> >> gpetracek (Gerhard Petracek}
> >>
> >> Thanks,
> >> Matt
> >>
> >>
> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >> For additional commands, e-mail: dev-help@commons.apache.org
> >>
> >>
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [sandbox] class scanning + karma requests
Posted by Simone Tripodi <si...@apache.org>.
Hi all guys!
That sounds really interesting!!!
I think that bringing the new ideas in existing [meiyo] APIs would
allow us releasing a new component soon!
Thanks in advance for your help and interesting, looking forward to
hear from you soon!!!
Simo
http://people.apache.org/~simonetripodi/
http://www.99soft.org/
On Fri, Jul 22, 2011 at 11:41 AM, Mark Struberg <st...@yahoo.de> wrote:
> Hi!
>
> Jakob Korherr (apacheId jakobk) is also interested and did some work in this area in MyFaces and OWB already.
>
> There are 2 main parts in this project
>
> a.) A classpath scanner which does _not_ use Class.getDeclaredXxx etc, since this allocates memory in the ClassLoader which cannot get freed up later. In a mid sized project this leads to getting 100MB of PermGenSpace easily! This is what Davids xbean-finder fixes already by just scanning the structure from the bytecode directly.
>
> b.) Lots of EE apps need to scan the classpath for annotations (e.g. MyFaces, OpenWebBeans, Tomcat, OpenEJB, etc). So this part is done redundantly a few times when starting the server. I hacked a quick API + SPI to do all this in one go. The trick is to introduce a register mechanism where classscan-clients can register their needs before the scan gets performed.
>
>
> We like to do this at commons because it's really decoupled from all the single projects and could be useful for a lot more projects which added some kind of annotation processing lately.
>
> LieGrue,
> strub
>
> --- On Fri, 7/22/11, Matt Benson <gu...@gmail.com> wrote:
>
>> From: Matt Benson <gu...@gmail.com>
>> Subject: [sandbox] class scanning + karma requests
>> To: dev@commons.apache.org
>> Cc: dblevins@apache.org, "Gerhard Petracek" <ge...@gmail.com>
>> Date: Friday, July 22, 2011, 4:30 AM
>> First, Simo added [meiyo] to the
>> sandbox. Now, some of the guys from
>> the various JEE-related communities have expressed interest
>> in working
>> with class scanning here at Commons. What we would
>> propose to do is
>> start with the code of Geronimo's xbean-finder, which scans
>> classes by
>> reading bytecode to populate meta-structures, thereby
>> sparing permgen
>> space whenever possible. We plan to carefully prune
>> and polish that
>> code, pull in the best ideas from [meiyo] (Simo and/or any
>> of the rest
>> of you Commons developers are of course welcome to
>> participate as
>> always!), and add a single-scan SPI strategy to allow the
>> majority of
>> consumers to share the impact of scanning. For my own
>> part, I hope to
>> be able to contribute such that the final API can
>> accommodate most if
>> not all conceivable use-cases for classpath scanning, while
>> not
>> becoming too unwieldy. We'd like to get started ASAP
>> (and obviously
>> nothing stops me from going ahead and taking the first
>> steps). I
>> would also request our new chair grant sandbox karma to:
>>
>> struberg (Mark Struberg)
>> dblevins (David Blevins)
>> gpetracek (Gerhard Petracek}
>>
>> Thanks,
>> Matt
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [sandbox] class scanning + karma requests
Posted by Mark Struberg <st...@yahoo.de>.
Hi!
Jakob Korherr (apacheId jakobk) is also interested and did some work in this area in MyFaces and OWB already.
There are 2 main parts in this project
a.) A classpath scanner which does _not_ use Class.getDeclaredXxx etc, since this allocates memory in the ClassLoader which cannot get freed up later. In a mid sized project this leads to getting 100MB of PermGenSpace easily! This is what Davids xbean-finder fixes already by just scanning the structure from the bytecode directly.
b.) Lots of EE apps need to scan the classpath for annotations (e.g. MyFaces, OpenWebBeans, Tomcat, OpenEJB, etc). So this part is done redundantly a few times when starting the server. I hacked a quick API + SPI to do all this in one go. The trick is to introduce a register mechanism where classscan-clients can register their needs before the scan gets performed.
We like to do this at commons because it's really decoupled from all the single projects and could be useful for a lot more projects which added some kind of annotation processing lately.
LieGrue,
strub
--- On Fri, 7/22/11, Matt Benson <gu...@gmail.com> wrote:
> From: Matt Benson <gu...@gmail.com>
> Subject: [sandbox] class scanning + karma requests
> To: dev@commons.apache.org
> Cc: dblevins@apache.org, "Gerhard Petracek" <ge...@gmail.com>
> Date: Friday, July 22, 2011, 4:30 AM
> First, Simo added [meiyo] to the
> sandbox. Now, some of the guys from
> the various JEE-related communities have expressed interest
> in working
> with class scanning here at Commons. What we would
> propose to do is
> start with the code of Geronimo's xbean-finder, which scans
> classes by
> reading bytecode to populate meta-structures, thereby
> sparing permgen
> space whenever possible. We plan to carefully prune
> and polish that
> code, pull in the best ideas from [meiyo] (Simo and/or any
> of the rest
> of you Commons developers are of course welcome to
> participate as
> always!), and add a single-scan SPI strategy to allow the
> majority of
> consumers to share the impact of scanning. For my own
> part, I hope to
> be able to contribute such that the final API can
> accommodate most if
> not all conceivable use-cases for classpath scanning, while
> not
> becoming too unwieldy. We'd like to get started ASAP
> (and obviously
> nothing stops me from going ahead and taking the first
> steps). I
> would also request our new chair grant sandbox karma to:
>
> struberg (Mark Struberg)
> dblevins (David Blevins)
> gpetracek (Gerhard Petracek}
>
> Thanks,
> Matt
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org