You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Mark Struberg <st...@yahoo.de> on 2012/06/06 23:37:50 UTC

[classscan] Metadata API discussion

Hi!

I now did read through the metadata classes of Chas' and Davids impls.

Both look pretty similar to some degree. A few key differences

* using AnnotatedElement instead of HasName() makes it possible to replace most 'old' code which does getAnnotations() etc 1:1
 Imo we should keep this feature.

* to interface or not to interface, that's the question. 

 The actual meta information should be as small as possible. There are tons of it. 

 1. Currently in Chas' impl there are bcel specific parts in it. Can those probably be extracted? I think so, or do you see any problems? If we cannot split then we should keep the interfaces. But remember that this takes (a bit) more space in mem.
 2. If Interfaces, then impls could point to bcel/asm internal strutures for storing the data. This would reduce space. But only if bcel/asm would keep this info anyway. Otherswise it might be better to throw the bcel/asm stuff away after repackaging to a small meta info holder.



(just to not loose some questions... more to come...)

LieGrue,
strub

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Re: [classscan] Metadata API discussion

Posted by Mark Struberg <st...@yahoo.de>.
Hi folks, quick reply from vacation :)


@sebb: yes correct. I was just thinking too complicated. We use the interface only for keeping the metainfo, and not in the metainfo itself. So it should itself not blow up the mem. So +1 for the interface.

There are 2 important consideration options left:

a.) Store the information we get from BCEL/ASM directly in the meta-info instances.

b.) Take the information we get from BCEL/ASM and store them e.g. in a SingleLinkedList or array. 

Which one is more effective also has to do whether BCEL/ASM throw away their information after they pass it to us or if they keep it internally for later reuse. If they store the information anyway, then  a.) is better - otherwise we should do b.)



LieGrue,
strub


----- Original Message -----
> From: "Honton, Charles" <Ch...@intuit.com>
> To: Commons Developers List <de...@commons.apache.org>
> Cc: 
> Sent: Friday, June 8, 2012 5:47 PM
> Subject: Re: [classscan] Metadata API discussion
> 
> In on my MacBook, I get the following counts for running just the unit
> tests:
> 
> Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-10M3527)
> Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
> 
> Cache:    1
> BootstrapClassLoader:    1
> WeakConcurrentHashMap:    1
> MetaUrlClassLoader:    2
> HashWeakReference:    5
> FileClassLocation:    5
> PrimitiveClass:    9
> JarClassLocation:    27
> SortedReadOnlySet:    38
> BcelEnumProperty:    296
> BcelProperty:    740
> AnnotationMap$1:    3217
> BcelAnnotation:    3417
> BcelClass$1:    9251
> DeferredMetaClass:    13620
> BcelClass$3:    18232
> BcelClass$2:    26096
> BcelClass:    26774
> ArrayMetaType:    33275
> ReadOnlySet:    54240
> BcelField:    110983
> BcelMethod:    231549
> BcelParameter:    264572
> ClassMetaType:    607104
> AnnotationMap:    633878
> 
> Regards,
> 
> chas
> 
> 
> On 6/7/12 5:27 PM, "James Carman" <jc...@carmanconsulting.com> 
> wrote:
> 
>> There can be quite a few if you have to have them for every class,
>> interface, annotation, method, field, etc.  Then again if you just reuse 1
>> adapter object all the time it wouldn't be that bad. You would just have
>> to
>> let users know that they cannot maintain references to those objects after
>> scanning is complete.
>> On Jun 7, 2012 8:10 PM, "sebb" <se...@gmail.com> wrote:
>> 
>>>  On 7 June 2012 21:11, James Carman <ja...@carmanconsulting.com> 
> wrote:
>>>  > On Thu, Jun 7, 2012 at 3:56 AM, sebb <se...@gmail.com> 
> wrote:
>>>  >>
>>>  >> Not sure I follow this. Why would an interface use extra 
> memory?
>>>  >> I can see that it might add a bit more to the static size of a 
> class,
>>>  >> but why would it add more to each instance of a class that 
> uses it?
>>>  >>
>>>  >
>>>  > It's not the interface itself.  It's the fact that you 
> have to have
>>>  > more objects loaded into memory when you use the interface-based
>>>  > approach.  For example, if BCEL has some object that represents a
>>>  > class' metadata, then we'll have to put some 
> "adapter" object in front
>>>  > of it that implements our metadata API interface and knows how to
>>>  > speak BCEL-speak to extract the information.
>>> 
>>>  Yes, but again, surely there won't be all that many such objects?
>>> 
>>>  I think some tests should be done of the two approaches before
>>>  deciding to abandon the interface-based design.
>>> 
>>>  > 
> ---------------------------------------------------------------------
>>>  > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>  > For additional commands, e-mail: dev-help@commons.apache.org
>>>  >
>>> 
>>>  ---------------------------------------------------------------------
>>>  To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>  For additional commands, e-mail: dev-help@commons.apache.org
>>> 
>>> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Re: [classscan] Metadata API discussion

Posted by "Honton, Charles" <Ch...@intuit.com>.
In on my MacBook, I get the following counts for running just the unit
tests:

Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-10M3527)
Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)

Cache:	1
BootstrapClassLoader:	1
WeakConcurrentHashMap:	1
MetaUrlClassLoader:	2
HashWeakReference:	5
FileClassLocation:	5
PrimitiveClass:	9
JarClassLocation:	27
SortedReadOnlySet:	38
BcelEnumProperty:	296
BcelProperty:	740
AnnotationMap$1:	3217
BcelAnnotation:	3417
BcelClass$1:	9251
DeferredMetaClass:	13620
BcelClass$3:	18232
BcelClass$2:	26096
BcelClass:	26774
ArrayMetaType:	33275
ReadOnlySet:	54240
BcelField:	110983
BcelMethod:	231549
BcelParameter:	264572
ClassMetaType:	607104
AnnotationMap:	633878

Regards,

chas


On 6/7/12 5:27 PM, "James Carman" <jc...@carmanconsulting.com> wrote:

>There can be quite a few if you have to have them for every class,
>interface, annotation, method, field, etc.  Then again if you just reuse 1
>adapter object all the time it wouldn't be that bad. You would just have
>to
>let users know that they cannot maintain references to those objects after
>scanning is complete.
>On Jun 7, 2012 8:10 PM, "sebb" <se...@gmail.com> wrote:
>
>> On 7 June 2012 21:11, James Carman <ja...@carmanconsulting.com> wrote:
>> > On Thu, Jun 7, 2012 at 3:56 AM, sebb <se...@gmail.com> wrote:
>> >>
>> >> Not sure I follow this. Why would an interface use extra memory?
>> >> I can see that it might add a bit more to the static size of a class,
>> >> but why would it add more to each instance of a class that uses it?
>> >>
>> >
>> > It's not the interface itself.  It's the fact that you have to have
>> > more objects loaded into memory when you use the interface-based
>> > approach.  For example, if BCEL has some object that represents a
>> > class' metadata, then we'll have to put some "adapter" object in front
>> > of it that implements our metadata API interface and knows how to
>> > speak BCEL-speak to extract the information.
>>
>> Yes, but again, surely there won't be all that many such objects?
>>
>> I think some tests should be done of the two approaches before
>> deciding to abandon the interface-based design.
>>
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> > For additional commands, e-mail: dev-help@commons.apache.org
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Re: [classscan] Metadata API discussion

Posted by James Carman <jc...@carmanconsulting.com>.
There can be quite a few if you have to have them for every class,
interface, annotation, method, field, etc.  Then again if you just reuse 1
adapter object all the time it wouldn't be that bad. You would just have to
let users know that they cannot maintain references to those objects after
scanning is complete.
On Jun 7, 2012 8:10 PM, "sebb" <se...@gmail.com> wrote:

> On 7 June 2012 21:11, James Carman <ja...@carmanconsulting.com> wrote:
> > On Thu, Jun 7, 2012 at 3:56 AM, sebb <se...@gmail.com> wrote:
> >>
> >> Not sure I follow this. Why would an interface use extra memory?
> >> I can see that it might add a bit more to the static size of a class,
> >> but why would it add more to each instance of a class that uses it?
> >>
> >
> > It's not the interface itself.  It's the fact that you have to have
> > more objects loaded into memory when you use the interface-based
> > approach.  For example, if BCEL has some object that represents a
> > class' metadata, then we'll have to put some "adapter" object in front
> > of it that implements our metadata API interface and knows how to
> > speak BCEL-speak to extract the information.
>
> Yes, but again, surely there won't be all that many such objects?
>
> I think some tests should be done of the two approaches before
> deciding to abandon the interface-based design.
>
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>

Re: [classscan] Metadata API discussion

Posted by sebb <se...@gmail.com>.
On 7 June 2012 21:11, James Carman <ja...@carmanconsulting.com> wrote:
> On Thu, Jun 7, 2012 at 3:56 AM, sebb <se...@gmail.com> wrote:
>>
>> Not sure I follow this. Why would an interface use extra memory?
>> I can see that it might add a bit more to the static size of a class,
>> but why would it add more to each instance of a class that uses it?
>>
>
> It's not the interface itself.  It's the fact that you have to have
> more objects loaded into memory when you use the interface-based
> approach.  For example, if BCEL has some object that represents a
> class' metadata, then we'll have to put some "adapter" object in front
> of it that implements our metadata API interface and knows how to
> speak BCEL-speak to extract the information.

Yes, but again, surely there won't be all that many such objects?

I think some tests should be done of the two approaches before
deciding to abandon the interface-based design.

> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Re: [classscan] Metadata API discussion

Posted by James Carman <ja...@carmanconsulting.com>.
On Thu, Jun 7, 2012 at 3:56 AM, sebb <se...@gmail.com> wrote:
>
> Not sure I follow this. Why would an interface use extra memory?
> I can see that it might add a bit more to the static size of a class,
> but why would it add more to each instance of a class that uses it?
>

It's not the interface itself.  It's the fact that you have to have
more objects loaded into memory when you use the interface-based
approach.  For example, if BCEL has some object that represents a
class' metadata, then we'll have to put some "adapter" object in front
of it that implements our metadata API interface and knows how to
speak BCEL-speak to extract the information.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Re: [classscan] Metadata API discussion

Posted by sebb <se...@gmail.com>.
On 7 June 2012 05:18, Mark Struberg <st...@yahoo.de> wrote:
>> You mention that interfaces take a bit more space in memory.  Is this
>> because there is an adapter class (implementing the interface) and the
>> delegated final implementor? Or are you pointing out some other memory
>> concern?
> It's not much actually. Merely 80 bytes + fully qualified class name length. But we have 100.000s of instances. That might sum up. We need to trace it finally. Otherwise +1 for interfaces.
>

Not sure I follow this. Why would an interface use extra memory?
I can see that it might add a bit more to the static size of a class,
but why would it add more to each instance of a class that uses it?

>
> LieGrue,
> strub
>
>
>
> ----- Original Message -----
>> From: "Honton, Charles" <Ch...@intuit.com>
>> To: Commons Developers List <de...@commons.apache.org>; Mark Struberg <st...@yahoo.de>
>> Cc:
>> Sent: Thursday, June 7, 2012 1:49 AM
>> Subject: Re: [classscan] Metadata API discussion
>>
>> Mark,
>>
>> In the BCEL implementation I originally used adapter classes which
>> implemented the appropriate interface and delegated to the BCEL object.
>> Unfortunately, this created a large memory footprint.  The BCEL objects
>> are not frugal with memory.
>>
>> I'm pretty sure we could tease apart the BCEL parts from the caching
>> parts.  During that split, we could introduce the scanner SPI.
>>
>> I am a big fan of interfaces.  Interfaces at the SPI would allow
>> techniques such as delayed instantiation, perhaps other flexibility that
>> will be useful.
>>
>> On the other hand, we may find each provider re-implement the same Meta
>> information classes.  We may wish to provide a default immutable
>> implementation for each meta information interface.
>>
>> You mention that interfaces take a bit more space in memory.  Is this
>> because there is an adapter class (implementing the interface) and the
>> delegated final implementor? Or are you pointing out some other memory
>> concern?
>>
>> Chas
>>
>>
>>
>> On 6/6/12 2:37 PM, "Mark Struberg" <st...@yahoo.de> wrote:
>>
>>> Hi!
>>>
>>> I now did read through the metadata classes of Chas' and Davids impls.
>>>
>>> Both look pretty similar to some degree. A few key differences
>>>
>>> * using AnnotatedElement instead of HasName() makes it possible to
>>> replace most 'old' code which does getAnnotations() etc 1:1
>>>  Imo we should keep this feature.
>>>
>>> * to interface or not to interface, that's the question.
>>>
>>>  The actual meta information should be as small as possible. There are
>>> tons of it.
>>>
>>>  1. Currently in Chas' impl there are bcel specific parts in it. Can
>>> those probably be extracted? I think so, or do you see any problems? If
>>> we cannot split then we should keep the interfaces. But remember that
>>> this takes (a bit) more space in mem.
>>>  2. If Interfaces, then impls could point to bcel/asm internal strutures
>>> for storing the data. This would reduce space. But only if bcel/asm would
>>> keep this info anyway. Otherswise it might be better to throw the
>>> bcel/asm stuff away after repackaging to a small meta info holder.
>>>
>>>
>>>
>>> (just to not loose some questions... more to come...)
>>>
>>> LieGrue,
>>> strub
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Re: [classscan] Metadata API discussion

Posted by Mark Struberg <st...@yahoo.de>.
> You mention that interfaces take a bit more space in memory.  Is this
> because there is an adapter class (implementing the interface) and the
> delegated final implementor? Or are you pointing out some other memory
> concern?
It's not much actually. Merely 80 bytes + fully qualified class name length. But we have 100.000s of instances. That might sum up. We need to trace it finally. Otherwise +1 for interfaces. 


LieGrue,
strub



----- Original Message -----
> From: "Honton, Charles" <Ch...@intuit.com>
> To: Commons Developers List <de...@commons.apache.org>; Mark Struberg <st...@yahoo.de>
> Cc: 
> Sent: Thursday, June 7, 2012 1:49 AM
> Subject: Re: [classscan] Metadata API discussion
> 
> Mark,
> 
> In the BCEL implementation I originally used adapter classes which
> implemented the appropriate interface and delegated to the BCEL object.
> Unfortunately, this created a large memory footprint.  The BCEL objects
> are not frugal with memory.
> 
> I'm pretty sure we could tease apart the BCEL parts from the caching
> parts.  During that split, we could introduce the scanner SPI.
> 
> I am a big fan of interfaces.  Interfaces at the SPI would allow
> techniques such as delayed instantiation, perhaps other flexibility that
> will be useful.
> 
> On the other hand, we may find each provider re-implement the same Meta
> information classes.  We may wish to provide a default immutable
> implementation for each meta information interface.
> 
> You mention that interfaces take a bit more space in memory.  Is this
> because there is an adapter class (implementing the interface) and the
> delegated final implementor? Or are you pointing out some other memory
> concern?
> 
> Chas
> 
> 
> 
> On 6/6/12 2:37 PM, "Mark Struberg" <st...@yahoo.de> wrote:
> 
>> Hi!
>> 
>> I now did read through the metadata classes of Chas' and Davids impls.
>> 
>> Both look pretty similar to some degree. A few key differences
>> 
>> * using AnnotatedElement instead of HasName() makes it possible to
>> replace most 'old' code which does getAnnotations() etc 1:1
>>  Imo we should keep this feature.
>> 
>> * to interface or not to interface, that's the question.
>> 
>>  The actual meta information should be as small as possible. There are
>> tons of it. 
>> 
>>  1. Currently in Chas' impl there are bcel specific parts in it. Can
>> those probably be extracted? I think so, or do you see any problems? If
>> we cannot split then we should keep the interfaces. But remember that
>> this takes (a bit) more space in mem.
>>  2. If Interfaces, then impls could point to bcel/asm internal strutures
>> for storing the data. This would reduce space. But only if bcel/asm would
>> keep this info anyway. Otherswise it might be better to throw the
>> bcel/asm stuff away after repackaging to a small meta info holder.
>> 
>> 
>> 
>> (just to not loose some questions... more to come...)
>> 
>> LieGrue,
>> strub
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Re: [classscan] Metadata API discussion

Posted by "Honton, Charles" <Ch...@intuit.com>.
Mark,

In the BCEL implementation I originally used adapter classes which
implemented the appropriate interface and delegated to the BCEL object.
Unfortunately, this created a large memory footprint.  The BCEL objects
are not frugal with memory.

I'm pretty sure we could tease apart the BCEL parts from the caching
parts.  During that split, we could introduce the scanner SPI.

I am a big fan of interfaces.  Interfaces at the SPI would allow
techniques such as delayed instantiation, perhaps other flexibility that
will be useful.

On the other hand, we may find each provider re-implement the same Meta
information classes.  We may wish to provide a default immutable
implementation for each meta information interface.

You mention that interfaces take a bit more space in memory.  Is this
because there is an adapter class (implementing the interface) and the
delegated final implementor? Or are you pointing out some other memory
concern?

Chas



On 6/6/12 2:37 PM, "Mark Struberg" <st...@yahoo.de> wrote:

>Hi!
>
>I now did read through the metadata classes of Chas' and Davids impls.
>
>Both look pretty similar to some degree. A few key differences
>
>* using AnnotatedElement instead of HasName() makes it possible to
>replace most 'old' code which does getAnnotations() etc 1:1
> Imo we should keep this feature.
>
>* to interface or not to interface, that's the question.
>
> The actual meta information should be as small as possible. There are
>tons of it. 
>
> 1. Currently in Chas' impl there are bcel specific parts in it. Can
>those probably be extracted? I think so, or do you see any problems? If
>we cannot split then we should keep the interfaces. But remember that
>this takes (a bit) more space in mem.
> 2. If Interfaces, then impls could point to bcel/asm internal strutures
>for storing the data. This would reduce space. But only if bcel/asm would
>keep this info anyway. Otherswise it might be better to throw the
>bcel/asm stuff away after repackaging to a small meta info holder.
>
>
>
>(just to not loose some questions... more to come...)
>
>LieGrue,
>strub
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>For additional commands, e-mail: dev-help@commons.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org