Annotated MBean causing significant performance issue with JMX.

[Kevin Burton <bu...@spinn3r.com>]

OK. I’ve fixed 2-3 significant bugs in ActiveMQ with large numbers of
queues and degraded performance. Most of theses are O(N^2) bugs so the more
queues you have the more this becomes VERY painful.

I don’t have an easy fix of this one though.

Queue creation right now is about 3x slower than it could be…. Mostly
because of AnnotatedMBean.

http://imgur.com/a/iS1Xb

This is a constant time issue though.  Not as urgent of a fix as the other
bugs.

This is ALL due to reflection.

What I was thinking of doing was building a cache so that we could lookup
the annotation info from the Class in the cache first, and if it’s absent,
read it from reflection.

Thoughts?

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>

Re: Annotated MBean causing significant performance issue with JMX.

[Tim Bain <tb...@alumni.duke.edu>]

Guava's caches include the ability to define eviction behavior, based on
timeouts, memory usage, etc.  Seriously, check them out if you haven't
previously, you'll be glad you did.

I'm not sure keying off the class name solves the problem (if there really
is one).  If you've got a situation where you're reloading a class, then
you can't guarantee the new version (which will have the same
fully-qualified name) will have the same reflection-based result as what's
in the cache, so the cache might be invalid after a reload.  Now that I
think it through a bit more, I think it's better to just set a cache
eviction policy based on time; the old class won't go out of scope
immediately, but you'll only have to wait for the eviction timeout, which
is probably fine.  And again, I'm not sure whether this is even a concern,
because I don't know how ActiveMQ deals with plugins; this may be a
non-issue (though a cache eviction policy handles it cleanly enough that
I'd probably do that even if it's not currently needed).

Tim
On May 3, 2015 8:03 PM, "Kevin Burton" <bu...@spinn3r.com> wrote:

> Ah.. What I could do is contain a reference to the Class name.. I think
> it’s just Class method and parameter annotations.  So those have full
> paths.
>
> And I was probably just going to use a simple ConcurrentHashMap for the
> case..
>
> I t *might* leak a bit of memory if it’s static.. but maybe I could make it
> so its not necessarily static.. Hm.
>
> Another idea is just a cache TTL. so entries are kept for 5 minutes then
> destroyed.
>
> In retrospect it’s not a MASSIVE performance increase though.  But it turns
> out that it takes 2-4x longer to create queues when JMX is enabled because
> of this.
>
> On Sun, May 3, 2015 at 6:40 PM, Tim Bain <tb...@alumni.duke.edu> wrote:
>
> > Guava's got some awesome support for caches; they may be the best feature
> > in the library (and Multisets/Multimaps are pretty awesome).  If you
> > implement this (and it sounds like a good enhancement), you should use
> them
> > for it.
> >
> > One thing to consider: this cache will hold references to classes, and
> > under certain circumstances (especially OSGI-style plugins with child
> > classloaders or webapps in an app server) that can result in classes not
> > being unloaded when they're no longer used.  I don't think any of those
> > situations applies here, but someone who knows the architecture of
> ActiveMQ
> > plugins should probably confirm that.
> >
> > Tim
> > On May 2, 2015 3:05 PM, "Kevin Burton" <bu...@spinn3r.com> wrote:
> >
> > > OK. I’ve fixed 2-3 significant bugs in ActiveMQ with large numbers of
> > > queues and degraded performance. Most of theses are O(N^2) bugs so the
> > more
> > > queues you have the more this becomes VERY painful.
> > >
> > > I don’t have an easy fix of this one though.
> > >
> > > Queue creation right now is about 3x slower than it could be…. Mostly
> > > because of AnnotatedMBean.
> > >
> > > http://imgur.com/a/iS1Xb
> > >
> > > This is a constant time issue though.  Not as urgent of a fix as the
> > other
> > > bugs.
> > >
> > > This is ALL due to reflection.
> > >
> > > What I was thinking of doing was building a cache so that we could
> lookup
> > > the annotation info from the Class in the cache first, and if it’s
> > absent,
> > > read it from reflection.
> > >
> > > Thoughts?
> > >
> > > --
> > >
> > > Founder/CEO Spinn3r.com
> > > Location: *San Francisco, CA*
> > > blog: http://burtonator.wordpress.com
> > > … or check out my Google+ profile
> > > <https://plus.google.com/102718274791889610666/posts>
> > >
> >
>
>
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
>

Re: Annotated MBean causing significant performance issue with JMX.

[Kevin Burton <bu...@spinn3r.com>]

Ah.. What I could do is contain a reference to the Class name.. I think
it’s just Class method and parameter annotations.  So those have full paths.

And I was probably just going to use a simple ConcurrentHashMap for the
case..

I t *might* leak a bit of memory if it’s static.. but maybe I could make it
so its not necessarily static.. Hm.

Another idea is just a cache TTL. so entries are kept for 5 minutes then
destroyed.

In retrospect it’s not a MASSIVE performance increase though.  But it turns
out that it takes 2-4x longer to create queues when JMX is enabled because
of this.

On Sun, May 3, 2015 at 6:40 PM, Tim Bain <tb...@alumni.duke.edu> wrote:

> Guava's got some awesome support for caches; they may be the best feature
> in the library (and Multisets/Multimaps are pretty awesome).  If you
> implement this (and it sounds like a good enhancement), you should use them
> for it.
>
> One thing to consider: this cache will hold references to classes, and
> under certain circumstances (especially OSGI-style plugins with child
> classloaders or webapps in an app server) that can result in classes not
> being unloaded when they're no longer used.  I don't think any of those
> situations applies here, but someone who knows the architecture of ActiveMQ
> plugins should probably confirm that.
>
> Tim
> On May 2, 2015 3:05 PM, "Kevin Burton" <bu...@spinn3r.com> wrote:
>
> > OK. I’ve fixed 2-3 significant bugs in ActiveMQ with large numbers of
> > queues and degraded performance. Most of theses are O(N^2) bugs so the
> more
> > queues you have the more this becomes VERY painful.
> >
> > I don’t have an easy fix of this one though.
> >
> > Queue creation right now is about 3x slower than it could be…. Mostly
> > because of AnnotatedMBean.
> >
> > http://imgur.com/a/iS1Xb
> >
> > This is a constant time issue though.  Not as urgent of a fix as the
> other
> > bugs.
> >
> > This is ALL due to reflection.
> >
> > What I was thinking of doing was building a cache so that we could lookup
> > the annotation info from the Class in the cache first, and if it’s
> absent,
> > read it from reflection.
> >
> > Thoughts?
> >
> > --
> >
> > Founder/CEO Spinn3r.com
> > Location: *San Francisco, CA*
> > blog: http://burtonator.wordpress.com
> > … or check out my Google+ profile
> > <https://plus.google.com/102718274791889610666/posts>
> >
>



-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>

Re: Annotated MBean causing significant performance issue with JMX.

[Tim Bain <tb...@alumni.duke.edu>]

Guava's got some awesome support for caches; they may be the best feature
in the library (and Multisets/Multimaps are pretty awesome).  If you
implement this (and it sounds like a good enhancement), you should use them
for it.

One thing to consider: this cache will hold references to classes, and
under certain circumstances (especially OSGI-style plugins with child
classloaders or webapps in an app server) that can result in classes not
being unloaded when they're no longer used.  I don't think any of those
situations applies here, but someone who knows the architecture of ActiveMQ
plugins should probably confirm that.

Tim
On May 2, 2015 3:05 PM, "Kevin Burton" <bu...@spinn3r.com> wrote:

> OK. I’ve fixed 2-3 significant bugs in ActiveMQ with large numbers of
> queues and degraded performance. Most of theses are O(N^2) bugs so the more
> queues you have the more this becomes VERY painful.
>
> I don’t have an easy fix of this one though.
>
> Queue creation right now is about 3x slower than it could be…. Mostly
> because of AnnotatedMBean.
>
> http://imgur.com/a/iS1Xb
>
> This is a constant time issue though.  Not as urgent of a fix as the other
> bugs.
>
> This is ALL due to reflection.
>
> What I was thinking of doing was building a cache so that we could lookup
> the annotation info from the Class in the cache first, and if it’s absent,
> read it from reflection.
>
> Thoughts?
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
>