You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Jamie McCrindle <ja...@gmail.com> on 2006/08/02 19:04:08 UTC
performance of message selectors
hi all,
first off, i realise that my usage pattern cuts against the grain but
here's the scenario nonetheless:
i have a queue of messages with an "id" message property (not to
confused with JMSMessageID) and i regularly need to pull a particular
message (with a particular "id") off of that queue. i do that by
creating a message consumer that has a selector something like "id =
'" + someid + "'".
performance is quite poor as the queue gets larger.
in the future, i'll put those messages into a table with an index
against id but for now i was wondering if there is any tuning i can
apply to get it to go faster. i've read
http://devzone.logicblaze.com/site/apache-activemq-performance-tuning-guide.html
and only setting the prefetch size to 1 seems applicable.
cheers,
j.
ps. anyone else using message selectors like this?
Re: performance of message selectors
Posted by Jamie McCrindle <ja...@gmail.com>.
> I guess the part of the broker which replays all pending messages to a
> new consumer on a queue needs to be changed to say - 'ah if the
> consumer is only for a specific mesage ID then just replay that one
> message'.
>
> Maybe we just need a new operation on a broker to allow a specific
> message ID to be consumed?
Either would be awesome. I'm happy to try and figure it out and
contribute a patch if you'd like. If so, any hints would be great :)
cheers,
j.
On 8/4/06, James Strachan <ja...@gmail.com> wrote:
> On 8/4/06, Hiram Chirino <hi...@hiramchirino.com> wrote:
> > On 8/4/06, Jamie McCrindle <ja...@gmail.com> wrote:
> > > Thanks Hiram,
> > >
> > > I'll definitely look into that. How difficult do you think it would be
> > > just to optimise the selector "JMSMessageID = '" + id + "'"?
> > > presumably there are some hooks for that already (e.g. indexes in
> > > db's)?
> > >
> >
> > Yep.. you just need to add something the analyzes the selector and
> > avoids the queue scan. Geez this starting to should like DB.
>
> I guess the part of the broker which replays all pending messages to a
> new consumer on a queue needs to be changed to say - 'ah if the
> consumer is only for a specific mesage ID then just replay that one
> message'.
>
> Maybe we just need a new operation on a broker to allow a specific
> message ID to be consumed?
> --
>
> James
> -------
> http://radio.weblogs.com/0112098/
>
Re: performance of message selectors
Posted by James Strachan <ja...@gmail.com>.
On 8/4/06, Hiram Chirino <hi...@hiramchirino.com> wrote:
> On 8/4/06, Jamie McCrindle <ja...@gmail.com> wrote:
> > Thanks Hiram,
> >
> > I'll definitely look into that. How difficult do you think it would be
> > just to optimise the selector "JMSMessageID = '" + id + "'"?
> > presumably there are some hooks for that already (e.g. indexes in
> > db's)?
> >
>
> Yep.. you just need to add something the analyzes the selector and
> avoids the queue scan. Geez this starting to should like DB.
I guess the part of the broker which replays all pending messages to a
new consumer on a queue needs to be changed to say - 'ah if the
consumer is only for a specific mesage ID then just replay that one
message'.
Maybe we just need a new operation on a broker to allow a specific
message ID to be consumed?
--
James
-------
http://radio.weblogs.com/0112098/
Re: performance of message selectors
Posted by Hiram Chirino <hi...@hiramchirino.com>.
On 8/4/06, Jamie McCrindle <ja...@gmail.com> wrote:
> Thanks Hiram,
>
> I'll definitely look into that. How difficult do you think it would be
> just to optimise the selector "JMSMessageID = '" + id + "'"?
> presumably there are some hooks for that already (e.g. indexes in
> db's)?
>
Yep.. you just need to add something the analyzes the selector and
avoids the queue scan. Geez this starting to should like DB.
> cheers,
> j.
>
> On 8/2/06, Hiram Chirino <hi...@hiramchirino.com> wrote:
> > Unfortunately.. right now, I don't think we can do anything to improve
> > your use case. If you feel like doing some hacking on ActiveMQ, you
> > could potentially implement a solution where some selector expressions
> > are optimized by maintaining indexes.
> >
> > Regards,
> > Hiram
> >
> > On 8/2/06, Jamie McCrindle <ja...@gmail.com> wrote:
> > > hi all,
> > >
> > > first off, i realise that my usage pattern cuts against the grain but
> > > here's the scenario nonetheless:
> > >
> > > i have a queue of messages with an "id" message property (not to
> > > confused with JMSMessageID) and i regularly need to pull a particular
> > > message (with a particular "id") off of that queue. i do that by
> > > creating a message consumer that has a selector something like "id =
> > > '" + someid + "'".
> > >
> > > performance is quite poor as the queue gets larger.
> > >
> > > in the future, i'll put those messages into a table with an index
> > > against id but for now i was wondering if there is any tuning i can
> > > apply to get it to go faster. i've read
> > > http://devzone.logicblaze.com/site/apache-activemq-performance-tuning-guide.html
> > > and only setting the prefetch size to 1 seems applicable.
> > >
> > > cheers,
> > > j.
> > >
> > > ps. anyone else using message selectors like this?
> > >
> >
> >
> > --
> > Regards,
> > Hiram
> >
> > Blog: http://hiramchirino.com
> >
>
--
Regards,
Hiram
Blog: http://hiramchirino.com
Re: performance of message selectors
Posted by Jamie McCrindle <ja...@gmail.com>.
Thanks Hiram,
I'll definitely look into that. How difficult do you think it would be
just to optimise the selector "JMSMessageID = '" + id + "'"?
presumably there are some hooks for that already (e.g. indexes in
db's)?
cheers,
j.
On 8/2/06, Hiram Chirino <hi...@hiramchirino.com> wrote:
> Unfortunately.. right now, I don't think we can do anything to improve
> your use case. If you feel like doing some hacking on ActiveMQ, you
> could potentially implement a solution where some selector expressions
> are optimized by maintaining indexes.
>
> Regards,
> Hiram
>
> On 8/2/06, Jamie McCrindle <ja...@gmail.com> wrote:
> > hi all,
> >
> > first off, i realise that my usage pattern cuts against the grain but
> > here's the scenario nonetheless:
> >
> > i have a queue of messages with an "id" message property (not to
> > confused with JMSMessageID) and i regularly need to pull a particular
> > message (with a particular "id") off of that queue. i do that by
> > creating a message consumer that has a selector something like "id =
> > '" + someid + "'".
> >
> > performance is quite poor as the queue gets larger.
> >
> > in the future, i'll put those messages into a table with an index
> > against id but for now i was wondering if there is any tuning i can
> > apply to get it to go faster. i've read
> > http://devzone.logicblaze.com/site/apache-activemq-performance-tuning-guide.html
> > and only setting the prefetch size to 1 seems applicable.
> >
> > cheers,
> > j.
> >
> > ps. anyone else using message selectors like this?
> >
>
>
> --
> Regards,
> Hiram
>
> Blog: http://hiramchirino.com
>
Re: performance of message selectors
Posted by Hiram Chirino <hi...@hiramchirino.com>.
Unfortunately.. right now, I don't think we can do anything to improve
your use case. If you feel like doing some hacking on ActiveMQ, you
could potentially implement a solution where some selector expressions
are optimized by maintaining indexes.
Regards,
Hiram
On 8/2/06, Jamie McCrindle <ja...@gmail.com> wrote:
> hi all,
>
> first off, i realise that my usage pattern cuts against the grain but
> here's the scenario nonetheless:
>
> i have a queue of messages with an "id" message property (not to
> confused with JMSMessageID) and i regularly need to pull a particular
> message (with a particular "id") off of that queue. i do that by
> creating a message consumer that has a selector something like "id =
> '" + someid + "'".
>
> performance is quite poor as the queue gets larger.
>
> in the future, i'll put those messages into a table with an index
> against id but for now i was wondering if there is any tuning i can
> apply to get it to go faster. i've read
> http://devzone.logicblaze.com/site/apache-activemq-performance-tuning-guide.html
> and only setting the prefetch size to 1 seems applicable.
>
> cheers,
> j.
>
> ps. anyone else using message selectors like this?
>
--
Regards,
Hiram
Blog: http://hiramchirino.com