You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@servicemix.apache.org by Felipe Gaucho <fg...@gmail.com> on 2007/02/14 10:29:09 UTC

performance question - 72’000 messages per hour

my system is being designed to support 72’000 messages per hour

it means: 20 messages per second, availability 24x7 (99%)

no problem for ServiceMix ? any experience >= that values ?

other question: 

the above number refer to 12 096 000 (12 million messages per week)

How to you archive such messages ? That means, I will keep the message data
persisted in a database for other processes (audit processes and history
log). From time to time, I supose I must dump the database or clear my log
directory.. do you do that through ServieMix or delegate for thid-party
processes ?? 
-- 
View this message in context: http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a8961467
Sent from the ServiceMix - User mailing list archive at Nabble.com.


Re: performance question - 72’000 messages per hour

Posted by Moksha <mo...@gmail.com>.
Sorry, forgot to add that currently i am using jms with jca. And whether i
set the BootstrapContextFactoryBean's  threadPoolSize to 25 or 200, there is
no effect on the performance.


Moksha wrote:
> 
> 
> Thanks for the reply. My requirement is such that i have to make the
> message go through XSL transformations as well and then put them on
> another topic, so i will have to put them on the bus.
> 
> And i tried moving to version 3.1 , but on running the build , i kept
> getting the error "work manager is not set". Dnt know whats going wrong
> there.
> 
> gnodet wrote:
>> 
>> The best way is to use a JCA endpoint with a fixed number of synchronous
>> receiver threads, so that all the jms messages are not consumed and put
>> in
>> the jbi bus, which would obviously be overloaded.  This will improve in
>> future
>> versions (currently, you have to use JCA to completely control that).
>> 
>> Btw, you should upgrade to ServiceMix 3.1 and configure the thread pools
>> to suite your needs
>> (http://incubator.apache.org/servicemix/thread-pools.html).
>> 
>> On 2/16/07, Moksha <mo...@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> I am trying to receive 50,000 messages per minute from an activemq topic
>>> using the jms receiver component . But the container hangs.I have
>>> configured
>>> a work manager as well but i am not quite sure how to make this receiver
>>> component aware of this work manager so that it can spawn more threads.
>>> Could you please tell me how do i do this?
>>>
>>> Cheers,
>>> Moksha
>>>
>>> gnodet wrote:
>>> >
>>> > I'm wondering how this can work with sendSync.
>>> > Well, the only option is to just not wait for it of course ...
>>> > And all errors would be lost, but yeah, this is a possibility,
>>> > but it completely breaks the JBI compliance ...
>>> >
>>> > On 2/15/07, jlo_gestalt <jl...@gestalt-llc.com> wrote:
>>> >>
>>> >> Guillaume,
>>> >> Here are some ideas from the Petals JBI container:
>>> >>
>>> http://wiki.petals.objectweb.org/xwiki/bin/view/Documentation.UserGuide/Optimisation.
>>> >>
>>> >> Might be some thoughts to incorporate into SM. I especially like the
>>> >> noack
>>> >> option. In fact I plan on submitting it to the jbi 2.0 spec group.
>>> >>
>>> >>
>>> >>
>>> >> gnodet wrote:
>>> >> >
>>> >> > No :-(
>>> >> > To enable it, you just have to change the autoStart attribute to
>>> true
>>> >> ...
>>> >> >
>>> >> > On 2/14/07, Felipe Gaucho <fg...@gmail.com> wrote:
>>> >> >>
>>> >> >> great.. Any documentation about how to use (enable)  this
>>> JdbcAudit
>>> >> >> component
>>> >> >> ?
>>> >> >>
>>> >> >> --
>>> >> >> View this message in context:
>>> >> >>
>>> >>
>>> http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a8962617
>>> >> >> Sent from the ServiceMix - User mailing list archive at
>>> Nabble.com.
>>> >> >>
>>> >> >>
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Cheers,
>>> >> > Guillaume Nodet
>>> >> > ------------------------
>>> >> > Architect, LogicBlaze (http://www.logicblaze.com/)
>>> >> > Blog: http://gnodet.blogspot.com/
>>> >> >
>>> >> >
>>> >>
>>> >> --
>>> >> View this message in context:
>>> >>
>>> http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a8987699
>>> >> Sent from the ServiceMix - User mailing list archive at Nabble.com.
>>> >>
>>> >>
>>> >
>>> >
>>> > --
>>> > Cheers,
>>> > Guillaume Nodet
>>> > ------------------------
>>> > Architect, LogicBlaze (http://www.logicblaze.com/)
>>> > Blog: http://gnodet.blogspot.com/
>>> >
>>> >
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a9013472
>>> Sent from the ServiceMix - User mailing list archive at Nabble.com.
>>>
>>>
>> 
>> 
>> -- 
>> Cheers,
>> Guillaume Nodet
>> ------------------------
>> Architect, LogicBlaze (http://www.logicblaze.com/)
>> Blog: http://gnodet.blogspot.com/
>> 
>> 
> 
> 

-- 
View this message in context: http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a9014694
Sent from the ServiceMix - User mailing list archive at Nabble.com.


Re: performance question - 72’000 messages per hour

Posted by Moksha <mo...@gmail.com>.

Thanks for the reply. My requirement is such that i have to make the message
go through XSL transformations as well and then put them on another topic,
so i will have to put them on the bus.

And i tried moving to version 3.1 , but on running the build , i kept
getting the error "work manager is not set". Dnt know whats going wrong
there.

gnodet wrote:
> 
> The best way is to use a JCA endpoint with a fixed number of synchronous
> receiver threads, so that all the jms messages are not consumed and put in
> the jbi bus, which would obviously be overloaded.  This will improve in
> future
> versions (currently, you have to use JCA to completely control that).
> 
> Btw, you should upgrade to ServiceMix 3.1 and configure the thread pools
> to suite your needs
> (http://incubator.apache.org/servicemix/thread-pools.html).
> 
> On 2/16/07, Moksha <mo...@gmail.com> wrote:
>>
>> Hi,
>>
>> I am trying to receive 50,000 messages per minute from an activemq topic
>> using the jms receiver component . But the container hangs.I have
>> configured
>> a work manager as well but i am not quite sure how to make this receiver
>> component aware of this work manager so that it can spawn more threads.
>> Could you please tell me how do i do this?
>>
>> Cheers,
>> Moksha
>>
>> gnodet wrote:
>> >
>> > I'm wondering how this can work with sendSync.
>> > Well, the only option is to just not wait for it of course ...
>> > And all errors would be lost, but yeah, this is a possibility,
>> > but it completely breaks the JBI compliance ...
>> >
>> > On 2/15/07, jlo_gestalt <jl...@gestalt-llc.com> wrote:
>> >>
>> >> Guillaume,
>> >> Here are some ideas from the Petals JBI container:
>> >>
>> http://wiki.petals.objectweb.org/xwiki/bin/view/Documentation.UserGuide/Optimisation.
>> >>
>> >> Might be some thoughts to incorporate into SM. I especially like the
>> >> noack
>> >> option. In fact I plan on submitting it to the jbi 2.0 spec group.
>> >>
>> >>
>> >>
>> >> gnodet wrote:
>> >> >
>> >> > No :-(
>> >> > To enable it, you just have to change the autoStart attribute to
>> true
>> >> ...
>> >> >
>> >> > On 2/14/07, Felipe Gaucho <fg...@gmail.com> wrote:
>> >> >>
>> >> >> great.. Any documentation about how to use (enable)  this JdbcAudit
>> >> >> component
>> >> >> ?
>> >> >>
>> >> >> --
>> >> >> View this message in context:
>> >> >>
>> >>
>> http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a8962617
>> >> >> Sent from the ServiceMix - User mailing list archive at Nabble.com.
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >> > --
>> >> > Cheers,
>> >> > Guillaume Nodet
>> >> > ------------------------
>> >> > Architect, LogicBlaze (http://www.logicblaze.com/)
>> >> > Blog: http://gnodet.blogspot.com/
>> >> >
>> >> >
>> >>
>> >> --
>> >> View this message in context:
>> >>
>> http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a8987699
>> >> Sent from the ServiceMix - User mailing list archive at Nabble.com.
>> >>
>> >>
>> >
>> >
>> > --
>> > Cheers,
>> > Guillaume Nodet
>> > ------------------------
>> > Architect, LogicBlaze (http://www.logicblaze.com/)
>> > Blog: http://gnodet.blogspot.com/
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a9013472
>> Sent from the ServiceMix - User mailing list archive at Nabble.com.
>>
>>
> 
> 
> -- 
> Cheers,
> Guillaume Nodet
> ------------------------
> Architect, LogicBlaze (http://www.logicblaze.com/)
> Blog: http://gnodet.blogspot.com/
> 
> 

-- 
View this message in context: http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a9014618
Sent from the ServiceMix - User mailing list archive at Nabble.com.


Re: performance question - 72’000 messages per hour

Posted by Guillaume Nodet <gn...@gmail.com>.
The best way is to use a JCA endpoint with a fixed number of synchronous
receiver threads, so that all the jms messages are not consumed and put in
the jbi bus, which would obviously be overloaded.  This will improve in future
versions (currently, you have to use JCA to completely control that).

Btw, you should upgrade to ServiceMix 3.1 and configure the thread pools
to suite your needs (http://incubator.apache.org/servicemix/thread-pools.html).

On 2/16/07, Moksha <mo...@gmail.com> wrote:
>
> Hi,
>
> I am trying to receive 50,000 messages per minute from an activemq topic
> using the jms receiver component . But the container hangs.I have configured
> a work manager as well but i am not quite sure how to make this receiver
> component aware of this work manager so that it can spawn more threads.
> Could you please tell me how do i do this?
>
> Cheers,
> Moksha
>
> gnodet wrote:
> >
> > I'm wondering how this can work with sendSync.
> > Well, the only option is to just not wait for it of course ...
> > And all errors would be lost, but yeah, this is a possibility,
> > but it completely breaks the JBI compliance ...
> >
> > On 2/15/07, jlo_gestalt <jl...@gestalt-llc.com> wrote:
> >>
> >> Guillaume,
> >> Here are some ideas from the Petals JBI container:
> >> http://wiki.petals.objectweb.org/xwiki/bin/view/Documentation.UserGuide/Optimisation.
> >>
> >> Might be some thoughts to incorporate into SM. I especially like the
> >> noack
> >> option. In fact I plan on submitting it to the jbi 2.0 spec group.
> >>
> >>
> >>
> >> gnodet wrote:
> >> >
> >> > No :-(
> >> > To enable it, you just have to change the autoStart attribute to true
> >> ...
> >> >
> >> > On 2/14/07, Felipe Gaucho <fg...@gmail.com> wrote:
> >> >>
> >> >> great.. Any documentation about how to use (enable)  this JdbcAudit
> >> >> component
> >> >> ?
> >> >>
> >> >> --
> >> >> View this message in context:
> >> >>
> >> http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a8962617
> >> >> Sent from the ServiceMix - User mailing list archive at Nabble.com.
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > Cheers,
> >> > Guillaume Nodet
> >> > ------------------------
> >> > Architect, LogicBlaze (http://www.logicblaze.com/)
> >> > Blog: http://gnodet.blogspot.com/
> >> >
> >> >
> >>
> >> --
> >> View this message in context:
> >> http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a8987699
> >> Sent from the ServiceMix - User mailing list archive at Nabble.com.
> >>
> >>
> >
> >
> > --
> > Cheers,
> > Guillaume Nodet
> > ------------------------
> > Architect, LogicBlaze (http://www.logicblaze.com/)
> > Blog: http://gnodet.blogspot.com/
> >
> >
>
> --
> View this message in context: http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a9013472
> Sent from the ServiceMix - User mailing list archive at Nabble.com.
>
>


-- 
Cheers,
Guillaume Nodet
------------------------
Architect, LogicBlaze (http://www.logicblaze.com/)
Blog: http://gnodet.blogspot.com/

Re: performance question - 72’000 messages per hour

Posted by Moksha <mo...@gmail.com>.
Hi,

I am trying to receive 50,000 messages per minute from an activemq topic
using the jms receiver component . But the container hangs.I have configured
a work manager as well but i am not quite sure how to make this receiver
component aware of this work manager so that it can spawn more threads. 
Could you please tell me how do i do this?

Cheers,
Moksha

gnodet wrote:
> 
> I'm wondering how this can work with sendSync.
> Well, the only option is to just not wait for it of course ...
> And all errors would be lost, but yeah, this is a possibility,
> but it completely breaks the JBI compliance ...
> 
> On 2/15/07, jlo_gestalt <jl...@gestalt-llc.com> wrote:
>>
>> Guillaume,
>> Here are some ideas from the Petals JBI container:
>> http://wiki.petals.objectweb.org/xwiki/bin/view/Documentation.UserGuide/Optimisation.
>>
>> Might be some thoughts to incorporate into SM. I especially like the
>> noack
>> option. In fact I plan on submitting it to the jbi 2.0 spec group.
>>
>>
>>
>> gnodet wrote:
>> >
>> > No :-(
>> > To enable it, you just have to change the autoStart attribute to true
>> ...
>> >
>> > On 2/14/07, Felipe Gaucho <fg...@gmail.com> wrote:
>> >>
>> >> great.. Any documentation about how to use (enable)  this JdbcAudit
>> >> component
>> >> ?
>> >>
>> >> --
>> >> View this message in context:
>> >>
>> http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a8962617
>> >> Sent from the ServiceMix - User mailing list archive at Nabble.com.
>> >>
>> >>
>> >
>> >
>> > --
>> > Cheers,
>> > Guillaume Nodet
>> > ------------------------
>> > Architect, LogicBlaze (http://www.logicblaze.com/)
>> > Blog: http://gnodet.blogspot.com/
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a8987699
>> Sent from the ServiceMix - User mailing list archive at Nabble.com.
>>
>>
> 
> 
> -- 
> Cheers,
> Guillaume Nodet
> ------------------------
> Architect, LogicBlaze (http://www.logicblaze.com/)
> Blog: http://gnodet.blogspot.com/
> 
> 

-- 
View this message in context: http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a9013472
Sent from the ServiceMix - User mailing list archive at Nabble.com.


Re: performance question - 72’000 messages per hour

Posted by Guillaume Nodet <gn...@gmail.com>.
I'm wondering how this can work with sendSync.
Well, the only option is to just not wait for it of course ...
And all errors would be lost, but yeah, this is a possibility,
but it completely breaks the JBI compliance ...

On 2/15/07, jlo_gestalt <jl...@gestalt-llc.com> wrote:
>
> Guillaume,
> Here are some ideas from the Petals JBI container:
> http://wiki.petals.objectweb.org/xwiki/bin/view/Documentation.UserGuide/Optimisation.
>
> Might be some thoughts to incorporate into SM. I especially like the noack
> option. In fact I plan on submitting it to the jbi 2.0 spec group.
>
>
>
> gnodet wrote:
> >
> > No :-(
> > To enable it, you just have to change the autoStart attribute to true ...
> >
> > On 2/14/07, Felipe Gaucho <fg...@gmail.com> wrote:
> >>
> >> great.. Any documentation about how to use (enable)  this JdbcAudit
> >> component
> >> ?
> >>
> >> --
> >> View this message in context:
> >> http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a8962617
> >> Sent from the ServiceMix - User mailing list archive at Nabble.com.
> >>
> >>
> >
> >
> > --
> > Cheers,
> > Guillaume Nodet
> > ------------------------
> > Architect, LogicBlaze (http://www.logicblaze.com/)
> > Blog: http://gnodet.blogspot.com/
> >
> >
>
> --
> View this message in context: http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a8987699
> Sent from the ServiceMix - User mailing list archive at Nabble.com.
>
>


-- 
Cheers,
Guillaume Nodet
------------------------
Architect, LogicBlaze (http://www.logicblaze.com/)
Blog: http://gnodet.blogspot.com/

Re: performance question - 72’000 messages per hour

Posted by jlo_gestalt <jl...@gestalt-llc.com>.
Guillaume,
Here are some ideas from the Petals JBI container:
http://wiki.petals.objectweb.org/xwiki/bin/view/Documentation.UserGuide/Optimisation.

Might be some thoughts to incorporate into SM. I especially like the noack
option. In fact I plan on submitting it to the jbi 2.0 spec group.



gnodet wrote:
> 
> No :-(
> To enable it, you just have to change the autoStart attribute to true ...
> 
> On 2/14/07, Felipe Gaucho <fg...@gmail.com> wrote:
>>
>> great.. Any documentation about how to use (enable)  this JdbcAudit
>> component
>> ?
>>
>> --
>> View this message in context:
>> http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a8962617
>> Sent from the ServiceMix - User mailing list archive at Nabble.com.
>>
>>
> 
> 
> -- 
> Cheers,
> Guillaume Nodet
> ------------------------
> Architect, LogicBlaze (http://www.logicblaze.com/)
> Blog: http://gnodet.blogspot.com/
> 
> 

-- 
View this message in context: http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a8987699
Sent from the ServiceMix - User mailing list archive at Nabble.com.


Re: performance question - 72’000 messages per hour

Posted by Guillaume Nodet <gn...@gmail.com>.
No :-(
To enable it, you just have to change the autoStart attribute to true ...

On 2/14/07, Felipe Gaucho <fg...@gmail.com> wrote:
>
> great.. Any documentation about how to use (enable)  this JdbcAudit component
> ?
>
> --
> View this message in context: http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a8962617
> Sent from the ServiceMix - User mailing list archive at Nabble.com.
>
>


-- 
Cheers,
Guillaume Nodet
------------------------
Architect, LogicBlaze (http://www.logicblaze.com/)
Blog: http://gnodet.blogspot.com/

Re: performance question - 72’000 messages per hour

Posted by Felipe Gaucho <fg...@gmail.com>.
great.. Any documentation about how to use (enable)  this JdbcAudit component
?

-- 
View this message in context: http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a8962617
Sent from the ServiceMix - User mailing list archive at Nabble.com.


Re: performance question - 72’000 messages per hour

Posted by Guillaume Nodet <gn...@gmail.com>.
The JBI container itself should be able to handle the load.
I think performances mainly depend on what kind of processing
the system will perform.  For example, if you need to keep
track of all messages, it will take some processing time ...

On this point, you can use the JdbcAuditor which is already
configured in the default configuration (but disabled).  You can
also modify it to suit your needs or implement another one
using the org.apache.servicemix.jbi.events package.

On 2/14/07, Felipe Gaucho <fg...@gmail.com> wrote:
>
> my system is being designed to support 72'000 messages per hour
>
> it means: 20 messages per second, availability 24x7 (99%)
>
> no problem for ServiceMix ? any experience >= that values ?
>
> other question:
>
> the above number refer to 12 096 000 (12 million messages per week)
>
> How to you archive such messages ? That means, I will keep the message data
> persisted in a database for other processes (audit processes and history
> log). From time to time, I supose I must dump the database or clear my log
> directory.. do you do that through ServieMix or delegate for thid-party
> processes ??
> --
> View this message in context: http://www.nabble.com/performance-question---72%E2%80%99000-messages-per-hour-tf3226134s12049.html#a8961467
> Sent from the ServiceMix - User mailing list archive at Nabble.com.
>
>


-- 
Cheers,
Guillaume Nodet
------------------------
Architect, LogicBlaze (http://www.logicblaze.com/)
Blog: http://gnodet.blogspot.com/