You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@servicemix.apache.org by boday <bo...@vektrel.com> on 2009/07/16 03:06:08 UTC
single threaded instead of multi-threaded execution
ActiveMQ is designed to work with multi-CPUs and operating system threads
correct? I'd have to assume so, but we are seeing some strange behavior...
We have our application deployed on a Solaris box. I'm working with an
administrator who knows Solaris very well. He is monitoring our application
and says its only using a single CPU and thread while others remain idle.
We are able to achieve only 25 msg/second on the box. I've tried different
combinations of just about everything I've seen suggested at this point.
Also worth noting that I'm getting 80 msg/second with the identical
build/configuration on my laptop...
Our message processing is as follows...
from("jetty:" + inboundURL)
.process(new HTTPInboundService())
.to("activemq:GatewayMsgQueue");
from("activemq:GatewayMsgQueue?maxConcurrentConsumers=20")
.convertBodyTo(DOMSource.class)
.to("jbi:service:http://www.test.com/eda/messageProcessor?mep=in-out")
.convertBodyTo(String.class)
.to("AMQPOOL:topic:PortalTopic");
If I simplify the route to just log all the HTTP requests (as follows)...I
get much better thread/CPU utilization and can process 800+/second
from("jetty:" + inboundURL)
.to("log:test?level=DEBUG");
The only other noteworthy code is the connection pool setup in the
MyRouteBuilder.java file.
String brokerURL = "vm://localhost:61616?marshal=false";
//String brokerURL = "tcp://localhost:61616";
PooledConnectionFactory pcf = new PooledConnectionFactory(brokerURL);
pcf.setMaxConnections(8);
JmsTransactionManager tm = new JmsTransactionManager();
tm.setConnectionFactory(pcf);
ActiveMQComponent amq = ActiveMQComponent.activeMQComponent(brokerURL);
amq.setConnectionFactory(pcf);
amq.setTransacted(true);
amq.setTransactionManager(tm);
amq.setMaxConcurrentConsumers(20);
//amq.setCacheLevel(3); //no difference
//amq.setAcknowledgementModeName("DUPS_OK_ACKNOWLEDGE"); //actually slower
this.getContext().addComponent("activemq",amq);
here is our environment...
Solaris 10 T2000, Niagara Processor, 8 core CPU, 4 execution threads/core,
8GB RAM
ServiceMix 3.3.1 (Camel 1.6.1)
-servicemix.sh changes...
JAVA_HOME="/usr/jdk/jdk1.6.0_11"
JAVA_OPTS="-server -Xmx1024M -Xms512M -XX:MaxPermSize=256m"
-servicemix.properties changes...
servicemix.corePoolSize = 100
servicemix.maximumPoolSize = -1
servicemix.queueSize = 1024
Is there anything else that needs to be done to achieve better
multi-threaded execution of this message processing application? We will
need to support 500+ msg/second at some point...
thanks in advance...
-----
Ben O'Day
Vektrel - Senior Consultant
--
View this message in context: http://www.nabble.com/single-threaded-instead-of-multi-threaded-execution-tp24508532p24508532.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.
Re: ServiceMix multi-threaded execution
Posted by rotgier <ro...@gmail.com>.
Hello,
You are saying that the HTTP traffic is consumed quickly. Maybe the
bottleneck is the ActiveMQ broker itself. Try to disable broker's
persistence as described in: http://activemq.apache.org/persistence.html .
Kind Regards,
Marek Psiuk
Gert Vanthienen wrote:
>
> Ben,
>
> Could you check the thread pool sizes with jconsole? Perhaps one of
> the pools just isn't large enough and is slowing everything down.
> Perhaps it's easier to start with one defaultConfig with some very
> high defaults (like core=16, queue=0, max=128) and then start
> fine-tuning things by adding more specific and strict configurations
> for some components. Also, if you're not using the JMS/JCA flow,
> perhaps you could disable those to ease the configuration effort.
>
> Regards,
>
> Gert Vanthienen
> ------------------------
> Open Source SOA: http://fusesource.com
> Blog: http://gertvanthienen.blogspot.com/
>
>
>
> 2009/7/31 boday <bo...@vektrel.com>:
>>
>> here is an update on this...
>>
>> By using a different HTTP endpoint (servicemix-http instead of Camel's)
>> I'm
>> able to get better multi-threaded execution and slightly higher CPU
>> utilization on this Solaris box (~10-15%). The issue is that my overall
>> throughput is still extremely low. Any ideas about what could cause
>> this?
>> Are there any known issues with Solaris?
>>
>> I'm able to process ~150-200 msg/sec on my laptop (2.4Ghz, Core 2 Duo,
>> 4GB),
>> but am only seeing roughly ~60 msg/sec with the identical build on this
>> Solaris box. Supposedly this box should be able to handle a
>> substantially
>> higher level of traffic (8 CPU core's, 8GB RAM, etc.). Have tried both
>> Sun
>> and JRockit JVMs as well...no difference.
>>
>> I have been playing around with the threading configuration (increases
>> 3-4x
>> what I have setup on my machine). Still the CPU load stays low on this
>> machine (10% vs. 80% on my laptop) and throughput is terrible. It is
>> able
>> to consume the HTTP traffic quickly, but just queues up in front of my
>> JBI
>> service that processes the data.
>>
>> The HTTP component is the only one able to really get high throughput,
>> everything else just queues up downstream. I have increased my thread
>> configuration as follows...
>>
>> <property name="configs">
>> <map>
>> <entry key="flow.jms.servicemix-http">
>> <bean
>> class="org.apache.servicemix.executors.impl.ExecutorConfig">
>> <property name="corePoolSize" value="2"/>
>> <property name="maximumPoolSize" value="4"/>
>> <property name="queueSize" value="1024"/>
>> </bean>
>> </entry>
>> <entry key="flow.jms.servicemix-jms">
>> <bean
>> class="org.apache.servicemix.executors.impl.ExecutorConfig">
>> <property name="corePoolSize" value="32"/>
>> <property name="maximumPoolSize" value="64"/>
>> <property name="queueSize" value="1024"/>
>> </bean>
>> </entry>
>> <entry key="flow.seda.servicemix-bean">
>> <bean
>> class="org.apache.servicemix.executors.impl.ExecutorConfig">
>> <property name="corePoolSize" value="64"/>
>> <property name="maximumPoolSize" value="128"/>
>> <property name="queueSize" value="1024"/>
>> </bean>
>> </entry>
>> <entry key="flow.seda.servicemix-camel">
>> <bean
>> class="org.apache.servicemix.executors.impl.ExecutorConfig">
>> <property name="corePoolSize" value="32"/>
>> <property name="maximumPoolSize" value="64"/>
>> <property name="queueSize" value="1024"/>
>> </bean>
>> </entry>
>> </map>
>> </property>
>>
>> thanks in advance...
>>
>> -----
>> Ben - Senior Consultant
>>
>> --
>> View this message in context:
>> http://www.nabble.com/ServiceMix-multi-threaded-execution-tp24508532p24759936.html
>> Sent from the ServiceMix - User mailing list archive at Nabble.com.
>>
>>
>
>
> -----
> ---
> Gert Vanthienen
> http://gertvanthienen.blogspot.com
>
--
View this message in context: http://www.nabble.com/ServiceMix-multi-threaded-execution-tp24508532p25764189.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.
Re: ServiceMix multi-threaded execution
Posted by Gert Vanthienen <ge...@gmail.com>.
Ben,
Could you check the thread pool sizes with jconsole? Perhaps one of
the pools just isn't large enough and is slowing everything down.
Perhaps it's easier to start with one defaultConfig with some very
high defaults (like core=16, queue=0, max=128) and then start
fine-tuning things by adding more specific and strict configurations
for some components. Also, if you're not using the JMS/JCA flow,
perhaps you could disable those to ease the configuration effort.
Regards,
Gert Vanthienen
------------------------
Open Source SOA: http://fusesource.com
Blog: http://gertvanthienen.blogspot.com/
2009/7/31 boday <bo...@vektrel.com>:
>
> here is an update on this...
>
> By using a different HTTP endpoint (servicemix-http instead of Camel's) I'm
> able to get better multi-threaded execution and slightly higher CPU
> utilization on this Solaris box (~10-15%). The issue is that my overall
> throughput is still extremely low. Any ideas about what could cause this?
> Are there any known issues with Solaris?
>
> I'm able to process ~150-200 msg/sec on my laptop (2.4Ghz, Core 2 Duo, 4GB),
> but am only seeing roughly ~60 msg/sec with the identical build on this
> Solaris box. Supposedly this box should be able to handle a substantially
> higher level of traffic (8 CPU core's, 8GB RAM, etc.). Have tried both Sun
> and JRockit JVMs as well...no difference.
>
> I have been playing around with the threading configuration (increases 3-4x
> what I have setup on my machine). Still the CPU load stays low on this
> machine (10% vs. 80% on my laptop) and throughput is terrible. It is able
> to consume the HTTP traffic quickly, but just queues up in front of my JBI
> service that processes the data.
>
> The HTTP component is the only one able to really get high throughput,
> everything else just queues up downstream. I have increased my thread
> configuration as follows...
>
> <property name="configs">
> <map>
> <entry key="flow.jms.servicemix-http">
> <bean
> class="org.apache.servicemix.executors.impl.ExecutorConfig">
> <property name="corePoolSize" value="2"/>
> <property name="maximumPoolSize" value="4"/>
> <property name="queueSize" value="1024"/>
> </bean>
> </entry>
> <entry key="flow.jms.servicemix-jms">
> <bean
> class="org.apache.servicemix.executors.impl.ExecutorConfig">
> <property name="corePoolSize" value="32"/>
> <property name="maximumPoolSize" value="64"/>
> <property name="queueSize" value="1024"/>
> </bean>
> </entry>
> <entry key="flow.seda.servicemix-bean">
> <bean
> class="org.apache.servicemix.executors.impl.ExecutorConfig">
> <property name="corePoolSize" value="64"/>
> <property name="maximumPoolSize" value="128"/>
> <property name="queueSize" value="1024"/>
> </bean>
> </entry>
> <entry key="flow.seda.servicemix-camel">
> <bean
> class="org.apache.servicemix.executors.impl.ExecutorConfig">
> <property name="corePoolSize" value="32"/>
> <property name="maximumPoolSize" value="64"/>
> <property name="queueSize" value="1024"/>
> </bean>
> </entry>
> </map>
> </property>
>
> thanks in advance...
>
> -----
> Ben - Senior Consultant
>
> --
> View this message in context: http://www.nabble.com/ServiceMix-multi-threaded-execution-tp24508532p24759936.html
> Sent from the ServiceMix - User mailing list archive at Nabble.com.
>
>
Re: ServiceMix multi-threaded execution
Posted by boday <bo...@vektrel.com>.
here is an update on this...
By using a different HTTP endpoint (servicemix-http instead of Camel's) I'm
able to get better multi-threaded execution and slightly higher CPU
utilization on this Solaris box (~10-15%). The issue is that my overall
throughput is still extremely low. Any ideas about what could cause this?
Are there any known issues with Solaris?
I'm able to process ~150-200 msg/sec on my laptop (2.4Ghz, Core 2 Duo, 4GB),
but am only seeing roughly ~60 msg/sec with the identical build on this
Solaris box. Supposedly this box should be able to handle a substantially
higher level of traffic (8 CPU core's, 8GB RAM, etc.). Have tried both Sun
and JRockit JVMs as well...no difference.
I have been playing around with the threading configuration (increases 3-4x
what I have setup on my machine). Still the CPU load stays low on this
machine (10% vs. 80% on my laptop) and throughput is terrible. It is able
to consume the HTTP traffic quickly, but just queues up in front of my JBI
service that processes the data.
The HTTP component is the only one able to really get high throughput,
everything else just queues up downstream. I have increased my thread
configuration as follows...
<property name="configs">
<map>
<entry key="flow.jms.servicemix-http">
<bean
class="org.apache.servicemix.executors.impl.ExecutorConfig">
<property name="corePoolSize" value="2"/>
<property name="maximumPoolSize" value="4"/>
<property name="queueSize" value="1024"/>
</bean>
</entry>
<entry key="flow.jms.servicemix-jms">
<bean
class="org.apache.servicemix.executors.impl.ExecutorConfig">
<property name="corePoolSize" value="32"/>
<property name="maximumPoolSize" value="64"/>
<property name="queueSize" value="1024"/>
</bean>
</entry>
<entry key="flow.seda.servicemix-bean">
<bean
class="org.apache.servicemix.executors.impl.ExecutorConfig">
<property name="corePoolSize" value="64"/>
<property name="maximumPoolSize" value="128"/>
<property name="queueSize" value="1024"/>
</bean>
</entry>
<entry key="flow.seda.servicemix-camel">
<bean
class="org.apache.servicemix.executors.impl.ExecutorConfig">
<property name="corePoolSize" value="32"/>
<property name="maximumPoolSize" value="64"/>
<property name="queueSize" value="1024"/>
</bean>
</entry>
</map>
</property>
thanks in advance...
-----
Ben - Senior Consultant
--
View this message in context: http://www.nabble.com/ServiceMix-multi-threaded-execution-tp24508532p24759936.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.