You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@servicemix.apache.org by KOS <ko...@student.ethz.ch> on 2009/02/18 14:25:42 UTC
Performance: low throughput, errors
Hi
(Please see my setting below)
I tried to find out how many messages ServiceMix can handle. After sending
more than about 1400 requests (in parallel with 100 connections) from an
external soap client to an external Web Service (through the ESB), SM raises
errors, for about every second request (see stacktrace below). If I send the
requests from only one client (one connection), it takes longer (about 2000
- 3000 requests) until SM raises errors.
The strange thing is: It makes no difference if I send 1000 requests, wait a
few seconds, and send another 1000, compared to sending 2000 without break.
It always fails after about 1400 requests. The only thing that helps is
restarting SM. Then, I can send another 1400 messages without fails. It
seems that SM does not "recover" from the high load.
My question:
1. Is ServiceMix (the cxf-bc) designed to raise errors when too many
requests are sent? Can't they be queued (before cxf-consumer, this one seems
to raise the error)?
2. How can I tune it (the cxf-bc, whatever would help)?
3. Raising Java VM memory settings helps, SM can then handle more requests.
But I have still the problem that SM raises a lot of errors when I once put
too much load. It seems "not to recover"... How can I solve this (without
restarting SM)?
4. Even worse, when I keep sending requests, SM throws this error (see
below: error-message and stacktrace for second error). Then, it closes the
ports and the service is not available anymore. I have to delete the data
folder and restart SM.
Thanks
My setting:
- ServiceMix 3.3.1 SNAPSHOT
- A WebService-Bridge (cxf-bc consumer and cxf-bc provider). I send
soap-requests from an external client to an external WebService through the
ESB.
- 2 Intel quad-core @ 2.33 GHz, >> 4 GB RAM
- Linux version 2.6.18
- OpenJDK Server VM build 1.6.0_0-b12
- java -server -Xms128M -Xmx512M (higher settings help, but the problem of
"not recovering" stays)
Stacktrace for exception:
org.apache.servicemix.cxfbc.interceptors.JbiFault: Fault occured
at
org.apache.servicemix.cxfbc.CxfBcConsumer$JbiPostInvokerInterceptor.handleMessage(CxfBcConsumer.java:653)
at
org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:220)
at
org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:78)
at
org.apache.cxf.transport.http_jetty.JettyHTTPDestination.serviceRequest(JettyHTTPDestination.java:278)
at
org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:252)
at
org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:70)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:722)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:206)
at org.mortbay.jetty.handler.HandlerList.handle(HandlerList.java:49)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:324)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:505)
at
org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:842)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:648)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:395)
at
org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:450)
error-message and stacktrace for second error:
<loc-message>Target component servicemix-cxf-bc for service unit
ontology-change-su is not installed</loc-message>
at
org.apache.servicemix.jbi.framework.ManagementSupport.failure(ManagementSupport.java:125)
at
org.apache.servicemix.jbi.framework.ManagementSupport.failure(ManagementSupport.java:111)
at
org.apache.servicemix.jbi.framework.ServiceAssemblyLifeCycle.start(ServiceAssemblyLifeCycle.java:146)
at
org.apache.servicemix.jbi.framework.ServiceAssemblyLifeCycle.restore(ServiceAssemblyLifeCycle.java:330)
at
org.apache.servicemix.jbi.framework.DeploymentService.start(DeploymentService.java:100)
at
org.apache.servicemix.jbi.container.JBIContainer.start(JBIContainer.java:670)
at
org.apache.servicemix.jbi.container.SpringJBIContainer.afterPropertiesSet(SpringJBIContainer.java:115)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1368)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1334)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:473)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
at java.security.AccessController.doPrivileged(Native Method)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:221)
at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:429)
at
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:729)
at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:381)
at
org.apache.xbean.spring.context.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:161)
at
org.apache.xbean.spring.context.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:51)
at org.apache.servicemix.Main.main(Main.java:53)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.codehaus.classworlds.Launcher.launchStandard(Launcher.java:353)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:264)
at
org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
--
View this message in context: http://www.nabble.com/Performance%3A-low-throughput%2C-errors-tp22078651p22078651.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.
Re: Performance: low throughput, errors
Posted by Freeman Fang <fr...@gmail.com>.
KOS wrote:
> Hi Freeman
>
> See my configuration below.
>
> The heap size is always about 100 - 150 MB, also when raising errors.
>
> I see, in addition to the stack trace I posted below, this error:
> WARNING: EXCEPTION
> java.io.IOException: Too many open files
> at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
> at
> sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
> at
> org.mortbay.jetty.nio.SelectChannelConnector$1.acceptChannel(SelectChannelConnector.java:75)
> at
> org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:475)
> at
> org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:166)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
> at
> org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:515)
> at
> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:450)
>
> cxf-bc configuration:
> <cxfbc:consumer targetService="ts" targetEndoint="tep" wsdl="localWsdlFile"
> />
> <cxfbc:provider service="serviceAsInWsdl" endoint="epAsInWsdl"
> wsdl="remoteWsdl" />
>
> I have two service units. So one for consumer, one for provider.
>
> Is there another way to tune the cxf-component? Or raising/lowering
> threadpool size to prevent the TOO MANY OPEN FILES error? Because sometimes,
> after having these errors, I even have to remove the data folder to be able
> to restart SM...
>
>
Hi,
You can configure a cxf bus with jetty thread pool using spring file to
control the threads, let's say the file name is jettyThreadPool.xml,
then set the busCfg="jettyThreadPool,xml" for your cxf bc consumer endpoint.
The jettyThreadPool.xml should be
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:sec="http://cxf.apache.org/configuration/security"
xmlns:http="http://cxf.apache.org/transports/http/configuration"
xmlns:httpj="http://cxf.apache.org/transports/http-jetty/configuration"
xmlns:jaxws="http://java.sun.com/xml/ns/jaxws"
xsi:schemaLocation="
http://cxf.apache.org/configuration/security
http://cxf.apache.org/schemas/configuration/security.xsd
http://cxf.apache.org/transports/http/configuration
http://cxf.apache.org/schemas/configuration/http-conf.xsd
http://cxf.apache.org/transports/http-jetty/configuration
http://cxf.apache.org/schemas/configuration/http-jetty.xsd
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">
<http:destination
name="{your_port_in_wsdl_namespace}port_name_in_wsdl.http-destination">
</http:destination>
<httpj:engine-factory bus="cxf">
<httpj:engine port="9000(the_port_your_cxf_bc_consumer_listening)">
<httpj:threadingParameters minThreads="1(or what you want)"
maxThreads="3(or what you want)" />
<httpj:connector>
<bean class="org.mortbay.jetty.nio.SelectChannelConnector">
<property name = "port" value="9000" />
<property name="threadPool">
<bean class="org.mortbay.thread.BoundedThreadPool"/>
</property>
</bean>
</httpj:connector>
</httpj:engine>
</httpj:engine-factory>
<!-- We need a bean named "cxf" -->
<bean id="cxf" class="org.apache.cxf.bus.CXFBusImpl"/>
</beans>
Freeman
> Thanks a lot
>
>
> Freeman Fang wrote:
>
>> Hi,
>> When you see the error throw, is the memory reach the heap max size?
>> Would you please append your configuration for cxf bc consumer/provider
>> also?
>> Freeman
>>
>> KOS wrote:
>>
>>> Hi
>>>
>>> (Please see my setting below)
>>>
>>> I tried to find out how many messages ServiceMix can handle. After
>>> sending
>>> more than about 1400 requests (in parallel with 100 connections) from an
>>> external soap client to an external Web Service (through the ESB), SM
>>> raises
>>> errors, for about every second request (see stacktrace below). If I send
>>> the
>>> requests from only one client (one connection), it takes longer (about
>>> 2000
>>> - 3000 requests) until SM raises errors.
>>> The strange thing is: It makes no difference if I send 1000 requests,
>>> wait a
>>> few seconds, and send another 1000, compared to sending 2000 without
>>> break.
>>> It always fails after about 1400 requests. The only thing that helps is
>>> restarting SM. Then, I can send another 1400 messages without fails. It
>>> seems that SM does not "recover" from the high load.
>>>
>>> My question:
>>> 1. Is ServiceMix (the cxf-bc) designed to raise errors when too many
>>> requests are sent? Can't they be queued (before cxf-consumer, this one
>>> seems
>>> to raise the error)?
>>> 2. How can I tune it (the cxf-bc, whatever would help)?
>>> 3. Raising Java VM memory settings helps, SM can then handle more
>>> requests.
>>> But I have still the problem that SM raises a lot of errors when I once
>>> put
>>> too much load. It seems "not to recover"... How can I solve this (without
>>> restarting SM)?
>>> 4. Even worse, when I keep sending requests, SM throws this error (see
>>> below: error-message and stacktrace for second error). Then, it closes
>>> the
>>> ports and the service is not available anymore. I have to delete the data
>>> folder and restart SM.
>>>
>>> Thanks
>>>
>>> My setting:
>>>
>>> - ServiceMix 3.3.1 SNAPSHOT
>>> - A WebService-Bridge (cxf-bc consumer and cxf-bc provider). I send
>>> soap-requests from an external client to an external WebService through
>>> the
>>> ESB.
>>> - 2 Intel quad-core @ 2.33 GHz, >> 4 GB RAM
>>> - Linux version 2.6.18
>>> - OpenJDK Server VM build 1.6.0_0-b12
>>> - java -server -Xms128M -Xmx512M (higher settings help, but the problem
>>> of
>>> "not recovering" stays)
>>>
>>>
>>> Stacktrace for exception:
>>> org.apache.servicemix.cxfbc.interceptors.JbiFault: Fault occured
>>> at
>>> org.apache.servicemix.cxfbc.CxfBcConsumer$JbiPostInvokerInterceptor.handleMessage(CxfBcConsumer.java:653)
>>> at
>>> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:220)
>>> at
>>> org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:78)
>>> at
>>> org.apache.cxf.transport.http_jetty.JettyHTTPDestination.serviceRequest(JettyHTTPDestination.java:278)
>>> at
>>> org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:252)
>>> at
>>> org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:70)
>>> at
>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:722)
>>> at
>>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:206)
>>> at
>>> org.mortbay.jetty.handler.HandlerList.handle(HandlerList.java:49)
>>> at
>>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
>>> at org.mortbay.jetty.Server.handle(Server.java:324)
>>> at
>>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:505)
>>> at
>>> org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:842)
>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:648)
>>> at
>>> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
>>> at
>>> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380)
>>> at
>>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:395)
>>> at
>>> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:450)
>>>
>>> error-message and stacktrace for second error:
>>> <loc-message>Target component servicemix-cxf-bc for service unit
>>> ontology-change-su is not installed</loc-message>
>>>
>>> at
>>> org.apache.servicemix.jbi.framework.ManagementSupport.failure(ManagementSupport.java:125)
>>> at
>>> org.apache.servicemix.jbi.framework.ManagementSupport.failure(ManagementSupport.java:111)
>>> at
>>> org.apache.servicemix.jbi.framework.ServiceAssemblyLifeCycle.start(ServiceAssemblyLifeCycle.java:146)
>>> at
>>> org.apache.servicemix.jbi.framework.ServiceAssemblyLifeCycle.restore(ServiceAssemblyLifeCycle.java:330)
>>> at
>>> org.apache.servicemix.jbi.framework.DeploymentService.start(DeploymentService.java:100)
>>> at
>>> org.apache.servicemix.jbi.container.JBIContainer.start(JBIContainer.java:670)
>>> at
>>> org.apache.servicemix.jbi.container.SpringJBIContainer.afterPropertiesSet(SpringJBIContainer.java:115)
>>> at
>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1368)
>>> at
>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1334)
>>> at
>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:473)
>>> at
>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at
>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
>>> at
>>> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
>>> at
>>> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:221)
>>> at
>>> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
>>> at
>>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
>>> at
>>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
>>> at
>>> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:429)
>>> at
>>> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:729)
>>> at
>>> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:381)
>>> at
>>> org.apache.xbean.spring.context.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:161)
>>> at
>>> org.apache.xbean.spring.context.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:51)
>>> at org.apache.servicemix.Main.main(Main.java:53)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:616)
>>> at
>>> org.codehaus.classworlds.Launcher.launchStandard(Launcher.java:353)
>>> at org.codehaus.classworlds.Launcher.launch(Launcher.java:264)
>>> at
>>> org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
>>> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
>>>
>>>
>>>
>>>
>>
>>
>
>
Re: Performance: low throughput, errors
Posted by KOS <ko...@student.ethz.ch>.
Hi Freeman
See my configuration below.
The heap size is always about 100 - 150 MB, also when raising errors.
I see, in addition to the stack trace I posted below, this error:
WARNING: EXCEPTION
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
at
org.mortbay.jetty.nio.SelectChannelConnector$1.acceptChannel(SelectChannelConnector.java:75)
at
org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:475)
at
org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:166)
at
org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
at
org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:515)
at
org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:450)
cxf-bc configuration:
<cxfbc:consumer targetService="ts" targetEndoint="tep" wsdl="localWsdlFile"
/>
<cxfbc:provider service="serviceAsInWsdl" endoint="epAsInWsdl"
wsdl="remoteWsdl" />
I have two service units. So one for consumer, one for provider.
Is there another way to tune the cxf-component? Or raising/lowering
threadpool size to prevent the TOO MANY OPEN FILES error? Because sometimes,
after having these errors, I even have to remove the data folder to be able
to restart SM...
Thanks a lot
Freeman Fang wrote:
>
> Hi,
> When you see the error throw, is the memory reach the heap max size?
> Would you please append your configuration for cxf bc consumer/provider
> also?
> Freeman
>
> KOS wrote:
>> Hi
>>
>> (Please see my setting below)
>>
>> I tried to find out how many messages ServiceMix can handle. After
>> sending
>> more than about 1400 requests (in parallel with 100 connections) from an
>> external soap client to an external Web Service (through the ESB), SM
>> raises
>> errors, for about every second request (see stacktrace below). If I send
>> the
>> requests from only one client (one connection), it takes longer (about
>> 2000
>> - 3000 requests) until SM raises errors.
>> The strange thing is: It makes no difference if I send 1000 requests,
>> wait a
>> few seconds, and send another 1000, compared to sending 2000 without
>> break.
>> It always fails after about 1400 requests. The only thing that helps is
>> restarting SM. Then, I can send another 1400 messages without fails. It
>> seems that SM does not "recover" from the high load.
>>
>> My question:
>> 1. Is ServiceMix (the cxf-bc) designed to raise errors when too many
>> requests are sent? Can't they be queued (before cxf-consumer, this one
>> seems
>> to raise the error)?
>> 2. How can I tune it (the cxf-bc, whatever would help)?
>> 3. Raising Java VM memory settings helps, SM can then handle more
>> requests.
>> But I have still the problem that SM raises a lot of errors when I once
>> put
>> too much load. It seems "not to recover"... How can I solve this (without
>> restarting SM)?
>> 4. Even worse, when I keep sending requests, SM throws this error (see
>> below: error-message and stacktrace for second error). Then, it closes
>> the
>> ports and the service is not available anymore. I have to delete the data
>> folder and restart SM.
>>
>> Thanks
>>
>> My setting:
>>
>> - ServiceMix 3.3.1 SNAPSHOT
>> - A WebService-Bridge (cxf-bc consumer and cxf-bc provider). I send
>> soap-requests from an external client to an external WebService through
>> the
>> ESB.
>> - 2 Intel quad-core @ 2.33 GHz, >> 4 GB RAM
>> - Linux version 2.6.18
>> - OpenJDK Server VM build 1.6.0_0-b12
>> - java -server -Xms128M -Xmx512M (higher settings help, but the problem
>> of
>> "not recovering" stays)
>>
>>
>> Stacktrace for exception:
>> org.apache.servicemix.cxfbc.interceptors.JbiFault: Fault occured
>> at
>> org.apache.servicemix.cxfbc.CxfBcConsumer$JbiPostInvokerInterceptor.handleMessage(CxfBcConsumer.java:653)
>> at
>> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:220)
>> at
>> org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:78)
>> at
>> org.apache.cxf.transport.http_jetty.JettyHTTPDestination.serviceRequest(JettyHTTPDestination.java:278)
>> at
>> org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:252)
>> at
>> org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:70)
>> at
>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:722)
>> at
>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:206)
>> at
>> org.mortbay.jetty.handler.HandlerList.handle(HandlerList.java:49)
>> at
>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
>> at org.mortbay.jetty.Server.handle(Server.java:324)
>> at
>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:505)
>> at
>> org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:842)
>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:648)
>> at
>> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
>> at
>> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380)
>> at
>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:395)
>> at
>> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:450)
>>
>> error-message and stacktrace for second error:
>> <loc-message>Target component servicemix-cxf-bc for service unit
>> ontology-change-su is not installed</loc-message>
>>
>> at
>> org.apache.servicemix.jbi.framework.ManagementSupport.failure(ManagementSupport.java:125)
>> at
>> org.apache.servicemix.jbi.framework.ManagementSupport.failure(ManagementSupport.java:111)
>> at
>> org.apache.servicemix.jbi.framework.ServiceAssemblyLifeCycle.start(ServiceAssemblyLifeCycle.java:146)
>> at
>> org.apache.servicemix.jbi.framework.ServiceAssemblyLifeCycle.restore(ServiceAssemblyLifeCycle.java:330)
>> at
>> org.apache.servicemix.jbi.framework.DeploymentService.start(DeploymentService.java:100)
>> at
>> org.apache.servicemix.jbi.container.JBIContainer.start(JBIContainer.java:670)
>> at
>> org.apache.servicemix.jbi.container.SpringJBIContainer.afterPropertiesSet(SpringJBIContainer.java:115)
>> at
>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1368)
>> at
>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1334)
>> at
>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:473)
>> at
>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at
>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
>> at
>> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
>> at
>> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:221)
>> at
>> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
>> at
>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
>> at
>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
>> at
>> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:429)
>> at
>> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:729)
>> at
>> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:381)
>> at
>> org.apache.xbean.spring.context.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:161)
>> at
>> org.apache.xbean.spring.context.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:51)
>> at org.apache.servicemix.Main.main(Main.java:53)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:616)
>> at
>> org.codehaus.classworlds.Launcher.launchStandard(Launcher.java:353)
>> at org.codehaus.classworlds.Launcher.launch(Launcher.java:264)
>> at
>> org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
>> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
>>
>>
>>
>
>
>
--
View this message in context: http://www.nabble.com/Performance%3A-low-throughput%2C-errors-tp22078651p22098121.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.
Re: Performance: low throughput, errors
Posted by Freeman Fang <fr...@gmail.com>.
Hi,
When you see the error throw, is the memory reach the heap max size?
Would you please append your configuration for cxf bc consumer/provider
also?
Freeman
KOS wrote:
> Hi
>
> (Please see my setting below)
>
> I tried to find out how many messages ServiceMix can handle. After sending
> more than about 1400 requests (in parallel with 100 connections) from an
> external soap client to an external Web Service (through the ESB), SM raises
> errors, for about every second request (see stacktrace below). If I send the
> requests from only one client (one connection), it takes longer (about 2000
> - 3000 requests) until SM raises errors.
> The strange thing is: It makes no difference if I send 1000 requests, wait a
> few seconds, and send another 1000, compared to sending 2000 without break.
> It always fails after about 1400 requests. The only thing that helps is
> restarting SM. Then, I can send another 1400 messages without fails. It
> seems that SM does not "recover" from the high load.
>
> My question:
> 1. Is ServiceMix (the cxf-bc) designed to raise errors when too many
> requests are sent? Can't they be queued (before cxf-consumer, this one seems
> to raise the error)?
> 2. How can I tune it (the cxf-bc, whatever would help)?
> 3. Raising Java VM memory settings helps, SM can then handle more requests.
> But I have still the problem that SM raises a lot of errors when I once put
> too much load. It seems "not to recover"... How can I solve this (without
> restarting SM)?
> 4. Even worse, when I keep sending requests, SM throws this error (see
> below: error-message and stacktrace for second error). Then, it closes the
> ports and the service is not available anymore. I have to delete the data
> folder and restart SM.
>
> Thanks
>
> My setting:
>
> - ServiceMix 3.3.1 SNAPSHOT
> - A WebService-Bridge (cxf-bc consumer and cxf-bc provider). I send
> soap-requests from an external client to an external WebService through the
> ESB.
> - 2 Intel quad-core @ 2.33 GHz, >> 4 GB RAM
> - Linux version 2.6.18
> - OpenJDK Server VM build 1.6.0_0-b12
> - java -server -Xms128M -Xmx512M (higher settings help, but the problem of
> "not recovering" stays)
>
>
> Stacktrace for exception:
> org.apache.servicemix.cxfbc.interceptors.JbiFault: Fault occured
> at
> org.apache.servicemix.cxfbc.CxfBcConsumer$JbiPostInvokerInterceptor.handleMessage(CxfBcConsumer.java:653)
> at
> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:220)
> at
> org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:78)
> at
> org.apache.cxf.transport.http_jetty.JettyHTTPDestination.serviceRequest(JettyHTTPDestination.java:278)
> at
> org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:252)
> at
> org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:70)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:722)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:206)
> at org.mortbay.jetty.handler.HandlerList.handle(HandlerList.java:49)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
> at org.mortbay.jetty.Server.handle(Server.java:324)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:505)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:842)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:648)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:395)
> at
> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:450)
>
> error-message and stacktrace for second error:
> <loc-message>Target component servicemix-cxf-bc for service unit
> ontology-change-su is not installed</loc-message>
>
> at
> org.apache.servicemix.jbi.framework.ManagementSupport.failure(ManagementSupport.java:125)
> at
> org.apache.servicemix.jbi.framework.ManagementSupport.failure(ManagementSupport.java:111)
> at
> org.apache.servicemix.jbi.framework.ServiceAssemblyLifeCycle.start(ServiceAssemblyLifeCycle.java:146)
> at
> org.apache.servicemix.jbi.framework.ServiceAssemblyLifeCycle.restore(ServiceAssemblyLifeCycle.java:330)
> at
> org.apache.servicemix.jbi.framework.DeploymentService.start(DeploymentService.java:100)
> at
> org.apache.servicemix.jbi.container.JBIContainer.start(JBIContainer.java:670)
> at
> org.apache.servicemix.jbi.container.SpringJBIContainer.afterPropertiesSet(SpringJBIContainer.java:115)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1368)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1334)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:473)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
> at
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:221)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
> at
> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:429)
> at
> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:729)
> at
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:381)
> at
> org.apache.xbean.spring.context.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:161)
> at
> org.apache.xbean.spring.context.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:51)
> at org.apache.servicemix.Main.main(Main.java:53)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at
> org.codehaus.classworlds.Launcher.launchStandard(Launcher.java:353)
> at org.codehaus.classworlds.Launcher.launch(Launcher.java:264)
> at
> org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
>
>
>