You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@servicemix.apache.org by rgavlin <rg...@yahoo.com> on 2008/08/24 10:55:08 UTC
smx-cxf-bc provider & camel-cxf scalability
Greetings,
I have an asynchronous pipeline using either an smx-cxf-bc provider or
camel-cxf as its transformer. Assume the pipeline is sending a large number
of async, in-out meps to the cxf-based transformer concurrently in order to
invoke an external, request-response cxf-hosted service using the jms
transport.
In order for this solution to scale, the cxf-based transformer cannot send
the request to the external service and then wait for the response on the
same thread. Based on Dan's post
"http://www.nabble.com/Does-SMX-CXF-BC-provider-w-JMS-transport-block-thread-waiting-for-a-reply--tp19102173p19105489.html",
it appears this is how the cxf "ClientImpl" works currently regardless of
the transport. Therefore, I suspect that the smx-cxf-bc provider and
camel-cxf behave similarly since they likely inherit this functionality from
the cxf "ClientImpl".
Is this true for the smx-cxf-bc provider?
Is it true for camel-cxf?
In order for the smx-cxf-bc provider and camel-cxf to scale, the thread
sending the request should complete after the request is successfully sent.
Then a thread or thread pool should be listening on the reply-to queue. When
the reply is received, it should be correlated to the original message
exchange and returned to the pipeline.
Must the cxf "ClientImpl" be upgraded to deliver this functionality or could
the smx-cxf-bc provider and camel-cxf be enhanced independently to provide
this functionality? It sounds like enhancing the cxf "ClientImpl" will
likely require API changes that would not be doable in a CXF 2.0.x or even
2.1.x branch. Of course, I would like to see this scalability issue
addressed sooner than later.
Let me know your thoughts.
- Ron
--
View this message in context: http://www.nabble.com/smx-cxf-bc-provider---camel-cxf-scalability-tp19128784p19128784.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.
Re: smx-cxf-bc provider & camel-cxf scalability
Posted by Freeman Fang <fr...@gmail.com>.
rgavlin wrote:
> Hi Freeman,
>
> Would it be appropriate to use the same Spring-JMS-based solution here as is
> used to fix SM-1520? See
> http://www.nabble.com/New-smx-jms-provider-scalability-handling-in-out-MEP-td19129424.html
> for the discussion. I have read that CXF plans to migrate their JMS
> transport to use Spring JMS in the future.
>
>
Waiting feedback from cxf-dev and discuss this issue there.
> Is the CXF conduit the right place to solve the problem or should it be done
> at a higher-level using common smx and camel patterns?
>
It should get resolved in cxf jms transport IMO, since servicemix-cxf-bc
leverage cxf jms transport and shouldn't see jms transport level details.
> - Ron
>
>
> Freeman Fang wrote:
>
>> Hi Ron,
>>
>> Yes, servicemix-cxf-bc provider almost do same thing as cxf ClientImpl
>> do (like create interceptor chain, invoke interceptor and register
>> incoming message observer).
>> We can add some code for servicemix-cxf-bc provider like create another
>> thread to invoke interceptor to send request so that the main thread
>> will not get blocked when waiting response, but I believe it's not what
>> you really want like your described in another mail
>> "It should send the JMS request on one thread and store the
>> messageExchange in an in-memory map keyed by correlationId. A a pool of
>> threads should be listening on the temporary reply-to queue..."
>>
>> servicemix-cxf-bc provider will reuse jms transport of cxf, so the
>> synchronized way how JMSConduit send out message and handleResponse
>> (block for response message until client timeout) currently may need
>> change. I will post thread on cxf-dev list to discuss this issue.
>>
>> Regards
>> Freeman
>>
>>
>>
>>
>> rgavlin wrote:
>>
>>> Greetings,
>>>
>>> I have an asynchronous pipeline using either an smx-cxf-bc provider or
>>> camel-cxf as its transformer. Assume the pipeline is sending a large
>>> number
>>> of async, in-out meps to the cxf-based transformer concurrently in order
>>> to
>>> invoke an external, request-response cxf-hosted service using the jms
>>> transport.
>>>
>>> In order for this solution to scale, the cxf-based transformer cannot
>>> send
>>> the request to the external service and then wait for the response on the
>>> same thread. Based on Dan's post
>>> "http://www.nabble.com/Does-SMX-CXF-BC-provider-w-JMS-transport-block-thread-waiting-for-a-reply--tp19102173p19105489.html",
>>> it appears this is how the cxf "ClientImpl" works currently regardless of
>>> the transport. Therefore, I suspect that the smx-cxf-bc provider and
>>> camel-cxf behave similarly since they likely inherit this functionality
>>> from
>>> the cxf "ClientImpl".
>>>
>>> Is this true for the smx-cxf-bc provider?
>>>
>>> Is it true for camel-cxf?
>>>
>>> In order for the smx-cxf-bc provider and camel-cxf to scale, the thread
>>> sending the request should complete after the request is successfully
>>> sent.
>>> Then a thread or thread pool should be listening on the reply-to queue.
>>> When
>>> the reply is received, it should be correlated to the original message
>>> exchange and returned to the pipeline.
>>>
>>> Must the cxf "ClientImpl" be upgraded to deliver this functionality or
>>> could
>>> the smx-cxf-bc provider and camel-cxf be enhanced independently to
>>> provide
>>> this functionality? It sounds like enhancing the cxf "ClientImpl" will
>>> likely require API changes that would not be doable in a CXF 2.0.x or
>>> even
>>> 2.1.x branch. Of course, I would like to see this scalability issue
>>> addressed sooner than later.
>>>
>>> Let me know your thoughts.
>>>
>>> - Ron
>>>
>>>
>>
>>
>
>
Re: smx-cxf-bc provider & camel-cxf scalability
Posted by rgavlin <rg...@yahoo.com>.
Hi Freeman,
Would it be appropriate to use the same Spring-JMS-based solution here as is
used to fix SM-1520? See
http://www.nabble.com/New-smx-jms-provider-scalability-handling-in-out-MEP-td19129424.html
for the discussion. I have read that CXF plans to migrate their JMS
transport to use Spring JMS in the future.
Is the CXF conduit the right place to solve the problem or should it be done
at a higher-level using common smx and camel patterns?
- Ron
Freeman Fang wrote:
>
> Hi Ron,
>
> Yes, servicemix-cxf-bc provider almost do same thing as cxf ClientImpl
> do (like create interceptor chain, invoke interceptor and register
> incoming message observer).
> We can add some code for servicemix-cxf-bc provider like create another
> thread to invoke interceptor to send request so that the main thread
> will not get blocked when waiting response, but I believe it's not what
> you really want like your described in another mail
> "It should send the JMS request on one thread and store the
> messageExchange in an in-memory map keyed by correlationId. A a pool of
> threads should be listening on the temporary reply-to queue..."
>
> servicemix-cxf-bc provider will reuse jms transport of cxf, so the
> synchronized way how JMSConduit send out message and handleResponse
> (block for response message until client timeout) currently may need
> change. I will post thread on cxf-dev list to discuss this issue.
>
> Regards
> Freeman
>
>
>
>
> rgavlin wrote:
>> Greetings,
>>
>> I have an asynchronous pipeline using either an smx-cxf-bc provider or
>> camel-cxf as its transformer. Assume the pipeline is sending a large
>> number
>> of async, in-out meps to the cxf-based transformer concurrently in order
>> to
>> invoke an external, request-response cxf-hosted service using the jms
>> transport.
>>
>> In order for this solution to scale, the cxf-based transformer cannot
>> send
>> the request to the external service and then wait for the response on the
>> same thread. Based on Dan's post
>> "http://www.nabble.com/Does-SMX-CXF-BC-provider-w-JMS-transport-block-thread-waiting-for-a-reply--tp19102173p19105489.html",
>> it appears this is how the cxf "ClientImpl" works currently regardless of
>> the transport. Therefore, I suspect that the smx-cxf-bc provider and
>> camel-cxf behave similarly since they likely inherit this functionality
>> from
>> the cxf "ClientImpl".
>>
>> Is this true for the smx-cxf-bc provider?
>>
>> Is it true for camel-cxf?
>>
>> In order for the smx-cxf-bc provider and camel-cxf to scale, the thread
>> sending the request should complete after the request is successfully
>> sent.
>> Then a thread or thread pool should be listening on the reply-to queue.
>> When
>> the reply is received, it should be correlated to the original message
>> exchange and returned to the pipeline.
>>
>> Must the cxf "ClientImpl" be upgraded to deliver this functionality or
>> could
>> the smx-cxf-bc provider and camel-cxf be enhanced independently to
>> provide
>> this functionality? It sounds like enhancing the cxf "ClientImpl" will
>> likely require API changes that would not be doable in a CXF 2.0.x or
>> even
>> 2.1.x branch. Of course, I would like to see this scalability issue
>> addressed sooner than later.
>>
>> Let me know your thoughts.
>>
>> - Ron
>>
>
>
>
--
View this message in context: http://www.nabble.com/smx-cxf-bc-provider---camel-cxf-scalability-tp19128784p19142064.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.
Re: smx-cxf-bc provider & camel-cxf scalability
Posted by Freeman Fang <fr...@gmail.com>.
Hi Ron,
Yes, servicemix-cxf-bc provider almost do same thing as cxf ClientImpl
do (like create interceptor chain, invoke interceptor and register
incoming message observer).
We can add some code for servicemix-cxf-bc provider like create another
thread to invoke interceptor to send request so that the main thread
will not get blocked when waiting response, but I believe it's not what
you really want like your described in another mail
"It should send the JMS request on one thread and store the
messageExchange in an in-memory map keyed by correlationId. A a pool of
threads should be listening on the temporary reply-to queue..."
servicemix-cxf-bc provider will reuse jms transport of cxf, so the
synchronized way how JMSConduit send out message and handleResponse
(block for response message until client timeout) currently may need
change. I will post thread on cxf-dev list to discuss this issue.
Regards
Freeman
rgavlin wrote:
> Greetings,
>
> I have an asynchronous pipeline using either an smx-cxf-bc provider or
> camel-cxf as its transformer. Assume the pipeline is sending a large number
> of async, in-out meps to the cxf-based transformer concurrently in order to
> invoke an external, request-response cxf-hosted service using the jms
> transport.
>
> In order for this solution to scale, the cxf-based transformer cannot send
> the request to the external service and then wait for the response on the
> same thread. Based on Dan's post
> "http://www.nabble.com/Does-SMX-CXF-BC-provider-w-JMS-transport-block-thread-waiting-for-a-reply--tp19102173p19105489.html",
> it appears this is how the cxf "ClientImpl" works currently regardless of
> the transport. Therefore, I suspect that the smx-cxf-bc provider and
> camel-cxf behave similarly since they likely inherit this functionality from
> the cxf "ClientImpl".
>
> Is this true for the smx-cxf-bc provider?
>
> Is it true for camel-cxf?
>
> In order for the smx-cxf-bc provider and camel-cxf to scale, the thread
> sending the request should complete after the request is successfully sent.
> Then a thread or thread pool should be listening on the reply-to queue. When
> the reply is received, it should be correlated to the original message
> exchange and returned to the pipeline.
>
> Must the cxf "ClientImpl" be upgraded to deliver this functionality or could
> the smx-cxf-bc provider and camel-cxf be enhanced independently to provide
> this functionality? It sounds like enhancing the cxf "ClientImpl" will
> likely require API changes that would not be doable in a CXF 2.0.x or even
> 2.1.x branch. Of course, I would like to see this scalability issue
> addressed sooner than later.
>
> Let me know your thoughts.
>
> - Ron
>