You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@servicemix.apache.org by boday <bo...@vektrel.com> on 2009/07/23 18:05:31 UTC
Re: JBI performance
Ashwin...
So you are basically saying that in high-volume scenarios, Camel shouldn't
use JBI components because there is a higher overhead associated with these
requests? I'm seeing similar performance issues using Camel and am tempted
to move all my JBI services into Camel components to eliminate as much
overhead as possible.
If I do this, then why do I need SMX at all. Camel supports all the binding
components (activemq, http, file, etc) that SMX does and seems to be add
more flexibility with routing and exception handling.
What is SMX providing otherwise? I'm starting to feel that Camel and SMX
are competing technologies that happen to work together (using
servicemix-camel binding). Can someone please elaborate on why I need SMX
if Camel components seem to do everything...
thanks in advance...
Ashwin Karpe wrote:
>
> Hi,
>
> Based on your example, the following will happen in JBI.
>
> 1> Camel SA1 invoking the JBI service myService will cause the following
> additional work to take place.It will cause the running thread in the
> servicemix-camel component to create an In-only Message exchange, copy the
> message from the processor to the exchange. The exchange will be sent to
> the NMR to locate and forward to myService.
> 2> Now myService though in another SA is also a servicemix-camel endpoint
> registered with the NMR and will cause the exchange to be picked up by
> another servicemix-camel thread and processed. The payload will be pulled
> out of the JBI Message exchange and moved into a Camel Exchange that is
> forwarded to the second processor. The JBI Message exchange is now set to
> Done and pushed back and sent to the original thread for Camel-SA1 for
> further processing and cleanup (due to JBI MEP processing rules).
> 3> The exchnage cleanup has to be done by the servicemix container after
> the servicemix-camel thread SA1 does nothing further with the exchange.
>
> The trouble you run into when it is all running in the same
> servicemix-camel container is that the code becomes re-entrant, more and
> more time is spent in exchange creation and management within
> servicemix-camel, exchanges start to pile up pretty quickly, thereby
> driving down the overall processing speed. Also, this activity chews up
> threads pretty quickly in a high volume scenario.
>
> Note that this does not happen in the pure JMS situation. The
> servicemix-camel component does not have to create any fresh JBI exchanges
> and manage them since it has no NMR involved. The camel routing scenario
> can entirely executed in a running thread that can grab a message from the
> Broker managing the Topic, process and and send it to another Topic.
> Another thread then picks up from the second Topic and processes the
> message. There is no need to create and manage JBI exchanges. I guess the
> difference is the difference between a routing and a bridging scenario.
> Hence the much better performance.
>
> Hope this helps. The only thing that surprised me was how much the cost
> differential was. You might try to increase the number of threads for the
> Camel component via configuration (servicemix.properties in the conf
> directory) and see if that makes a difference. I bet increasing the thread
> pool will improve performance, however the performance of the pure-JMS
> solution should improve as well :).
>
> Cheers,
>
> Ashwin...
>
>
-----
Ben O'Day
Vektrel - Senior Consultant
--
View this message in context: http://www.nabble.com/JBI-performance-tp21615804p24629079.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.
Re: JBI performance
Posted by Ashwin Karpe <as...@progress.com>.
Hi,
Jose, you bring up a good point. It is not that binary payloads cannot be
sent with SMX. They cannot be sent in the message body. They need to be sent
as message attachments (MTOM). The message body format however must be
strictly XML. This is not so in Camel. In Camel the message body (payload)
itself maybe of any format ...
Cheers,
Ashwin...
jose romero-2 wrote:
>
> Just my experience on JBI and performance,
>
> I'm using extensively dynamic endpoint resolution so I can have a very
> flexible endpoint addressing which is required for me because I'm
> using endpoint in a rest-like way. It looks like camel is a little
> more restrictive on this point but I might be wrong
>
> As for jbi being only xml, I would say it's not exactly true. Any JBI
> message can handle attachments (to manage source/stream on large
> external data blob for instance) or even properties (java objects). So
> using the right one is important : attachments for large data,
> properties for native java objects, xml for xml structured payloads
> (by the way using vtd-xml for small jbi xml payload is way faster than
> standard sax/stax/dom).The payload can be only an xml descriptor on
> properties or attachments data.
> I wrapped my custom jbi component with data input and output adapters
> so I can easily adapt the data input / output behavior and use the
> fastest one.
>
> Also :
> - the jbi lifecycle (and the osgi one) is very useful for dynamic
> deployment/undeployment
> - the MEP support in JBI allows for easy asynchronous conversation
> (and so maximize parallelism even for IN-OUT mep)
>
> Also for me SMX 4 jbi clustering and OSGI NMR is a great way to ensure
> robust and fast enterprise integration flows.
>
> cheers
>
> On Fri, Jul 24, 2009 at 9:53 AM, Madesclair
> Vivian<vm...@sopragroup.com> wrote:
>> Hello,
>>
>> I agree with vincent.
>> And could you please explain a bit more what you said about smx4 & OSGI?
>> You mean the problem you mention a few posts ago about bridging with
>> camel disappear with smx 4? How is that?
>>
>> Regards,
>> Vivian
>>
>>
>> -----Message d'origine-----
>> De : Vincent GIRARDREYDET [mailto:vincent.girardreydet@thalesgroup.com]
>> Envoyé : vendredi 24 juillet 2009 09:11
>> À : users@servicemix.apache.org
>> Objet : Re: JBI performance
>>
>> Ashwin,
>>
>> Is there a way you could put this on a wiki page in the servicemix wiki ?
>> I find your explanations very clear and in few sentences you have
>> explained me everything I had'nt understood before about the Camel vs
>> ServiceMix difference. I guess this might be usefull to other people as
>> well :-).
>>
>> Vincent
>>
>> Ashwin Karpe a écrit :
>>> Hi,
>>>
>>> Camel and Servicemix are both fundamentally about applying Enterprise
>>> Integration patterns to solve complex integration problems across
>>> different technology sets.
>>>
>>> However, they do this by employing different styles and approaches.
>>> The broad distinctions between Camel and Servicemix JBI technology
>>> stack, in my view, are the following
>>> - Camel is standardized via DSL (Domain Specific Language),
>>> Servicemix on the other hand is an implementation of the JBI standard
>>> - Both use message exchanges internally to move payloads
>>> between endpoints using routes (Camel) and workflows (JBI). However,
>>> in Camel, a MessageExchange is an Interface which is implemented by
>>> different technology sets in their own way i.e HTTP Exchange, FTP
>>> Exchange etc and employs type conversion to normalize data between
>>> these message exchanges. In Servicemix, a Message Exchange is a
>>> standardized Data structure with XML as its message payload.
>>> - Messages in Camel may be of any type (binary, text,
>>> serialized object etc). Messages in Servicemix are XML only.
>>> - Camel utilizes URI's to express endpoints, Servicemix uses
>>> the WSDL conventions (ServiceEndpointReference) to express endpoints.
>>> - Camel has a broader palette due to a simpler/more efficient
>>> and expressive way of wiring endpoints using routes. It is more easily
>>> embeddable in traditional Java Apps and can be run from a command line.
>>> Servicemix on the other hand is fundamentally based on a framework
>>> (JBI) and its services/endpoints are deployment in a JBI/OSGi based
>>> container.
>>> - The core JBI framework provides its endpoints key
>>> infrastructure services/facilities out of the box (location
>>> transarency, mediated messaging via NMR) and is more suited for
>>> viewing as a traditional ESB. Camel is more suited for being viewed as
>>> an easy to use, yet sophisticated piece of DSL based integration
>>> technology.
>>>
>>> BTW, Servicemix 4 (OSGi) blends/brings together both these
>>> technologies beautifully. Servicemix-Camel and Camel can seamlessly
>>> coexist and provide seamless navigation between JBI and Camel routes.
>>>
>>> I like both for their own quirks, elegance & oddities. To each his
>>> own, I guess :).
>>>
>>> Cheers,
>>>
>>> Ashwin...
>>>
>>>
>>> boday wrote:
>>>
>>>> Ashwin...
>>>>
>>>> So you are basically saying that in high-volume scenarios, Camel
>>>> shouldn't use JBI components because there is a higher overhead
>>>> associated with these requests? I'm seeing similar performance
>>>> issues using Camel and am tempted to move all my JBI services into
>>>> Camel components to eliminate as much overhead as possible.
>>>>
>>>> If I do this, then why do I need SMX at all. Camel supports all the
>>>> binding components (activemq, http, file, etc) that SMX does and
>>>> seems to be add more flexibility with routing and exception handling.
>>>>
>>>> What is SMX providing otherwise? I'm starting to feel that Camel and
>>>> SMX are competing technologies that happen to work together (using
>>>> servicemix-camel binding). Can someone please elaborate on why I
>>>> need SMX if Camel components seem to do everything...
>>>>
>>>> thanks in advance...
>>>>
>>>>
>>>> Ashwin Karpe wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Based on your example, the following will happen in JBI.
>>>>>
>>>>> 1> Camel SA1 invoking the JBI service myService will cause the
>>>>> following additional work to take place.It will cause the running
>>>>> thread in the servicemix-camel component to create an In-only
>>>>> Message exchange, copy the message from the processor to the
>>>>> exchange. The exchange will be sent to the NMR to locate and forward
>>>>> to myService.
>>>>> 2> Now myService though in another SA is also a servicemix-camel
>>>>> endpoint registered with the NMR and will cause the exchange to be
>>>>> picked up by another servicemix-camel thread and processed. The
>>>>> payload will be pulled out of the JBI Message exchange and moved
>>>>> into a Camel Exchange that is forwarded to the second processor. The
>>>>> JBI Message exchange is now set to Done and pushed back and sent to
>>>>> the original thread for
>>>>> Camel-SA1 for further processing and cleanup (due to JBI MEP
>>>>> processing rules).
>>>>> 3> The exchnage cleanup has to be done by the servicemix container
>>>>> after the servicemix-camel thread SA1 does nothing further with the
>>>>> exchange.
>>>>>
>>>>> The trouble you run into when it is all running in the same
>>>>> servicemix-camel container is that the code becomes re-entrant, more
>>>>> and more time is spent in exchange creation and management within
>>>>> servicemix-camel, exchanges start to pile up pretty quickly, thereby
>>>>> driving down the overall processing speed. Also, this activity chews
>>>>> up threads pretty quickly in a high volume scenario.
>>>>>
>>>>> Note that this does not happen in the pure JMS situation. The
>>>>> servicemix-camel component does not have to create any fresh JBI
>>>>> exchanges and manage them since it has no NMR involved. The camel
>>>>> routing scenario can entirely executed in a running thread that can
>>>>> grab a message from the Broker managing the Topic, process and and
>>>>> send it to another Topic. Another thread then picks up from the
>>>>> second Topic and processes the message. There is no need to create
>>>>> and manage JBI exchanges. I guess the difference is the difference
>>>>> between a routing and a bridging scenario. Hence the much better
>>>>> performance.
>>>>>
>>>>> Hope this helps. The only thing that surprised me was how much the
>>>>> cost differential was. You might try to increase the number of
>>>>> threads for the Camel component via configuration
>>>>> (servicemix.properties in the conf
>>>>> directory) and see if that makes a difference. I bet increasing the
>>>>> thread pool will improve performance, however the performance of the
>>>>> pure-JMS solution should improve as well :).
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Ashwin...
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>> -----
>>> ---
>>> Ashwin Karpe, Principal Consultant, PS - Opensource Center of
>>> Competence Progress Software Corporation
>>> 14 Oak Park Drive
>>> Bedford, MA 01730
>>> ---
>>> +1-972-304-9084 (Office)
>>> +1-972-971-1700 (Mobile)
>>> ----
>>> Blog: http://opensourceknowledge.blogspot.com/
>>>
>>>
>>>
>>
>
>
-----
---
Ashwin Karpe, Principal Consultant, PS - Opensource Center of Competence
Progress Software Corporation
14 Oak Park Drive
Bedford, MA 01730
---
+1-972-304-9084 (Office)
+1-972-971-1700 (Mobile)
----
Blog: http://opensourceknowledge.blogspot.com/
--
View this message in context: http://www.nabble.com/JBI-performance-tp21615804p24727368.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.
Re: JBI performance
Posted by jose romero <jo...@gmail.com>.
Just my experience on JBI and performance,
I'm using extensively dynamic endpoint resolution so I can have a very
flexible endpoint addressing which is required for me because I'm
using endpoint in a rest-like way. It looks like camel is a little
more restrictive on this point but I might be wrong
As for jbi being only xml, I would say it's not exactly true. Any JBI
message can handle attachments (to manage source/stream on large
external data blob for instance) or even properties (java objects). So
using the right one is important : attachments for large data,
properties for native java objects, xml for xml structured payloads
(by the way using vtd-xml for small jbi xml payload is way faster than
standard sax/stax/dom).The payload can be only an xml descriptor on
properties or attachments data.
I wrapped my custom jbi component with data input and output adapters
so I can easily adapt the data input / output behavior and use the
fastest one.
Also :
- the jbi lifecycle (and the osgi one) is very useful for dynamic
deployment/undeployment
- the MEP support in JBI allows for easy asynchronous conversation
(and so maximize parallelism even for IN-OUT mep)
Also for me SMX 4 jbi clustering and OSGI NMR is a great way to ensure
robust and fast enterprise integration flows.
cheers
On Fri, Jul 24, 2009 at 9:53 AM, Madesclair
Vivian<vm...@sopragroup.com> wrote:
> Hello,
>
> I agree with vincent.
> And could you please explain a bit more what you said about smx4 & OSGI? You mean the problem you mention a few posts ago about bridging with camel disappear with smx 4? How is that?
>
> Regards,
> Vivian
>
>
> -----Message d'origine-----
> De : Vincent GIRARDREYDET [mailto:vincent.girardreydet@thalesgroup.com]
> Envoyé : vendredi 24 juillet 2009 09:11
> À : users@servicemix.apache.org
> Objet : Re: JBI performance
>
> Ashwin,
>
> Is there a way you could put this on a wiki page in the servicemix wiki ? I find your explanations very clear and in few sentences you have explained me everything I had'nt understood before about the Camel vs ServiceMix difference. I guess this might be usefull to other people as well :-).
>
> Vincent
>
> Ashwin Karpe a écrit :
>> Hi,
>>
>> Camel and Servicemix are both fundamentally about applying Enterprise
>> Integration patterns to solve complex integration problems across
>> different technology sets.
>>
>> However, they do this by employing different styles and approaches.
>> The broad distinctions between Camel and Servicemix JBI technology
>> stack, in my view, are the following
>> - Camel is standardized via DSL (Domain Specific Language),
>> Servicemix on the other hand is an implementation of the JBI standard
>> - Both use message exchanges internally to move payloads
>> between endpoints using routes (Camel) and workflows (JBI). However,
>> in Camel, a MessageExchange is an Interface which is implemented by
>> different technology sets in their own way i.e HTTP Exchange, FTP
>> Exchange etc and employs type conversion to normalize data between
>> these message exchanges. In Servicemix, a Message Exchange is a
>> standardized Data structure with XML as its message payload.
>> - Messages in Camel may be of any type (binary, text,
>> serialized object etc). Messages in Servicemix are XML only.
>> - Camel utilizes URI's to express endpoints, Servicemix uses
>> the WSDL conventions (ServiceEndpointReference) to express endpoints.
>> - Camel has a broader palette due to a simpler/more efficient
>> and expressive way of wiring endpoints using routes. It is more easily
>> embeddable in traditional Java Apps and can be run from a command line.
>> Servicemix on the other hand is fundamentally based on a framework
>> (JBI) and its services/endpoints are deployment in a JBI/OSGi based container.
>> - The core JBI framework provides its endpoints key
>> infrastructure services/facilities out of the box (location
>> transarency, mediated messaging via NMR) and is more suited for
>> viewing as a traditional ESB. Camel is more suited for being viewed as
>> an easy to use, yet sophisticated piece of DSL based integration technology.
>>
>> BTW, Servicemix 4 (OSGi) blends/brings together both these
>> technologies beautifully. Servicemix-Camel and Camel can seamlessly
>> coexist and provide seamless navigation between JBI and Camel routes.
>>
>> I like both for their own quirks, elegance & oddities. To each his
>> own, I guess :).
>>
>> Cheers,
>>
>> Ashwin...
>>
>>
>> boday wrote:
>>
>>> Ashwin...
>>>
>>> So you are basically saying that in high-volume scenarios, Camel
>>> shouldn't use JBI components because there is a higher overhead
>>> associated with these requests? I'm seeing similar performance
>>> issues using Camel and am tempted to move all my JBI services into
>>> Camel components to eliminate as much overhead as possible.
>>>
>>> If I do this, then why do I need SMX at all. Camel supports all the
>>> binding components (activemq, http, file, etc) that SMX does and
>>> seems to be add more flexibility with routing and exception handling.
>>>
>>> What is SMX providing otherwise? I'm starting to feel that Camel and
>>> SMX are competing technologies that happen to work together (using
>>> servicemix-camel binding). Can someone please elaborate on why I
>>> need SMX if Camel components seem to do everything...
>>>
>>> thanks in advance...
>>>
>>>
>>> Ashwin Karpe wrote:
>>>
>>>> Hi,
>>>>
>>>> Based on your example, the following will happen in JBI.
>>>>
>>>> 1> Camel SA1 invoking the JBI service myService will cause the
>>>> following additional work to take place.It will cause the running
>>>> thread in the servicemix-camel component to create an In-only
>>>> Message exchange, copy the message from the processor to the
>>>> exchange. The exchange will be sent to the NMR to locate and forward to myService.
>>>> 2> Now myService though in another SA is also a servicemix-camel
>>>> endpoint registered with the NMR and will cause the exchange to be
>>>> picked up by another servicemix-camel thread and processed. The
>>>> payload will be pulled out of the JBI Message exchange and moved
>>>> into a Camel Exchange that is forwarded to the second processor. The
>>>> JBI Message exchange is now set to Done and pushed back and sent to
>>>> the original thread for
>>>> Camel-SA1 for further processing and cleanup (due to JBI MEP
>>>> processing rules).
>>>> 3> The exchnage cleanup has to be done by the servicemix container
>>>> after the servicemix-camel thread SA1 does nothing further with the exchange.
>>>>
>>>> The trouble you run into when it is all running in the same
>>>> servicemix-camel container is that the code becomes re-entrant, more
>>>> and more time is spent in exchange creation and management within
>>>> servicemix-camel, exchanges start to pile up pretty quickly, thereby
>>>> driving down the overall processing speed. Also, this activity chews
>>>> up threads pretty quickly in a high volume scenario.
>>>>
>>>> Note that this does not happen in the pure JMS situation. The
>>>> servicemix-camel component does not have to create any fresh JBI
>>>> exchanges and manage them since it has no NMR involved. The camel
>>>> routing scenario can entirely executed in a running thread that can
>>>> grab a message from the Broker managing the Topic, process and and
>>>> send it to another Topic. Another thread then picks up from the
>>>> second Topic and processes the message. There is no need to create
>>>> and manage JBI exchanges. I guess the difference is the difference
>>>> between a routing and a bridging scenario. Hence the much better performance.
>>>>
>>>> Hope this helps. The only thing that surprised me was how much the
>>>> cost differential was. You might try to increase the number of
>>>> threads for the Camel component via configuration
>>>> (servicemix.properties in the conf
>>>> directory) and see if that makes a difference. I bet increasing the
>>>> thread pool will improve performance, however the performance of the
>>>> pure-JMS solution should improve as well :).
>>>>
>>>> Cheers,
>>>>
>>>> Ashwin...
>>>>
>>>>
>>>>
>>>
>>
>>
>> -----
>> ---
>> Ashwin Karpe, Principal Consultant, PS - Opensource Center of
>> Competence Progress Software Corporation
>> 14 Oak Park Drive
>> Bedford, MA 01730
>> ---
>> +1-972-304-9084 (Office)
>> +1-972-971-1700 (Mobile)
>> ----
>> Blog: http://opensourceknowledge.blogspot.com/
>>
>>
>>
>
RE: JBI performance
Posted by Madesclair Vivian <vm...@sopragroup.com>.
Hello,
I agree with vincent.
And could you please explain a bit more what you said about smx4 & OSGI? You mean the problem you mention a few posts ago about bridging with camel disappear with smx 4? How is that?
Regards,
Vivian
-----Message d'origine-----
De : Vincent GIRARDREYDET [mailto:vincent.girardreydet@thalesgroup.com]
Envoyé : vendredi 24 juillet 2009 09:11
À : users@servicemix.apache.org
Objet : Re: JBI performance
Ashwin,
Is there a way you could put this on a wiki page in the servicemix wiki ? I find your explanations very clear and in few sentences you have explained me everything I had'nt understood before about the Camel vs ServiceMix difference. I guess this might be usefull to other people as well :-).
Vincent
Ashwin Karpe a écrit :
> Hi,
>
> Camel and Servicemix are both fundamentally about applying Enterprise
> Integration patterns to solve complex integration problems across
> different technology sets.
>
> However, they do this by employing different styles and approaches.
> The broad distinctions between Camel and Servicemix JBI technology
> stack, in my view, are the following
> - Camel is standardized via DSL (Domain Specific Language),
> Servicemix on the other hand is an implementation of the JBI standard
> - Both use message exchanges internally to move payloads
> between endpoints using routes (Camel) and workflows (JBI). However,
> in Camel, a MessageExchange is an Interface which is implemented by
> different technology sets in their own way i.e HTTP Exchange, FTP
> Exchange etc and employs type conversion to normalize data between
> these message exchanges. In Servicemix, a Message Exchange is a
> standardized Data structure with XML as its message payload.
> - Messages in Camel may be of any type (binary, text,
> serialized object etc). Messages in Servicemix are XML only.
> - Camel utilizes URI's to express endpoints, Servicemix uses
> the WSDL conventions (ServiceEndpointReference) to express endpoints.
> - Camel has a broader palette due to a simpler/more efficient
> and expressive way of wiring endpoints using routes. It is more easily
> embeddable in traditional Java Apps and can be run from a command line.
> Servicemix on the other hand is fundamentally based on a framework
> (JBI) and its services/endpoints are deployment in a JBI/OSGi based container.
> - The core JBI framework provides its endpoints key
> infrastructure services/facilities out of the box (location
> transarency, mediated messaging via NMR) and is more suited for
> viewing as a traditional ESB. Camel is more suited for being viewed as
> an easy to use, yet sophisticated piece of DSL based integration technology.
>
> BTW, Servicemix 4 (OSGi) blends/brings together both these
> technologies beautifully. Servicemix-Camel and Camel can seamlessly
> coexist and provide seamless navigation between JBI and Camel routes.
>
> I like both for their own quirks, elegance & oddities. To each his
> own, I guess :).
>
> Cheers,
>
> Ashwin...
>
>
> boday wrote:
>
>> Ashwin...
>>
>> So you are basically saying that in high-volume scenarios, Camel
>> shouldn't use JBI components because there is a higher overhead
>> associated with these requests? I'm seeing similar performance
>> issues using Camel and am tempted to move all my JBI services into
>> Camel components to eliminate as much overhead as possible.
>>
>> If I do this, then why do I need SMX at all. Camel supports all the
>> binding components (activemq, http, file, etc) that SMX does and
>> seems to be add more flexibility with routing and exception handling.
>>
>> What is SMX providing otherwise? I'm starting to feel that Camel and
>> SMX are competing technologies that happen to work together (using
>> servicemix-camel binding). Can someone please elaborate on why I
>> need SMX if Camel components seem to do everything...
>>
>> thanks in advance...
>>
>>
>> Ashwin Karpe wrote:
>>
>>> Hi,
>>>
>>> Based on your example, the following will happen in JBI.
>>>
>>> 1> Camel SA1 invoking the JBI service myService will cause the
>>> following additional work to take place.It will cause the running
>>> thread in the servicemix-camel component to create an In-only
>>> Message exchange, copy the message from the processor to the
>>> exchange. The exchange will be sent to the NMR to locate and forward to myService.
>>> 2> Now myService though in another SA is also a servicemix-camel
>>> endpoint registered with the NMR and will cause the exchange to be
>>> picked up by another servicemix-camel thread and processed. The
>>> payload will be pulled out of the JBI Message exchange and moved
>>> into a Camel Exchange that is forwarded to the second processor. The
>>> JBI Message exchange is now set to Done and pushed back and sent to
>>> the original thread for
>>> Camel-SA1 for further processing and cleanup (due to JBI MEP
>>> processing rules).
>>> 3> The exchnage cleanup has to be done by the servicemix container
>>> after the servicemix-camel thread SA1 does nothing further with the exchange.
>>>
>>> The trouble you run into when it is all running in the same
>>> servicemix-camel container is that the code becomes re-entrant, more
>>> and more time is spent in exchange creation and management within
>>> servicemix-camel, exchanges start to pile up pretty quickly, thereby
>>> driving down the overall processing speed. Also, this activity chews
>>> up threads pretty quickly in a high volume scenario.
>>>
>>> Note that this does not happen in the pure JMS situation. The
>>> servicemix-camel component does not have to create any fresh JBI
>>> exchanges and manage them since it has no NMR involved. The camel
>>> routing scenario can entirely executed in a running thread that can
>>> grab a message from the Broker managing the Topic, process and and
>>> send it to another Topic. Another thread then picks up from the
>>> second Topic and processes the message. There is no need to create
>>> and manage JBI exchanges. I guess the difference is the difference
>>> between a routing and a bridging scenario. Hence the much better performance.
>>>
>>> Hope this helps. The only thing that surprised me was how much the
>>> cost differential was. You might try to increase the number of
>>> threads for the Camel component via configuration
>>> (servicemix.properties in the conf
>>> directory) and see if that makes a difference. I bet increasing the
>>> thread pool will improve performance, however the performance of the
>>> pure-JMS solution should improve as well :).
>>>
>>> Cheers,
>>>
>>> Ashwin...
>>>
>>>
>>>
>>
>
>
> -----
> ---
> Ashwin Karpe, Principal Consultant, PS - Opensource Center of
> Competence Progress Software Corporation
> 14 Oak Park Drive
> Bedford, MA 01730
> ---
> +1-972-304-9084 (Office)
> +1-972-971-1700 (Mobile)
> ----
> Blog: http://opensourceknowledge.blogspot.com/
>
>
>
Re: JBI performance
Posted by Ashwin Karpe <as...@progress.com>.
Hi,
I will check with Guillaume/Claus as to the prpoper place wher I could put a
concise wiki entry.
I will also try to write a blog article on my blog site capturing this
information.
I am glad you liked it.
Cheers,
Ashwin...
====================
TheWinch wrote:
>
> Ashwin,
>
> Is there a way you could put this on a wiki page in the servicemix wiki
> ? I find your explanations very clear and in few sentences you have
> explained me everything I had'nt understood before about the Camel vs
> ServiceMix difference. I guess this might be usefull to other people as
> well :-).
>
> Vincent
>
> Ashwin Karpe a écrit :
>> Hi,
>>
>> Camel and Servicemix are both fundamentally about applying Enterprise
>> Integration patterns to solve complex integration problems across
>> different
>> technology sets.
>>
>> However, they do this by employing different styles and approaches. The
>> broad distinctions between Camel and Servicemix JBI technology stack, in
>> my
>> view, are the following
>> - Camel is standardized via DSL (Domain Specific Language),
>> Servicemix on the other hand is an implementation of the JBI standard
>> - Both use message exchanges internally to move payloads between
>> endpoints using routes (Camel) and workflows (JBI). However, in Camel, a
>> MessageExchange is an Interface which is implemented by different
>> technology
>> sets in their own way i.e HTTP Exchange, FTP Exchange etc and employs
>> type
>> conversion to normalize data between these message exchanges. In
>> Servicemix,
>> a Message Exchange is a standardized Data structure with XML as its
>> message
>> payload.
>> - Messages in Camel may be of any type (binary, text, serialized
>> object etc). Messages in Servicemix are XML only.
>> - Camel utilizes URI's to express endpoints, Servicemix uses the
>> WSDL conventions (ServiceEndpointReference) to express endpoints.
>> - Camel has a broader palette due to a simpler/more efficient and
>> expressive way of wiring endpoints using routes. It is more easily
>> embeddable in traditional Java Apps and can be run from a command line.
>> Servicemix on the other hand is fundamentally based on a framework (JBI)
>> and
>> its services/endpoints are deployment in a JBI/OSGi based container.
>> - The core JBI framework provides its endpoints key
>> infrastructure
>> services/facilities out of the box (location transarency, mediated
>> messaging
>> via NMR) and is more suited for viewing as a traditional ESB. Camel is
>> more
>> suited for being viewed as an easy to use, yet sophisticated piece of DSL
>> based integration technology.
>>
>> BTW, Servicemix 4 (OSGi) blends/brings together both these technologies
>> beautifully. Servicemix-Camel and Camel can seamlessly coexist and
>> provide
>> seamless navigation between JBI and Camel routes.
>>
>> I like both for their own quirks, elegance & oddities. To each his own, I
>> guess :).
>>
>> Cheers,
>>
>> Ashwin...
>>
>>
>> boday wrote:
>>
>>> Ashwin...
>>>
>>> So you are basically saying that in high-volume scenarios, Camel
>>> shouldn't
>>> use JBI components because there is a higher overhead associated with
>>> these requests? I'm seeing similar performance issues using Camel and
>>> am
>>> tempted to move all my JBI services into Camel components to eliminate
>>> as
>>> much overhead as possible.
>>>
>>> If I do this, then why do I need SMX at all. Camel supports all the
>>> binding components (activemq, http, file, etc) that SMX does and seems
>>> to
>>> be add more flexibility with routing and exception handling.
>>>
>>> What is SMX providing otherwise? I'm starting to feel that Camel and
>>> SMX
>>> are competing technologies that happen to work together (using
>>> servicemix-camel binding). Can someone please elaborate on why I need
>>> SMX
>>> if Camel components seem to do everything...
>>>
>>> thanks in advance...
>>>
>>>
>>> Ashwin Karpe wrote:
>>>
>>>> Hi,
>>>>
>>>> Based on your example, the following will happen in JBI.
>>>>
>>>> 1> Camel SA1 invoking the JBI service myService will cause the
>>>> following
>>>> additional work to take place.It will cause the running thread in the
>>>> servicemix-camel component to create an In-only Message exchange, copy
>>>> the message from the processor to the exchange. The exchange will be
>>>> sent
>>>> to the NMR to locate and forward to myService.
>>>> 2> Now myService though in another SA is also a servicemix-camel
>>>> endpoint registered with the NMR and will cause the exchange to be
>>>> picked
>>>> up by another servicemix-camel thread and processed. The payload will
>>>> be
>>>> pulled out of the JBI Message exchange and moved into a Camel Exchange
>>>> that is forwarded to the second processor. The JBI Message exchange is
>>>> now set to Done and pushed back and sent to the original thread for
>>>> Camel-SA1 for further processing and cleanup (due to JBI MEP processing
>>>> rules).
>>>> 3> The exchnage cleanup has to be done by the servicemix container
>>>> after
>>>> the servicemix-camel thread SA1 does nothing further with the exchange.
>>>>
>>>> The trouble you run into when it is all running in the same
>>>> servicemix-camel container is that the code becomes re-entrant, more
>>>> and
>>>> more time is spent in exchange creation and management within
>>>> servicemix-camel, exchanges start to pile up pretty quickly, thereby
>>>> driving down the overall processing speed. Also, this activity chews up
>>>> threads pretty quickly in a high volume scenario.
>>>>
>>>> Note that this does not happen in the pure JMS situation. The
>>>> servicemix-camel component does not have to create any fresh JBI
>>>> exchanges and manage them since it has no NMR involved. The camel
>>>> routing
>>>> scenario can entirely executed in a running thread that can grab a
>>>> message from the Broker managing the Topic, process and and send it to
>>>> another Topic. Another thread then picks up from the second Topic and
>>>> processes the message. There is no need to create and manage JBI
>>>> exchanges. I guess the difference is the difference between a routing
>>>> and
>>>> a bridging scenario. Hence the much better performance.
>>>>
>>>> Hope this helps. The only thing that surprised me was how much the cost
>>>> differential was. You might try to increase the number of threads for
>>>> the
>>>> Camel component via configuration (servicemix.properties in the conf
>>>> directory) and see if that makes a difference. I bet increasing the
>>>> thread pool will improve performance, however the performance of the
>>>> pure-JMS solution should improve as well :).
>>>>
>>>> Cheers,
>>>>
>>>> Ashwin...
>>>>
>>>>
>>>>
>>>
>>
>>
>> -----
>> ---
>> Ashwin Karpe, Principal Consultant, PS - Opensource Center of Competence
>> Progress Software Corporation
>> 14 Oak Park Drive
>> Bedford, MA 01730
>> ---
>> +1-972-304-9084 (Office)
>> +1-972-971-1700 (Mobile)
>> ----
>> Blog: http://opensourceknowledge.blogspot.com/
>>
>>
>>
>
>
-----
---
Ashwin Karpe, Principal Consultant, PS - Opensource Center of Competence
Progress Software Corporation
14 Oak Park Drive
Bedford, MA 01730
---
+1-972-304-9084 (Office)
+1-972-971-1700 (Mobile)
----
Blog: http://opensourceknowledge.blogspot.com/
--
View this message in context: http://www.nabble.com/JBI-performance-tp21615804p24727311.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.
Re: JBI performance
Posted by Vincent GIRARDREYDET <vi...@thalesgroup.com>.
Ashwin,
Is there a way you could put this on a wiki page in the servicemix wiki
? I find your explanations very clear and in few sentences you have
explained me everything I had'nt understood before about the Camel vs
ServiceMix difference. I guess this might be usefull to other people as
well :-).
Vincent
Ashwin Karpe a écrit :
> Hi,
>
> Camel and Servicemix are both fundamentally about applying Enterprise
> Integration patterns to solve complex integration problems across different
> technology sets.
>
> However, they do this by employing different styles and approaches. The
> broad distinctions between Camel and Servicemix JBI technology stack, in my
> view, are the following
> - Camel is standardized via DSL (Domain Specific Language),
> Servicemix on the other hand is an implementation of the JBI standard
> - Both use message exchanges internally to move payloads between
> endpoints using routes (Camel) and workflows (JBI). However, in Camel, a
> MessageExchange is an Interface which is implemented by different technology
> sets in their own way i.e HTTP Exchange, FTP Exchange etc and employs type
> conversion to normalize data between these message exchanges. In Servicemix,
> a Message Exchange is a standardized Data structure with XML as its message
> payload.
> - Messages in Camel may be of any type (binary, text, serialized
> object etc). Messages in Servicemix are XML only.
> - Camel utilizes URI's to express endpoints, Servicemix uses the
> WSDL conventions (ServiceEndpointReference) to express endpoints.
> - Camel has a broader palette due to a simpler/more efficient and
> expressive way of wiring endpoints using routes. It is more easily
> embeddable in traditional Java Apps and can be run from a command line.
> Servicemix on the other hand is fundamentally based on a framework (JBI) and
> its services/endpoints are deployment in a JBI/OSGi based container.
> - The core JBI framework provides its endpoints key infrastructure
> services/facilities out of the box (location transarency, mediated messaging
> via NMR) and is more suited for viewing as a traditional ESB. Camel is more
> suited for being viewed as an easy to use, yet sophisticated piece of DSL
> based integration technology.
>
> BTW, Servicemix 4 (OSGi) blends/brings together both these technologies
> beautifully. Servicemix-Camel and Camel can seamlessly coexist and provide
> seamless navigation between JBI and Camel routes.
>
> I like both for their own quirks, elegance & oddities. To each his own, I
> guess :).
>
> Cheers,
>
> Ashwin...
>
>
> boday wrote:
>
>> Ashwin...
>>
>> So you are basically saying that in high-volume scenarios, Camel shouldn't
>> use JBI components because there is a higher overhead associated with
>> these requests? I'm seeing similar performance issues using Camel and am
>> tempted to move all my JBI services into Camel components to eliminate as
>> much overhead as possible.
>>
>> If I do this, then why do I need SMX at all. Camel supports all the
>> binding components (activemq, http, file, etc) that SMX does and seems to
>> be add more flexibility with routing and exception handling.
>>
>> What is SMX providing otherwise? I'm starting to feel that Camel and SMX
>> are competing technologies that happen to work together (using
>> servicemix-camel binding). Can someone please elaborate on why I need SMX
>> if Camel components seem to do everything...
>>
>> thanks in advance...
>>
>>
>> Ashwin Karpe wrote:
>>
>>> Hi,
>>>
>>> Based on your example, the following will happen in JBI.
>>>
>>> 1> Camel SA1 invoking the JBI service myService will cause the following
>>> additional work to take place.It will cause the running thread in the
>>> servicemix-camel component to create an In-only Message exchange, copy
>>> the message from the processor to the exchange. The exchange will be sent
>>> to the NMR to locate and forward to myService.
>>> 2> Now myService though in another SA is also a servicemix-camel
>>> endpoint registered with the NMR and will cause the exchange to be picked
>>> up by another servicemix-camel thread and processed. The payload will be
>>> pulled out of the JBI Message exchange and moved into a Camel Exchange
>>> that is forwarded to the second processor. The JBI Message exchange is
>>> now set to Done and pushed back and sent to the original thread for
>>> Camel-SA1 for further processing and cleanup (due to JBI MEP processing
>>> rules).
>>> 3> The exchnage cleanup has to be done by the servicemix container after
>>> the servicemix-camel thread SA1 does nothing further with the exchange.
>>>
>>> The trouble you run into when it is all running in the same
>>> servicemix-camel container is that the code becomes re-entrant, more and
>>> more time is spent in exchange creation and management within
>>> servicemix-camel, exchanges start to pile up pretty quickly, thereby
>>> driving down the overall processing speed. Also, this activity chews up
>>> threads pretty quickly in a high volume scenario.
>>>
>>> Note that this does not happen in the pure JMS situation. The
>>> servicemix-camel component does not have to create any fresh JBI
>>> exchanges and manage them since it has no NMR involved. The camel routing
>>> scenario can entirely executed in a running thread that can grab a
>>> message from the Broker managing the Topic, process and and send it to
>>> another Topic. Another thread then picks up from the second Topic and
>>> processes the message. There is no need to create and manage JBI
>>> exchanges. I guess the difference is the difference between a routing and
>>> a bridging scenario. Hence the much better performance.
>>>
>>> Hope this helps. The only thing that surprised me was how much the cost
>>> differential was. You might try to increase the number of threads for the
>>> Camel component via configuration (servicemix.properties in the conf
>>> directory) and see if that makes a difference. I bet increasing the
>>> thread pool will improve performance, however the performance of the
>>> pure-JMS solution should improve as well :).
>>>
>>> Cheers,
>>>
>>> Ashwin...
>>>
>>>
>>>
>>
>
>
> -----
> ---
> Ashwin Karpe, Principal Consultant, PS - Opensource Center of Competence
> Progress Software Corporation
> 14 Oak Park Drive
> Bedford, MA 01730
> ---
> +1-972-304-9084 (Office)
> +1-972-971-1700 (Mobile)
> ----
> Blog: http://opensourceknowledge.blogspot.com/
>
>
>
Re: JBI performance
Posted by Ashwin Karpe <as...@progress.com>.
Hi,
Camel and Servicemix are both fundamentally about applying Enterprise
Integration patterns to solve complex integration problems across different
technology sets.
However, they do this by employing different styles and approaches. The
broad distinctions between Camel and Servicemix JBI technology stack, in my
view, are the following
- Camel is standardized via DSL (Domain Specific Language),
Servicemix on the other hand is an implementation of the JBI standard
- Both use message exchanges internally to move payloads between
endpoints using routes (Camel) and workflows (JBI). However, in Camel, a
MessageExchange is an Interface which is implemented by different technology
sets in their own way i.e HTTP Exchange, FTP Exchange etc and employs type
conversion to normalize data between these message exchanges. In Servicemix,
a Message Exchange is a standardized Data structure with XML as its message
payload.
- Messages in Camel may be of any type (binary, text, serialized
object etc). Messages in Servicemix are XML only.
- Camel utilizes URI's to express endpoints, Servicemix uses the
WSDL conventions (ServiceEndpointReference) to express endpoints.
- Camel has a broader palette due to a simpler/more efficient and
expressive way of wiring endpoints using routes. It is more easily
embeddable in traditional Java Apps and can be run from a command line.
Servicemix on the other hand is fundamentally based on a framework (JBI) and
its services/endpoints are deployment in a JBI/OSGi based container.
- The core JBI framework provides its endpoints key infrastructure
services/facilities out of the box (location transarency, mediated messaging
via NMR) and is more suited for viewing as a traditional ESB. Camel is more
suited for being viewed as an easy to use, yet sophisticated piece of DSL
based integration technology.
BTW, Servicemix 4 (OSGi) blends/brings together both these technologies
beautifully. Servicemix-Camel and Camel can seamlessly coexist and provide
seamless navigation between JBI and Camel routes.
I like both for their own quirks, elegance & oddities. To each his own, I
guess :).
Cheers,
Ashwin...
boday wrote:
>
> Ashwin...
>
> So you are basically saying that in high-volume scenarios, Camel shouldn't
> use JBI components because there is a higher overhead associated with
> these requests? I'm seeing similar performance issues using Camel and am
> tempted to move all my JBI services into Camel components to eliminate as
> much overhead as possible.
>
> If I do this, then why do I need SMX at all. Camel supports all the
> binding components (activemq, http, file, etc) that SMX does and seems to
> be add more flexibility with routing and exception handling.
>
> What is SMX providing otherwise? I'm starting to feel that Camel and SMX
> are competing technologies that happen to work together (using
> servicemix-camel binding). Can someone please elaborate on why I need SMX
> if Camel components seem to do everything...
>
> thanks in advance...
>
>
> Ashwin Karpe wrote:
>>
>> Hi,
>>
>> Based on your example, the following will happen in JBI.
>>
>> 1> Camel SA1 invoking the JBI service myService will cause the following
>> additional work to take place.It will cause the running thread in the
>> servicemix-camel component to create an In-only Message exchange, copy
>> the message from the processor to the exchange. The exchange will be sent
>> to the NMR to locate and forward to myService.
>> 2> Now myService though in another SA is also a servicemix-camel
>> endpoint registered with the NMR and will cause the exchange to be picked
>> up by another servicemix-camel thread and processed. The payload will be
>> pulled out of the JBI Message exchange and moved into a Camel Exchange
>> that is forwarded to the second processor. The JBI Message exchange is
>> now set to Done and pushed back and sent to the original thread for
>> Camel-SA1 for further processing and cleanup (due to JBI MEP processing
>> rules).
>> 3> The exchnage cleanup has to be done by the servicemix container after
>> the servicemix-camel thread SA1 does nothing further with the exchange.
>>
>> The trouble you run into when it is all running in the same
>> servicemix-camel container is that the code becomes re-entrant, more and
>> more time is spent in exchange creation and management within
>> servicemix-camel, exchanges start to pile up pretty quickly, thereby
>> driving down the overall processing speed. Also, this activity chews up
>> threads pretty quickly in a high volume scenario.
>>
>> Note that this does not happen in the pure JMS situation. The
>> servicemix-camel component does not have to create any fresh JBI
>> exchanges and manage them since it has no NMR involved. The camel routing
>> scenario can entirely executed in a running thread that can grab a
>> message from the Broker managing the Topic, process and and send it to
>> another Topic. Another thread then picks up from the second Topic and
>> processes the message. There is no need to create and manage JBI
>> exchanges. I guess the difference is the difference between a routing and
>> a bridging scenario. Hence the much better performance.
>>
>> Hope this helps. The only thing that surprised me was how much the cost
>> differential was. You might try to increase the number of threads for the
>> Camel component via configuration (servicemix.properties in the conf
>> directory) and see if that makes a difference. I bet increasing the
>> thread pool will improve performance, however the performance of the
>> pure-JMS solution should improve as well :).
>>
>> Cheers,
>>
>> Ashwin...
>>
>>
>
>
-----
---
Ashwin Karpe, Principal Consultant, PS - Opensource Center of Competence
Progress Software Corporation
14 Oak Park Drive
Bedford, MA 01730
---
+1-972-304-9084 (Office)
+1-972-971-1700 (Mobile)
----
Blog: http://opensourceknowledge.blogspot.com/
--
View this message in context: http://www.nabble.com/JBI-performance-tp21615804p24638358.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.