You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@servicemix.apache.org by Marek Psiuk <ro...@gmail.com> on 2008/04/26 16:13:56 UTC
JBI Profiling Tool
Dear ServiceMix Users,
Together with my friend I'm working on a subject of monitoring a JBI
compliant ESB. It is actually a subject of out Master Thesis :>. So we
were focusing on ServiceMix and OpenESB implementations. At the
beginning we were total noobs ;) (maybe we still are) but we cracked
through the JBI specification and analyzed how things look in
ServiceMix and OpenESB.
After some time of research we have decided to create a solution that
would be 100% (or close to that :>) JBI compliant and would not assume
anything about specific mechanisms (either SMix or OpenESB)
[Unfortunately that was not truly possible :|. For example SMix has
this not JBI compliant 'processInbound' method which is often used in
DeliveryChannelImpl]. We decided to go AOP way. We created an sniffy
aspect which intercepts everything that goes through JBI's
DeliveryChannel implementations. It turned out that it was quite easy.
We went a step further and provided a mechanism of correlation
MessageExchanges that are related - in fact it is very similiar to
ServiceMix's correlationId (we examined ThreadLocal's oriented
implementaion of it).
So at the end of the day we have a mechanism that without a slight
code modification of JBI Container itself is able to intercept all
MessageExchanges flying through the components. At the beginning of
our work the assumption was that our solution should be inline with
the idea of Application Response Measurement
(http://www.opengroup.org/arm). We studied the ARM specification and
decided that it is far to complicated for our purposes but we can use
the concepts invented there !! An Idea of profiling tool for JBI came
up !!
We were wondering how the JBI profiling session could look like and
what would be most useful ?? We thought that it would be interesting
if at the end our GUI would be able to draw something like Sequence
Diagram (http://upload.wikimedia.org/wikipedia/en/3/32/CheckEmail.png).
The entities that would be on such diagrams could be specific
endpoints in componets(BC and SE). Each arrow would be connected with
some ME exchange and would be described by the time that elapsed and
maybe some other information (we are still working on it).
So why I am telling You all this things. We decided that in scope of
our MasterT we want to make same useful tool that could be really
handy for both Developers and Users of JBI Containers. My question is:
Could such JBI Profiling Tool be useful ?
I am aware of ServiceMix's visualizations: dotViewEndpointListener and
dotViewFlowListener (they are based on MessageExchangeListeners).
Those were not an options for us because we wanted JBI compliance. But
I am aware that functionalities of our mechanism would overlap with
those dotViewThingies
So what do You think about it ?
Thanks in advace,
Kind Regards,
Marek Psiuk
Re: JBI Profiling Tool
Posted by rotgier <ro...@gmail.com>.
Yeah we had a lot of problems while trying to correlate MEs flying through
Apache Ode.
At first time I have noticed response to my question in this thread:
http://cwiki.apache.org/SM/discussion-forums.html#nabble-to16908206%7Ca17293632
so I given some details of our implementation there.
I did not know to which list I should post my question so I posted to both
User and Dev. Should we focus on discussing the JBI Profiling Idea on one of
those list ? Or is it ok ?
As I have explained on Dev list we are tracing not only the ME accessed in
threads but also NormalizedMessages which are related to MEs that we are
interested in. Such analysis allows to implement CorrelationId in OpenESB
(which does not have it at all !!) and in most of ServiceMix components.
However it is just an imperfect heuristic :(. Maybe You have some idea how
to make it better ?
Kind Regards,
Marek Psiuk
gnodet wrote:
>
> I'm wondering a bit about your thread local aop processing.
> I don't really understand how this is possible in pure JBI way: as a
> example, i've tried to implement the correlationId for Apache Ode JBI
> component, but in this case, the thread local bit can not be used
> because the internal engine uses its own thread pool to process
> requests. Even worst, sending a JBI exchange as a result of the
> processing of an incoming exchange is not performed in the same
> thread, so all thread based correlation does not work (afaik).
> How do you make that work ?
>
> On Sat, Apr 26, 2008 at 4:13 PM, Marek Psiuk <ro...@gmail.com> wrote:
>> Dear ServiceMix Users,
>>
>>
>> Together with my friend I'm working on a subject of monitoring a JBI
>> compliant ESB. It is actually a subject of out Master Thesis :>. So we
>> were focusing on ServiceMix and OpenESB implementations. At the
>> beginning we were total noobs ;) (maybe we still are) but we cracked
>> through the JBI specification and analyzed how things look in
>> ServiceMix and OpenESB.
>> After some time of research we have decided to create a solution that
>> would be 100% (or close to that :>) JBI compliant and would not assume
>> anything about specific mechanisms (either SMix or OpenESB)
>> [Unfortunately that was not truly possible :|. For example SMix has
>> this not JBI compliant 'processInbound' method which is often used in
>> DeliveryChannelImpl]. We decided to go AOP way. We created an sniffy
>> aspect which intercepts everything that goes through JBI's
>> DeliveryChannel implementations. It turned out that it was quite easy.
>> We went a step further and provided a mechanism of correlation
>> MessageExchanges that are related - in fact it is very similiar to
>> ServiceMix's correlationId (we examined ThreadLocal's oriented
>> implementaion of it).
>> So at the end of the day we have a mechanism that without a slight
>> code modification of JBI Container itself is able to intercept all
>> MessageExchanges flying through the components. At the beginning of
>> our work the assumption was that our solution should be inline with
>> the idea of Application Response Measurement
>> (http://www.opengroup.org/arm). We studied the ARM specification and
>> decided that it is far to complicated for our purposes but we can use
>> the concepts invented there !! An Idea of profiling tool for JBI came
>> up !!
>>
>> We were wondering how the JBI profiling session could look like and
>> what would be most useful ?? We thought that it would be interesting
>> if at the end our GUI would be able to draw something like Sequence
>> Diagram (http://upload.wikimedia.org/wikipedia/en/3/32/CheckEmail.png).
>> The entities that would be on such diagrams could be specific
>> endpoints in componets(BC and SE). Each arrow would be connected with
>> some ME exchange and would be described by the time that elapsed and
>> maybe some other information (we are still working on it).
>>
>>
>> So why I am telling You all this things. We decided that in scope of
>> our MasterT we want to make same useful tool that could be really
>> handy for both Developers and Users of JBI Containers. My question is:
>> Could such JBI Profiling Tool be useful ?
>>
>> I am aware of ServiceMix's visualizations: dotViewEndpointListener and
>> dotViewFlowListener (they are based on MessageExchangeListeners).
>> Those were not an options for us because we wanted JBI compliance. But
>> I am aware that functionalities of our mechanism would overlap with
>> those dotViewThingies
>>
>>
>> So what do You think about it ?
>>
>>
>> Thanks in advace,
>>
>>
>> Kind Regards,
>>
>> Marek Psiuk
>>
>
>
>
> --
> Cheers,
> Guillaume Nodet
> ------------------------
> Blog: http://gnodet.blogspot.com/
>
>
--
View this message in context: http://www.nabble.com/JBI-Profiling-Tool-tp16913351p17328941.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.
Re: JBI Profiling Tool
Posted by Guillaume Nodet <gn...@gmail.com>.
I'm wondering a bit about your thread local aop processing.
I don't really understand how this is possible in pure JBI way: as a
example, i've tried to implement the correlationId for Apache Ode JBI
component, but in this case, the thread local bit can not be used
because the internal engine uses its own thread pool to process
requests. Even worst, sending a JBI exchange as a result of the
processing of an incoming exchange is not performed in the same
thread, so all thread based correlation does not work (afaik).
How do you make that work ?
On Sat, Apr 26, 2008 at 4:13 PM, Marek Psiuk <ro...@gmail.com> wrote:
> Dear ServiceMix Users,
>
>
> Together with my friend I'm working on a subject of monitoring a JBI
> compliant ESB. It is actually a subject of out Master Thesis :>. So we
> were focusing on ServiceMix and OpenESB implementations. At the
> beginning we were total noobs ;) (maybe we still are) but we cracked
> through the JBI specification and analyzed how things look in
> ServiceMix and OpenESB.
> After some time of research we have decided to create a solution that
> would be 100% (or close to that :>) JBI compliant and would not assume
> anything about specific mechanisms (either SMix or OpenESB)
> [Unfortunately that was not truly possible :|. For example SMix has
> this not JBI compliant 'processInbound' method which is often used in
> DeliveryChannelImpl]. We decided to go AOP way. We created an sniffy
> aspect which intercepts everything that goes through JBI's
> DeliveryChannel implementations. It turned out that it was quite easy.
> We went a step further and provided a mechanism of correlation
> MessageExchanges that are related - in fact it is very similiar to
> ServiceMix's correlationId (we examined ThreadLocal's oriented
> implementaion of it).
> So at the end of the day we have a mechanism that without a slight
> code modification of JBI Container itself is able to intercept all
> MessageExchanges flying through the components. At the beginning of
> our work the assumption was that our solution should be inline with
> the idea of Application Response Measurement
> (http://www.opengroup.org/arm). We studied the ARM specification and
> decided that it is far to complicated for our purposes but we can use
> the concepts invented there !! An Idea of profiling tool for JBI came
> up !!
>
> We were wondering how the JBI profiling session could look like and
> what would be most useful ?? We thought that it would be interesting
> if at the end our GUI would be able to draw something like Sequence
> Diagram (http://upload.wikimedia.org/wikipedia/en/3/32/CheckEmail.png).
> The entities that would be on such diagrams could be specific
> endpoints in componets(BC and SE). Each arrow would be connected with
> some ME exchange and would be described by the time that elapsed and
> maybe some other information (we are still working on it).
>
>
> So why I am telling You all this things. We decided that in scope of
> our MasterT we want to make same useful tool that could be really
> handy for both Developers and Users of JBI Containers. My question is:
> Could such JBI Profiling Tool be useful ?
>
> I am aware of ServiceMix's visualizations: dotViewEndpointListener and
> dotViewFlowListener (they are based on MessageExchangeListeners).
> Those were not an options for us because we wanted JBI compliance. But
> I am aware that functionalities of our mechanism would overlap with
> those dotViewThingies
>
>
> So what do You think about it ?
>
>
> Thanks in advace,
>
>
> Kind Regards,
>
> Marek Psiuk
>
--
Cheers,
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
Re: JBI Profiling Tool
Posted by rotgier <ro...@gmail.com>.
bsnyder wrote:
>
> On Sat, Apr 26, 2008 at 8:13 AM, Marek Psiuk <ro...@gmail.com> wrote:
>> Dear ServiceMix Users,
>>
>>
>> Together with my friend I'm working on a subject of monitoring a JBI
>> compliant ESB. It is actually a subject of out Master Thesis :>. So we
>> were focusing on ServiceMix and OpenESB implementations. At the
>> beginning we were total noobs ;) (maybe we still are) but we cracked
>> through the JBI specification and analyzed how things look in
>> ServiceMix and OpenESB.
>> After some time of research we have decided to create a solution that
>> would be 100% (or close to that :>) JBI compliant and would not assume
>> anything about specific mechanisms (either SMix or OpenESB)
>> [Unfortunately that was not truly possible :|. For example SMix has
>> this not JBI compliant 'processInbound' method which is often used in
>> DeliveryChannelImpl]. We decided to go AOP way. We created an sniffy
>> aspect which intercepts everything that goes through JBI's
>> DeliveryChannel implementations. It turned out that it was quite easy.
>> We went a step further and provided a mechanism of correlation
>> MessageExchanges that are related - in fact it is very similiar to
>> ServiceMix's correlationId (we examined ThreadLocal's oriented
>> implementaion of it).
>> So at the end of the day we have a mechanism that without a slight
>> code modification of JBI Container itself is able to intercept all
>> MessageExchanges flying through the components. At the beginning of
>> our work the assumption was that our solution should be inline with
>> the idea of Application Response Measurement
>> (http://www.opengroup.org/arm). We studied the ARM specification and
>> decided that it is far to complicated for our purposes but we can use
>> the concepts invented there !! An Idea of profiling tool for JBI came
>> up !!
>>
>> We were wondering how the JBI profiling session could look like and
>> what would be most useful ?? We thought that it would be interesting
>> if at the end our GUI would be able to draw something like Sequence
>> Diagram (http://upload.wikimedia.org/wikipedia/en/3/32/CheckEmail.png).
>> The entities that would be on such diagrams could be specific
>> endpoints in componets(BC and SE). Each arrow would be connected with
>> some ME exchange and would be described by the time that elapsed and
>> maybe some other information (we are still working on it).
>>
>>
>> So why I am telling You all this things. We decided that in scope of
>> our MasterT we want to make same useful tool that could be really
>> handy for both Developers and Users of JBI Containers. My question is:
>> Could such JBI Profiling Tool be useful ?
>>
>> I am aware of ServiceMix's visualizations: dotViewEndpointListener and
>> dotViewFlowListener (they are based on MessageExchangeListeners).
>> Those were not an options for us because we wanted JBI compliance. But
>> I am aware that functionalities of our mechanism would overlap with
>> those dotViewThingies
>>
>>
>> So what do You think about it ?
>
> This is an interesting idea. I have built some stuff around a similar
> idea in the past for ServiceMix. Take a look at the servicemix-audit
> component. I wasn't ever concerned with JBI compliance though, so I
> just used a MessageExchangeListener.
>
> IMO, you should to develop this in stages. Get the most important
> items working correctly before moving on to the nice-to-have type of
> features like the diagram stuff.
>
> Bruce
>
>
Thank You for your response Bruce.
I took a look into the code of servicemix-audit. Seems to be quite useful
mechanism for persisting all the MessageExchanges (I found JDBC and Lucene
implementations) and exposing them by JMX (various queries included) !!
As You saing I found that You were implementing it as an ExchangeListener.
Yeah it is not JBI compliant but we Use AOP to directly pointcut
DeliveryChannel methods just to achieve the same result as You did (having
exchangeSent and exchangeAccepted called when ME is passing by ;)... oo
actually I see that You do not use the exchangeAccepted entrypoint [I have
the source code from 3.1.2 release]).
We were wondering If a direct storing in DB (using for example JDBC) will
not slow down the process of sending and accepting ME. I do not know exactly
how are the MessageListeners implemented but I suppose that You have proffed
them against situation when some listener process the exchangeSent or
exchangeAccepted for some long time period (but in the source of
servicemix-core I recall some regular for loop which just loops through all
the Listeners and executes notification method)?
When we are pointcuting the DeliveryMethod so we are directly interfering
when sending and recieving threads so we are worrying about the performance
:(. To do not block the thread we have decided to send necessary data to the
other processing Engine. We chose regular TCP socket to make things faster.
What do You think about such decision ? We tried to evaluate JMS for this
purpose but it turned out to be a lot slower !!
Regards,
Marek Psiuk
P.S.
I have put some more details about our idea in this thread:
http://cwiki.apache.org/SM/discussion-forums.html#nabble-to16908206%7Ca17293632
--
View this message in context: http://www.nabble.com/JBI-Profiling-Tool-tp16913351p17329248.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.
Re: JBI Profiling Tool
Posted by Bruce Snyder <br...@gmail.com>.
On Sat, Apr 26, 2008 at 8:13 AM, Marek Psiuk <ro...@gmail.com> wrote:
> Dear ServiceMix Users,
>
>
> Together with my friend I'm working on a subject of monitoring a JBI
> compliant ESB. It is actually a subject of out Master Thesis :>. So we
> were focusing on ServiceMix and OpenESB implementations. At the
> beginning we were total noobs ;) (maybe we still are) but we cracked
> through the JBI specification and analyzed how things look in
> ServiceMix and OpenESB.
> After some time of research we have decided to create a solution that
> would be 100% (or close to that :>) JBI compliant and would not assume
> anything about specific mechanisms (either SMix or OpenESB)
> [Unfortunately that was not truly possible :|. For example SMix has
> this not JBI compliant 'processInbound' method which is often used in
> DeliveryChannelImpl]. We decided to go AOP way. We created an sniffy
> aspect which intercepts everything that goes through JBI's
> DeliveryChannel implementations. It turned out that it was quite easy.
> We went a step further and provided a mechanism of correlation
> MessageExchanges that are related - in fact it is very similiar to
> ServiceMix's correlationId (we examined ThreadLocal's oriented
> implementaion of it).
> So at the end of the day we have a mechanism that without a slight
> code modification of JBI Container itself is able to intercept all
> MessageExchanges flying through the components. At the beginning of
> our work the assumption was that our solution should be inline with
> the idea of Application Response Measurement
> (http://www.opengroup.org/arm). We studied the ARM specification and
> decided that it is far to complicated for our purposes but we can use
> the concepts invented there !! An Idea of profiling tool for JBI came
> up !!
>
> We were wondering how the JBI profiling session could look like and
> what would be most useful ?? We thought that it would be interesting
> if at the end our GUI would be able to draw something like Sequence
> Diagram (http://upload.wikimedia.org/wikipedia/en/3/32/CheckEmail.png).
> The entities that would be on such diagrams could be specific
> endpoints in componets(BC and SE). Each arrow would be connected with
> some ME exchange and would be described by the time that elapsed and
> maybe some other information (we are still working on it).
>
>
> So why I am telling You all this things. We decided that in scope of
> our MasterT we want to make same useful tool that could be really
> handy for both Developers and Users of JBI Containers. My question is:
> Could such JBI Profiling Tool be useful ?
>
> I am aware of ServiceMix's visualizations: dotViewEndpointListener and
> dotViewFlowListener (they are based on MessageExchangeListeners).
> Those were not an options for us because we wanted JBI compliance. But
> I am aware that functionalities of our mechanism would overlap with
> those dotViewThingies
>
>
> So what do You think about it ?
This is an interesting idea. I have built some stuff around a similar
idea in the past for ServiceMix. Take a look at the servicemix-audit
component. I wasn't ever concerned with JBI compliance though, so I
just used a MessageExchangeListener.
IMO, you should to develop this in stages. Get the most important
items working correctly before moving on to the nice-to-have type of
features like the diagram stuff.
Bruce
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'
Apache ActiveMQ - http://activemq.org/
Apache Camel - http://activemq.org/camel/
Apache ServiceMix - http://servicemix.org/
Apache Geronimo - http://geronimo.apache.org/
Blog: http://bruceblog.org/