You are viewing a plain text version of this content. The canonical link for it is here.

Posted to kandula-dev@ws.apache.org by nilupa bandara <ni...@gmail.com> on 2010/03/24 23:03:39 UTC

[GSoC Project Proposal] Distributed TCP Monitor

Hello,

I am graduate student at Politecnica de Madrid and I am thinking of
proposing a GSoC project this summer. I would like to take project
"Distribute TCP Monitor" descried[1] which is very interesting and
also should be helpful to any Java developer using Apache Axis2 Web
services middleware. I will submit more detailed proposal later. I
would really like to hear any feedback from you. Any suggestions or
features that you would like to see in a "Distributed TCP Monitor" are
most welcome.

Best Regards,
Nilupa Bandara

[1]

Distributed TCP monitor

To use TCP monitor we have to tweak the endpoints, which is bit hard
sometimes (e.g. two channels Async case for Axis2). We can solve the
problem by writing a Axis2 module (Handler) that intercept messages
comes in and goes out of a Axis2 server and send to a UI (via a
pub/sub channel or a message Box e.g. ) or record them so user can
pull those messages. Also, we can do something to turn the module on
and off remotely, so users can have his module deployed, but turn it
on only when they want to debug the system.

Then we can take this to next level by adding this module to all
services in a system , configuring modules to send all collected
messages to a pub/sub channel, and subscribing to those messages via a
UI, and depicting those messages through a UI. Then users have a
TCPMon for a whole system.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Thu, Apr 1, 2010 at 4:47 AM, nilupa bandara <ni...@gmail.com> wrote:

> On Wed, Mar 31, 2010 at 4:51 PM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> >> Then we can take this to next level by adding this module to all
> >> services in a system , configuring modules to send all collected
> >> messages to a pub/sub channel, and subscribing to those messages via a
> >> UI, and depicting those messages through a UI.
> >
> > What pub/sub model do you plan to use?  WS-Eventing?
>
> Something which allows the publisher ( or the Log service which
> collects log messages from different nodes) to notify its subscribers
> (or the clients which show the topology of particular message
> sequence) the arrival of a message which belongs to a particular
> sequence of messages that they are interested in. Is far as I
> understood this notification will be based on SOAP. WS-Eventing sounds
> like something that can be used but I guess I have to explore more
> (rather than reading some introductory notes on WS-Eventing) to give a
> concrete answer.
>

If you are using eventing, you might want to look at what we have done in
BAM.  We basically use an Axis2 module with Axis2 based eventing API to
pub/sub.

Since this is an Axis2 module, you can use the code as it is. We do not have
any other dependency.

Samisa....

>
>
> Best Regards,
> Nilupa
>
> >
> > Samisa...
> > --
> > blog: http://samisa-abeysinghe.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Thu, Apr 1, 2010 at 4:47 AM, nilupa bandara <ni...@gmail.com> wrote:

> On Wed, Mar 31, 2010 at 4:51 PM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> >> Then we can take this to next level by adding this module to all
> >> services in a system , configuring modules to send all collected
> >> messages to a pub/sub channel, and subscribing to those messages via a
> >> UI, and depicting those messages through a UI.
> >
> > What pub/sub model do you plan to use?  WS-Eventing?
>
> Something which allows the publisher ( or the Log service which
> collects log messages from different nodes) to notify its subscribers
> (or the clients which show the topology of particular message
> sequence) the arrival of a message which belongs to a particular
> sequence of messages that they are interested in. Is far as I
> understood this notification will be based on SOAP. WS-Eventing sounds
> like something that can be used but I guess I have to explore more
> (rather than reading some introductory notes on WS-Eventing) to give a
> concrete answer.
>

If you are using eventing, you might want to look at what we have done in
BAM.  We basically use an Axis2 module with Axis2 based eventing API to
pub/sub.

Since this is an Axis2 module, you can use the code as it is. We do not have
any other dependency.

Samisa....

>
>
> Best Regards,
> Nilupa
>
> >
> > Samisa...
> > --
> > blog: http://samisa-abeysinghe.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Thu, Apr 1, 2010 at 4:47 AM, nilupa bandara <ni...@gmail.com> wrote:

> On Wed, Mar 31, 2010 at 4:51 PM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> >> Then we can take this to next level by adding this module to all
> >> services in a system , configuring modules to send all collected
> >> messages to a pub/sub channel, and subscribing to those messages via a
> >> UI, and depicting those messages through a UI.
> >
> > What pub/sub model do you plan to use?  WS-Eventing?
>
> Something which allows the publisher ( or the Log service which
> collects log messages from different nodes) to notify its subscribers
> (or the clients which show the topology of particular message
> sequence) the arrival of a message which belongs to a particular
> sequence of messages that they are interested in. Is far as I
> understood this notification will be based on SOAP. WS-Eventing sounds
> like something that can be used but I guess I have to explore more
> (rather than reading some introductory notes on WS-Eventing) to give a
> concrete answer.
>

If you are using eventing, you might want to look at what we have done in
BAM.  We basically use an Axis2 module with Axis2 based eventing API to
pub/sub.

Since this is an Axis2 module, you can use the code as it is. We do not have
any other dependency.

Samisa....

>
>
> Best Regards,
> Nilupa
>
> >
> > Samisa...
> > --
> > blog: http://samisa-abeysinghe.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Thu, Apr 1, 2010 at 4:47 AM, nilupa bandara <ni...@gmail.com> wrote:

> On Wed, Mar 31, 2010 at 4:51 PM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> >> Then we can take this to next level by adding this module to all
> >> services in a system , configuring modules to send all collected
> >> messages to a pub/sub channel, and subscribing to those messages via a
> >> UI, and depicting those messages through a UI.
> >
> > What pub/sub model do you plan to use?  WS-Eventing?
>
> Something which allows the publisher ( or the Log service which
> collects log messages from different nodes) to notify its subscribers
> (or the clients which show the topology of particular message
> sequence) the arrival of a message which belongs to a particular
> sequence of messages that they are interested in. Is far as I
> understood this notification will be based on SOAP. WS-Eventing sounds
> like something that can be used but I guess I have to explore more
> (rather than reading some introductory notes on WS-Eventing) to give a
> concrete answer.
>

If you are using eventing, you might want to look at what we have done in
BAM.  We basically use an Axis2 module with Axis2 based eventing API to
pub/sub.

Since this is an Axis2 module, you can use the code as it is. We do not have
any other dependency.

Samisa....

>
>
> Best Regards,
> Nilupa
>
> >
> > Samisa...
> > --
> > blog: http://samisa-abeysinghe.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Thu, Apr 1, 2010 at 4:47 AM, nilupa bandara <ni...@gmail.com> wrote:

> On Wed, Mar 31, 2010 at 4:51 PM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> >> Then we can take this to next level by adding this module to all
> >> services in a system , configuring modules to send all collected
> >> messages to a pub/sub channel, and subscribing to those messages via a
> >> UI, and depicting those messages through a UI.
> >
> > What pub/sub model do you plan to use?  WS-Eventing?
>
> Something which allows the publisher ( or the Log service which
> collects log messages from different nodes) to notify its subscribers
> (or the clients which show the topology of particular message
> sequence) the arrival of a message which belongs to a particular
> sequence of messages that they are interested in. Is far as I
> understood this notification will be based on SOAP. WS-Eventing sounds
> like something that can be used but I guess I have to explore more
> (rather than reading some introductory notes on WS-Eventing) to give a
> concrete answer.
>

If you are using eventing, you might want to look at what we have done in
BAM.  We basically use an Axis2 module with Axis2 based eventing API to
pub/sub.

Since this is an Axis2 module, you can use the code as it is. We do not have
any other dependency.

Samisa....

>
>
> Best Regards,
> Nilupa
>
> >
> > Samisa...
> > --
> > blog: http://samisa-abeysinghe.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

On Wed, Mar 31, 2010 at 4:51 PM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI.
>
> What pub/sub model do you plan to use?  WS-Eventing?

Something which allows the publisher ( or the Log service which
collects log messages from different nodes) to notify its subscribers
(or the clients which show the topology of particular message
sequence) the arrival of a message which belongs to a particular
sequence of messages that they are interested in. Is far as I
understood this notification will be based on SOAP. WS-Eventing sounds
like something that can be used but I guess I have to explore more
(rather than reading some introductory notes on WS-Eventing) to give a
concrete answer.

Best Regards,
Nilupa

>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

On Wed, Apr 7, 2010 at 2:41 PM, Amila Suriarachchi
<am...@gmail.com> wrote:
>
>
> On Mon, Apr 5, 2010 at 5:27 AM, nilupa bandara <ni...@gmail.com> wrote:
>>
>> Hello Everyone,
>>
>> Let me thank all of you once again for your valuable inputs.
>>
>> Based on the feedback I've got on initial project idea, I've come up
>> with an updated version of the project proposal which is listed
>> below[1],[2].
>
> Can you please add an diagram which shows the deployment architecture of the
> system. This make it more
> clear about your work.

I've just updated the proposal with design diagram of the system.
Could you please check whether it is reasonable ?

Best Regards,
Nilupa


>
> thanks,
> Amila.
>>
>> Best Regards,
>> Nilupa
>>
>> [1] http://wiki.apache.org/general/nilupa/gsoc2010/distributed_tcpmon
>> [2]
>> http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/nilupa/t127042423893
>>
>>
>> On Fri, Apr 2, 2010 at 5:01 AM, Samisa Abeysinghe
>> <sa...@gmail.com> wrote:
>> >
>> >
>> > On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe
>> > <sa...@gmail.com> wrote:
>> >>
>> >>
>> >> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com>
>> >> wrote:
>> >>>
>> >>> Hi nilupa;
>> >>>
>> >>> IMO this thread has more than one concept. Initially, the version I
>> >>> discussed with you is more of a message monitoring tool, which allow
>> >>> an user to see which message goes through where in the system. It is
>> >>> purely used for debugging services only, and a actually it distributed
>> >>> TCP monitor that helps people debug their distributed application.
>> >>> Features I had in my mind was to
>> >>>
>> >>> 1. Ability to turn on/off message interception
>> >>> 2. When tuned on, it show the system in a UI and let me see all
>> >>> messages went through each link I selected.
>> >>> 3. If messages have a some ID that identify a message sequence, it
>> >>> should let me select a message and find sequence that generated/ lead
>> >>> to that message.
>> >>>
>> >>> Basically, as I mentioned before, it is debugging tool and not a
>> >>> monitoring tool.
>> >>
>> >> If it is a debugging tool, that is going to be useful.
>> >
>> > And I think, it would be useful to list out the features it supports.
>> > I would also personally prefer some additions to the existing TCPMon
>> > where
>> > we can add plain text display of secure messages. However, I know that
>> > it is
>> > complicated, as the middle man is not supposed to change the ttl and the
>> > like in WS Security.
>> > Samisa...
>> >>
>> >> Samisa...
>> >>
>> >>>
>> >>> Then, there is a BAM/System monitoring aspect we heavily discussed in
>> >>> this thread, which is used to monitor a running system. IMHO they are
>> >>> different,yet very useful. However, they are targeted at two audiences
>> >>> and you should not do half of each.
>> >>>
>> >>> Personlly, I am interested in both topics, and willing to
>> >>> mentor/co-mentor either, however, IMO it two projects, not one.
>> >>>
>> >>> Thanks
>> >>> Srinath
>> >>>
>> >>>
>> >>>
>> >>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
>> >>> <an...@gmail.com> wrote:
>> >>> > Probably the right answer to this question is: none. Why would the
>> >>> > individual nodes intercepting the messages use publish/subscribe?
>> >>> > They
>> >>> > should all send them to a central place (or back to the sender of
>> >>> > the
>> >>> > original request).
>> >>> >
>> >>> > Andreas
>> >>> >
>> >>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
>> >>> > <sa...@gmail.com> wrote:
>> >>> >>> Then we can take this to next level by adding this module to all
>> >>> >>> services in a system , configuring modules to send all collected
>> >>> >>> messages to a pub/sub channel, and subscribing to those messages
>> >>> >>> via
>> >>> >>> a
>> >>> >>> UI, and depicting those messages through a UI.
>> >>> >>
>> >>> >> What pub/sub model do you plan to use? WS-Eventing?
>> >>> >>
>> >>> >> Samisa...
>> >>> >> --
>> >>> >> blog: http://samisa-abeysinghe.blogspot.com/
>> >>> >
>> >>> >
>> >>> > ---------------------------------------------------------------------
>> >>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>> >
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> ============================
>> >>> Srinath Perera, Ph.D.
>> >>>   WSO2 Inc. http://wso2.com
>> >>>   Blog: http://srinathsview.blogspot.com/
>> >>>
>> >>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>
>> >> --
>> >> Samisa Abeysinghe
>> >> Director, Engineering - WSO2 Inc.
>> >>
>> >> http://wso2.com/ - "lean . enterprise . middleware"
>> >
>> > --
>> > Samisa Abeysinghe
>> > Director, Engineering - WSO2 Inc.
>> >
>> > http://wso2.com/ - "lean . enterprise . middleware"
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
>
>
> --
> Amila Suriarachchi
> WSO2 Inc.
> blog: http://amilachinthaka.blogspot.com/
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

On Wed, Apr 7, 2010 at 2:41 PM, Amila Suriarachchi
<am...@gmail.com> wrote:
>
>
> On Mon, Apr 5, 2010 at 5:27 AM, nilupa bandara <ni...@gmail.com> wrote:
>>
>> Hello Everyone,
>>
>> Let me thank all of you once again for your valuable inputs.
>>
>> Based on the feedback I've got on initial project idea, I've come up
>> with an updated version of the project proposal which is listed
>> below[1],[2].
>
> Can you please add an diagram which shows the deployment architecture of the
> system. This make it more
> clear about your work.

I've just updated the proposal with design diagram of the system.
Could you please check whether it is reasonable ?

Best Regards,
Nilupa


>
> thanks,
> Amila.
>>
>> Best Regards,
>> Nilupa
>>
>> [1] http://wiki.apache.org/general/nilupa/gsoc2010/distributed_tcpmon
>> [2]
>> http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/nilupa/t127042423893
>>
>>
>> On Fri, Apr 2, 2010 at 5:01 AM, Samisa Abeysinghe
>> <sa...@gmail.com> wrote:
>> >
>> >
>> > On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe
>> > <sa...@gmail.com> wrote:
>> >>
>> >>
>> >> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com>
>> >> wrote:
>> >>>
>> >>> Hi nilupa;
>> >>>
>> >>> IMO this thread has more than one concept. Initially, the version I
>> >>> discussed with you is more of a message monitoring tool, which allow
>> >>> an user to see which message goes through where in the system. It is
>> >>> purely used for debugging services only, and a actually it distributed
>> >>> TCP monitor that helps people debug their distributed application.
>> >>> Features I had in my mind was to
>> >>>
>> >>> 1. Ability to turn on/off message interception
>> >>> 2. When tuned on, it show the system in a UI and let me see all
>> >>> messages went through each link I selected.
>> >>> 3. If messages have a some ID that identify a message sequence, it
>> >>> should let me select a message and find sequence that generated/ lead
>> >>> to that message.
>> >>>
>> >>> Basically, as I mentioned before, it is debugging tool and not a
>> >>> monitoring tool.
>> >>
>> >> If it is a debugging tool, that is going to be useful.
>> >
>> > And I think, it would be useful to list out the features it supports.
>> > I would also personally prefer some additions to the existing TCPMon
>> > where
>> > we can add plain text display of secure messages. However, I know that
>> > it is
>> > complicated, as the middle man is not supposed to change the ttl and the
>> > like in WS Security.
>> > Samisa...
>> >>
>> >> Samisa...
>> >>
>> >>>
>> >>> Then, there is a BAM/System monitoring aspect we heavily discussed in
>> >>> this thread, which is used to monitor a running system. IMHO they are
>> >>> different,yet very useful. However, they are targeted at two audiences
>> >>> and you should not do half of each.
>> >>>
>> >>> Personlly, I am interested in both topics, and willing to
>> >>> mentor/co-mentor either, however, IMO it two projects, not one.
>> >>>
>> >>> Thanks
>> >>> Srinath
>> >>>
>> >>>
>> >>>
>> >>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
>> >>> <an...@gmail.com> wrote:
>> >>> > Probably the right answer to this question is: none. Why would the
>> >>> > individual nodes intercepting the messages use publish/subscribe?
>> >>> > They
>> >>> > should all send them to a central place (or back to the sender of
>> >>> > the
>> >>> > original request).
>> >>> >
>> >>> > Andreas
>> >>> >
>> >>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
>> >>> > <sa...@gmail.com> wrote:
>> >>> >>> Then we can take this to next level by adding this module to all
>> >>> >>> services in a system , configuring modules to send all collected
>> >>> >>> messages to a pub/sub channel, and subscribing to those messages
>> >>> >>> via
>> >>> >>> a
>> >>> >>> UI, and depicting those messages through a UI.
>> >>> >>
>> >>> >> What pub/sub model do you plan to use? WS-Eventing?
>> >>> >>
>> >>> >> Samisa...
>> >>> >> --
>> >>> >> blog: http://samisa-abeysinghe.blogspot.com/
>> >>> >
>> >>> >
>> >>> > ---------------------------------------------------------------------
>> >>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>> >
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> ============================
>> >>> Srinath Perera, Ph.D.
>> >>>   WSO2 Inc. http://wso2.com
>> >>>   Blog: http://srinathsview.blogspot.com/
>> >>>
>> >>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>
>> >> --
>> >> Samisa Abeysinghe
>> >> Director, Engineering - WSO2 Inc.
>> >>
>> >> http://wso2.com/ - "lean . enterprise . middleware"
>> >
>> > --
>> > Samisa Abeysinghe
>> > Director, Engineering - WSO2 Inc.
>> >
>> > http://wso2.com/ - "lean . enterprise . middleware"
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
>
>
> --
> Amila Suriarachchi
> WSO2 Inc.
> blog: http://amilachinthaka.blogspot.com/
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

On Wed, Apr 7, 2010 at 2:41 PM, Amila Suriarachchi
<am...@gmail.com> wrote:
>
>
> On Mon, Apr 5, 2010 at 5:27 AM, nilupa bandara <ni...@gmail.com> wrote:
>>
>> Hello Everyone,
>>
>> Let me thank all of you once again for your valuable inputs.
>>
>> Based on the feedback I've got on initial project idea, I've come up
>> with an updated version of the project proposal which is listed
>> below[1],[2].
>
> Can you please add an diagram which shows the deployment architecture of the
> system. This make it more
> clear about your work.

I've just updated the proposal with design diagram of the system.
Could you please check whether it is reasonable ?

Best Regards,
Nilupa


>
> thanks,
> Amila.
>>
>> Best Regards,
>> Nilupa
>>
>> [1] http://wiki.apache.org/general/nilupa/gsoc2010/distributed_tcpmon
>> [2]
>> http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/nilupa/t127042423893
>>
>>
>> On Fri, Apr 2, 2010 at 5:01 AM, Samisa Abeysinghe
>> <sa...@gmail.com> wrote:
>> >
>> >
>> > On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe
>> > <sa...@gmail.com> wrote:
>> >>
>> >>
>> >> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com>
>> >> wrote:
>> >>>
>> >>> Hi nilupa;
>> >>>
>> >>> IMO this thread has more than one concept. Initially, the version I
>> >>> discussed with you is more of a message monitoring tool, which allow
>> >>> an user to see which message goes through where in the system. It is
>> >>> purely used for debugging services only, and a actually it distributed
>> >>> TCP monitor that helps people debug their distributed application.
>> >>> Features I had in my mind was to
>> >>>
>> >>> 1. Ability to turn on/off message interception
>> >>> 2. When tuned on, it show the system in a UI and let me see all
>> >>> messages went through each link I selected.
>> >>> 3. If messages have a some ID that identify a message sequence, it
>> >>> should let me select a message and find sequence that generated/ lead
>> >>> to that message.
>> >>>
>> >>> Basically, as I mentioned before, it is debugging tool and not a
>> >>> monitoring tool.
>> >>
>> >> If it is a debugging tool, that is going to be useful.
>> >
>> > And I think, it would be useful to list out the features it supports.
>> > I would also personally prefer some additions to the existing TCPMon
>> > where
>> > we can add plain text display of secure messages. However, I know that
>> > it is
>> > complicated, as the middle man is not supposed to change the ttl and the
>> > like in WS Security.
>> > Samisa...
>> >>
>> >> Samisa...
>> >>
>> >>>
>> >>> Then, there is a BAM/System monitoring aspect we heavily discussed in
>> >>> this thread, which is used to monitor a running system. IMHO they are
>> >>> different,yet very useful. However, they are targeted at two audiences
>> >>> and you should not do half of each.
>> >>>
>> >>> Personlly, I am interested in both topics, and willing to
>> >>> mentor/co-mentor either, however, IMO it two projects, not one.
>> >>>
>> >>> Thanks
>> >>> Srinath
>> >>>
>> >>>
>> >>>
>> >>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
>> >>> <an...@gmail.com> wrote:
>> >>> > Probably the right answer to this question is: none. Why would the
>> >>> > individual nodes intercepting the messages use publish/subscribe?
>> >>> > They
>> >>> > should all send them to a central place (or back to the sender of
>> >>> > the
>> >>> > original request).
>> >>> >
>> >>> > Andreas
>> >>> >
>> >>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
>> >>> > <sa...@gmail.com> wrote:
>> >>> >>> Then we can take this to next level by adding this module to all
>> >>> >>> services in a system , configuring modules to send all collected
>> >>> >>> messages to a pub/sub channel, and subscribing to those messages
>> >>> >>> via
>> >>> >>> a
>> >>> >>> UI, and depicting those messages through a UI.
>> >>> >>
>> >>> >> What pub/sub model do you plan to use? WS-Eventing?
>> >>> >>
>> >>> >> Samisa...
>> >>> >> --
>> >>> >> blog: http://samisa-abeysinghe.blogspot.com/
>> >>> >
>> >>> >
>> >>> > ---------------------------------------------------------------------
>> >>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>> >
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> ============================
>> >>> Srinath Perera, Ph.D.
>> >>>   WSO2 Inc. http://wso2.com
>> >>>   Blog: http://srinathsview.blogspot.com/
>> >>>
>> >>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>
>> >> --
>> >> Samisa Abeysinghe
>> >> Director, Engineering - WSO2 Inc.
>> >>
>> >> http://wso2.com/ - "lean . enterprise . middleware"
>> >
>> > --
>> > Samisa Abeysinghe
>> > Director, Engineering - WSO2 Inc.
>> >
>> > http://wso2.com/ - "lean . enterprise . middleware"
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
>
>
> --
> Amila Suriarachchi
> WSO2 Inc.
> blog: http://amilachinthaka.blogspot.com/
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

On Wed, Apr 7, 2010 at 2:41 PM, Amila Suriarachchi
<am...@gmail.com> wrote:
>
>
> On Mon, Apr 5, 2010 at 5:27 AM, nilupa bandara <ni...@gmail.com> wrote:
>>
>> Hello Everyone,
>>
>> Let me thank all of you once again for your valuable inputs.
>>
>> Based on the feedback I've got on initial project idea, I've come up
>> with an updated version of the project proposal which is listed
>> below[1],[2].
>
> Can you please add an diagram which shows the deployment architecture of the
> system. This make it more
> clear about your work.

I've just updated the proposal with design diagram of the system.
Could you please check whether it is reasonable ?

Best Regards,
Nilupa


>
> thanks,
> Amila.
>>
>> Best Regards,
>> Nilupa
>>
>> [1] http://wiki.apache.org/general/nilupa/gsoc2010/distributed_tcpmon
>> [2]
>> http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/nilupa/t127042423893
>>
>>
>> On Fri, Apr 2, 2010 at 5:01 AM, Samisa Abeysinghe
>> <sa...@gmail.com> wrote:
>> >
>> >
>> > On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe
>> > <sa...@gmail.com> wrote:
>> >>
>> >>
>> >> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com>
>> >> wrote:
>> >>>
>> >>> Hi nilupa;
>> >>>
>> >>> IMO this thread has more than one concept. Initially, the version I
>> >>> discussed with you is more of a message monitoring tool, which allow
>> >>> an user to see which message goes through where in the system. It is
>> >>> purely used for debugging services only, and a actually it distributed
>> >>> TCP monitor that helps people debug their distributed application.
>> >>> Features I had in my mind was to
>> >>>
>> >>> 1. Ability to turn on/off message interception
>> >>> 2. When tuned on, it show the system in a UI and let me see all
>> >>> messages went through each link I selected.
>> >>> 3. If messages have a some ID that identify a message sequence, it
>> >>> should let me select a message and find sequence that generated/ lead
>> >>> to that message.
>> >>>
>> >>> Basically, as I mentioned before, it is debugging tool and not a
>> >>> monitoring tool.
>> >>
>> >> If it is a debugging tool, that is going to be useful.
>> >
>> > And I think, it would be useful to list out the features it supports.
>> > I would also personally prefer some additions to the existing TCPMon
>> > where
>> > we can add plain text display of secure messages. However, I know that
>> > it is
>> > complicated, as the middle man is not supposed to change the ttl and the
>> > like in WS Security.
>> > Samisa...
>> >>
>> >> Samisa...
>> >>
>> >>>
>> >>> Then, there is a BAM/System monitoring aspect we heavily discussed in
>> >>> this thread, which is used to monitor a running system. IMHO they are
>> >>> different,yet very useful. However, they are targeted at two audiences
>> >>> and you should not do half of each.
>> >>>
>> >>> Personlly, I am interested in both topics, and willing to
>> >>> mentor/co-mentor either, however, IMO it two projects, not one.
>> >>>
>> >>> Thanks
>> >>> Srinath
>> >>>
>> >>>
>> >>>
>> >>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
>> >>> <an...@gmail.com> wrote:
>> >>> > Probably the right answer to this question is: none. Why would the
>> >>> > individual nodes intercepting the messages use publish/subscribe?
>> >>> > They
>> >>> > should all send them to a central place (or back to the sender of
>> >>> > the
>> >>> > original request).
>> >>> >
>> >>> > Andreas
>> >>> >
>> >>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
>> >>> > <sa...@gmail.com> wrote:
>> >>> >>> Then we can take this to next level by adding this module to all
>> >>> >>> services in a system , configuring modules to send all collected
>> >>> >>> messages to a pub/sub channel, and subscribing to those messages
>> >>> >>> via
>> >>> >>> a
>> >>> >>> UI, and depicting those messages through a UI.
>> >>> >>
>> >>> >> What pub/sub model do you plan to use? WS-Eventing?
>> >>> >>
>> >>> >> Samisa...
>> >>> >> --
>> >>> >> blog: http://samisa-abeysinghe.blogspot.com/
>> >>> >
>> >>> >
>> >>> > ---------------------------------------------------------------------
>> >>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>> >
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> ============================
>> >>> Srinath Perera, Ph.D.
>> >>>   WSO2 Inc. http://wso2.com
>> >>>   Blog: http://srinathsview.blogspot.com/
>> >>>
>> >>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>
>> >> --
>> >> Samisa Abeysinghe
>> >> Director, Engineering - WSO2 Inc.
>> >>
>> >> http://wso2.com/ - "lean . enterprise . middleware"
>> >
>> > --
>> > Samisa Abeysinghe
>> > Director, Engineering - WSO2 Inc.
>> >
>> > http://wso2.com/ - "lean . enterprise . middleware"
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
>
>
> --
> Amila Suriarachchi
> WSO2 Inc.
> blog: http://amilachinthaka.blogspot.com/
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

On Wed, Apr 7, 2010 at 2:41 PM, Amila Suriarachchi
<am...@gmail.com> wrote:
>
>
> On Mon, Apr 5, 2010 at 5:27 AM, nilupa bandara <ni...@gmail.com> wrote:
>>
>> Hello Everyone,
>>
>> Let me thank all of you once again for your valuable inputs.
>>
>> Based on the feedback I've got on initial project idea, I've come up
>> with an updated version of the project proposal which is listed
>> below[1],[2].
>
> Can you please add an diagram which shows the deployment architecture of the
> system. This make it more
> clear about your work.

I've just updated the proposal with design diagram of the system.
Could you please check whether it is reasonable ?

Best Regards,
Nilupa


>
> thanks,
> Amila.
>>
>> Best Regards,
>> Nilupa
>>
>> [1] http://wiki.apache.org/general/nilupa/gsoc2010/distributed_tcpmon
>> [2]
>> http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/nilupa/t127042423893
>>
>>
>> On Fri, Apr 2, 2010 at 5:01 AM, Samisa Abeysinghe
>> <sa...@gmail.com> wrote:
>> >
>> >
>> > On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe
>> > <sa...@gmail.com> wrote:
>> >>
>> >>
>> >> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com>
>> >> wrote:
>> >>>
>> >>> Hi nilupa;
>> >>>
>> >>> IMO this thread has more than one concept. Initially, the version I
>> >>> discussed with you is more of a message monitoring tool, which allow
>> >>> an user to see which message goes through where in the system. It is
>> >>> purely used for debugging services only, and a actually it distributed
>> >>> TCP monitor that helps people debug their distributed application.
>> >>> Features I had in my mind was to
>> >>>
>> >>> 1. Ability to turn on/off message interception
>> >>> 2. When tuned on, it show the system in a UI and let me see all
>> >>> messages went through each link I selected.
>> >>> 3. If messages have a some ID that identify a message sequence, it
>> >>> should let me select a message and find sequence that generated/ lead
>> >>> to that message.
>> >>>
>> >>> Basically, as I mentioned before, it is debugging tool and not a
>> >>> monitoring tool.
>> >>
>> >> If it is a debugging tool, that is going to be useful.
>> >
>> > And I think, it would be useful to list out the features it supports.
>> > I would also personally prefer some additions to the existing TCPMon
>> > where
>> > we can add plain text display of secure messages. However, I know that
>> > it is
>> > complicated, as the middle man is not supposed to change the ttl and the
>> > like in WS Security.
>> > Samisa...
>> >>
>> >> Samisa...
>> >>
>> >>>
>> >>> Then, there is a BAM/System monitoring aspect we heavily discussed in
>> >>> this thread, which is used to monitor a running system. IMHO they are
>> >>> different,yet very useful. However, they are targeted at two audiences
>> >>> and you should not do half of each.
>> >>>
>> >>> Personlly, I am interested in both topics, and willing to
>> >>> mentor/co-mentor either, however, IMO it two projects, not one.
>> >>>
>> >>> Thanks
>> >>> Srinath
>> >>>
>> >>>
>> >>>
>> >>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
>> >>> <an...@gmail.com> wrote:
>> >>> > Probably the right answer to this question is: none. Why would the
>> >>> > individual nodes intercepting the messages use publish/subscribe?
>> >>> > They
>> >>> > should all send them to a central place (or back to the sender of
>> >>> > the
>> >>> > original request).
>> >>> >
>> >>> > Andreas
>> >>> >
>> >>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
>> >>> > <sa...@gmail.com> wrote:
>> >>> >>> Then we can take this to next level by adding this module to all
>> >>> >>> services in a system , configuring modules to send all collected
>> >>> >>> messages to a pub/sub channel, and subscribing to those messages
>> >>> >>> via
>> >>> >>> a
>> >>> >>> UI, and depicting those messages through a UI.
>> >>> >>
>> >>> >> What pub/sub model do you plan to use? WS-Eventing?
>> >>> >>
>> >>> >> Samisa...
>> >>> >> --
>> >>> >> blog: http://samisa-abeysinghe.blogspot.com/
>> >>> >
>> >>> >
>> >>> > ---------------------------------------------------------------------
>> >>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>> >
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> ============================
>> >>> Srinath Perera, Ph.D.
>> >>>   WSO2 Inc. http://wso2.com
>> >>>   Blog: http://srinathsview.blogspot.com/
>> >>>
>> >>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>
>> >> --
>> >> Samisa Abeysinghe
>> >> Director, Engineering - WSO2 Inc.
>> >>
>> >> http://wso2.com/ - "lean . enterprise . middleware"
>> >
>> > --
>> > Samisa Abeysinghe
>> > Director, Engineering - WSO2 Inc.
>> >
>> > http://wso2.com/ - "lean . enterprise . middleware"
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
>
>
> --
> Amila Suriarachchi
> WSO2 Inc.
> blog: http://amilachinthaka.blogspot.com/
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Amila Suriarachchi <am...@gmail.com>.

On Mon, Apr 5, 2010 at 5:27 AM, nilupa bandara <ni...@gmail.com> wrote:

> Hello Everyone,
>
> Let me thank all of you once again for your valuable inputs.
>
> Based on the feedback I've got on initial project idea, I've come up
> with an updated version of the project proposal which is listed
> below[1],[2].
>

Can you please add an diagram which shows the deployment architecture of the
system. This make it more
clear about your work.

thanks,
Amila.

>
> Best Regards,
> Nilupa
>
> [1] http://wiki.apache.org/general/nilupa/gsoc2010/distributed_tcpmon
> [2]
> http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/nilupa/t127042423893
>
>
> On Fri, Apr 2, 2010 at 5:01 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> >
> >
> > On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe
> > <sa...@gmail.com> wrote:
> >>
> >>
> >> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com>
> wrote:
> >>>
> >>> Hi nilupa;
> >>>
> >>> IMO this thread has more than one concept. Initially, the version I
> >>> discussed with you is more of a message monitoring tool, which allow
> >>> an user to see which message goes through where in the system. It is
> >>> purely used for debugging services only, and a actually it distributed
> >>> TCP monitor that helps people debug their distributed application.
> >>> Features I had in my mind was to
> >>>
> >>> 1. Ability to turn on/off message interception
> >>> 2. When tuned on, it show the system in a UI and let me see all
> >>> messages went through each link I selected.
> >>> 3. If messages have a some ID that identify a message sequence, it
> >>> should let me select a message and find sequence that generated/ lead
> >>> to that message.
> >>>
> >>> Basically, as I mentioned before, it is debugging tool and not a
> >>> monitoring tool.
> >>
> >> If it is a debugging tool, that is going to be useful.
> >
> > And I think, it would be useful to list out the features it supports.
> > I would also personally prefer some additions to the existing TCPMon
> where
> > we can add plain text display of secure messages. However, I know that it
> is
> > complicated, as the middle man is not supposed to change the ttl and the
> > like in WS Security.
> > Samisa...
> >>
> >> Samisa...
> >>
> >>>
> >>> Then, there is a BAM/System monitoring aspect we heavily discussed in
> >>> this thread, which is used to monitor a running system. IMHO they are
> >>> different,yet very useful. However, they are targeted at two audiences
> >>> and you should not do half of each.
> >>>
> >>> Personlly, I am interested in both topics, and willing to
> >>> mentor/co-mentor either, however, IMO it two projects, not one.
> >>>
> >>> Thanks
> >>> Srinath
> >>>
> >>>
> >>>
> >>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
> >>> <an...@gmail.com> wrote:
> >>> > Probably the right answer to this question is: none. Why would the
> >>> > individual nodes intercepting the messages use publish/subscribe?
> They
> >>> > should all send them to a central place (or back to the sender of the
> >>> > original request).
> >>> >
> >>> > Andreas
> >>> >
> >>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> >>> > <sa...@gmail.com> wrote:
> >>> >>> Then we can take this to next level by adding this module to all
> >>> >>> services in a system , configuring modules to send all collected
> >>> >>> messages to a pub/sub channel, and subscribing to those messages
> via
> >>> >>> a
> >>> >>> UI, and depicting those messages through a UI.
> >>> >>
> >>> >> What pub/sub model do you plan to use? WS-Eventing?
> >>> >>
> >>> >> Samisa...
> >>> >> --
> >>> >> blog: http://samisa-abeysinghe.blogspot.com/
> >>> >
> >>> > ---------------------------------------------------------------------
> >>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> ============================
> >>> Srinath Perera, Ph.D.
> >>>   WSO2 Inc. http://wso2.com
> >>>   Blog: http://srinathsview.blogspot.com/
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>
> >> --
> >> Samisa Abeysinghe
> >> Director, Engineering - WSO2 Inc.
> >>
> >> http://wso2.com/ - "lean . enterprise . middleware"
> >
> > --
> > Samisa Abeysinghe
> > Director, Engineering - WSO2 Inc.
> >
> > http://wso2.com/ - "lean . enterprise . middleware"
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>


-- 
Amila Suriarachchi
WSO2 Inc.
blog: http://amilachinthaka.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Amila Suriarachchi <am...@gmail.com>.

On Mon, Apr 5, 2010 at 5:27 AM, nilupa bandara <ni...@gmail.com> wrote:

> Hello Everyone,
>
> Let me thank all of you once again for your valuable inputs.
>
> Based on the feedback I've got on initial project idea, I've come up
> with an updated version of the project proposal which is listed
> below[1],[2].
>

Can you please add an diagram which shows the deployment architecture of the
system. This make it more
clear about your work.

thanks,
Amila.

>
> Best Regards,
> Nilupa
>
> [1] http://wiki.apache.org/general/nilupa/gsoc2010/distributed_tcpmon
> [2]
> http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/nilupa/t127042423893
>
>
> On Fri, Apr 2, 2010 at 5:01 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> >
> >
> > On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe
> > <sa...@gmail.com> wrote:
> >>
> >>
> >> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com>
> wrote:
> >>>
> >>> Hi nilupa;
> >>>
> >>> IMO this thread has more than one concept. Initially, the version I
> >>> discussed with you is more of a message monitoring tool, which allow
> >>> an user to see which message goes through where in the system. It is
> >>> purely used for debugging services only, and a actually it distributed
> >>> TCP monitor that helps people debug their distributed application.
> >>> Features I had in my mind was to
> >>>
> >>> 1. Ability to turn on/off message interception
> >>> 2. When tuned on, it show the system in a UI and let me see all
> >>> messages went through each link I selected.
> >>> 3. If messages have a some ID that identify a message sequence, it
> >>> should let me select a message and find sequence that generated/ lead
> >>> to that message.
> >>>
> >>> Basically, as I mentioned before, it is debugging tool and not a
> >>> monitoring tool.
> >>
> >> If it is a debugging tool, that is going to be useful.
> >
> > And I think, it would be useful to list out the features it supports.
> > I would also personally prefer some additions to the existing TCPMon
> where
> > we can add plain text display of secure messages. However, I know that it
> is
> > complicated, as the middle man is not supposed to change the ttl and the
> > like in WS Security.
> > Samisa...
> >>
> >> Samisa...
> >>
> >>>
> >>> Then, there is a BAM/System monitoring aspect we heavily discussed in
> >>> this thread, which is used to monitor a running system. IMHO they are
> >>> different,yet very useful. However, they are targeted at two audiences
> >>> and you should not do half of each.
> >>>
> >>> Personlly, I am interested in both topics, and willing to
> >>> mentor/co-mentor either, however, IMO it two projects, not one.
> >>>
> >>> Thanks
> >>> Srinath
> >>>
> >>>
> >>>
> >>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
> >>> <an...@gmail.com> wrote:
> >>> > Probably the right answer to this question is: none. Why would the
> >>> > individual nodes intercepting the messages use publish/subscribe?
> They
> >>> > should all send them to a central place (or back to the sender of the
> >>> > original request).
> >>> >
> >>> > Andreas
> >>> >
> >>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> >>> > <sa...@gmail.com> wrote:
> >>> >>> Then we can take this to next level by adding this module to all
> >>> >>> services in a system , configuring modules to send all collected
> >>> >>> messages to a pub/sub channel, and subscribing to those messages
> via
> >>> >>> a
> >>> >>> UI, and depicting those messages through a UI.
> >>> >>
> >>> >> What pub/sub model do you plan to use? WS-Eventing?
> >>> >>
> >>> >> Samisa...
> >>> >> --
> >>> >> blog: http://samisa-abeysinghe.blogspot.com/
> >>> >
> >>> > ---------------------------------------------------------------------
> >>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> ============================
> >>> Srinath Perera, Ph.D.
> >>>   WSO2 Inc. http://wso2.com
> >>>   Blog: http://srinathsview.blogspot.com/
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>
> >> --
> >> Samisa Abeysinghe
> >> Director, Engineering - WSO2 Inc.
> >>
> >> http://wso2.com/ - "lean . enterprise . middleware"
> >
> > --
> > Samisa Abeysinghe
> > Director, Engineering - WSO2 Inc.
> >
> > http://wso2.com/ - "lean . enterprise . middleware"
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>


-- 
Amila Suriarachchi
WSO2 Inc.
blog: http://amilachinthaka.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Amila Suriarachchi <am...@gmail.com>.

On Mon, Apr 5, 2010 at 5:27 AM, nilupa bandara <ni...@gmail.com> wrote:

> Hello Everyone,
>
> Let me thank all of you once again for your valuable inputs.
>
> Based on the feedback I've got on initial project idea, I've come up
> with an updated version of the project proposal which is listed
> below[1],[2].
>

Can you please add an diagram which shows the deployment architecture of the
system. This make it more
clear about your work.

thanks,
Amila.

>
> Best Regards,
> Nilupa
>
> [1] http://wiki.apache.org/general/nilupa/gsoc2010/distributed_tcpmon
> [2]
> http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/nilupa/t127042423893
>
>
> On Fri, Apr 2, 2010 at 5:01 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> >
> >
> > On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe
> > <sa...@gmail.com> wrote:
> >>
> >>
> >> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com>
> wrote:
> >>>
> >>> Hi nilupa;
> >>>
> >>> IMO this thread has more than one concept. Initially, the version I
> >>> discussed with you is more of a message monitoring tool, which allow
> >>> an user to see which message goes through where in the system. It is
> >>> purely used for debugging services only, and a actually it distributed
> >>> TCP monitor that helps people debug their distributed application.
> >>> Features I had in my mind was to
> >>>
> >>> 1. Ability to turn on/off message interception
> >>> 2. When tuned on, it show the system in a UI and let me see all
> >>> messages went through each link I selected.
> >>> 3. If messages have a some ID that identify a message sequence, it
> >>> should let me select a message and find sequence that generated/ lead
> >>> to that message.
> >>>
> >>> Basically, as I mentioned before, it is debugging tool and not a
> >>> monitoring tool.
> >>
> >> If it is a debugging tool, that is going to be useful.
> >
> > And I think, it would be useful to list out the features it supports.
> > I would also personally prefer some additions to the existing TCPMon
> where
> > we can add plain text display of secure messages. However, I know that it
> is
> > complicated, as the middle man is not supposed to change the ttl and the
> > like in WS Security.
> > Samisa...
> >>
> >> Samisa...
> >>
> >>>
> >>> Then, there is a BAM/System monitoring aspect we heavily discussed in
> >>> this thread, which is used to monitor a running system. IMHO they are
> >>> different,yet very useful. However, they are targeted at two audiences
> >>> and you should not do half of each.
> >>>
> >>> Personlly, I am interested in both topics, and willing to
> >>> mentor/co-mentor either, however, IMO it two projects, not one.
> >>>
> >>> Thanks
> >>> Srinath
> >>>
> >>>
> >>>
> >>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
> >>> <an...@gmail.com> wrote:
> >>> > Probably the right answer to this question is: none. Why would the
> >>> > individual nodes intercepting the messages use publish/subscribe?
> They
> >>> > should all send them to a central place (or back to the sender of the
> >>> > original request).
> >>> >
> >>> > Andreas
> >>> >
> >>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> >>> > <sa...@gmail.com> wrote:
> >>> >>> Then we can take this to next level by adding this module to all
> >>> >>> services in a system , configuring modules to send all collected
> >>> >>> messages to a pub/sub channel, and subscribing to those messages
> via
> >>> >>> a
> >>> >>> UI, and depicting those messages through a UI.
> >>> >>
> >>> >> What pub/sub model do you plan to use? WS-Eventing?
> >>> >>
> >>> >> Samisa...
> >>> >> --
> >>> >> blog: http://samisa-abeysinghe.blogspot.com/
> >>> >
> >>> > ---------------------------------------------------------------------
> >>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> ============================
> >>> Srinath Perera, Ph.D.
> >>>   WSO2 Inc. http://wso2.com
> >>>   Blog: http://srinathsview.blogspot.com/
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>
> >> --
> >> Samisa Abeysinghe
> >> Director, Engineering - WSO2 Inc.
> >>
> >> http://wso2.com/ - "lean . enterprise . middleware"
> >
> > --
> > Samisa Abeysinghe
> > Director, Engineering - WSO2 Inc.
> >
> > http://wso2.com/ - "lean . enterprise . middleware"
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>


-- 
Amila Suriarachchi
WSO2 Inc.
blog: http://amilachinthaka.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Amila Suriarachchi <am...@gmail.com>.

On Mon, Apr 5, 2010 at 5:27 AM, nilupa bandara <ni...@gmail.com> wrote:

> Hello Everyone,
>
> Let me thank all of you once again for your valuable inputs.
>
> Based on the feedback I've got on initial project idea, I've come up
> with an updated version of the project proposal which is listed
> below[1],[2].
>

Can you please add an diagram which shows the deployment architecture of the
system. This make it more
clear about your work.

thanks,
Amila.

>
> Best Regards,
> Nilupa
>
> [1] http://wiki.apache.org/general/nilupa/gsoc2010/distributed_tcpmon
> [2]
> http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/nilupa/t127042423893
>
>
> On Fri, Apr 2, 2010 at 5:01 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> >
> >
> > On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe
> > <sa...@gmail.com> wrote:
> >>
> >>
> >> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com>
> wrote:
> >>>
> >>> Hi nilupa;
> >>>
> >>> IMO this thread has more than one concept. Initially, the version I
> >>> discussed with you is more of a message monitoring tool, which allow
> >>> an user to see which message goes through where in the system. It is
> >>> purely used for debugging services only, and a actually it distributed
> >>> TCP monitor that helps people debug their distributed application.
> >>> Features I had in my mind was to
> >>>
> >>> 1. Ability to turn on/off message interception
> >>> 2. When tuned on, it show the system in a UI and let me see all
> >>> messages went through each link I selected.
> >>> 3. If messages have a some ID that identify a message sequence, it
> >>> should let me select a message and find sequence that generated/ lead
> >>> to that message.
> >>>
> >>> Basically, as I mentioned before, it is debugging tool and not a
> >>> monitoring tool.
> >>
> >> If it is a debugging tool, that is going to be useful.
> >
> > And I think, it would be useful to list out the features it supports.
> > I would also personally prefer some additions to the existing TCPMon
> where
> > we can add plain text display of secure messages. However, I know that it
> is
> > complicated, as the middle man is not supposed to change the ttl and the
> > like in WS Security.
> > Samisa...
> >>
> >> Samisa...
> >>
> >>>
> >>> Then, there is a BAM/System monitoring aspect we heavily discussed in
> >>> this thread, which is used to monitor a running system. IMHO they are
> >>> different,yet very useful. However, they are targeted at two audiences
> >>> and you should not do half of each.
> >>>
> >>> Personlly, I am interested in both topics, and willing to
> >>> mentor/co-mentor either, however, IMO it two projects, not one.
> >>>
> >>> Thanks
> >>> Srinath
> >>>
> >>>
> >>>
> >>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
> >>> <an...@gmail.com> wrote:
> >>> > Probably the right answer to this question is: none. Why would the
> >>> > individual nodes intercepting the messages use publish/subscribe?
> They
> >>> > should all send them to a central place (or back to the sender of the
> >>> > original request).
> >>> >
> >>> > Andreas
> >>> >
> >>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> >>> > <sa...@gmail.com> wrote:
> >>> >>> Then we can take this to next level by adding this module to all
> >>> >>> services in a system , configuring modules to send all collected
> >>> >>> messages to a pub/sub channel, and subscribing to those messages
> via
> >>> >>> a
> >>> >>> UI, and depicting those messages through a UI.
> >>> >>
> >>> >> What pub/sub model do you plan to use? WS-Eventing?
> >>> >>
> >>> >> Samisa...
> >>> >> --
> >>> >> blog: http://samisa-abeysinghe.blogspot.com/
> >>> >
> >>> > ---------------------------------------------------------------------
> >>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> ============================
> >>> Srinath Perera, Ph.D.
> >>>   WSO2 Inc. http://wso2.com
> >>>   Blog: http://srinathsview.blogspot.com/
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>
> >> --
> >> Samisa Abeysinghe
> >> Director, Engineering - WSO2 Inc.
> >>
> >> http://wso2.com/ - "lean . enterprise . middleware"
> >
> > --
> > Samisa Abeysinghe
> > Director, Engineering - WSO2 Inc.
> >
> > http://wso2.com/ - "lean . enterprise . middleware"
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>


-- 
Amila Suriarachchi
WSO2 Inc.
blog: http://amilachinthaka.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Amila Suriarachchi <am...@gmail.com>.

On Mon, Apr 5, 2010 at 5:27 AM, nilupa bandara <ni...@gmail.com> wrote:

> Hello Everyone,
>
> Let me thank all of you once again for your valuable inputs.
>
> Based on the feedback I've got on initial project idea, I've come up
> with an updated version of the project proposal which is listed
> below[1],[2].
>

Can you please add an diagram which shows the deployment architecture of the
system. This make it more
clear about your work.

thanks,
Amila.

>
> Best Regards,
> Nilupa
>
> [1] http://wiki.apache.org/general/nilupa/gsoc2010/distributed_tcpmon
> [2]
> http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/nilupa/t127042423893
>
>
> On Fri, Apr 2, 2010 at 5:01 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> >
> >
> > On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe
> > <sa...@gmail.com> wrote:
> >>
> >>
> >> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com>
> wrote:
> >>>
> >>> Hi nilupa;
> >>>
> >>> IMO this thread has more than one concept. Initially, the version I
> >>> discussed with you is more of a message monitoring tool, which allow
> >>> an user to see which message goes through where in the system. It is
> >>> purely used for debugging services only, and a actually it distributed
> >>> TCP monitor that helps people debug their distributed application.
> >>> Features I had in my mind was to
> >>>
> >>> 1. Ability to turn on/off message interception
> >>> 2. When tuned on, it show the system in a UI and let me see all
> >>> messages went through each link I selected.
> >>> 3. If messages have a some ID that identify a message sequence, it
> >>> should let me select a message and find sequence that generated/ lead
> >>> to that message.
> >>>
> >>> Basically, as I mentioned before, it is debugging tool and not a
> >>> monitoring tool.
> >>
> >> If it is a debugging tool, that is going to be useful.
> >
> > And I think, it would be useful to list out the features it supports.
> > I would also personally prefer some additions to the existing TCPMon
> where
> > we can add plain text display of secure messages. However, I know that it
> is
> > complicated, as the middle man is not supposed to change the ttl and the
> > like in WS Security.
> > Samisa...
> >>
> >> Samisa...
> >>
> >>>
> >>> Then, there is a BAM/System monitoring aspect we heavily discussed in
> >>> this thread, which is used to monitor a running system. IMHO they are
> >>> different,yet very useful. However, they are targeted at two audiences
> >>> and you should not do half of each.
> >>>
> >>> Personlly, I am interested in both topics, and willing to
> >>> mentor/co-mentor either, however, IMO it two projects, not one.
> >>>
> >>> Thanks
> >>> Srinath
> >>>
> >>>
> >>>
> >>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
> >>> <an...@gmail.com> wrote:
> >>> > Probably the right answer to this question is: none. Why would the
> >>> > individual nodes intercepting the messages use publish/subscribe?
> They
> >>> > should all send them to a central place (or back to the sender of the
> >>> > original request).
> >>> >
> >>> > Andreas
> >>> >
> >>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> >>> > <sa...@gmail.com> wrote:
> >>> >>> Then we can take this to next level by adding this module to all
> >>> >>> services in a system , configuring modules to send all collected
> >>> >>> messages to a pub/sub channel, and subscribing to those messages
> via
> >>> >>> a
> >>> >>> UI, and depicting those messages through a UI.
> >>> >>
> >>> >> What pub/sub model do you plan to use? WS-Eventing?
> >>> >>
> >>> >> Samisa...
> >>> >> --
> >>> >> blog: http://samisa-abeysinghe.blogspot.com/
> >>> >
> >>> > ---------------------------------------------------------------------
> >>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> ============================
> >>> Srinath Perera, Ph.D.
> >>>   WSO2 Inc. http://wso2.com
> >>>   Blog: http://srinathsview.blogspot.com/
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>
> >> --
> >> Samisa Abeysinghe
> >> Director, Engineering - WSO2 Inc.
> >>
> >> http://wso2.com/ - "lean . enterprise . middleware"
> >
> > --
> > Samisa Abeysinghe
> > Director, Engineering - WSO2 Inc.
> >
> > http://wso2.com/ - "lean . enterprise . middleware"
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>


-- 
Amila Suriarachchi
WSO2 Inc.
blog: http://amilachinthaka.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

Hello Everyone,

Let me thank all of you once again for your valuable inputs.

Based on the feedback I've got on initial project idea, I've come up
with an updated version of the project proposal which is listed
below[1],[2].

Best Regards,
Nilupa

[1] http://wiki.apache.org/general/nilupa/gsoc2010/distributed_tcpmon
[2] http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/nilupa/t127042423893


On Fri, Apr 2, 2010 at 5:01 AM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>
>
> On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>
>>
>> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com> wrote:
>>>
>>> Hi nilupa;
>>>
>>> IMO this thread has more than one concept. Initially, the version I
>>> discussed with you is more of a message monitoring tool, which allow
>>> an user to see which message goes through where in the system. It is
>>> purely used for debugging services only, and a actually it distributed
>>> TCP monitor that helps people debug their distributed application.
>>> Features I had in my mind was to
>>>
>>> 1. Ability to turn on/off message interception
>>> 2. When tuned on, it show the system in a UI and let me see all
>>> messages went through each link I selected.
>>> 3. If messages have a some ID that identify a message sequence, it
>>> should let me select a message and find sequence that generated/ lead
>>> to that message.
>>>
>>> Basically, as I mentioned before, it is debugging tool and not a
>>> monitoring tool.
>>
>> If it is a debugging tool, that is going to be useful.
>
> And I think, it would be useful to list out the features it supports.
> I would also personally prefer some additions to the existing TCPMon where
> we can add plain text display of secure messages. However, I know that it is
> complicated, as the middle man is not supposed to change the ttl and the
> like in WS Security.
> Samisa...
>>
>> Samisa...
>>
>>>
>>> Then, there is a BAM/System monitoring aspect we heavily discussed in
>>> this thread, which is used to monitor a running system. IMHO they are
>>> different,yet very useful. However, they are targeted at two audiences
>>> and you should not do half of each.
>>>
>>> Personlly, I am interested in both topics, and willing to
>>> mentor/co-mentor either, however, IMO it two projects, not one.
>>>
>>> Thanks
>>> Srinath
>>>
>>>
>>>
>>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
>>> <an...@gmail.com> wrote:
>>> > Probably the right answer to this question is: none. Why would the
>>> > individual nodes intercepting the messages use publish/subscribe? They
>>> > should all send them to a central place (or back to the sender of the
>>> > original request).
>>> >
>>> > Andreas
>>> >
>>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
>>> > <sa...@gmail.com> wrote:
>>> >>> Then we can take this to next level by adding this module to all
>>> >>> services in a system , configuring modules to send all collected
>>> >>> messages to a pub/sub channel, and subscribing to those messages via
>>> >>> a
>>> >>> UI, and depicting those messages through a UI.
>>> >>
>>> >> What pub/sub model do you plan to use? WS-Eventing?
>>> >>
>>> >> Samisa...
>>> >> --
>>> >> blog: http://samisa-abeysinghe.blogspot.com/
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> ============================
>>> Srinath Perera, Ph.D.
>>>   WSO2 Inc. http://wso2.com
>>>   Blog: http://srinathsview.blogspot.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>> --
>> Samisa Abeysinghe
>> Director, Engineering - WSO2 Inc.
>>
>> http://wso2.com/ - "lean . enterprise . middleware"
>
> --
> Samisa Abeysinghe
> Director, Engineering - WSO2 Inc.
>
> http://wso2.com/ - "lean . enterprise . middleware"
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

Hello Everyone,

Let me thank all of you once again for your valuable inputs.

Based on the feedback I've got on initial project idea, I've come up
with an updated version of the project proposal which is listed
below[1],[2].

Best Regards,
Nilupa

[1] http://wiki.apache.org/general/nilupa/gsoc2010/distributed_tcpmon
[2] http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/nilupa/t127042423893


On Fri, Apr 2, 2010 at 5:01 AM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>
>
> On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>
>>
>> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com> wrote:
>>>
>>> Hi nilupa;
>>>
>>> IMO this thread has more than one concept. Initially, the version I
>>> discussed with you is more of a message monitoring tool, which allow
>>> an user to see which message goes through where in the system. It is
>>> purely used for debugging services only, and a actually it distributed
>>> TCP monitor that helps people debug their distributed application.
>>> Features I had in my mind was to
>>>
>>> 1. Ability to turn on/off message interception
>>> 2. When tuned on, it show the system in a UI and let me see all
>>> messages went through each link I selected.
>>> 3. If messages have a some ID that identify a message sequence, it
>>> should let me select a message and find sequence that generated/ lead
>>> to that message.
>>>
>>> Basically, as I mentioned before, it is debugging tool and not a
>>> monitoring tool.
>>
>> If it is a debugging tool, that is going to be useful.
>
> And I think, it would be useful to list out the features it supports.
> I would also personally prefer some additions to the existing TCPMon where
> we can add plain text display of secure messages. However, I know that it is
> complicated, as the middle man is not supposed to change the ttl and the
> like in WS Security.
> Samisa...
>>
>> Samisa...
>>
>>>
>>> Then, there is a BAM/System monitoring aspect we heavily discussed in
>>> this thread, which is used to monitor a running system. IMHO they are
>>> different,yet very useful. However, they are targeted at two audiences
>>> and you should not do half of each.
>>>
>>> Personlly, I am interested in both topics, and willing to
>>> mentor/co-mentor either, however, IMO it two projects, not one.
>>>
>>> Thanks
>>> Srinath
>>>
>>>
>>>
>>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
>>> <an...@gmail.com> wrote:
>>> > Probably the right answer to this question is: none. Why would the
>>> > individual nodes intercepting the messages use publish/subscribe? They
>>> > should all send them to a central place (or back to the sender of the
>>> > original request).
>>> >
>>> > Andreas
>>> >
>>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
>>> > <sa...@gmail.com> wrote:
>>> >>> Then we can take this to next level by adding this module to all
>>> >>> services in a system , configuring modules to send all collected
>>> >>> messages to a pub/sub channel, and subscribing to those messages via
>>> >>> a
>>> >>> UI, and depicting those messages through a UI.
>>> >>
>>> >> What pub/sub model do you plan to use? WS-Eventing?
>>> >>
>>> >> Samisa...
>>> >> --
>>> >> blog: http://samisa-abeysinghe.blogspot.com/
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> ============================
>>> Srinath Perera, Ph.D.
>>>   WSO2 Inc. http://wso2.com
>>>   Blog: http://srinathsview.blogspot.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>> --
>> Samisa Abeysinghe
>> Director, Engineering - WSO2 Inc.
>>
>> http://wso2.com/ - "lean . enterprise . middleware"
>
> --
> Samisa Abeysinghe
> Director, Engineering - WSO2 Inc.
>
> http://wso2.com/ - "lean . enterprise . middleware"
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

Hello Everyone,

Let me thank all of you once again for your valuable inputs.

Based on the feedback I've got on initial project idea, I've come up
with an updated version of the project proposal which is listed
below[1],[2].

Best Regards,
Nilupa

[1] http://wiki.apache.org/general/nilupa/gsoc2010/distributed_tcpmon
[2] http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/nilupa/t127042423893


On Fri, Apr 2, 2010 at 5:01 AM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>
>
> On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>
>>
>> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com> wrote:
>>>
>>> Hi nilupa;
>>>
>>> IMO this thread has more than one concept. Initially, the version I
>>> discussed with you is more of a message monitoring tool, which allow
>>> an user to see which message goes through where in the system. It is
>>> purely used for debugging services only, and a actually it distributed
>>> TCP monitor that helps people debug their distributed application.
>>> Features I had in my mind was to
>>>
>>> 1. Ability to turn on/off message interception
>>> 2. When tuned on, it show the system in a UI and let me see all
>>> messages went through each link I selected.
>>> 3. If messages have a some ID that identify a message sequence, it
>>> should let me select a message and find sequence that generated/ lead
>>> to that message.
>>>
>>> Basically, as I mentioned before, it is debugging tool and not a
>>> monitoring tool.
>>
>> If it is a debugging tool, that is going to be useful.
>
> And I think, it would be useful to list out the features it supports.
> I would also personally prefer some additions to the existing TCPMon where
> we can add plain text display of secure messages. However, I know that it is
> complicated, as the middle man is not supposed to change the ttl and the
> like in WS Security.
> Samisa...
>>
>> Samisa...
>>
>>>
>>> Then, there is a BAM/System monitoring aspect we heavily discussed in
>>> this thread, which is used to monitor a running system. IMHO they are
>>> different,yet very useful. However, they are targeted at two audiences
>>> and you should not do half of each.
>>>
>>> Personlly, I am interested in both topics, and willing to
>>> mentor/co-mentor either, however, IMO it two projects, not one.
>>>
>>> Thanks
>>> Srinath
>>>
>>>
>>>
>>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
>>> <an...@gmail.com> wrote:
>>> > Probably the right answer to this question is: none. Why would the
>>> > individual nodes intercepting the messages use publish/subscribe? They
>>> > should all send them to a central place (or back to the sender of the
>>> > original request).
>>> >
>>> > Andreas
>>> >
>>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
>>> > <sa...@gmail.com> wrote:
>>> >>> Then we can take this to next level by adding this module to all
>>> >>> services in a system , configuring modules to send all collected
>>> >>> messages to a pub/sub channel, and subscribing to those messages via
>>> >>> a
>>> >>> UI, and depicting those messages through a UI.
>>> >>
>>> >> What pub/sub model do you plan to use? WS-Eventing?
>>> >>
>>> >> Samisa...
>>> >> --
>>> >> blog: http://samisa-abeysinghe.blogspot.com/
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> ============================
>>> Srinath Perera, Ph.D.
>>>   WSO2 Inc. http://wso2.com
>>>   Blog: http://srinathsview.blogspot.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>> --
>> Samisa Abeysinghe
>> Director, Engineering - WSO2 Inc.
>>
>> http://wso2.com/ - "lean . enterprise . middleware"
>
> --
> Samisa Abeysinghe
> Director, Engineering - WSO2 Inc.
>
> http://wso2.com/ - "lean . enterprise . middleware"
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

Hello Everyone,

Let me thank all of you once again for your valuable inputs.

Based on the feedback I've got on initial project idea, I've come up
with an updated version of the project proposal which is listed
below[1],[2].

Best Regards,
Nilupa

[1] http://wiki.apache.org/general/nilupa/gsoc2010/distributed_tcpmon
[2] http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/nilupa/t127042423893


On Fri, Apr 2, 2010 at 5:01 AM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>
>
> On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>
>>
>> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com> wrote:
>>>
>>> Hi nilupa;
>>>
>>> IMO this thread has more than one concept. Initially, the version I
>>> discussed with you is more of a message monitoring tool, which allow
>>> an user to see which message goes through where in the system. It is
>>> purely used for debugging services only, and a actually it distributed
>>> TCP monitor that helps people debug their distributed application.
>>> Features I had in my mind was to
>>>
>>> 1. Ability to turn on/off message interception
>>> 2. When tuned on, it show the system in a UI and let me see all
>>> messages went through each link I selected.
>>> 3. If messages have a some ID that identify a message sequence, it
>>> should let me select a message and find sequence that generated/ lead
>>> to that message.
>>>
>>> Basically, as I mentioned before, it is debugging tool and not a
>>> monitoring tool.
>>
>> If it is a debugging tool, that is going to be useful.
>
> And I think, it would be useful to list out the features it supports.
> I would also personally prefer some additions to the existing TCPMon where
> we can add plain text display of secure messages. However, I know that it is
> complicated, as the middle man is not supposed to change the ttl and the
> like in WS Security.
> Samisa...
>>
>> Samisa...
>>
>>>
>>> Then, there is a BAM/System monitoring aspect we heavily discussed in
>>> this thread, which is used to monitor a running system. IMHO they are
>>> different,yet very useful. However, they are targeted at two audiences
>>> and you should not do half of each.
>>>
>>> Personlly, I am interested in both topics, and willing to
>>> mentor/co-mentor either, however, IMO it two projects, not one.
>>>
>>> Thanks
>>> Srinath
>>>
>>>
>>>
>>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
>>> <an...@gmail.com> wrote:
>>> > Probably the right answer to this question is: none. Why would the
>>> > individual nodes intercepting the messages use publish/subscribe? They
>>> > should all send them to a central place (or back to the sender of the
>>> > original request).
>>> >
>>> > Andreas
>>> >
>>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
>>> > <sa...@gmail.com> wrote:
>>> >>> Then we can take this to next level by adding this module to all
>>> >>> services in a system , configuring modules to send all collected
>>> >>> messages to a pub/sub channel, and subscribing to those messages via
>>> >>> a
>>> >>> UI, and depicting those messages through a UI.
>>> >>
>>> >> What pub/sub model do you plan to use? WS-Eventing?
>>> >>
>>> >> Samisa...
>>> >> --
>>> >> blog: http://samisa-abeysinghe.blogspot.com/
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> ============================
>>> Srinath Perera, Ph.D.
>>>   WSO2 Inc. http://wso2.com
>>>   Blog: http://srinathsview.blogspot.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>> --
>> Samisa Abeysinghe
>> Director, Engineering - WSO2 Inc.
>>
>> http://wso2.com/ - "lean . enterprise . middleware"
>
> --
> Samisa Abeysinghe
> Director, Engineering - WSO2 Inc.
>
> http://wso2.com/ - "lean . enterprise . middleware"
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

Hello Everyone,

Let me thank all of you once again for your valuable inputs.

Based on the feedback I've got on initial project idea, I've come up
with an updated version of the project proposal which is listed
below[1],[2].

Best Regards,
Nilupa

[1] http://wiki.apache.org/general/nilupa/gsoc2010/distributed_tcpmon
[2] http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/nilupa/t127042423893


On Fri, Apr 2, 2010 at 5:01 AM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>
>
> On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>
>>
>> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com> wrote:
>>>
>>> Hi nilupa;
>>>
>>> IMO this thread has more than one concept. Initially, the version I
>>> discussed with you is more of a message monitoring tool, which allow
>>> an user to see which message goes through where in the system. It is
>>> purely used for debugging services only, and a actually it distributed
>>> TCP monitor that helps people debug their distributed application.
>>> Features I had in my mind was to
>>>
>>> 1. Ability to turn on/off message interception
>>> 2. When tuned on, it show the system in a UI and let me see all
>>> messages went through each link I selected.
>>> 3. If messages have a some ID that identify a message sequence, it
>>> should let me select a message and find sequence that generated/ lead
>>> to that message.
>>>
>>> Basically, as I mentioned before, it is debugging tool and not a
>>> monitoring tool.
>>
>> If it is a debugging tool, that is going to be useful.
>
> And I think, it would be useful to list out the features it supports.
> I would also personally prefer some additions to the existing TCPMon where
> we can add plain text display of secure messages. However, I know that it is
> complicated, as the middle man is not supposed to change the ttl and the
> like in WS Security.
> Samisa...
>>
>> Samisa...
>>
>>>
>>> Then, there is a BAM/System monitoring aspect we heavily discussed in
>>> this thread, which is used to monitor a running system. IMHO they are
>>> different,yet very useful. However, they are targeted at two audiences
>>> and you should not do half of each.
>>>
>>> Personlly, I am interested in both topics, and willing to
>>> mentor/co-mentor either, however, IMO it two projects, not one.
>>>
>>> Thanks
>>> Srinath
>>>
>>>
>>>
>>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
>>> <an...@gmail.com> wrote:
>>> > Probably the right answer to this question is: none. Why would the
>>> > individual nodes intercepting the messages use publish/subscribe? They
>>> > should all send them to a central place (or back to the sender of the
>>> > original request).
>>> >
>>> > Andreas
>>> >
>>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
>>> > <sa...@gmail.com> wrote:
>>> >>> Then we can take this to next level by adding this module to all
>>> >>> services in a system , configuring modules to send all collected
>>> >>> messages to a pub/sub channel, and subscribing to those messages via
>>> >>> a
>>> >>> UI, and depicting those messages through a UI.
>>> >>
>>> >> What pub/sub model do you plan to use? WS-Eventing?
>>> >>
>>> >> Samisa...
>>> >> --
>>> >> blog: http://samisa-abeysinghe.blogspot.com/
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> ============================
>>> Srinath Perera, Ph.D.
>>>   WSO2 Inc. http://wso2.com
>>>   Blog: http://srinathsview.blogspot.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>> --
>> Samisa Abeysinghe
>> Director, Engineering - WSO2 Inc.
>>
>> http://wso2.com/ - "lean . enterprise . middleware"
>
> --
> Samisa Abeysinghe
> Director, Engineering - WSO2 Inc.
>
> http://wso2.com/ - "lean . enterprise . middleware"
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe <
samisa.abeysinghe@gmail.com> wrote:

>
>
> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com> wrote:
>
>> Hi nilupa;
>>
>> IMO this thread has more than one concept. Initially, the version I
>> discussed with you is more of a message monitoring tool, which allow
>> an user to see which message goes through where in the system. It is
>> purely used for debugging services only, and a actually it distributed
>> TCP monitor that helps people debug their distributed application.
>> Features I had in my mind was to
>>
>> 1. Ability to turn on/off message interception
>> 2. When tuned on, it show the system in a UI and let me see all
>> messages went through each link I selected.
>> 3. If messages have a some ID that identify a message sequence, it
>> should let me select a message and find sequence that generated/ lead
>> to that message.
>>
>> Basically, as I mentioned before, it is debugging tool and not a
>> monitoring tool.
>>
>
> If it is a debugging tool, that is going to be useful.
>

And I think, it would be useful to list out the features it supports.

I would also personally prefer some additions to the existing TCPMon where
we can add plain text display of secure messages. However, I know that it is
complicated, as the middle man is not supposed to change the ttl and the
like in WS Security.

Samisa...

>
> Samisa...
>
>
>>
>> Then, there is a BAM/System monitoring aspect we heavily discussed in
>> this thread, which is used to monitor a running system. IMHO they are
>> different,yet very useful. However, they are targeted at two audiences
>> and you should not do half of each.
>>
>> Personlly, I am interested in both topics, and willing to
>> mentor/co-mentor either, however, IMO it two projects, not one.
>>
>> Thanks
>> Srinath
>>
>>
>>
>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
>> <an...@gmail.com> wrote:
>> > Probably the right answer to this question is: none. Why would the
>> > individual nodes intercepting the messages use publish/subscribe? They
>> > should all send them to a central place (or back to the sender of the
>> > original request).
>> >
>> > Andreas
>> >
>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
>> > <sa...@gmail.com> wrote:
>> >>> Then we can take this to next level by adding this module to all
>> >>> services in a system , configuring modules to send all collected
>> >>> messages to a pub/sub channel, and subscribing to those messages via a
>> >>> UI, and depicting those messages through a UI.
>> >>
>> >> What pub/sub model do you plan to use? WS-Eventing?
>> >>
>> >> Samisa...
>> >> --
>> >> blog: http://samisa-abeysinghe.blogspot.com/
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >
>> >
>>
>>
>>
>> --
>> ============================
>> Srinath Perera, Ph.D.
>>   WSO2 Inc. http://wso2.com
>>    Blog: http://srinathsview.blogspot.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>> --
> Samisa Abeysinghe
> Director, Engineering - WSO2 Inc.
>
> http://wso2.com/ - "lean . enterprise . middleware"
>
> --
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe <
samisa.abeysinghe@gmail.com> wrote:

>
>
> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com> wrote:
>
>> Hi nilupa;
>>
>> IMO this thread has more than one concept. Initially, the version I
>> discussed with you is more of a message monitoring tool, which allow
>> an user to see which message goes through where in the system. It is
>> purely used for debugging services only, and a actually it distributed
>> TCP monitor that helps people debug their distributed application.
>> Features I had in my mind was to
>>
>> 1. Ability to turn on/off message interception
>> 2. When tuned on, it show the system in a UI and let me see all
>> messages went through each link I selected.
>> 3. If messages have a some ID that identify a message sequence, it
>> should let me select a message and find sequence that generated/ lead
>> to that message.
>>
>> Basically, as I mentioned before, it is debugging tool and not a
>> monitoring tool.
>>
>
> If it is a debugging tool, that is going to be useful.
>

And I think, it would be useful to list out the features it supports.

I would also personally prefer some additions to the existing TCPMon where
we can add plain text display of secure messages. However, I know that it is
complicated, as the middle man is not supposed to change the ttl and the
like in WS Security.

Samisa...

>
> Samisa...
>
>
>>
>> Then, there is a BAM/System monitoring aspect we heavily discussed in
>> this thread, which is used to monitor a running system. IMHO they are
>> different,yet very useful. However, they are targeted at two audiences
>> and you should not do half of each.
>>
>> Personlly, I am interested in both topics, and willing to
>> mentor/co-mentor either, however, IMO it two projects, not one.
>>
>> Thanks
>> Srinath
>>
>>
>>
>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
>> <an...@gmail.com> wrote:
>> > Probably the right answer to this question is: none. Why would the
>> > individual nodes intercepting the messages use publish/subscribe? They
>> > should all send them to a central place (or back to the sender of the
>> > original request).
>> >
>> > Andreas
>> >
>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
>> > <sa...@gmail.com> wrote:
>> >>> Then we can take this to next level by adding this module to all
>> >>> services in a system , configuring modules to send all collected
>> >>> messages to a pub/sub channel, and subscribing to those messages via a
>> >>> UI, and depicting those messages through a UI.
>> >>
>> >> What pub/sub model do you plan to use? WS-Eventing?
>> >>
>> >> Samisa...
>> >> --
>> >> blog: http://samisa-abeysinghe.blogspot.com/
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >
>> >
>>
>>
>>
>> --
>> ============================
>> Srinath Perera, Ph.D.
>>   WSO2 Inc. http://wso2.com
>>    Blog: http://srinathsview.blogspot.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>> --
> Samisa Abeysinghe
> Director, Engineering - WSO2 Inc.
>
> http://wso2.com/ - "lean . enterprise . middleware"
>
> --
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe <
samisa.abeysinghe@gmail.com> wrote:

>
>
> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com> wrote:
>
>> Hi nilupa;
>>
>> IMO this thread has more than one concept. Initially, the version I
>> discussed with you is more of a message monitoring tool, which allow
>> an user to see which message goes through where in the system. It is
>> purely used for debugging services only, and a actually it distributed
>> TCP monitor that helps people debug their distributed application.
>> Features I had in my mind was to
>>
>> 1. Ability to turn on/off message interception
>> 2. When tuned on, it show the system in a UI and let me see all
>> messages went through each link I selected.
>> 3. If messages have a some ID that identify a message sequence, it
>> should let me select a message and find sequence that generated/ lead
>> to that message.
>>
>> Basically, as I mentioned before, it is debugging tool and not a
>> monitoring tool.
>>
>
> If it is a debugging tool, that is going to be useful.
>

And I think, it would be useful to list out the features it supports.

I would also personally prefer some additions to the existing TCPMon where
we can add plain text display of secure messages. However, I know that it is
complicated, as the middle man is not supposed to change the ttl and the
like in WS Security.

Samisa...

>
> Samisa...
>
>
>>
>> Then, there is a BAM/System monitoring aspect we heavily discussed in
>> this thread, which is used to monitor a running system. IMHO they are
>> different,yet very useful. However, they are targeted at two audiences
>> and you should not do half of each.
>>
>> Personlly, I am interested in both topics, and willing to
>> mentor/co-mentor either, however, IMO it two projects, not one.
>>
>> Thanks
>> Srinath
>>
>>
>>
>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
>> <an...@gmail.com> wrote:
>> > Probably the right answer to this question is: none. Why would the
>> > individual nodes intercepting the messages use publish/subscribe? They
>> > should all send them to a central place (or back to the sender of the
>> > original request).
>> >
>> > Andreas
>> >
>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
>> > <sa...@gmail.com> wrote:
>> >>> Then we can take this to next level by adding this module to all
>> >>> services in a system , configuring modules to send all collected
>> >>> messages to a pub/sub channel, and subscribing to those messages via a
>> >>> UI, and depicting those messages through a UI.
>> >>
>> >> What pub/sub model do you plan to use? WS-Eventing?
>> >>
>> >> Samisa...
>> >> --
>> >> blog: http://samisa-abeysinghe.blogspot.com/
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >
>> >
>>
>>
>>
>> --
>> ============================
>> Srinath Perera, Ph.D.
>>   WSO2 Inc. http://wso2.com
>>    Blog: http://srinathsview.blogspot.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>> --
> Samisa Abeysinghe
> Director, Engineering - WSO2 Inc.
>
> http://wso2.com/ - "lean . enterprise . middleware"
>
> --
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe <
samisa.abeysinghe@gmail.com> wrote:

>
>
> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com> wrote:
>
>> Hi nilupa;
>>
>> IMO this thread has more than one concept. Initially, the version I
>> discussed with you is more of a message monitoring tool, which allow
>> an user to see which message goes through where in the system. It is
>> purely used for debugging services only, and a actually it distributed
>> TCP monitor that helps people debug their distributed application.
>> Features I had in my mind was to
>>
>> 1. Ability to turn on/off message interception
>> 2. When tuned on, it show the system in a UI and let me see all
>> messages went through each link I selected.
>> 3. If messages have a some ID that identify a message sequence, it
>> should let me select a message and find sequence that generated/ lead
>> to that message.
>>
>> Basically, as I mentioned before, it is debugging tool and not a
>> monitoring tool.
>>
>
> If it is a debugging tool, that is going to be useful.
>

And I think, it would be useful to list out the features it supports.

I would also personally prefer some additions to the existing TCPMon where
we can add plain text display of secure messages. However, I know that it is
complicated, as the middle man is not supposed to change the ttl and the
like in WS Security.

Samisa...

>
> Samisa...
>
>
>>
>> Then, there is a BAM/System monitoring aspect we heavily discussed in
>> this thread, which is used to monitor a running system. IMHO they are
>> different,yet very useful. However, they are targeted at two audiences
>> and you should not do half of each.
>>
>> Personlly, I am interested in both topics, and willing to
>> mentor/co-mentor either, however, IMO it two projects, not one.
>>
>> Thanks
>> Srinath
>>
>>
>>
>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
>> <an...@gmail.com> wrote:
>> > Probably the right answer to this question is: none. Why would the
>> > individual nodes intercepting the messages use publish/subscribe? They
>> > should all send them to a central place (or back to the sender of the
>> > original request).
>> >
>> > Andreas
>> >
>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
>> > <sa...@gmail.com> wrote:
>> >>> Then we can take this to next level by adding this module to all
>> >>> services in a system , configuring modules to send all collected
>> >>> messages to a pub/sub channel, and subscribing to those messages via a
>> >>> UI, and depicting those messages through a UI.
>> >>
>> >> What pub/sub model do you plan to use? WS-Eventing?
>> >>
>> >> Samisa...
>> >> --
>> >> blog: http://samisa-abeysinghe.blogspot.com/
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >
>> >
>>
>>
>>
>> --
>> ============================
>> Srinath Perera, Ph.D.
>>   WSO2 Inc. http://wso2.com
>>    Blog: http://srinathsview.blogspot.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>> --
> Samisa Abeysinghe
> Director, Engineering - WSO2 Inc.
>
> http://wso2.com/ - "lean . enterprise . middleware"
>
> --
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Fri, Apr 2, 2010 at 8:23 AM, Samisa Abeysinghe <
samisa.abeysinghe@gmail.com> wrote:

>
>
> On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com> wrote:
>
>> Hi nilupa;
>>
>> IMO this thread has more than one concept. Initially, the version I
>> discussed with you is more of a message monitoring tool, which allow
>> an user to see which message goes through where in the system. It is
>> purely used for debugging services only, and a actually it distributed
>> TCP monitor that helps people debug their distributed application.
>> Features I had in my mind was to
>>
>> 1. Ability to turn on/off message interception
>> 2. When tuned on, it show the system in a UI and let me see all
>> messages went through each link I selected.
>> 3. If messages have a some ID that identify a message sequence, it
>> should let me select a message and find sequence that generated/ lead
>> to that message.
>>
>> Basically, as I mentioned before, it is debugging tool and not a
>> monitoring tool.
>>
>
> If it is a debugging tool, that is going to be useful.
>

And I think, it would be useful to list out the features it supports.

I would also personally prefer some additions to the existing TCPMon where
we can add plain text display of secure messages. However, I know that it is
complicated, as the middle man is not supposed to change the ttl and the
like in WS Security.

Samisa...

>
> Samisa...
>
>
>>
>> Then, there is a BAM/System monitoring aspect we heavily discussed in
>> this thread, which is used to monitor a running system. IMHO they are
>> different,yet very useful. However, they are targeted at two audiences
>> and you should not do half of each.
>>
>> Personlly, I am interested in both topics, and willing to
>> mentor/co-mentor either, however, IMO it two projects, not one.
>>
>> Thanks
>> Srinath
>>
>>
>>
>> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
>> <an...@gmail.com> wrote:
>> > Probably the right answer to this question is: none. Why would the
>> > individual nodes intercepting the messages use publish/subscribe? They
>> > should all send them to a central place (or back to the sender of the
>> > original request).
>> >
>> > Andreas
>> >
>> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
>> > <sa...@gmail.com> wrote:
>> >>> Then we can take this to next level by adding this module to all
>> >>> services in a system , configuring modules to send all collected
>> >>> messages to a pub/sub channel, and subscribing to those messages via a
>> >>> UI, and depicting those messages through a UI.
>> >>
>> >> What pub/sub model do you plan to use? WS-Eventing?
>> >>
>> >> Samisa...
>> >> --
>> >> blog: http://samisa-abeysinghe.blogspot.com/
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >
>> >
>>
>>
>>
>> --
>> ============================
>> Srinath Perera, Ph.D.
>>   WSO2 Inc. http://wso2.com
>>    Blog: http://srinathsview.blogspot.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>> --
> Samisa Abeysinghe
> Director, Engineering - WSO2 Inc.
>
> http://wso2.com/ - "lean . enterprise . middleware"
>
> --
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com> wrote:

> Hi nilupa;
>
> IMO this thread has more than one concept. Initially, the version I
> discussed with you is more of a message monitoring tool, which allow
> an user to see which message goes through where in the system. It is
> purely used for debugging services only, and a actually it distributed
> TCP monitor that helps people debug their distributed application.
> Features I had in my mind was to
>
> 1. Ability to turn on/off message interception
> 2. When tuned on, it show the system in a UI and let me see all
> messages went through each link I selected.
> 3. If messages have a some ID that identify a message sequence, it
> should let me select a message and find sequence that generated/ lead
> to that message.
>
> Basically, as I mentioned before, it is debugging tool and not a
> monitoring tool.
>

If it is a debugging tool, that is going to be useful.

Samisa...


>
> Then, there is a BAM/System monitoring aspect we heavily discussed in
> this thread, which is used to monitor a running system. IMHO they are
> different,yet very useful. However, they are targeted at two audiences
> and you should not do half of each.
>
> Personlly, I am interested in both topics, and willing to
> mentor/co-mentor either, however, IMO it two projects, not one.
>
> Thanks
> Srinath
>
>
>
> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
> <an...@gmail.com> wrote:
> > Probably the right answer to this question is: none. Why would the
> > individual nodes intercepting the messages use publish/subscribe? They
> > should all send them to a central place (or back to the sender of the
> > original request).
> >
> > Andreas
> >
> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> > <sa...@gmail.com> wrote:
> >>> Then we can take this to next level by adding this module to all
> >>> services in a system , configuring modules to send all collected
> >>> messages to a pub/sub channel, and subscribing to those messages via a
> >>> UI, and depicting those messages through a UI.
> >>
> >> What pub/sub model do you plan to use? WS-Eventing?
> >>
> >> Samisa...
> >> --
> >> blog: http://samisa-abeysinghe.blogspot.com/
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >
> >
>
>
>
> --
> ============================
> Srinath Perera, Ph.D.
>   WSO2 Inc. http://wso2.com
>    Blog: http://srinathsview.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> --
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com> wrote:

> Hi nilupa;
>
> IMO this thread has more than one concept. Initially, the version I
> discussed with you is more of a message monitoring tool, which allow
> an user to see which message goes through where in the system. It is
> purely used for debugging services only, and a actually it distributed
> TCP monitor that helps people debug their distributed application.
> Features I had in my mind was to
>
> 1. Ability to turn on/off message interception
> 2. When tuned on, it show the system in a UI and let me see all
> messages went through each link I selected.
> 3. If messages have a some ID that identify a message sequence, it
> should let me select a message and find sequence that generated/ lead
> to that message.
>
> Basically, as I mentioned before, it is debugging tool and not a
> monitoring tool.
>

If it is a debugging tool, that is going to be useful.

Samisa...


>
> Then, there is a BAM/System monitoring aspect we heavily discussed in
> this thread, which is used to monitor a running system. IMHO they are
> different,yet very useful. However, they are targeted at two audiences
> and you should not do half of each.
>
> Personlly, I am interested in both topics, and willing to
> mentor/co-mentor either, however, IMO it two projects, not one.
>
> Thanks
> Srinath
>
>
>
> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
> <an...@gmail.com> wrote:
> > Probably the right answer to this question is: none. Why would the
> > individual nodes intercepting the messages use publish/subscribe? They
> > should all send them to a central place (or back to the sender of the
> > original request).
> >
> > Andreas
> >
> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> > <sa...@gmail.com> wrote:
> >>> Then we can take this to next level by adding this module to all
> >>> services in a system , configuring modules to send all collected
> >>> messages to a pub/sub channel, and subscribing to those messages via a
> >>> UI, and depicting those messages through a UI.
> >>
> >> What pub/sub model do you plan to use? WS-Eventing?
> >>
> >> Samisa...
> >> --
> >> blog: http://samisa-abeysinghe.blogspot.com/
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >
> >
>
>
>
> --
> ============================
> Srinath Perera, Ph.D.
>   WSO2 Inc. http://wso2.com
>    Blog: http://srinathsview.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> --
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com> wrote:

> Hi nilupa;
>
> IMO this thread has more than one concept. Initially, the version I
> discussed with you is more of a message monitoring tool, which allow
> an user to see which message goes through where in the system. It is
> purely used for debugging services only, and a actually it distributed
> TCP monitor that helps people debug their distributed application.
> Features I had in my mind was to
>
> 1. Ability to turn on/off message interception
> 2. When tuned on, it show the system in a UI and let me see all
> messages went through each link I selected.
> 3. If messages have a some ID that identify a message sequence, it
> should let me select a message and find sequence that generated/ lead
> to that message.
>
> Basically, as I mentioned before, it is debugging tool and not a
> monitoring tool.
>

If it is a debugging tool, that is going to be useful.

Samisa...


>
> Then, there is a BAM/System monitoring aspect we heavily discussed in
> this thread, which is used to monitor a running system. IMHO they are
> different,yet very useful. However, they are targeted at two audiences
> and you should not do half of each.
>
> Personlly, I am interested in both topics, and willing to
> mentor/co-mentor either, however, IMO it two projects, not one.
>
> Thanks
> Srinath
>
>
>
> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
> <an...@gmail.com> wrote:
> > Probably the right answer to this question is: none. Why would the
> > individual nodes intercepting the messages use publish/subscribe? They
> > should all send them to a central place (or back to the sender of the
> > original request).
> >
> > Andreas
> >
> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> > <sa...@gmail.com> wrote:
> >>> Then we can take this to next level by adding this module to all
> >>> services in a system , configuring modules to send all collected
> >>> messages to a pub/sub channel, and subscribing to those messages via a
> >>> UI, and depicting those messages through a UI.
> >>
> >> What pub/sub model do you plan to use? WS-Eventing?
> >>
> >> Samisa...
> >> --
> >> blog: http://samisa-abeysinghe.blogspot.com/
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >
> >
>
>
>
> --
> ============================
> Srinath Perera, Ph.D.
>   WSO2 Inc. http://wso2.com
>    Blog: http://srinathsview.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> --
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com> wrote:

> Hi nilupa;
>
> IMO this thread has more than one concept. Initially, the version I
> discussed with you is more of a message monitoring tool, which allow
> an user to see which message goes through where in the system. It is
> purely used for debugging services only, and a actually it distributed
> TCP monitor that helps people debug their distributed application.
> Features I had in my mind was to
>
> 1. Ability to turn on/off message interception
> 2. When tuned on, it show the system in a UI and let me see all
> messages went through each link I selected.
> 3. If messages have a some ID that identify a message sequence, it
> should let me select a message and find sequence that generated/ lead
> to that message.
>
> Basically, as I mentioned before, it is debugging tool and not a
> monitoring tool.
>

If it is a debugging tool, that is going to be useful.

Samisa...


>
> Then, there is a BAM/System monitoring aspect we heavily discussed in
> this thread, which is used to monitor a running system. IMHO they are
> different,yet very useful. However, they are targeted at two audiences
> and you should not do half of each.
>
> Personlly, I am interested in both topics, and willing to
> mentor/co-mentor either, however, IMO it two projects, not one.
>
> Thanks
> Srinath
>
>
>
> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
> <an...@gmail.com> wrote:
> > Probably the right answer to this question is: none. Why would the
> > individual nodes intercepting the messages use publish/subscribe? They
> > should all send them to a central place (or back to the sender of the
> > original request).
> >
> > Andreas
> >
> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> > <sa...@gmail.com> wrote:
> >>> Then we can take this to next level by adding this module to all
> >>> services in a system , configuring modules to send all collected
> >>> messages to a pub/sub channel, and subscribing to those messages via a
> >>> UI, and depicting those messages through a UI.
> >>
> >> What pub/sub model do you plan to use? WS-Eventing?
> >>
> >> Samisa...
> >> --
> >> blog: http://samisa-abeysinghe.blogspot.com/
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >
> >
>
>
>
> --
> ============================
> Srinath Perera, Ph.D.
>   WSO2 Inc. http://wso2.com
>    Blog: http://srinathsview.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> --
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Fri, Apr 2, 2010 at 7:12 AM, Srinath Perera <he...@gmail.com> wrote:

> Hi nilupa;
>
> IMO this thread has more than one concept. Initially, the version I
> discussed with you is more of a message monitoring tool, which allow
> an user to see which message goes through where in the system. It is
> purely used for debugging services only, and a actually it distributed
> TCP monitor that helps people debug their distributed application.
> Features I had in my mind was to
>
> 1. Ability to turn on/off message interception
> 2. When tuned on, it show the system in a UI and let me see all
> messages went through each link I selected.
> 3. If messages have a some ID that identify a message sequence, it
> should let me select a message and find sequence that generated/ lead
> to that message.
>
> Basically, as I mentioned before, it is debugging tool and not a
> monitoring tool.
>

If it is a debugging tool, that is going to be useful.

Samisa...


>
> Then, there is a BAM/System monitoring aspect we heavily discussed in
> this thread, which is used to monitor a running system. IMHO they are
> different,yet very useful. However, they are targeted at two audiences
> and you should not do half of each.
>
> Personlly, I am interested in both topics, and willing to
> mentor/co-mentor either, however, IMO it two projects, not one.
>
> Thanks
> Srinath
>
>
>
> On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
> <an...@gmail.com> wrote:
> > Probably the right answer to this question is: none. Why would the
> > individual nodes intercepting the messages use publish/subscribe? They
> > should all send them to a central place (or back to the sender of the
> > original request).
> >
> > Andreas
> >
> > On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> > <sa...@gmail.com> wrote:
> >>> Then we can take this to next level by adding this module to all
> >>> services in a system , configuring modules to send all collected
> >>> messages to a pub/sub channel, and subscribing to those messages via a
> >>> UI, and depicting those messages through a UI.
> >>
> >> What pub/sub model do you plan to use? WS-Eventing?
> >>
> >> Samisa...
> >> --
> >> blog: http://samisa-abeysinghe.blogspot.com/
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >
> >
>
>
>
> --
> ============================
> Srinath Perera, Ph.D.
>   WSO2 Inc. http://wso2.com
>    Blog: http://srinathsview.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> --
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Srinath Perera <he...@gmail.com>.

Hi nilupa;

IMO this thread has more than one concept. Initially, the version I
discussed with you is more of a message monitoring tool, which allow
an user to see which message goes through where in the system. It is
purely used for debugging services only, and a actually it distributed
TCP monitor that helps people debug their distributed application.
Features I had in my mind was to

1. Ability to turn on/off message interception
2. When tuned on, it show the system in a UI and let me see all
messages went through each link I selected.
3. If messages have a some ID that identify a message sequence, it
should let me select a message and find sequence that generated/ lead
to that message.

Basically, as I mentioned before, it is debugging tool and not a
monitoring tool.

Then, there is a BAM/System monitoring aspect we heavily discussed in
this thread, which is used to monitor a running system. IMHO they are
different,yet very useful. However, they are targeted at two audiences
and you should not do half of each.

Personlly, I am interested in both topics, and willing to
mentor/co-mentor either, however, IMO it two projects, not one.

Thanks
Srinath

On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
<an...@gmail.com> wrote:
> Probably the right answer to this question is: none. Why would the
> individual nodes intercepting the messages use publish/subscribe? They
> should all send them to a central place (or back to the sender of the
> original request).
>
> Andreas
>
> On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>> Then we can take this to next level by adding this module to all
>>> services in a system , configuring modules to send all collected
>>> messages to a pub/sub channel, and subscribing to those messages via a
>>> UI, and depicting those messages through a UI.
>>
>> What pub/sub model do you plan to use? WS-Eventing?
>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

-- 
============================
Srinath Perera, Ph.D.
   WSO2 Inc. http://wso2.com
   Blog: http://srinathsview.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Srinath Perera <he...@gmail.com>.

Hi nilupa;

IMO this thread has more than one concept. Initially, the version I
discussed with you is more of a message monitoring tool, which allow
an user to see which message goes through where in the system. It is
purely used for debugging services only, and a actually it distributed
TCP monitor that helps people debug their distributed application.
Features I had in my mind was to

1. Ability to turn on/off message interception
2. When tuned on, it show the system in a UI and let me see all
messages went through each link I selected.
3. If messages have a some ID that identify a message sequence, it
should let me select a message and find sequence that generated/ lead
to that message.

Basically, as I mentioned before, it is debugging tool and not a
monitoring tool.

Then, there is a BAM/System monitoring aspect we heavily discussed in
this thread, which is used to monitor a running system. IMHO they are
different,yet very useful. However, they are targeted at two audiences
and you should not do half of each.

Personlly, I am interested in both topics, and willing to
mentor/co-mentor either, however, IMO it two projects, not one.

Thanks
Srinath

On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
<an...@gmail.com> wrote:
> Probably the right answer to this question is: none. Why would the
> individual nodes intercepting the messages use publish/subscribe? They
> should all send them to a central place (or back to the sender of the
> original request).
>
> Andreas
>
> On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>> Then we can take this to next level by adding this module to all
>>> services in a system , configuring modules to send all collected
>>> messages to a pub/sub channel, and subscribing to those messages via a
>>> UI, and depicting those messages through a UI.
>>
>> What pub/sub model do you plan to use? WS-Eventing?
>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

-- 
============================
Srinath Perera, Ph.D.
   WSO2 Inc. http://wso2.com
   Blog: http://srinathsview.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

This is true if it is a debugging tool and fires all messages to the
'central' place.

However, still, the 'firewall' aspect needs to be taken into account.

For example, the current dual channel is quite useless when not in same
network and there are firewalls. So async dual channel might not work in a
real setup debug.

Samisa...


On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
<an...@gmail.com>wrote:

> Probably the right answer to this question is: none. Why would the
> individual nodes intercepting the messages use publish/subscribe? They
> should all send them to a central place (or back to the sender of the
> original request).
>
> Andreas
>
> On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> >> Then we can take this to next level by adding this module to all
> >> services in a system , configuring modules to send all collected
> >> messages to a pub/sub channel, and subscribing to those messages via a
> >> UI, and depicting those messages through a UI.
> >
> > What pub/sub model do you plan to use? WS-Eventing?
> >
> > Samisa...
> > --
> > blog: http://samisa-abeysinghe.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> --
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Srinath Perera <he...@gmail.com>.

Hi nilupa;

IMO this thread has more than one concept. Initially, the version I
discussed with you is more of a message monitoring tool, which allow
an user to see which message goes through where in the system. It is
purely used for debugging services only, and a actually it distributed
TCP monitor that helps people debug their distributed application.
Features I had in my mind was to

1. Ability to turn on/off message interception
2. When tuned on, it show the system in a UI and let me see all
messages went through each link I selected.
3. If messages have a some ID that identify a message sequence, it
should let me select a message and find sequence that generated/ lead
to that message.

Basically, as I mentioned before, it is debugging tool and not a
monitoring tool.

Then, there is a BAM/System monitoring aspect we heavily discussed in
this thread, which is used to monitor a running system. IMHO they are
different,yet very useful. However, they are targeted at two audiences
and you should not do half of each.

Personlly, I am interested in both topics, and willing to
mentor/co-mentor either, however, IMO it two projects, not one.

Thanks
Srinath

On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
<an...@gmail.com> wrote:
> Probably the right answer to this question is: none. Why would the
> individual nodes intercepting the messages use publish/subscribe? They
> should all send them to a central place (or back to the sender of the
> original request).
>
> Andreas
>
> On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>> Then we can take this to next level by adding this module to all
>>> services in a system , configuring modules to send all collected
>>> messages to a pub/sub channel, and subscribing to those messages via a
>>> UI, and depicting those messages through a UI.
>>
>> What pub/sub model do you plan to use? WS-Eventing?
>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

-- 
============================
Srinath Perera, Ph.D.
   WSO2 Inc. http://wso2.com
   Blog: http://srinathsview.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

This is true if it is a debugging tool and fires all messages to the
'central' place.

However, still, the 'firewall' aspect needs to be taken into account.

For example, the current dual channel is quite useless when not in same
network and there are firewalls. So async dual channel might not work in a
real setup debug.

Samisa...


On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
<an...@gmail.com>wrote:

> Probably the right answer to this question is: none. Why would the
> individual nodes intercepting the messages use publish/subscribe? They
> should all send them to a central place (or back to the sender of the
> original request).
>
> Andreas
>
> On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> >> Then we can take this to next level by adding this module to all
> >> services in a system , configuring modules to send all collected
> >> messages to a pub/sub channel, and subscribing to those messages via a
> >> UI, and depicting those messages through a UI.
> >
> > What pub/sub model do you plan to use? WS-Eventing?
> >
> > Samisa...
> > --
> > blog: http://samisa-abeysinghe.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> --
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

This is true if it is a debugging tool and fires all messages to the
'central' place.

However, still, the 'firewall' aspect needs to be taken into account.

For example, the current dual channel is quite useless when not in same
network and there are firewalls. So async dual channel might not work in a
real setup debug.

Samisa...


On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
<an...@gmail.com>wrote:

> Probably the right answer to this question is: none. Why would the
> individual nodes intercepting the messages use publish/subscribe? They
> should all send them to a central place (or back to the sender of the
> original request).
>
> Andreas
>
> On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> >> Then we can take this to next level by adding this module to all
> >> services in a system , configuring modules to send all collected
> >> messages to a pub/sub channel, and subscribing to those messages via a
> >> UI, and depicting those messages through a UI.
> >
> > What pub/sub model do you plan to use? WS-Eventing?
> >
> > Samisa...
> > --
> > blog: http://samisa-abeysinghe.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> --
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

This is true if it is a debugging tool and fires all messages to the
'central' place.

However, still, the 'firewall' aspect needs to be taken into account.

For example, the current dual channel is quite useless when not in same
network and there are firewalls. So async dual channel might not work in a
real setup debug.

Samisa...


On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
<an...@gmail.com>wrote:

> Probably the right answer to this question is: none. Why would the
> individual nodes intercepting the messages use publish/subscribe? They
> should all send them to a central place (or back to the sender of the
> original request).
>
> Andreas
>
> On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> >> Then we can take this to next level by adding this module to all
> >> services in a system , configuring modules to send all collected
> >> messages to a pub/sub channel, and subscribing to those messages via a
> >> UI, and depicting those messages through a UI.
> >
> > What pub/sub model do you plan to use? WS-Eventing?
> >
> > Samisa...
> > --
> > blog: http://samisa-abeysinghe.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> --
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Srinath Perera <he...@gmail.com>.

Hi nilupa;

IMO this thread has more than one concept. Initially, the version I
discussed with you is more of a message monitoring tool, which allow
an user to see which message goes through where in the system. It is
purely used for debugging services only, and a actually it distributed
TCP monitor that helps people debug their distributed application.
Features I had in my mind was to

1. Ability to turn on/off message interception
2. When tuned on, it show the system in a UI and let me see all
messages went through each link I selected.
3. If messages have a some ID that identify a message sequence, it
should let me select a message and find sequence that generated/ lead
to that message.

Basically, as I mentioned before, it is debugging tool and not a
monitoring tool.

Then, there is a BAM/System monitoring aspect we heavily discussed in
this thread, which is used to monitor a running system. IMHO they are
different,yet very useful. However, they are targeted at two audiences
and you should not do half of each.

Personlly, I am interested in both topics, and willing to
mentor/co-mentor either, however, IMO it two projects, not one.

Thanks
Srinath

On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
<an...@gmail.com> wrote:
> Probably the right answer to this question is: none. Why would the
> individual nodes intercepting the messages use publish/subscribe? They
> should all send them to a central place (or back to the sender of the
> original request).
>
> Andreas
>
> On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>> Then we can take this to next level by adding this module to all
>>> services in a system , configuring modules to send all collected
>>> messages to a pub/sub channel, and subscribing to those messages via a
>>> UI, and depicting those messages through a UI.
>>
>> What pub/sub model do you plan to use? WS-Eventing?
>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

-- 
============================
Srinath Perera, Ph.D.
   WSO2 Inc. http://wso2.com
   Blog: http://srinathsview.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Srinath Perera <he...@gmail.com>.

Hi nilupa;

IMO this thread has more than one concept. Initially, the version I
discussed with you is more of a message monitoring tool, which allow
an user to see which message goes through where in the system. It is
purely used for debugging services only, and a actually it distributed
TCP monitor that helps people debug their distributed application.
Features I had in my mind was to

1. Ability to turn on/off message interception
2. When tuned on, it show the system in a UI and let me see all
messages went through each link I selected.
3. If messages have a some ID that identify a message sequence, it
should let me select a message and find sequence that generated/ lead
to that message.

Basically, as I mentioned before, it is debugging tool and not a
monitoring tool.

Then, there is a BAM/System monitoring aspect we heavily discussed in
this thread, which is used to monitor a running system. IMHO they are
different,yet very useful. However, they are targeted at two audiences
and you should not do half of each.

Personlly, I am interested in both topics, and willing to
mentor/co-mentor either, however, IMO it two projects, not one.

Thanks
Srinath

On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
<an...@gmail.com> wrote:
> Probably the right answer to this question is: none. Why would the
> individual nodes intercepting the messages use publish/subscribe? They
> should all send them to a central place (or back to the sender of the
> original request).
>
> Andreas
>
> On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>> Then we can take this to next level by adding this module to all
>>> services in a system , configuring modules to send all collected
>>> messages to a pub/sub channel, and subscribing to those messages via a
>>> UI, and depicting those messages through a UI.
>>
>> What pub/sub model do you plan to use? WS-Eventing?
>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

-- 
============================
Srinath Perera, Ph.D.
   WSO2 Inc. http://wso2.com
   Blog: http://srinathsview.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

This is true if it is a debugging tool and fires all messages to the
'central' place.

However, still, the 'firewall' aspect needs to be taken into account.

For example, the current dual channel is quite useless when not in same
network and there are firewalls. So async dual channel might not work in a
real setup debug.

Samisa...


On Fri, Apr 2, 2010 at 12:17 AM, Andreas Veithen
<an...@gmail.com>wrote:

> Probably the right answer to this question is: none. Why would the
> individual nodes intercepting the messages use publish/subscribe? They
> should all send them to a central place (or back to the sender of the
> original request).
>
> Andreas
>
> On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> >> Then we can take this to next level by adding this module to all
> >> services in a system , configuring modules to send all collected
> >> messages to a pub/sub channel, and subscribing to those messages via a
> >> UI, and depicting those messages through a UI.
> >
> > What pub/sub model do you plan to use? WS-Eventing?
> >
> > Samisa...
> > --
> > blog: http://samisa-abeysinghe.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> --
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

Probably the right answer to this question is: none. Why would the
individual nodes intercepting the messages use publish/subscribe? They
should all send them to a central place (or back to the sender of the
original request).

Andreas

On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI.
>
> What pub/sub model do you plan to use? WS-Eventing?
>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

Probably the right answer to this question is: none. Why would the
individual nodes intercepting the messages use publish/subscribe? They
should all send them to a central place (or back to the sender of the
original request).

Andreas

On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI.
>
> What pub/sub model do you plan to use? WS-Eventing?
>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Wed, Mar 31, 2010 at 8:25 PM, Sanjiva Weerawarana
<sa...@opensource.lk>wrote:

> Samisa can you describe the architecture of the WSO2 BAM server? Maybe we
> can build on that .. some of what Glen described in his blog is a cool way
> to achieve correlation. Its also a clever idea to capture the logs thru
> log4j instead of thru a module.
>

We pump data using an eventing model to a database, from server to BAM.

Data is stored in the DB.

Expose the data in the DB as data services.

Use those data services and pull information to the presentation layer, and
present using Shinding based gadget technology.

This, of course is 30K ft view.

Samisa...



>
> Sanjiva.
>
>
> On Wed, Mar 31, 2010 at 8:21 PM, Samisa Abeysinghe <
> samisa.abeysinghe@gmail.com> wrote:
>
>> Then we can take this to next level by adding this module to all
>>> services in a system , configuring modules to send all collected
>>> messages to a pub/sub channel, and subscribing to those messages via a
>>> UI, and depicting those messages through a UI.
>>
>>
>> What pub/sub model do you plan to use? WS-Eventing?
>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>>
>
>
>
> --
> Sanjiva Weerawarana, Ph.D.
> Founder, Director & Chief Scientist; Lanka Software Foundation;
> http://www.opensource.lk/
> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> Member; Apache Software Foundation; http://www.apache.org/
> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>
> Blog: http://sanjiva.weerawarana.org/
>
-- 
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Wed, Mar 31, 2010 at 8:25 PM, Sanjiva Weerawarana
<sa...@opensource.lk>wrote:

> Samisa can you describe the architecture of the WSO2 BAM server? Maybe we
> can build on that .. some of what Glen described in his blog is a cool way
> to achieve correlation. Its also a clever idea to capture the logs thru
> log4j instead of thru a module.
>

We pump data using an eventing model to a database, from server to BAM.

Data is stored in the DB.

Expose the data in the DB as data services.

Use those data services and pull information to the presentation layer, and
present using Shinding based gadget technology.

This, of course is 30K ft view.

Samisa...



>
> Sanjiva.
>
>
> On Wed, Mar 31, 2010 at 8:21 PM, Samisa Abeysinghe <
> samisa.abeysinghe@gmail.com> wrote:
>
>> Then we can take this to next level by adding this module to all
>>> services in a system , configuring modules to send all collected
>>> messages to a pub/sub channel, and subscribing to those messages via a
>>> UI, and depicting those messages through a UI.
>>
>>
>> What pub/sub model do you plan to use? WS-Eventing?
>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>>
>
>
>
> --
> Sanjiva Weerawarana, Ph.D.
> Founder, Director & Chief Scientist; Lanka Software Foundation;
> http://www.opensource.lk/
> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> Member; Apache Software Foundation; http://www.apache.org/
> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>
> Blog: http://sanjiva.weerawarana.org/
>
-- 
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Wed, Mar 31, 2010 at 8:25 PM, Sanjiva Weerawarana
<sa...@opensource.lk>wrote:

> Samisa can you describe the architecture of the WSO2 BAM server? Maybe we
> can build on that .. some of what Glen described in his blog is a cool way
> to achieve correlation. Its also a clever idea to capture the logs thru
> log4j instead of thru a module.
>

We pump data using an eventing model to a database, from server to BAM.

Data is stored in the DB.

Expose the data in the DB as data services.

Use those data services and pull information to the presentation layer, and
present using Shinding based gadget technology.

This, of course is 30K ft view.

Samisa...



>
> Sanjiva.
>
>
> On Wed, Mar 31, 2010 at 8:21 PM, Samisa Abeysinghe <
> samisa.abeysinghe@gmail.com> wrote:
>
>> Then we can take this to next level by adding this module to all
>>> services in a system , configuring modules to send all collected
>>> messages to a pub/sub channel, and subscribing to those messages via a
>>> UI, and depicting those messages through a UI.
>>
>>
>> What pub/sub model do you plan to use? WS-Eventing?
>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>>
>
>
>
> --
> Sanjiva Weerawarana, Ph.D.
> Founder, Director & Chief Scientist; Lanka Software Foundation;
> http://www.opensource.lk/
> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> Member; Apache Software Foundation; http://www.apache.org/
> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>
> Blog: http://sanjiva.weerawarana.org/
>
-- 
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Wed, Mar 31, 2010 at 8:25 PM, Sanjiva Weerawarana
<sa...@opensource.lk>wrote:

> Samisa can you describe the architecture of the WSO2 BAM server? Maybe we
> can build on that .. some of what Glen described in his blog is a cool way
> to achieve correlation. Its also a clever idea to capture the logs thru
> log4j instead of thru a module.
>

We pump data using an eventing model to a database, from server to BAM.

Data is stored in the DB.

Expose the data in the DB as data services.

Use those data services and pull information to the presentation layer, and
present using Shinding based gadget technology.

This, of course is 30K ft view.

Samisa...



>
> Sanjiva.
>
>
> On Wed, Mar 31, 2010 at 8:21 PM, Samisa Abeysinghe <
> samisa.abeysinghe@gmail.com> wrote:
>
>> Then we can take this to next level by adding this module to all
>>> services in a system , configuring modules to send all collected
>>> messages to a pub/sub channel, and subscribing to those messages via a
>>> UI, and depicting those messages through a UI.
>>
>>
>> What pub/sub model do you plan to use? WS-Eventing?
>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>>
>
>
>
> --
> Sanjiva Weerawarana, Ph.D.
> Founder, Director & Chief Scientist; Lanka Software Foundation;
> http://www.opensource.lk/
> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> Member; Apache Software Foundation; http://www.apache.org/
> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>
> Blog: http://sanjiva.weerawarana.org/
>
-- 
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Wed, Mar 31, 2010 at 8:25 PM, Sanjiva Weerawarana
<sa...@opensource.lk>wrote:

> Samisa can you describe the architecture of the WSO2 BAM server? Maybe we
> can build on that .. some of what Glen described in his blog is a cool way
> to achieve correlation. Its also a clever idea to capture the logs thru
> log4j instead of thru a module.
>

We pump data using an eventing model to a database, from server to BAM.

Data is stored in the DB.

Expose the data in the DB as data services.

Use those data services and pull information to the presentation layer, and
present using Shinding based gadget technology.

This, of course is 30K ft view.

Samisa...



>
> Sanjiva.
>
>
> On Wed, Mar 31, 2010 at 8:21 PM, Samisa Abeysinghe <
> samisa.abeysinghe@gmail.com> wrote:
>
>> Then we can take this to next level by adding this module to all
>>> services in a system , configuring modules to send all collected
>>> messages to a pub/sub channel, and subscribing to those messages via a
>>> UI, and depicting those messages through a UI.
>>
>>
>> What pub/sub model do you plan to use? WS-Eventing?
>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>>
>
>
>
> --
> Sanjiva Weerawarana, Ph.D.
> Founder, Director & Chief Scientist; Lanka Software Foundation;
> http://www.opensource.lk/
> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> Member; Apache Software Foundation; http://www.apache.org/
> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>
> Blog: http://sanjiva.weerawarana.org/
>
-- 
Samisa Abeysinghe
Director, Engineering - WSO2 Inc.

http://wso2.com/ - "lean . enterprise . middleware"

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanjiva Weerawarana <sa...@opensource.lk>.

Samisa can you describe the architecture of the WSO2 BAM server? Maybe we
can build on that .. some of what Glen described in his blog is a cool way
to achieve correlation. Its also a clever idea to capture the logs thru
log4j instead of thru a module.

Sanjiva.

On Wed, Mar 31, 2010 at 8:21 PM, Samisa Abeysinghe <
samisa.abeysinghe@gmail.com> wrote:

> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI.
>
>
> What pub/sub model do you plan to use? WS-Eventing?
>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/
>



-- 
Sanjiva Weerawarana, Ph.D.
Founder, Director & Chief Scientist; Lanka Software Foundation;
http://www.opensource.lk/
Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
Member; Apache Software Foundation; http://www.apache.org/
Director; Sahana Software Foundation; http://www.sahanafoundation.org/
Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/

Blog: http://sanjiva.weerawarana.org/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanjiva Weerawarana <sa...@opensource.lk>.

Samisa can you describe the architecture of the WSO2 BAM server? Maybe we
can build on that .. some of what Glen described in his blog is a cool way
to achieve correlation. Its also a clever idea to capture the logs thru
log4j instead of thru a module.

Sanjiva.

On Wed, Mar 31, 2010 at 8:21 PM, Samisa Abeysinghe <
samisa.abeysinghe@gmail.com> wrote:

> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI.
>
>
> What pub/sub model do you plan to use? WS-Eventing?
>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/
>



-- 
Sanjiva Weerawarana, Ph.D.
Founder, Director & Chief Scientist; Lanka Software Foundation;
http://www.opensource.lk/
Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
Member; Apache Software Foundation; http://www.apache.org/
Director; Sahana Software Foundation; http://www.sahanafoundation.org/
Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/

Blog: http://sanjiva.weerawarana.org/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

Probably the right answer to this question is: none. Why would the
individual nodes intercepting the messages use publish/subscribe? They
should all send them to a central place (or back to the sender of the
original request).

Andreas

On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI.
>
> What pub/sub model do you plan to use? WS-Eventing?
>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

On Wed, Mar 31, 2010 at 4:51 PM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI.
>
> What pub/sub model do you plan to use?  WS-Eventing?

Something which allows the publisher ( or the Log service which
collects log messages from different nodes) to notify its subscribers
(or the clients which show the topology of particular message
sequence) the arrival of a message which belongs to a particular
sequence of messages that they are interested in. Is far as I
understood this notification will be based on SOAP. WS-Eventing sounds
like something that can be used but I guess I have to explore more
(rather than reading some introductory notes on WS-Eventing) to give a
concrete answer.

Best Regards,
Nilupa

>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

On Wed, Mar 31, 2010 at 4:51 PM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI.
>
> What pub/sub model do you plan to use?  WS-Eventing?

Something which allows the publisher ( or the Log service which
collects log messages from different nodes) to notify its subscribers
(or the clients which show the topology of particular message
sequence) the arrival of a message which belongs to a particular
sequence of messages that they are interested in. Is far as I
understood this notification will be based on SOAP. WS-Eventing sounds
like something that can be used but I guess I have to explore more
(rather than reading some introductory notes on WS-Eventing) to give a
concrete answer.

Best Regards,
Nilupa

>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

Probably the right answer to this question is: none. Why would the
individual nodes intercepting the messages use publish/subscribe? They
should all send them to a central place (or back to the sender of the
original request).

Andreas

On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI.
>
> What pub/sub model do you plan to use? WS-Eventing?
>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

On Wed, Mar 31, 2010 at 4:51 PM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI.
>
> What pub/sub model do you plan to use?  WS-Eventing?

Something which allows the publisher ( or the Log service which
collects log messages from different nodes) to notify its subscribers
(or the clients which show the topology of particular message
sequence) the arrival of a message which belongs to a particular
sequence of messages that they are interested in. Is far as I
understood this notification will be based on SOAP. WS-Eventing sounds
like something that can be used but I guess I have to explore more
(rather than reading some introductory notes on WS-Eventing) to give a
concrete answer.

Best Regards,
Nilupa

>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

Probably the right answer to this question is: none. Why would the
individual nodes intercepting the messages use publish/subscribe? They
should all send them to a central place (or back to the sender of the
original request).

Andreas

On Wed, Mar 31, 2010 at 16:51, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI.
>
> What pub/sub model do you plan to use? WS-Eventing?
>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanjiva Weerawarana <sa...@opensource.lk>.

Samisa can you describe the architecture of the WSO2 BAM server? Maybe we
can build on that .. some of what Glen described in his blog is a cool way
to achieve correlation. Its also a clever idea to capture the logs thru
log4j instead of thru a module.

Sanjiva.

On Wed, Mar 31, 2010 at 8:21 PM, Samisa Abeysinghe <
samisa.abeysinghe@gmail.com> wrote:

> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI.
>
>
> What pub/sub model do you plan to use? WS-Eventing?
>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/
>



-- 
Sanjiva Weerawarana, Ph.D.
Founder, Director & Chief Scientist; Lanka Software Foundation;
http://www.opensource.lk/
Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
Member; Apache Software Foundation; http://www.apache.org/
Director; Sahana Software Foundation; http://www.sahanafoundation.org/
Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/

Blog: http://sanjiva.weerawarana.org/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

On Wed, Mar 31, 2010 at 4:51 PM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI.
>
> What pub/sub model do you plan to use?  WS-Eventing?

Something which allows the publisher ( or the Log service which
collects log messages from different nodes) to notify its subscribers
(or the clients which show the topology of particular message
sequence) the arrival of a message which belongs to a particular
sequence of messages that they are interested in. Is far as I
understood this notification will be based on SOAP. WS-Eventing sounds
like something that can be used but I guess I have to explore more
(rather than reading some introductory notes on WS-Eventing) to give a
concrete answer.

Best Regards,
Nilupa

>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanjiva Weerawarana <sa...@opensource.lk>.

Samisa can you describe the architecture of the WSO2 BAM server? Maybe we
can build on that .. some of what Glen described in his blog is a cool way
to achieve correlation. Its also a clever idea to capture the logs thru
log4j instead of thru a module.

Sanjiva.

On Wed, Mar 31, 2010 at 8:21 PM, Samisa Abeysinghe <
samisa.abeysinghe@gmail.com> wrote:

> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI.
>
>
> What pub/sub model do you plan to use? WS-Eventing?
>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/
>



-- 
Sanjiva Weerawarana, Ph.D.
Founder, Director & Chief Scientist; Lanka Software Foundation;
http://www.opensource.lk/
Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
Member; Apache Software Foundation; http://www.apache.org/
Director; Sahana Software Foundation; http://www.sahanafoundation.org/
Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/

Blog: http://sanjiva.weerawarana.org/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanjiva Weerawarana <sa...@opensource.lk>.

Samisa can you describe the architecture of the WSO2 BAM server? Maybe we
can build on that .. some of what Glen described in his blog is a cool way
to achieve correlation. Its also a clever idea to capture the logs thru
log4j instead of thru a module.

Sanjiva.

On Wed, Mar 31, 2010 at 8:21 PM, Samisa Abeysinghe <
samisa.abeysinghe@gmail.com> wrote:

> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI.
>
>
> What pub/sub model do you plan to use? WS-Eventing?
>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/
>



-- 
Sanjiva Weerawarana, Ph.D.
Founder, Director & Chief Scientist; Lanka Software Foundation;
http://www.opensource.lk/
Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
Member; Apache Software Foundation; http://www.apache.org/
Director; Sahana Software Foundation; http://www.sahanafoundation.org/
Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/

Blog: http://sanjiva.weerawarana.org/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

>
> Then we can take this to next level by adding this module to all
> services in a system , configuring modules to send all collected
> messages to a pub/sub channel, and subscribing to those messages via a
> UI, and depicting those messages through a UI.


What pub/sub model do you plan to use? WS-Eventing?

Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

On Tue, Mar 30, 2010 at 15:11, Glen Daniels <gl...@thoughtcraft.com> wrote:
> I'm a really terribly inconsistent blogger.  However, my last actual entry,
> from 2009, was about exactly this:
>
> http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html

That comes pretty damn close to what I was thinking about.

> I like Andreas' suggestion below, but I think it might be a bit heavyweight
> for a GSoC project(?).  Maybe we could lower the bar to something like this:

Hey, they get 5000$, so they should sweat a bit ;-)

> * Build a distributed logging web service which can accept log messages and
> correlationIDs, logging to a file per correlationID.
>
> * Build a Module that processes incoming <debug:logInfo> headers and makes an
> appropriate RemoteLogger (or whatever) available.  This could, at first, go
> in one of Axis2's contexts (OperationContext?) as a demonstration.
> Components who call RemoteLogger.log() send messages (hopefully on another
> thread) to the central log service.
>
> * The Module could automatically put the right metadata on outgoing messages,
> but only the responses, so you still have the problem Andreas points out
> about needing the app to transfer the logging info over to new
> ServiceClients.  I was imagining that could be achieved by also setting a
> ThreadLocal - so if your clients run on the same thread you're all set, and
> if there are new threads it's up to the framework to copy that ThreadLocal over.

Exactly. A simple ThreadLocal will probably cover the vast majority of
use cases, and for asynchronous executions, one can probably implement
some magic that propagates the context from one thread to the other.

> * Regardless of the discovery mechanism, you'd want something static like
> RemoteLogger.getLogger() to hide it.
>
> This approach is definitely a bit more intrusive, as it relies on the
> components themselves to be looking for a RemoteLogger, but it seems like a
> reasonable starting point which could be evolved.  However, maybe the AOP
> stuff isn't as challenging as I'm imagining....

It all depends on the layer in which you want to intercept the
messages (one extreme would be an around advice on
GenericServlet#service) and the protocols you want to cover (think
about non-SOAP JMS messages, remote EJB calls, etc.).

> Thoughts?
>
> --Glen
>
> On 3/30/2010 4:57 AM, Andreas Veithen wrote:
>> I was actually referring to scenarios where a service may call other
>> services, which in turn may call again other services. In these
>> scenarios, it is not sufficient to simply collect received/sent
>> messages on different hosts, unless the system is isolated
>> (development environment) or the request rate is sufficiently low so
>> that messages can be correlated based on time. Here is a concrete
>> scenario from a project in my company:
>>
>> - Request comes in on an ESB that does security and validation.
>> - Request is processed by an application server which persists the
>> received information and publishes a JMS message (pub/sub events).
>> - The event is consumed by one or more components that may in turn
>> interact with other services.
>>
>> If you want track this flow, it is not sufficient to intercept the
>> messages on the different hosts: in addition, you need to instrument
>> the services so that the outgoing requests are correlated with the
>> incoming requests (by adding SOAP and/or transport headers). What I
>> would like to be able to do in the above scenario is to enable
>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> HTTP header to the initial request to enable logging and this
>> information is propagated along the chain. Every service/component
>> then sends a copy of the incoming/outgoing requests/responses to a
>> central place where the sequence of events is reconstructed.
>>
>> One way this could be achieved is with a tool that postprocesses the
>> artifacts from the build process. For each artifact, the tool would
>> disassemble the artifact, instrument the code using a set of AspectJ
>> aspects and reassemble the artifact for deployment. The responsibility
>> of the aspects would be to intercept messages, log them and modify
>> them to ensure proper correlation. Of course this only works if the
>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> use AspectJ for a similar use case (although at a much smaller scale)
>> in the transport test suites [1]. Interestingly, this approach would
>> allow to cover interactions that are not SOAP based, e.g. one could
>> even intercept the database queries executed during the flow.
>>
>> Andreas
>>
>> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>
>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>> Hi Nilupa;
>>>
>>> When we collect messages from a one location by installing proper
>>> handler that will intercept and send messages to to that one location
>>> (this one location can be a single server, pub/sub channel etc). There
>>> is many ways to make sense of those collected messages. What Andreas
>>> mentioned (following complete transaction) is a one possibility.
>>>
>>> I think you should come up with few scenarios on how you would make
>>> sense of the message.
>>>
>>> Thanks
>>> Srinath
>>>
>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> First let me thank you for commenting.
>>>>
>>>> As far as I understood, what you would like to see from the proposed
>>>> tool is to view set of messages that are exchanged in reponse to a
>>>> particular input message. With the understanding that I am having at
>>>> the momnet, one way to do it is to filter out the central repository
>>>> of messages based on 'To' , 'From' headers and try to contruct the
>>>> message chain from it. We can allow the client GUI wich connects to
>>>> the central repository to provide the paramenters (For instance the
>>>> value of 'To' header) from which an intelligent filtering can be done
>>>> for the set of messages avialable at the central repository.
>>>>
>>>> Perhaps someone has an idea of a better way of doing it and is willing
>>>> to share it with us. It would be really nice to hear from them.
>>>>
>>>> Thanks in advance ..!!
>>>>
>>>> Best Regards,
>>>> Nilupa
>>>>
>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>> <an...@gmail.com> wrote:
>>>>> Personally, I think that the added value of extending the SOAP monitor
>>>>> module to collect messages in a central place is too limited to
>>>>> attract enough interest from the user community, so that it will be
>>>>> difficult to ensure the evolution of such a project in the future.
>>>>> However, what many people are looking for is a tool that allows to
>>>>> track the flow of a message through a distributed system, or more
>>>>> generally to track the sequence of events triggered by a given input
>>>>> message (sort of end-to-end transaction monitoring).
>>>>>
>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>>> proposing a GSoC project this summer. I would like to take project
>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>>> services middleware. I will submit more detailed proposal later. I
>>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>>> most welcome.
>>>>>>
>>>>>> Best Regards,
>>>>>> Nilupa Bandara
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>> Distributed TCP monitor
>>>>>>
>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>>> pull those messages. Also, we can do something to turn the module on
>>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>>> on only when they want to debug the system.
>>>>>>
>>>>>> Then we can take this to next level by adding this module to all
>>>>>> services in a system , configuring modules to send all collected
>>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>>> TCPMon for a whole system.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> ============================
>>> Srinath Perera, Ph.D.
>>>   WSO2 Inc. http://wso2.com
>>>   Blog: http://srinathsview.blogspot.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

On Tue, Mar 30, 2010 at 15:11, Glen Daniels <gl...@thoughtcraft.com> wrote:
> I'm a really terribly inconsistent blogger.  However, my last actual entry,
> from 2009, was about exactly this:
>
> http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html

That comes pretty damn close to what I was thinking about.

> I like Andreas' suggestion below, but I think it might be a bit heavyweight
> for a GSoC project(?).  Maybe we could lower the bar to something like this:

Hey, they get 5000$, so they should sweat a bit ;-)

> * Build a distributed logging web service which can accept log messages and
> correlationIDs, logging to a file per correlationID.
>
> * Build a Module that processes incoming <debug:logInfo> headers and makes an
> appropriate RemoteLogger (or whatever) available.  This could, at first, go
> in one of Axis2's contexts (OperationContext?) as a demonstration.
> Components who call RemoteLogger.log() send messages (hopefully on another
> thread) to the central log service.
>
> * The Module could automatically put the right metadata on outgoing messages,
> but only the responses, so you still have the problem Andreas points out
> about needing the app to transfer the logging info over to new
> ServiceClients.  I was imagining that could be achieved by also setting a
> ThreadLocal - so if your clients run on the same thread you're all set, and
> if there are new threads it's up to the framework to copy that ThreadLocal over.

Exactly. A simple ThreadLocal will probably cover the vast majority of
use cases, and for asynchronous executions, one can probably implement
some magic that propagates the context from one thread to the other.

> * Regardless of the discovery mechanism, you'd want something static like
> RemoteLogger.getLogger() to hide it.
>
> This approach is definitely a bit more intrusive, as it relies on the
> components themselves to be looking for a RemoteLogger, but it seems like a
> reasonable starting point which could be evolved.  However, maybe the AOP
> stuff isn't as challenging as I'm imagining....

It all depends on the layer in which you want to intercept the
messages (one extreme would be an around advice on
GenericServlet#service) and the protocols you want to cover (think
about non-SOAP JMS messages, remote EJB calls, etc.).

> Thoughts?
>
> --Glen
>
> On 3/30/2010 4:57 AM, Andreas Veithen wrote:
>> I was actually referring to scenarios where a service may call other
>> services, which in turn may call again other services. In these
>> scenarios, it is not sufficient to simply collect received/sent
>> messages on different hosts, unless the system is isolated
>> (development environment) or the request rate is sufficiently low so
>> that messages can be correlated based on time. Here is a concrete
>> scenario from a project in my company:
>>
>> - Request comes in on an ESB that does security and validation.
>> - Request is processed by an application server which persists the
>> received information and publishes a JMS message (pub/sub events).
>> - The event is consumed by one or more components that may in turn
>> interact with other services.
>>
>> If you want track this flow, it is not sufficient to intercept the
>> messages on the different hosts: in addition, you need to instrument
>> the services so that the outgoing requests are correlated with the
>> incoming requests (by adding SOAP and/or transport headers). What I
>> would like to be able to do in the above scenario is to enable
>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> HTTP header to the initial request to enable logging and this
>> information is propagated along the chain. Every service/component
>> then sends a copy of the incoming/outgoing requests/responses to a
>> central place where the sequence of events is reconstructed.
>>
>> One way this could be achieved is with a tool that postprocesses the
>> artifacts from the build process. For each artifact, the tool would
>> disassemble the artifact, instrument the code using a set of AspectJ
>> aspects and reassemble the artifact for deployment. The responsibility
>> of the aspects would be to intercept messages, log them and modify
>> them to ensure proper correlation. Of course this only works if the
>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> use AspectJ for a similar use case (although at a much smaller scale)
>> in the transport test suites [1]. Interestingly, this approach would
>> allow to cover interactions that are not SOAP based, e.g. one could
>> even intercept the database queries executed during the flow.
>>
>> Andreas
>>
>> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>
>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>> Hi Nilupa;
>>>
>>> When we collect messages from a one location by installing proper
>>> handler that will intercept and send messages to to that one location
>>> (this one location can be a single server, pub/sub channel etc). There
>>> is many ways to make sense of those collected messages. What Andreas
>>> mentioned (following complete transaction) is a one possibility.
>>>
>>> I think you should come up with few scenarios on how you would make
>>> sense of the message.
>>>
>>> Thanks
>>> Srinath
>>>
>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> First let me thank you for commenting.
>>>>
>>>> As far as I understood, what you would like to see from the proposed
>>>> tool is to view set of messages that are exchanged in reponse to a
>>>> particular input message. With the understanding that I am having at
>>>> the momnet, one way to do it is to filter out the central repository
>>>> of messages based on 'To' , 'From' headers and try to contruct the
>>>> message chain from it. We can allow the client GUI wich connects to
>>>> the central repository to provide the paramenters (For instance the
>>>> value of 'To' header) from which an intelligent filtering can be done
>>>> for the set of messages avialable at the central repository.
>>>>
>>>> Perhaps someone has an idea of a better way of doing it and is willing
>>>> to share it with us. It would be really nice to hear from them.
>>>>
>>>> Thanks in advance ..!!
>>>>
>>>> Best Regards,
>>>> Nilupa
>>>>
>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>> <an...@gmail.com> wrote:
>>>>> Personally, I think that the added value of extending the SOAP monitor
>>>>> module to collect messages in a central place is too limited to
>>>>> attract enough interest from the user community, so that it will be
>>>>> difficult to ensure the evolution of such a project in the future.
>>>>> However, what many people are looking for is a tool that allows to
>>>>> track the flow of a message through a distributed system, or more
>>>>> generally to track the sequence of events triggered by a given input
>>>>> message (sort of end-to-end transaction monitoring).
>>>>>
>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>>> proposing a GSoC project this summer. I would like to take project
>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>>> services middleware. I will submit more detailed proposal later. I
>>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>>> most welcome.
>>>>>>
>>>>>> Best Regards,
>>>>>> Nilupa Bandara
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>> Distributed TCP monitor
>>>>>>
>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>>> pull those messages. Also, we can do something to turn the module on
>>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>>> on only when they want to debug the system.
>>>>>>
>>>>>> Then we can take this to next level by adding this module to all
>>>>>> services in a system , configuring modules to send all collected
>>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>>> TCPMon for a whole system.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> ============================
>>> Srinath Perera, Ph.D.
>>>   WSO2 Inc. http://wso2.com
>>>   Blog: http://srinathsview.blogspot.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

On Tue, Mar 30, 2010 at 15:11, Glen Daniels <gl...@thoughtcraft.com> wrote:
> I'm a really terribly inconsistent blogger.  However, my last actual entry,
> from 2009, was about exactly this:
>
> http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html

That comes pretty damn close to what I was thinking about.

> I like Andreas' suggestion below, but I think it might be a bit heavyweight
> for a GSoC project(?).  Maybe we could lower the bar to something like this:

Hey, they get 5000$, so they should sweat a bit ;-)

> * Build a distributed logging web service which can accept log messages and
> correlationIDs, logging to a file per correlationID.
>
> * Build a Module that processes incoming <debug:logInfo> headers and makes an
> appropriate RemoteLogger (or whatever) available.  This could, at first, go
> in one of Axis2's contexts (OperationContext?) as a demonstration.
> Components who call RemoteLogger.log() send messages (hopefully on another
> thread) to the central log service.
>
> * The Module could automatically put the right metadata on outgoing messages,
> but only the responses, so you still have the problem Andreas points out
> about needing the app to transfer the logging info over to new
> ServiceClients.  I was imagining that could be achieved by also setting a
> ThreadLocal - so if your clients run on the same thread you're all set, and
> if there are new threads it's up to the framework to copy that ThreadLocal over.

Exactly. A simple ThreadLocal will probably cover the vast majority of
use cases, and for asynchronous executions, one can probably implement
some magic that propagates the context from one thread to the other.

> * Regardless of the discovery mechanism, you'd want something static like
> RemoteLogger.getLogger() to hide it.
>
> This approach is definitely a bit more intrusive, as it relies on the
> components themselves to be looking for a RemoteLogger, but it seems like a
> reasonable starting point which could be evolved.  However, maybe the AOP
> stuff isn't as challenging as I'm imagining....

It all depends on the layer in which you want to intercept the
messages (one extreme would be an around advice on
GenericServlet#service) and the protocols you want to cover (think
about non-SOAP JMS messages, remote EJB calls, etc.).

> Thoughts?
>
> --Glen
>
> On 3/30/2010 4:57 AM, Andreas Veithen wrote:
>> I was actually referring to scenarios where a service may call other
>> services, which in turn may call again other services. In these
>> scenarios, it is not sufficient to simply collect received/sent
>> messages on different hosts, unless the system is isolated
>> (development environment) or the request rate is sufficiently low so
>> that messages can be correlated based on time. Here is a concrete
>> scenario from a project in my company:
>>
>> - Request comes in on an ESB that does security and validation.
>> - Request is processed by an application server which persists the
>> received information and publishes a JMS message (pub/sub events).
>> - The event is consumed by one or more components that may in turn
>> interact with other services.
>>
>> If you want track this flow, it is not sufficient to intercept the
>> messages on the different hosts: in addition, you need to instrument
>> the services so that the outgoing requests are correlated with the
>> incoming requests (by adding SOAP and/or transport headers). What I
>> would like to be able to do in the above scenario is to enable
>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> HTTP header to the initial request to enable logging and this
>> information is propagated along the chain. Every service/component
>> then sends a copy of the incoming/outgoing requests/responses to a
>> central place where the sequence of events is reconstructed.
>>
>> One way this could be achieved is with a tool that postprocesses the
>> artifacts from the build process. For each artifact, the tool would
>> disassemble the artifact, instrument the code using a set of AspectJ
>> aspects and reassemble the artifact for deployment. The responsibility
>> of the aspects would be to intercept messages, log them and modify
>> them to ensure proper correlation. Of course this only works if the
>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> use AspectJ for a similar use case (although at a much smaller scale)
>> in the transport test suites [1]. Interestingly, this approach would
>> allow to cover interactions that are not SOAP based, e.g. one could
>> even intercept the database queries executed during the flow.
>>
>> Andreas
>>
>> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>
>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>> Hi Nilupa;
>>>
>>> When we collect messages from a one location by installing proper
>>> handler that will intercept and send messages to to that one location
>>> (this one location can be a single server, pub/sub channel etc). There
>>> is many ways to make sense of those collected messages. What Andreas
>>> mentioned (following complete transaction) is a one possibility.
>>>
>>> I think you should come up with few scenarios on how you would make
>>> sense of the message.
>>>
>>> Thanks
>>> Srinath
>>>
>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> First let me thank you for commenting.
>>>>
>>>> As far as I understood, what you would like to see from the proposed
>>>> tool is to view set of messages that are exchanged in reponse to a
>>>> particular input message. With the understanding that I am having at
>>>> the momnet, one way to do it is to filter out the central repository
>>>> of messages based on 'To' , 'From' headers and try to contruct the
>>>> message chain from it. We can allow the client GUI wich connects to
>>>> the central repository to provide the paramenters (For instance the
>>>> value of 'To' header) from which an intelligent filtering can be done
>>>> for the set of messages avialable at the central repository.
>>>>
>>>> Perhaps someone has an idea of a better way of doing it and is willing
>>>> to share it with us. It would be really nice to hear from them.
>>>>
>>>> Thanks in advance ..!!
>>>>
>>>> Best Regards,
>>>> Nilupa
>>>>
>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>> <an...@gmail.com> wrote:
>>>>> Personally, I think that the added value of extending the SOAP monitor
>>>>> module to collect messages in a central place is too limited to
>>>>> attract enough interest from the user community, so that it will be
>>>>> difficult to ensure the evolution of such a project in the future.
>>>>> However, what many people are looking for is a tool that allows to
>>>>> track the flow of a message through a distributed system, or more
>>>>> generally to track the sequence of events triggered by a given input
>>>>> message (sort of end-to-end transaction monitoring).
>>>>>
>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>>> proposing a GSoC project this summer. I would like to take project
>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>>> services middleware. I will submit more detailed proposal later. I
>>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>>> most welcome.
>>>>>>
>>>>>> Best Regards,
>>>>>> Nilupa Bandara
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>> Distributed TCP monitor
>>>>>>
>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>>> pull those messages. Also, we can do something to turn the module on
>>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>>> on only when they want to debug the system.
>>>>>>
>>>>>> Then we can take this to next level by adding this module to all
>>>>>> services in a system , configuring modules to send all collected
>>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>>> TCPMon for a whole system.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> ============================
>>> Srinath Perera, Ph.D.
>>>   WSO2 Inc. http://wso2.com
>>>   Blog: http://srinathsview.blogspot.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

Probably we are already going too far into implementation details. I
think that the project would have 3 parts:

1. Establishing the general (implementation independent) framework:

- Define protocols: SOAP headers, but also mappings to HTTP headers
(for REST) and JMS properties.
- Define the structure of the correlation identifiers so that the
entire sequence can be reconstructed in a meaningful way.
- Look for strategies to detect and/or avoid duplicate logging
(because of redundant interceptors or simply because a message is
logged at the transmitter and receiver side).
- Find solutions for cases where the message is not XML.
- Think about what happens if one of the services doesn't understand
the protocol.
- Think about security implications.
- ...

2. Build a generic Java framework that supports this style of logging
(with components for remote logging, classes that represent the
context, tools to browse the intercepted messages, etc.).

3. Build a set of interceptors that make use of this framework (and
that log the messages and propagate the context). This can be Axis2
modules, JAX-WS handlers or AspectJ aspects.

On Tue, Mar 30, 2010 at 23:28, nilupa bandara <ni...@gmail.com> wrote:
> Hello Everyone,
>
> First let me thank all of you for your valuable inputs.
>
> Now let me ask few questions to make sure that I understood the
> suggested approach correctly.
>
> - What are the components that you meant by 'Components who call
> RemoteLogger.log()'  ? A Custom Message Receiver which logs the
> messages that are going in (and out) to a Service Impl perhaps ?
>
> - Does Service Impl needs check the availability of RemoteLogger (via
> a thread local variable) and log the message it sends ( and receive)
> using Service Clients to other applications ? I mean is it the case
> where service author needs to insert 'RemoteLogger.Log()' in
> appropriate places to ensure complete traceability ?
>
> Using AOP - I just read few documentation on AspectJ and something
> like the following could be done (But I'm not absolutely sure though
> !!)
>
> We could use some AOP code (package in a jar) which will, just before
> each Service Client invocation insert the appropriate debug info
> headers to outgoing payload , plus logs the response it gets.
>
> This AOP could be instrumented (or weaved) into all the service
> archive files in a given repository.
>
> We could provide a script (and AOP jar) to automate this. For instance
> ./our_script AOP.jar <path-to-repository> will make all the services
> that are available in the repository traceable.
>
> Any suggestions ?
>
> Nilupa
>
>
> On Tue, Mar 30, 2010 at 3:11 PM, Glen Daniels <gl...@thoughtcraft.com> wrote:
>> I'm a really terribly inconsistent blogger.  However, my last actual entry,
>> from 2009, was about exactly this:
>>
>> http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html
>>
>> I like Andreas' suggestion below, but I think it might be a bit heavyweight
>> for a GSoC project(?).  Maybe we could lower the bar to something like this:
>>
>> * Build a distributed logging web service which can accept log messages and
>> correlationIDs, logging to a file per correlationID.
>>
>> * Build a Module that processes incoming <debug:logInfo> headers and makes an
>> appropriate RemoteLogger (or whatever) available.  This could, at first, go
>> in one of Axis2's contexts (OperationContext?) as a demonstration.
>> Components who call RemoteLogger.log() send messages (hopefully on another
>> thread) to the central log service.
>>
>> * The Module could automatically put the right metadata on outgoing messages,
>> but only the responses, so you still have the problem Andreas points out
>> about needing the app to transfer the logging info over to new
>> ServiceClients.  I was imagining that could be achieved by also setting a
>> ThreadLocal - so if your clients run on the same thread you're all set, and
>> if there are new threads it's up to the framework to copy that ThreadLocal over.
>>
>> * Regardless of the discovery mechanism, you'd want something static like
>> RemoteLogger.getLogger() to hide it.
>>
>> This approach is definitely a bit more intrusive, as it relies on the
>> components themselves to be looking for a RemoteLogger, but it seems like a
>> reasonable starting point which could be evolved.  However, maybe the AOP
>> stuff isn't as challenging as I'm imagining....
>>
>> Thoughts?
>>
>> --Glen
>>
>> On 3/30/2010 4:57 AM, Andreas Veithen wrote:
>>> I was actually referring to scenarios where a service may call other
>>> services, which in turn may call again other services. In these
>>> scenarios, it is not sufficient to simply collect received/sent
>>> messages on different hosts, unless the system is isolated
>>> (development environment) or the request rate is sufficiently low so
>>> that messages can be correlated based on time. Here is a concrete
>>> scenario from a project in my company:
>>>
>>> - Request comes in on an ESB that does security and validation.
>>> - Request is processed by an application server which persists the
>>> received information and publishes a JMS message (pub/sub events).
>>> - The event is consumed by one or more components that may in turn
>>> interact with other services.
>>>
>>> If you want track this flow, it is not sufficient to intercept the
>>> messages on the different hosts: in addition, you need to instrument
>>> the services so that the outgoing requests are correlated with the
>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> would like to be able to do in the above scenario is to enable
>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> HTTP header to the initial request to enable logging and this
>>> information is propagated along the chain. Every service/component
>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> central place where the sequence of events is reconstructed.
>>>
>>> One way this could be achieved is with a tool that postprocesses the
>>> artifacts from the build process. For each artifact, the tool would
>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> aspects and reassemble the artifact for deployment. The responsibility
>>> of the aspects would be to intercept messages, log them and modify
>>> them to ensure proper correlation. Of course this only works if the
>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> in the transport test suites [1]. Interestingly, this approach would
>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> even intercept the database queries executed during the flow.
>>>
>>> Andreas
>>>
>>> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>>
>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>>> Hi Nilupa;
>>>>
>>>> When we collect messages from a one location by installing proper
>>>> handler that will intercept and send messages to to that one location
>>>> (this one location can be a single server, pub/sub channel etc). There
>>>> is many ways to make sense of those collected messages. What Andreas
>>>> mentioned (following complete transaction) is a one possibility.
>>>>
>>>> I think you should come up with few scenarios on how you would make
>>>> sense of the message.
>>>>
>>>> Thanks
>>>> Srinath
>>>>
>>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> First let me thank you for commenting.
>>>>>
>>>>> As far as I understood, what you would like to see from the proposed
>>>>> tool is to view set of messages that are exchanged in reponse to a
>>>>> particular input message. With the understanding that I am having at
>>>>> the momnet, one way to do it is to filter out the central repository
>>>>> of messages based on 'To' , 'From' headers and try to contruct the
>>>>> message chain from it. We can allow the client GUI wich connects to
>>>>> the central repository to provide the paramenters (For instance the
>>>>> value of 'To' header) from which an intelligent filtering can be done
>>>>> for the set of messages avialable at the central repository.
>>>>>
>>>>> Perhaps someone has an idea of a better way of doing it and is willing
>>>>> to share it with us. It would be really nice to hear from them.
>>>>>
>>>>> Thanks in advance ..!!
>>>>>
>>>>> Best Regards,
>>>>> Nilupa
>>>>>
>>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>>> <an...@gmail.com> wrote:
>>>>>> Personally, I think that the added value of extending the SOAP monitor
>>>>>> module to collect messages in a central place is too limited to
>>>>>> attract enough interest from the user community, so that it will be
>>>>>> difficult to ensure the evolution of such a project in the future.
>>>>>> However, what many people are looking for is a tool that allows to
>>>>>> track the flow of a message through a distributed system, or more
>>>>>> generally to track the sequence of events triggered by a given input
>>>>>> message (sort of end-to-end transaction monitoring).
>>>>>>
>>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>>>> proposing a GSoC project this summer. I would like to take project
>>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>>>> services middleware. I will submit more detailed proposal later. I
>>>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>>>> most welcome.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Nilupa Bandara
>>>>>>>
>>>>>>> [1]
>>>>>>>
>>>>>>> Distributed TCP monitor
>>>>>>>
>>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>>>> pull those messages. Also, we can do something to turn the module on
>>>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>>>> on only when they want to debug the system.
>>>>>>>
>>>>>>> Then we can take this to next level by adding this module to all
>>>>>>> services in a system , configuring modules to send all collected
>>>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>>>> TCPMon for a whole system.
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> ============================
>>>> Srinath Perera, Ph.D.
>>>>   WSO2 Inc. http://wso2.com
>>>>   Blog: http://srinathsview.blogspot.com/
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

Probably we are already going too far into implementation details. I
think that the project would have 3 parts:

1. Establishing the general (implementation independent) framework:

- Define protocols: SOAP headers, but also mappings to HTTP headers
(for REST) and JMS properties.
- Define the structure of the correlation identifiers so that the
entire sequence can be reconstructed in a meaningful way.
- Look for strategies to detect and/or avoid duplicate logging
(because of redundant interceptors or simply because a message is
logged at the transmitter and receiver side).
- Find solutions for cases where the message is not XML.
- Think about what happens if one of the services doesn't understand
the protocol.
- Think about security implications.
- ...

2. Build a generic Java framework that supports this style of logging
(with components for remote logging, classes that represent the
context, tools to browse the intercepted messages, etc.).

3. Build a set of interceptors that make use of this framework (and
that log the messages and propagate the context). This can be Axis2
modules, JAX-WS handlers or AspectJ aspects.

On Tue, Mar 30, 2010 at 23:28, nilupa bandara <ni...@gmail.com> wrote:
> Hello Everyone,
>
> First let me thank all of you for your valuable inputs.
>
> Now let me ask few questions to make sure that I understood the
> suggested approach correctly.
>
> - What are the components that you meant by 'Components who call
> RemoteLogger.log()'  ? A Custom Message Receiver which logs the
> messages that are going in (and out) to a Service Impl perhaps ?
>
> - Does Service Impl needs check the availability of RemoteLogger (via
> a thread local variable) and log the message it sends ( and receive)
> using Service Clients to other applications ? I mean is it the case
> where service author needs to insert 'RemoteLogger.Log()' in
> appropriate places to ensure complete traceability ?
>
> Using AOP - I just read few documentation on AspectJ and something
> like the following could be done (But I'm not absolutely sure though
> !!)
>
> We could use some AOP code (package in a jar) which will, just before
> each Service Client invocation insert the appropriate debug info
> headers to outgoing payload , plus logs the response it gets.
>
> This AOP could be instrumented (or weaved) into all the service
> archive files in a given repository.
>
> We could provide a script (and AOP jar) to automate this. For instance
> ./our_script AOP.jar <path-to-repository> will make all the services
> that are available in the repository traceable.
>
> Any suggestions ?
>
> Nilupa
>
>
> On Tue, Mar 30, 2010 at 3:11 PM, Glen Daniels <gl...@thoughtcraft.com> wrote:
>> I'm a really terribly inconsistent blogger.  However, my last actual entry,
>> from 2009, was about exactly this:
>>
>> http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html
>>
>> I like Andreas' suggestion below, but I think it might be a bit heavyweight
>> for a GSoC project(?).  Maybe we could lower the bar to something like this:
>>
>> * Build a distributed logging web service which can accept log messages and
>> correlationIDs, logging to a file per correlationID.
>>
>> * Build a Module that processes incoming <debug:logInfo> headers and makes an
>> appropriate RemoteLogger (or whatever) available.  This could, at first, go
>> in one of Axis2's contexts (OperationContext?) as a demonstration.
>> Components who call RemoteLogger.log() send messages (hopefully on another
>> thread) to the central log service.
>>
>> * The Module could automatically put the right metadata on outgoing messages,
>> but only the responses, so you still have the problem Andreas points out
>> about needing the app to transfer the logging info over to new
>> ServiceClients.  I was imagining that could be achieved by also setting a
>> ThreadLocal - so if your clients run on the same thread you're all set, and
>> if there are new threads it's up to the framework to copy that ThreadLocal over.
>>
>> * Regardless of the discovery mechanism, you'd want something static like
>> RemoteLogger.getLogger() to hide it.
>>
>> This approach is definitely a bit more intrusive, as it relies on the
>> components themselves to be looking for a RemoteLogger, but it seems like a
>> reasonable starting point which could be evolved.  However, maybe the AOP
>> stuff isn't as challenging as I'm imagining....
>>
>> Thoughts?
>>
>> --Glen
>>
>> On 3/30/2010 4:57 AM, Andreas Veithen wrote:
>>> I was actually referring to scenarios where a service may call other
>>> services, which in turn may call again other services. In these
>>> scenarios, it is not sufficient to simply collect received/sent
>>> messages on different hosts, unless the system is isolated
>>> (development environment) or the request rate is sufficiently low so
>>> that messages can be correlated based on time. Here is a concrete
>>> scenario from a project in my company:
>>>
>>> - Request comes in on an ESB that does security and validation.
>>> - Request is processed by an application server which persists the
>>> received information and publishes a JMS message (pub/sub events).
>>> - The event is consumed by one or more components that may in turn
>>> interact with other services.
>>>
>>> If you want track this flow, it is not sufficient to intercept the
>>> messages on the different hosts: in addition, you need to instrument
>>> the services so that the outgoing requests are correlated with the
>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> would like to be able to do in the above scenario is to enable
>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> HTTP header to the initial request to enable logging and this
>>> information is propagated along the chain. Every service/component
>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> central place where the sequence of events is reconstructed.
>>>
>>> One way this could be achieved is with a tool that postprocesses the
>>> artifacts from the build process. For each artifact, the tool would
>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> aspects and reassemble the artifact for deployment. The responsibility
>>> of the aspects would be to intercept messages, log them and modify
>>> them to ensure proper correlation. Of course this only works if the
>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> in the transport test suites [1]. Interestingly, this approach would
>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> even intercept the database queries executed during the flow.
>>>
>>> Andreas
>>>
>>> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>>
>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>>> Hi Nilupa;
>>>>
>>>> When we collect messages from a one location by installing proper
>>>> handler that will intercept and send messages to to that one location
>>>> (this one location can be a single server, pub/sub channel etc). There
>>>> is many ways to make sense of those collected messages. What Andreas
>>>> mentioned (following complete transaction) is a one possibility.
>>>>
>>>> I think you should come up with few scenarios on how you would make
>>>> sense of the message.
>>>>
>>>> Thanks
>>>> Srinath
>>>>
>>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> First let me thank you for commenting.
>>>>>
>>>>> As far as I understood, what you would like to see from the proposed
>>>>> tool is to view set of messages that are exchanged in reponse to a
>>>>> particular input message. With the understanding that I am having at
>>>>> the momnet, one way to do it is to filter out the central repository
>>>>> of messages based on 'To' , 'From' headers and try to contruct the
>>>>> message chain from it. We can allow the client GUI wich connects to
>>>>> the central repository to provide the paramenters (For instance the
>>>>> value of 'To' header) from which an intelligent filtering can be done
>>>>> for the set of messages avialable at the central repository.
>>>>>
>>>>> Perhaps someone has an idea of a better way of doing it and is willing
>>>>> to share it with us. It would be really nice to hear from them.
>>>>>
>>>>> Thanks in advance ..!!
>>>>>
>>>>> Best Regards,
>>>>> Nilupa
>>>>>
>>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>>> <an...@gmail.com> wrote:
>>>>>> Personally, I think that the added value of extending the SOAP monitor
>>>>>> module to collect messages in a central place is too limited to
>>>>>> attract enough interest from the user community, so that it will be
>>>>>> difficult to ensure the evolution of such a project in the future.
>>>>>> However, what many people are looking for is a tool that allows to
>>>>>> track the flow of a message through a distributed system, or more
>>>>>> generally to track the sequence of events triggered by a given input
>>>>>> message (sort of end-to-end transaction monitoring).
>>>>>>
>>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>>>> proposing a GSoC project this summer. I would like to take project
>>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>>>> services middleware. I will submit more detailed proposal later. I
>>>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>>>> most welcome.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Nilupa Bandara
>>>>>>>
>>>>>>> [1]
>>>>>>>
>>>>>>> Distributed TCP monitor
>>>>>>>
>>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>>>> pull those messages. Also, we can do something to turn the module on
>>>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>>>> on only when they want to debug the system.
>>>>>>>
>>>>>>> Then we can take this to next level by adding this module to all
>>>>>>> services in a system , configuring modules to send all collected
>>>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>>>> TCPMon for a whole system.
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> ============================
>>>> Srinath Perera, Ph.D.
>>>>   WSO2 Inc. http://wso2.com
>>>>   Blog: http://srinathsview.blogspot.com/
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

Probably we are already going too far into implementation details. I
think that the project would have 3 parts:

1. Establishing the general (implementation independent) framework:

- Define protocols: SOAP headers, but also mappings to HTTP headers
(for REST) and JMS properties.
- Define the structure of the correlation identifiers so that the
entire sequence can be reconstructed in a meaningful way.
- Look for strategies to detect and/or avoid duplicate logging
(because of redundant interceptors or simply because a message is
logged at the transmitter and receiver side).
- Find solutions for cases where the message is not XML.
- Think about what happens if one of the services doesn't understand
the protocol.
- Think about security implications.
- ...

2. Build a generic Java framework that supports this style of logging
(with components for remote logging, classes that represent the
context, tools to browse the intercepted messages, etc.).

3. Build a set of interceptors that make use of this framework (and
that log the messages and propagate the context). This can be Axis2
modules, JAX-WS handlers or AspectJ aspects.

On Tue, Mar 30, 2010 at 23:28, nilupa bandara <ni...@gmail.com> wrote:
> Hello Everyone,
>
> First let me thank all of you for your valuable inputs.
>
> Now let me ask few questions to make sure that I understood the
> suggested approach correctly.
>
> - What are the components that you meant by 'Components who call
> RemoteLogger.log()'  ? A Custom Message Receiver which logs the
> messages that are going in (and out) to a Service Impl perhaps ?
>
> - Does Service Impl needs check the availability of RemoteLogger (via
> a thread local variable) and log the message it sends ( and receive)
> using Service Clients to other applications ? I mean is it the case
> where service author needs to insert 'RemoteLogger.Log()' in
> appropriate places to ensure complete traceability ?
>
> Using AOP - I just read few documentation on AspectJ and something
> like the following could be done (But I'm not absolutely sure though
> !!)
>
> We could use some AOP code (package in a jar) which will, just before
> each Service Client invocation insert the appropriate debug info
> headers to outgoing payload , plus logs the response it gets.
>
> This AOP could be instrumented (or weaved) into all the service
> archive files in a given repository.
>
> We could provide a script (and AOP jar) to automate this. For instance
> ./our_script AOP.jar <path-to-repository> will make all the services
> that are available in the repository traceable.
>
> Any suggestions ?
>
> Nilupa
>
>
> On Tue, Mar 30, 2010 at 3:11 PM, Glen Daniels <gl...@thoughtcraft.com> wrote:
>> I'm a really terribly inconsistent blogger.  However, my last actual entry,
>> from 2009, was about exactly this:
>>
>> http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html
>>
>> I like Andreas' suggestion below, but I think it might be a bit heavyweight
>> for a GSoC project(?).  Maybe we could lower the bar to something like this:
>>
>> * Build a distributed logging web service which can accept log messages and
>> correlationIDs, logging to a file per correlationID.
>>
>> * Build a Module that processes incoming <debug:logInfo> headers and makes an
>> appropriate RemoteLogger (or whatever) available.  This could, at first, go
>> in one of Axis2's contexts (OperationContext?) as a demonstration.
>> Components who call RemoteLogger.log() send messages (hopefully on another
>> thread) to the central log service.
>>
>> * The Module could automatically put the right metadata on outgoing messages,
>> but only the responses, so you still have the problem Andreas points out
>> about needing the app to transfer the logging info over to new
>> ServiceClients.  I was imagining that could be achieved by also setting a
>> ThreadLocal - so if your clients run on the same thread you're all set, and
>> if there are new threads it's up to the framework to copy that ThreadLocal over.
>>
>> * Regardless of the discovery mechanism, you'd want something static like
>> RemoteLogger.getLogger() to hide it.
>>
>> This approach is definitely a bit more intrusive, as it relies on the
>> components themselves to be looking for a RemoteLogger, but it seems like a
>> reasonable starting point which could be evolved.  However, maybe the AOP
>> stuff isn't as challenging as I'm imagining....
>>
>> Thoughts?
>>
>> --Glen
>>
>> On 3/30/2010 4:57 AM, Andreas Veithen wrote:
>>> I was actually referring to scenarios where a service may call other
>>> services, which in turn may call again other services. In these
>>> scenarios, it is not sufficient to simply collect received/sent
>>> messages on different hosts, unless the system is isolated
>>> (development environment) or the request rate is sufficiently low so
>>> that messages can be correlated based on time. Here is a concrete
>>> scenario from a project in my company:
>>>
>>> - Request comes in on an ESB that does security and validation.
>>> - Request is processed by an application server which persists the
>>> received information and publishes a JMS message (pub/sub events).
>>> - The event is consumed by one or more components that may in turn
>>> interact with other services.
>>>
>>> If you want track this flow, it is not sufficient to intercept the
>>> messages on the different hosts: in addition, you need to instrument
>>> the services so that the outgoing requests are correlated with the
>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> would like to be able to do in the above scenario is to enable
>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> HTTP header to the initial request to enable logging and this
>>> information is propagated along the chain. Every service/component
>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> central place where the sequence of events is reconstructed.
>>>
>>> One way this could be achieved is with a tool that postprocesses the
>>> artifacts from the build process. For each artifact, the tool would
>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> aspects and reassemble the artifact for deployment. The responsibility
>>> of the aspects would be to intercept messages, log them and modify
>>> them to ensure proper correlation. Of course this only works if the
>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> in the transport test suites [1]. Interestingly, this approach would
>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> even intercept the database queries executed during the flow.
>>>
>>> Andreas
>>>
>>> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>>
>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>>> Hi Nilupa;
>>>>
>>>> When we collect messages from a one location by installing proper
>>>> handler that will intercept and send messages to to that one location
>>>> (this one location can be a single server, pub/sub channel etc). There
>>>> is many ways to make sense of those collected messages. What Andreas
>>>> mentioned (following complete transaction) is a one possibility.
>>>>
>>>> I think you should come up with few scenarios on how you would make
>>>> sense of the message.
>>>>
>>>> Thanks
>>>> Srinath
>>>>
>>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> First let me thank you for commenting.
>>>>>
>>>>> As far as I understood, what you would like to see from the proposed
>>>>> tool is to view set of messages that are exchanged in reponse to a
>>>>> particular input message. With the understanding that I am having at
>>>>> the momnet, one way to do it is to filter out the central repository
>>>>> of messages based on 'To' , 'From' headers and try to contruct the
>>>>> message chain from it. We can allow the client GUI wich connects to
>>>>> the central repository to provide the paramenters (For instance the
>>>>> value of 'To' header) from which an intelligent filtering can be done
>>>>> for the set of messages avialable at the central repository.
>>>>>
>>>>> Perhaps someone has an idea of a better way of doing it and is willing
>>>>> to share it with us. It would be really nice to hear from them.
>>>>>
>>>>> Thanks in advance ..!!
>>>>>
>>>>> Best Regards,
>>>>> Nilupa
>>>>>
>>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>>> <an...@gmail.com> wrote:
>>>>>> Personally, I think that the added value of extending the SOAP monitor
>>>>>> module to collect messages in a central place is too limited to
>>>>>> attract enough interest from the user community, so that it will be
>>>>>> difficult to ensure the evolution of such a project in the future.
>>>>>> However, what many people are looking for is a tool that allows to
>>>>>> track the flow of a message through a distributed system, or more
>>>>>> generally to track the sequence of events triggered by a given input
>>>>>> message (sort of end-to-end transaction monitoring).
>>>>>>
>>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>>>> proposing a GSoC project this summer. I would like to take project
>>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>>>> services middleware. I will submit more detailed proposal later. I
>>>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>>>> most welcome.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Nilupa Bandara
>>>>>>>
>>>>>>> [1]
>>>>>>>
>>>>>>> Distributed TCP monitor
>>>>>>>
>>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>>>> pull those messages. Also, we can do something to turn the module on
>>>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>>>> on only when they want to debug the system.
>>>>>>>
>>>>>>> Then we can take this to next level by adding this module to all
>>>>>>> services in a system , configuring modules to send all collected
>>>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>>>> TCPMon for a whole system.
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> ============================
>>>> Srinath Perera, Ph.D.
>>>>   WSO2 Inc. http://wso2.com
>>>>   Blog: http://srinathsview.blogspot.com/
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

Probably we are already going too far into implementation details. I
think that the project would have 3 parts:

1. Establishing the general (implementation independent) framework:

- Define protocols: SOAP headers, but also mappings to HTTP headers
(for REST) and JMS properties.
- Define the structure of the correlation identifiers so that the
entire sequence can be reconstructed in a meaningful way.
- Look for strategies to detect and/or avoid duplicate logging
(because of redundant interceptors or simply because a message is
logged at the transmitter and receiver side).
- Find solutions for cases where the message is not XML.
- Think about what happens if one of the services doesn't understand
the protocol.
- Think about security implications.
- ...

2. Build a generic Java framework that supports this style of logging
(with components for remote logging, classes that represent the
context, tools to browse the intercepted messages, etc.).

3. Build a set of interceptors that make use of this framework (and
that log the messages and propagate the context). This can be Axis2
modules, JAX-WS handlers or AspectJ aspects.

On Tue, Mar 30, 2010 at 23:28, nilupa bandara <ni...@gmail.com> wrote:
> Hello Everyone,
>
> First let me thank all of you for your valuable inputs.
>
> Now let me ask few questions to make sure that I understood the
> suggested approach correctly.
>
> - What are the components that you meant by 'Components who call
> RemoteLogger.log()'  ? A Custom Message Receiver which logs the
> messages that are going in (and out) to a Service Impl perhaps ?
>
> - Does Service Impl needs check the availability of RemoteLogger (via
> a thread local variable) and log the message it sends ( and receive)
> using Service Clients to other applications ? I mean is it the case
> where service author needs to insert 'RemoteLogger.Log()' in
> appropriate places to ensure complete traceability ?
>
> Using AOP - I just read few documentation on AspectJ and something
> like the following could be done (But I'm not absolutely sure though
> !!)
>
> We could use some AOP code (package in a jar) which will, just before
> each Service Client invocation insert the appropriate debug info
> headers to outgoing payload , plus logs the response it gets.
>
> This AOP could be instrumented (or weaved) into all the service
> archive files in a given repository.
>
> We could provide a script (and AOP jar) to automate this. For instance
> ./our_script AOP.jar <path-to-repository> will make all the services
> that are available in the repository traceable.
>
> Any suggestions ?
>
> Nilupa
>
>
> On Tue, Mar 30, 2010 at 3:11 PM, Glen Daniels <gl...@thoughtcraft.com> wrote:
>> I'm a really terribly inconsistent blogger.  However, my last actual entry,
>> from 2009, was about exactly this:
>>
>> http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html
>>
>> I like Andreas' suggestion below, but I think it might be a bit heavyweight
>> for a GSoC project(?).  Maybe we could lower the bar to something like this:
>>
>> * Build a distributed logging web service which can accept log messages and
>> correlationIDs, logging to a file per correlationID.
>>
>> * Build a Module that processes incoming <debug:logInfo> headers and makes an
>> appropriate RemoteLogger (or whatever) available.  This could, at first, go
>> in one of Axis2's contexts (OperationContext?) as a demonstration.
>> Components who call RemoteLogger.log() send messages (hopefully on another
>> thread) to the central log service.
>>
>> * The Module could automatically put the right metadata on outgoing messages,
>> but only the responses, so you still have the problem Andreas points out
>> about needing the app to transfer the logging info over to new
>> ServiceClients.  I was imagining that could be achieved by also setting a
>> ThreadLocal - so if your clients run on the same thread you're all set, and
>> if there are new threads it's up to the framework to copy that ThreadLocal over.
>>
>> * Regardless of the discovery mechanism, you'd want something static like
>> RemoteLogger.getLogger() to hide it.
>>
>> This approach is definitely a bit more intrusive, as it relies on the
>> components themselves to be looking for a RemoteLogger, but it seems like a
>> reasonable starting point which could be evolved.  However, maybe the AOP
>> stuff isn't as challenging as I'm imagining....
>>
>> Thoughts?
>>
>> --Glen
>>
>> On 3/30/2010 4:57 AM, Andreas Veithen wrote:
>>> I was actually referring to scenarios where a service may call other
>>> services, which in turn may call again other services. In these
>>> scenarios, it is not sufficient to simply collect received/sent
>>> messages on different hosts, unless the system is isolated
>>> (development environment) or the request rate is sufficiently low so
>>> that messages can be correlated based on time. Here is a concrete
>>> scenario from a project in my company:
>>>
>>> - Request comes in on an ESB that does security and validation.
>>> - Request is processed by an application server which persists the
>>> received information and publishes a JMS message (pub/sub events).
>>> - The event is consumed by one or more components that may in turn
>>> interact with other services.
>>>
>>> If you want track this flow, it is not sufficient to intercept the
>>> messages on the different hosts: in addition, you need to instrument
>>> the services so that the outgoing requests are correlated with the
>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> would like to be able to do in the above scenario is to enable
>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> HTTP header to the initial request to enable logging and this
>>> information is propagated along the chain. Every service/component
>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> central place where the sequence of events is reconstructed.
>>>
>>> One way this could be achieved is with a tool that postprocesses the
>>> artifacts from the build process. For each artifact, the tool would
>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> aspects and reassemble the artifact for deployment. The responsibility
>>> of the aspects would be to intercept messages, log them and modify
>>> them to ensure proper correlation. Of course this only works if the
>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> in the transport test suites [1]. Interestingly, this approach would
>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> even intercept the database queries executed during the flow.
>>>
>>> Andreas
>>>
>>> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>>
>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>>> Hi Nilupa;
>>>>
>>>> When we collect messages from a one location by installing proper
>>>> handler that will intercept and send messages to to that one location
>>>> (this one location can be a single server, pub/sub channel etc). There
>>>> is many ways to make sense of those collected messages. What Andreas
>>>> mentioned (following complete transaction) is a one possibility.
>>>>
>>>> I think you should come up with few scenarios on how you would make
>>>> sense of the message.
>>>>
>>>> Thanks
>>>> Srinath
>>>>
>>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> First let me thank you for commenting.
>>>>>
>>>>> As far as I understood, what you would like to see from the proposed
>>>>> tool is to view set of messages that are exchanged in reponse to a
>>>>> particular input message. With the understanding that I am having at
>>>>> the momnet, one way to do it is to filter out the central repository
>>>>> of messages based on 'To' , 'From' headers and try to contruct the
>>>>> message chain from it. We can allow the client GUI wich connects to
>>>>> the central repository to provide the paramenters (For instance the
>>>>> value of 'To' header) from which an intelligent filtering can be done
>>>>> for the set of messages avialable at the central repository.
>>>>>
>>>>> Perhaps someone has an idea of a better way of doing it and is willing
>>>>> to share it with us. It would be really nice to hear from them.
>>>>>
>>>>> Thanks in advance ..!!
>>>>>
>>>>> Best Regards,
>>>>> Nilupa
>>>>>
>>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>>> <an...@gmail.com> wrote:
>>>>>> Personally, I think that the added value of extending the SOAP monitor
>>>>>> module to collect messages in a central place is too limited to
>>>>>> attract enough interest from the user community, so that it will be
>>>>>> difficult to ensure the evolution of such a project in the future.
>>>>>> However, what many people are looking for is a tool that allows to
>>>>>> track the flow of a message through a distributed system, or more
>>>>>> generally to track the sequence of events triggered by a given input
>>>>>> message (sort of end-to-end transaction monitoring).
>>>>>>
>>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>>>> proposing a GSoC project this summer. I would like to take project
>>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>>>> services middleware. I will submit more detailed proposal later. I
>>>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>>>> most welcome.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Nilupa Bandara
>>>>>>>
>>>>>>> [1]
>>>>>>>
>>>>>>> Distributed TCP monitor
>>>>>>>
>>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>>>> pull those messages. Also, we can do something to turn the module on
>>>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>>>> on only when they want to debug the system.
>>>>>>>
>>>>>>> Then we can take this to next level by adding this module to all
>>>>>>> services in a system , configuring modules to send all collected
>>>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>>>> TCPMon for a whole system.
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> ============================
>>>> Srinath Perera, Ph.D.
>>>>   WSO2 Inc. http://wso2.com
>>>>   Blog: http://srinathsview.blogspot.com/
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

Probably we are already going too far into implementation details. I
think that the project would have 3 parts:

1. Establishing the general (implementation independent) framework:

- Define protocols: SOAP headers, but also mappings to HTTP headers
(for REST) and JMS properties.
- Define the structure of the correlation identifiers so that the
entire sequence can be reconstructed in a meaningful way.
- Look for strategies to detect and/or avoid duplicate logging
(because of redundant interceptors or simply because a message is
logged at the transmitter and receiver side).
- Find solutions for cases where the message is not XML.
- Think about what happens if one of the services doesn't understand
the protocol.
- Think about security implications.
- ...

2. Build a generic Java framework that supports this style of logging
(with components for remote logging, classes that represent the
context, tools to browse the intercepted messages, etc.).

3. Build a set of interceptors that make use of this framework (and
that log the messages and propagate the context). This can be Axis2
modules, JAX-WS handlers or AspectJ aspects.

On Tue, Mar 30, 2010 at 23:28, nilupa bandara <ni...@gmail.com> wrote:
> Hello Everyone,
>
> First let me thank all of you for your valuable inputs.
>
> Now let me ask few questions to make sure that I understood the
> suggested approach correctly.
>
> - What are the components that you meant by 'Components who call
> RemoteLogger.log()'  ? A Custom Message Receiver which logs the
> messages that are going in (and out) to a Service Impl perhaps ?
>
> - Does Service Impl needs check the availability of RemoteLogger (via
> a thread local variable) and log the message it sends ( and receive)
> using Service Clients to other applications ? I mean is it the case
> where service author needs to insert 'RemoteLogger.Log()' in
> appropriate places to ensure complete traceability ?
>
> Using AOP - I just read few documentation on AspectJ and something
> like the following could be done (But I'm not absolutely sure though
> !!)
>
> We could use some AOP code (package in a jar) which will, just before
> each Service Client invocation insert the appropriate debug info
> headers to outgoing payload , plus logs the response it gets.
>
> This AOP could be instrumented (or weaved) into all the service
> archive files in a given repository.
>
> We could provide a script (and AOP jar) to automate this. For instance
> ./our_script AOP.jar <path-to-repository> will make all the services
> that are available in the repository traceable.
>
> Any suggestions ?
>
> Nilupa
>
>
> On Tue, Mar 30, 2010 at 3:11 PM, Glen Daniels <gl...@thoughtcraft.com> wrote:
>> I'm a really terribly inconsistent blogger.  However, my last actual entry,
>> from 2009, was about exactly this:
>>
>> http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html
>>
>> I like Andreas' suggestion below, but I think it might be a bit heavyweight
>> for a GSoC project(?).  Maybe we could lower the bar to something like this:
>>
>> * Build a distributed logging web service which can accept log messages and
>> correlationIDs, logging to a file per correlationID.
>>
>> * Build a Module that processes incoming <debug:logInfo> headers and makes an
>> appropriate RemoteLogger (or whatever) available.  This could, at first, go
>> in one of Axis2's contexts (OperationContext?) as a demonstration.
>> Components who call RemoteLogger.log() send messages (hopefully on another
>> thread) to the central log service.
>>
>> * The Module could automatically put the right metadata on outgoing messages,
>> but only the responses, so you still have the problem Andreas points out
>> about needing the app to transfer the logging info over to new
>> ServiceClients.  I was imagining that could be achieved by also setting a
>> ThreadLocal - so if your clients run on the same thread you're all set, and
>> if there are new threads it's up to the framework to copy that ThreadLocal over.
>>
>> * Regardless of the discovery mechanism, you'd want something static like
>> RemoteLogger.getLogger() to hide it.
>>
>> This approach is definitely a bit more intrusive, as it relies on the
>> components themselves to be looking for a RemoteLogger, but it seems like a
>> reasonable starting point which could be evolved.  However, maybe the AOP
>> stuff isn't as challenging as I'm imagining....
>>
>> Thoughts?
>>
>> --Glen
>>
>> On 3/30/2010 4:57 AM, Andreas Veithen wrote:
>>> I was actually referring to scenarios where a service may call other
>>> services, which in turn may call again other services. In these
>>> scenarios, it is not sufficient to simply collect received/sent
>>> messages on different hosts, unless the system is isolated
>>> (development environment) or the request rate is sufficiently low so
>>> that messages can be correlated based on time. Here is a concrete
>>> scenario from a project in my company:
>>>
>>> - Request comes in on an ESB that does security and validation.
>>> - Request is processed by an application server which persists the
>>> received information and publishes a JMS message (pub/sub events).
>>> - The event is consumed by one or more components that may in turn
>>> interact with other services.
>>>
>>> If you want track this flow, it is not sufficient to intercept the
>>> messages on the different hosts: in addition, you need to instrument
>>> the services so that the outgoing requests are correlated with the
>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> would like to be able to do in the above scenario is to enable
>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> HTTP header to the initial request to enable logging and this
>>> information is propagated along the chain. Every service/component
>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> central place where the sequence of events is reconstructed.
>>>
>>> One way this could be achieved is with a tool that postprocesses the
>>> artifacts from the build process. For each artifact, the tool would
>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> aspects and reassemble the artifact for deployment. The responsibility
>>> of the aspects would be to intercept messages, log them and modify
>>> them to ensure proper correlation. Of course this only works if the
>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> in the transport test suites [1]. Interestingly, this approach would
>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> even intercept the database queries executed during the flow.
>>>
>>> Andreas
>>>
>>> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>>
>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>>> Hi Nilupa;
>>>>
>>>> When we collect messages from a one location by installing proper
>>>> handler that will intercept and send messages to to that one location
>>>> (this one location can be a single server, pub/sub channel etc). There
>>>> is many ways to make sense of those collected messages. What Andreas
>>>> mentioned (following complete transaction) is a one possibility.
>>>>
>>>> I think you should come up with few scenarios on how you would make
>>>> sense of the message.
>>>>
>>>> Thanks
>>>> Srinath
>>>>
>>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> First let me thank you for commenting.
>>>>>
>>>>> As far as I understood, what you would like to see from the proposed
>>>>> tool is to view set of messages that are exchanged in reponse to a
>>>>> particular input message. With the understanding that I am having at
>>>>> the momnet, one way to do it is to filter out the central repository
>>>>> of messages based on 'To' , 'From' headers and try to contruct the
>>>>> message chain from it. We can allow the client GUI wich connects to
>>>>> the central repository to provide the paramenters (For instance the
>>>>> value of 'To' header) from which an intelligent filtering can be done
>>>>> for the set of messages avialable at the central repository.
>>>>>
>>>>> Perhaps someone has an idea of a better way of doing it and is willing
>>>>> to share it with us. It would be really nice to hear from them.
>>>>>
>>>>> Thanks in advance ..!!
>>>>>
>>>>> Best Regards,
>>>>> Nilupa
>>>>>
>>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>>> <an...@gmail.com> wrote:
>>>>>> Personally, I think that the added value of extending the SOAP monitor
>>>>>> module to collect messages in a central place is too limited to
>>>>>> attract enough interest from the user community, so that it will be
>>>>>> difficult to ensure the evolution of such a project in the future.
>>>>>> However, what many people are looking for is a tool that allows to
>>>>>> track the flow of a message through a distributed system, or more
>>>>>> generally to track the sequence of events triggered by a given input
>>>>>> message (sort of end-to-end transaction monitoring).
>>>>>>
>>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>>>> proposing a GSoC project this summer. I would like to take project
>>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>>>> services middleware. I will submit more detailed proposal later. I
>>>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>>>> most welcome.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Nilupa Bandara
>>>>>>>
>>>>>>> [1]
>>>>>>>
>>>>>>> Distributed TCP monitor
>>>>>>>
>>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>>>> pull those messages. Also, we can do something to turn the module on
>>>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>>>> on only when they want to debug the system.
>>>>>>>
>>>>>>> Then we can take this to next level by adding this module to all
>>>>>>> services in a system , configuring modules to send all collected
>>>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>>>> TCPMon for a whole system.
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> ============================
>>>> Srinath Perera, Ph.D.
>>>>   WSO2 Inc. http://wso2.com
>>>>   Blog: http://srinathsview.blogspot.com/
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

Hello Everyone,

First let me thank all of you for your valuable inputs.

Now let me ask few questions to make sure that I understood the
suggested approach correctly.

- What are the components that you meant by 'Components who call
RemoteLogger.log()'  ? A Custom Message Receiver which logs the
messages that are going in (and out) to a Service Impl perhaps ?

- Does Service Impl needs check the availability of RemoteLogger (via
a thread local variable) and log the message it sends ( and receive)
using Service Clients to other applications ? I mean is it the case
where service author needs to insert 'RemoteLogger.Log()' in
appropriate places to ensure complete traceability ?

Using AOP - I just read few documentation on AspectJ and something
like the following could be done (But I'm not absolutely sure though
!!)

We could use some AOP code (package in a jar) which will, just before
each Service Client invocation insert the appropriate debug info
headers to outgoing payload , plus logs the response it gets.

This AOP could be instrumented (or weaved) into all the service
archive files in a given repository.

We could provide a script (and AOP jar) to automate this. For instance
./our_script AOP.jar <path-to-repository> will make all the services
that are available in the repository traceable.

Any suggestions ?

Nilupa


On Tue, Mar 30, 2010 at 3:11 PM, Glen Daniels <gl...@thoughtcraft.com> wrote:
> I'm a really terribly inconsistent blogger.  However, my last actual entry,
> from 2009, was about exactly this:
>
> http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html
>
> I like Andreas' suggestion below, but I think it might be a bit heavyweight
> for a GSoC project(?).  Maybe we could lower the bar to something like this:
>
> * Build a distributed logging web service which can accept log messages and
> correlationIDs, logging to a file per correlationID.
>
> * Build a Module that processes incoming <debug:logInfo> headers and makes an
> appropriate RemoteLogger (or whatever) available.  This could, at first, go
> in one of Axis2's contexts (OperationContext?) as a demonstration.
> Components who call RemoteLogger.log() send messages (hopefully on another
> thread) to the central log service.
>
> * The Module could automatically put the right metadata on outgoing messages,
> but only the responses, so you still have the problem Andreas points out
> about needing the app to transfer the logging info over to new
> ServiceClients.  I was imagining that could be achieved by also setting a
> ThreadLocal - so if your clients run on the same thread you're all set, and
> if there are new threads it's up to the framework to copy that ThreadLocal over.
>
> * Regardless of the discovery mechanism, you'd want something static like
> RemoteLogger.getLogger() to hide it.
>
> This approach is definitely a bit more intrusive, as it relies on the
> components themselves to be looking for a RemoteLogger, but it seems like a
> reasonable starting point which could be evolved.  However, maybe the AOP
> stuff isn't as challenging as I'm imagining....
>
> Thoughts?
>
> --Glen
>
> On 3/30/2010 4:57 AM, Andreas Veithen wrote:
>> I was actually referring to scenarios where a service may call other
>> services, which in turn may call again other services. In these
>> scenarios, it is not sufficient to simply collect received/sent
>> messages on different hosts, unless the system is isolated
>> (development environment) or the request rate is sufficiently low so
>> that messages can be correlated based on time. Here is a concrete
>> scenario from a project in my company:
>>
>> - Request comes in on an ESB that does security and validation.
>> - Request is processed by an application server which persists the
>> received information and publishes a JMS message (pub/sub events).
>> - The event is consumed by one or more components that may in turn
>> interact with other services.
>>
>> If you want track this flow, it is not sufficient to intercept the
>> messages on the different hosts: in addition, you need to instrument
>> the services so that the outgoing requests are correlated with the
>> incoming requests (by adding SOAP and/or transport headers). What I
>> would like to be able to do in the above scenario is to enable
>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> HTTP header to the initial request to enable logging and this
>> information is propagated along the chain. Every service/component
>> then sends a copy of the incoming/outgoing requests/responses to a
>> central place where the sequence of events is reconstructed.
>>
>> One way this could be achieved is with a tool that postprocesses the
>> artifacts from the build process. For each artifact, the tool would
>> disassemble the artifact, instrument the code using a set of AspectJ
>> aspects and reassemble the artifact for deployment. The responsibility
>> of the aspects would be to intercept messages, log them and modify
>> them to ensure proper correlation. Of course this only works if the
>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> use AspectJ for a similar use case (although at a much smaller scale)
>> in the transport test suites [1]. Interestingly, this approach would
>> allow to cover interactions that are not SOAP based, e.g. one could
>> even intercept the database queries executed during the flow.
>>
>> Andreas
>>
>> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>
>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>> Hi Nilupa;
>>>
>>> When we collect messages from a one location by installing proper
>>> handler that will intercept and send messages to to that one location
>>> (this one location can be a single server, pub/sub channel etc). There
>>> is many ways to make sense of those collected messages. What Andreas
>>> mentioned (following complete transaction) is a one possibility.
>>>
>>> I think you should come up with few scenarios on how you would make
>>> sense of the message.
>>>
>>> Thanks
>>> Srinath
>>>
>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> First let me thank you for commenting.
>>>>
>>>> As far as I understood, what you would like to see from the proposed
>>>> tool is to view set of messages that are exchanged in reponse to a
>>>> particular input message. With the understanding that I am having at
>>>> the momnet, one way to do it is to filter out the central repository
>>>> of messages based on 'To' , 'From' headers and try to contruct the
>>>> message chain from it. We can allow the client GUI wich connects to
>>>> the central repository to provide the paramenters (For instance the
>>>> value of 'To' header) from which an intelligent filtering can be done
>>>> for the set of messages avialable at the central repository.
>>>>
>>>> Perhaps someone has an idea of a better way of doing it and is willing
>>>> to share it with us. It would be really nice to hear from them.
>>>>
>>>> Thanks in advance ..!!
>>>>
>>>> Best Regards,
>>>> Nilupa
>>>>
>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>> <an...@gmail.com> wrote:
>>>>> Personally, I think that the added value of extending the SOAP monitor
>>>>> module to collect messages in a central place is too limited to
>>>>> attract enough interest from the user community, so that it will be
>>>>> difficult to ensure the evolution of such a project in the future.
>>>>> However, what many people are looking for is a tool that allows to
>>>>> track the flow of a message through a distributed system, or more
>>>>> generally to track the sequence of events triggered by a given input
>>>>> message (sort of end-to-end transaction monitoring).
>>>>>
>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>>> proposing a GSoC project this summer. I would like to take project
>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>>> services middleware. I will submit more detailed proposal later. I
>>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>>> most welcome.
>>>>>>
>>>>>> Best Regards,
>>>>>> Nilupa Bandara
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>> Distributed TCP monitor
>>>>>>
>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>>> pull those messages. Also, we can do something to turn the module on
>>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>>> on only when they want to debug the system.
>>>>>>
>>>>>> Then we can take this to next level by adding this module to all
>>>>>> services in a system , configuring modules to send all collected
>>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>>> TCPMon for a whole system.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> ============================
>>> Srinath Perera, Ph.D.
>>>   WSO2 Inc. http://wso2.com
>>>   Blog: http://srinathsview.blogspot.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

On Tue, Mar 30, 2010 at 15:11, Glen Daniels <gl...@thoughtcraft.com> wrote:
> I'm a really terribly inconsistent blogger.  However, my last actual entry,
> from 2009, was about exactly this:
>
> http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html

That comes pretty damn close to what I was thinking about.

> I like Andreas' suggestion below, but I think it might be a bit heavyweight
> for a GSoC project(?).  Maybe we could lower the bar to something like this:

Hey, they get 5000$, so they should sweat a bit ;-)

> * Build a distributed logging web service which can accept log messages and
> correlationIDs, logging to a file per correlationID.
>
> * Build a Module that processes incoming <debug:logInfo> headers and makes an
> appropriate RemoteLogger (or whatever) available.  This could, at first, go
> in one of Axis2's contexts (OperationContext?) as a demonstration.
> Components who call RemoteLogger.log() send messages (hopefully on another
> thread) to the central log service.
>
> * The Module could automatically put the right metadata on outgoing messages,
> but only the responses, so you still have the problem Andreas points out
> about needing the app to transfer the logging info over to new
> ServiceClients.  I was imagining that could be achieved by also setting a
> ThreadLocal - so if your clients run on the same thread you're all set, and
> if there are new threads it's up to the framework to copy that ThreadLocal over.

Exactly. A simple ThreadLocal will probably cover the vast majority of
use cases, and for asynchronous executions, one can probably implement
some magic that propagates the context from one thread to the other.

> * Regardless of the discovery mechanism, you'd want something static like
> RemoteLogger.getLogger() to hide it.
>
> This approach is definitely a bit more intrusive, as it relies on the
> components themselves to be looking for a RemoteLogger, but it seems like a
> reasonable starting point which could be evolved.  However, maybe the AOP
> stuff isn't as challenging as I'm imagining....

It all depends on the layer in which you want to intercept the
messages (one extreme would be an around advice on
GenericServlet#service) and the protocols you want to cover (think
about non-SOAP JMS messages, remote EJB calls, etc.).

> Thoughts?
>
> --Glen
>
> On 3/30/2010 4:57 AM, Andreas Veithen wrote:
>> I was actually referring to scenarios where a service may call other
>> services, which in turn may call again other services. In these
>> scenarios, it is not sufficient to simply collect received/sent
>> messages on different hosts, unless the system is isolated
>> (development environment) or the request rate is sufficiently low so
>> that messages can be correlated based on time. Here is a concrete
>> scenario from a project in my company:
>>
>> - Request comes in on an ESB that does security and validation.
>> - Request is processed by an application server which persists the
>> received information and publishes a JMS message (pub/sub events).
>> - The event is consumed by one or more components that may in turn
>> interact with other services.
>>
>> If you want track this flow, it is not sufficient to intercept the
>> messages on the different hosts: in addition, you need to instrument
>> the services so that the outgoing requests are correlated with the
>> incoming requests (by adding SOAP and/or transport headers). What I
>> would like to be able to do in the above scenario is to enable
>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> HTTP header to the initial request to enable logging and this
>> information is propagated along the chain. Every service/component
>> then sends a copy of the incoming/outgoing requests/responses to a
>> central place where the sequence of events is reconstructed.
>>
>> One way this could be achieved is with a tool that postprocesses the
>> artifacts from the build process. For each artifact, the tool would
>> disassemble the artifact, instrument the code using a set of AspectJ
>> aspects and reassemble the artifact for deployment. The responsibility
>> of the aspects would be to intercept messages, log them and modify
>> them to ensure proper correlation. Of course this only works if the
>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> use AspectJ for a similar use case (although at a much smaller scale)
>> in the transport test suites [1]. Interestingly, this approach would
>> allow to cover interactions that are not SOAP based, e.g. one could
>> even intercept the database queries executed during the flow.
>>
>> Andreas
>>
>> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>
>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>> Hi Nilupa;
>>>
>>> When we collect messages from a one location by installing proper
>>> handler that will intercept and send messages to to that one location
>>> (this one location can be a single server, pub/sub channel etc). There
>>> is many ways to make sense of those collected messages. What Andreas
>>> mentioned (following complete transaction) is a one possibility.
>>>
>>> I think you should come up with few scenarios on how you would make
>>> sense of the message.
>>>
>>> Thanks
>>> Srinath
>>>
>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> First let me thank you for commenting.
>>>>
>>>> As far as I understood, what you would like to see from the proposed
>>>> tool is to view set of messages that are exchanged in reponse to a
>>>> particular input message. With the understanding that I am having at
>>>> the momnet, one way to do it is to filter out the central repository
>>>> of messages based on 'To' , 'From' headers and try to contruct the
>>>> message chain from it. We can allow the client GUI wich connects to
>>>> the central repository to provide the paramenters (For instance the
>>>> value of 'To' header) from which an intelligent filtering can be done
>>>> for the set of messages avialable at the central repository.
>>>>
>>>> Perhaps someone has an idea of a better way of doing it and is willing
>>>> to share it with us. It would be really nice to hear from them.
>>>>
>>>> Thanks in advance ..!!
>>>>
>>>> Best Regards,
>>>> Nilupa
>>>>
>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>> <an...@gmail.com> wrote:
>>>>> Personally, I think that the added value of extending the SOAP monitor
>>>>> module to collect messages in a central place is too limited to
>>>>> attract enough interest from the user community, so that it will be
>>>>> difficult to ensure the evolution of such a project in the future.
>>>>> However, what many people are looking for is a tool that allows to
>>>>> track the flow of a message through a distributed system, or more
>>>>> generally to track the sequence of events triggered by a given input
>>>>> message (sort of end-to-end transaction monitoring).
>>>>>
>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>>> proposing a GSoC project this summer. I would like to take project
>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>>> services middleware. I will submit more detailed proposal later. I
>>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>>> most welcome.
>>>>>>
>>>>>> Best Regards,
>>>>>> Nilupa Bandara
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>> Distributed TCP monitor
>>>>>>
>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>>> pull those messages. Also, we can do something to turn the module on
>>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>>> on only when they want to debug the system.
>>>>>>
>>>>>> Then we can take this to next level by adding this module to all
>>>>>> services in a system , configuring modules to send all collected
>>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>>> TCPMon for a whole system.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> ============================
>>> Srinath Perera, Ph.D.
>>>   WSO2 Inc. http://wso2.com
>>>   Blog: http://srinathsview.blogspot.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

Hello Everyone,

First let me thank all of you for your valuable inputs.

Now let me ask few questions to make sure that I understood the
suggested approach correctly.

- What are the components that you meant by 'Components who call
RemoteLogger.log()'  ? A Custom Message Receiver which logs the
messages that are going in (and out) to a Service Impl perhaps ?

- Does Service Impl needs check the availability of RemoteLogger (via
a thread local variable) and log the message it sends ( and receive)
using Service Clients to other applications ? I mean is it the case
where service author needs to insert 'RemoteLogger.Log()' in
appropriate places to ensure complete traceability ?

Using AOP - I just read few documentation on AspectJ and something
like the following could be done (But I'm not absolutely sure though
!!)

We could use some AOP code (package in a jar) which will, just before
each Service Client invocation insert the appropriate debug info
headers to outgoing payload , plus logs the response it gets.

This AOP could be instrumented (or weaved) into all the service
archive files in a given repository.

We could provide a script (and AOP jar) to automate this. For instance
./our_script AOP.jar <path-to-repository> will make all the services
that are available in the repository traceable.

Any suggestions ?

Nilupa


On Tue, Mar 30, 2010 at 3:11 PM, Glen Daniels <gl...@thoughtcraft.com> wrote:
> I'm a really terribly inconsistent blogger.  However, my last actual entry,
> from 2009, was about exactly this:
>
> http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html
>
> I like Andreas' suggestion below, but I think it might be a bit heavyweight
> for a GSoC project(?).  Maybe we could lower the bar to something like this:
>
> * Build a distributed logging web service which can accept log messages and
> correlationIDs, logging to a file per correlationID.
>
> * Build a Module that processes incoming <debug:logInfo> headers and makes an
> appropriate RemoteLogger (or whatever) available.  This could, at first, go
> in one of Axis2's contexts (OperationContext?) as a demonstration.
> Components who call RemoteLogger.log() send messages (hopefully on another
> thread) to the central log service.
>
> * The Module could automatically put the right metadata on outgoing messages,
> but only the responses, so you still have the problem Andreas points out
> about needing the app to transfer the logging info over to new
> ServiceClients.  I was imagining that could be achieved by also setting a
> ThreadLocal - so if your clients run on the same thread you're all set, and
> if there are new threads it's up to the framework to copy that ThreadLocal over.
>
> * Regardless of the discovery mechanism, you'd want something static like
> RemoteLogger.getLogger() to hide it.
>
> This approach is definitely a bit more intrusive, as it relies on the
> components themselves to be looking for a RemoteLogger, but it seems like a
> reasonable starting point which could be evolved.  However, maybe the AOP
> stuff isn't as challenging as I'm imagining....
>
> Thoughts?
>
> --Glen
>
> On 3/30/2010 4:57 AM, Andreas Veithen wrote:
>> I was actually referring to scenarios where a service may call other
>> services, which in turn may call again other services. In these
>> scenarios, it is not sufficient to simply collect received/sent
>> messages on different hosts, unless the system is isolated
>> (development environment) or the request rate is sufficiently low so
>> that messages can be correlated based on time. Here is a concrete
>> scenario from a project in my company:
>>
>> - Request comes in on an ESB that does security and validation.
>> - Request is processed by an application server which persists the
>> received information and publishes a JMS message (pub/sub events).
>> - The event is consumed by one or more components that may in turn
>> interact with other services.
>>
>> If you want track this flow, it is not sufficient to intercept the
>> messages on the different hosts: in addition, you need to instrument
>> the services so that the outgoing requests are correlated with the
>> incoming requests (by adding SOAP and/or transport headers). What I
>> would like to be able to do in the above scenario is to enable
>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> HTTP header to the initial request to enable logging and this
>> information is propagated along the chain. Every service/component
>> then sends a copy of the incoming/outgoing requests/responses to a
>> central place where the sequence of events is reconstructed.
>>
>> One way this could be achieved is with a tool that postprocesses the
>> artifacts from the build process. For each artifact, the tool would
>> disassemble the artifact, instrument the code using a set of AspectJ
>> aspects and reassemble the artifact for deployment. The responsibility
>> of the aspects would be to intercept messages, log them and modify
>> them to ensure proper correlation. Of course this only works if the
>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> use AspectJ for a similar use case (although at a much smaller scale)
>> in the transport test suites [1]. Interestingly, this approach would
>> allow to cover interactions that are not SOAP based, e.g. one could
>> even intercept the database queries executed during the flow.
>>
>> Andreas
>>
>> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>
>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>> Hi Nilupa;
>>>
>>> When we collect messages from a one location by installing proper
>>> handler that will intercept and send messages to to that one location
>>> (this one location can be a single server, pub/sub channel etc). There
>>> is many ways to make sense of those collected messages. What Andreas
>>> mentioned (following complete transaction) is a one possibility.
>>>
>>> I think you should come up with few scenarios on how you would make
>>> sense of the message.
>>>
>>> Thanks
>>> Srinath
>>>
>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> First let me thank you for commenting.
>>>>
>>>> As far as I understood, what you would like to see from the proposed
>>>> tool is to view set of messages that are exchanged in reponse to a
>>>> particular input message. With the understanding that I am having at
>>>> the momnet, one way to do it is to filter out the central repository
>>>> of messages based on 'To' , 'From' headers and try to contruct the
>>>> message chain from it. We can allow the client GUI wich connects to
>>>> the central repository to provide the paramenters (For instance the
>>>> value of 'To' header) from which an intelligent filtering can be done
>>>> for the set of messages avialable at the central repository.
>>>>
>>>> Perhaps someone has an idea of a better way of doing it and is willing
>>>> to share it with us. It would be really nice to hear from them.
>>>>
>>>> Thanks in advance ..!!
>>>>
>>>> Best Regards,
>>>> Nilupa
>>>>
>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>> <an...@gmail.com> wrote:
>>>>> Personally, I think that the added value of extending the SOAP monitor
>>>>> module to collect messages in a central place is too limited to
>>>>> attract enough interest from the user community, so that it will be
>>>>> difficult to ensure the evolution of such a project in the future.
>>>>> However, what many people are looking for is a tool that allows to
>>>>> track the flow of a message through a distributed system, or more
>>>>> generally to track the sequence of events triggered by a given input
>>>>> message (sort of end-to-end transaction monitoring).
>>>>>
>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>>> proposing a GSoC project this summer. I would like to take project
>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>>> services middleware. I will submit more detailed proposal later. I
>>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>>> most welcome.
>>>>>>
>>>>>> Best Regards,
>>>>>> Nilupa Bandara
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>> Distributed TCP monitor
>>>>>>
>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>>> pull those messages. Also, we can do something to turn the module on
>>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>>> on only when they want to debug the system.
>>>>>>
>>>>>> Then we can take this to next level by adding this module to all
>>>>>> services in a system , configuring modules to send all collected
>>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>>> TCPMon for a whole system.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> ============================
>>> Srinath Perera, Ph.D.
>>>   WSO2 Inc. http://wso2.com
>>>   Blog: http://srinathsview.blogspot.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

Hello Everyone,

First let me thank all of you for your valuable inputs.

Now let me ask few questions to make sure that I understood the
suggested approach correctly.

- What are the components that you meant by 'Components who call
RemoteLogger.log()'  ? A Custom Message Receiver which logs the
messages that are going in (and out) to a Service Impl perhaps ?

- Does Service Impl needs check the availability of RemoteLogger (via
a thread local variable) and log the message it sends ( and receive)
using Service Clients to other applications ? I mean is it the case
where service author needs to insert 'RemoteLogger.Log()' in
appropriate places to ensure complete traceability ?

Using AOP - I just read few documentation on AspectJ and something
like the following could be done (But I'm not absolutely sure though
!!)

We could use some AOP code (package in a jar) which will, just before
each Service Client invocation insert the appropriate debug info
headers to outgoing payload , plus logs the response it gets.

This AOP could be instrumented (or weaved) into all the service
archive files in a given repository.

We could provide a script (and AOP jar) to automate this. For instance
./our_script AOP.jar <path-to-repository> will make all the services
that are available in the repository traceable.

Any suggestions ?

Nilupa


On Tue, Mar 30, 2010 at 3:11 PM, Glen Daniels <gl...@thoughtcraft.com> wrote:
> I'm a really terribly inconsistent blogger.  However, my last actual entry,
> from 2009, was about exactly this:
>
> http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html
>
> I like Andreas' suggestion below, but I think it might be a bit heavyweight
> for a GSoC project(?).  Maybe we could lower the bar to something like this:
>
> * Build a distributed logging web service which can accept log messages and
> correlationIDs, logging to a file per correlationID.
>
> * Build a Module that processes incoming <debug:logInfo> headers and makes an
> appropriate RemoteLogger (or whatever) available.  This could, at first, go
> in one of Axis2's contexts (OperationContext?) as a demonstration.
> Components who call RemoteLogger.log() send messages (hopefully on another
> thread) to the central log service.
>
> * The Module could automatically put the right metadata on outgoing messages,
> but only the responses, so you still have the problem Andreas points out
> about needing the app to transfer the logging info over to new
> ServiceClients.  I was imagining that could be achieved by also setting a
> ThreadLocal - so if your clients run on the same thread you're all set, and
> if there are new threads it's up to the framework to copy that ThreadLocal over.
>
> * Regardless of the discovery mechanism, you'd want something static like
> RemoteLogger.getLogger() to hide it.
>
> This approach is definitely a bit more intrusive, as it relies on the
> components themselves to be looking for a RemoteLogger, but it seems like a
> reasonable starting point which could be evolved.  However, maybe the AOP
> stuff isn't as challenging as I'm imagining....
>
> Thoughts?
>
> --Glen
>
> On 3/30/2010 4:57 AM, Andreas Veithen wrote:
>> I was actually referring to scenarios where a service may call other
>> services, which in turn may call again other services. In these
>> scenarios, it is not sufficient to simply collect received/sent
>> messages on different hosts, unless the system is isolated
>> (development environment) or the request rate is sufficiently low so
>> that messages can be correlated based on time. Here is a concrete
>> scenario from a project in my company:
>>
>> - Request comes in on an ESB that does security and validation.
>> - Request is processed by an application server which persists the
>> received information and publishes a JMS message (pub/sub events).
>> - The event is consumed by one or more components that may in turn
>> interact with other services.
>>
>> If you want track this flow, it is not sufficient to intercept the
>> messages on the different hosts: in addition, you need to instrument
>> the services so that the outgoing requests are correlated with the
>> incoming requests (by adding SOAP and/or transport headers). What I
>> would like to be able to do in the above scenario is to enable
>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> HTTP header to the initial request to enable logging and this
>> information is propagated along the chain. Every service/component
>> then sends a copy of the incoming/outgoing requests/responses to a
>> central place where the sequence of events is reconstructed.
>>
>> One way this could be achieved is with a tool that postprocesses the
>> artifacts from the build process. For each artifact, the tool would
>> disassemble the artifact, instrument the code using a set of AspectJ
>> aspects and reassemble the artifact for deployment. The responsibility
>> of the aspects would be to intercept messages, log them and modify
>> them to ensure proper correlation. Of course this only works if the
>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> use AspectJ for a similar use case (although at a much smaller scale)
>> in the transport test suites [1]. Interestingly, this approach would
>> allow to cover interactions that are not SOAP based, e.g. one could
>> even intercept the database queries executed during the flow.
>>
>> Andreas
>>
>> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>
>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>> Hi Nilupa;
>>>
>>> When we collect messages from a one location by installing proper
>>> handler that will intercept and send messages to to that one location
>>> (this one location can be a single server, pub/sub channel etc). There
>>> is many ways to make sense of those collected messages. What Andreas
>>> mentioned (following complete transaction) is a one possibility.
>>>
>>> I think you should come up with few scenarios on how you would make
>>> sense of the message.
>>>
>>> Thanks
>>> Srinath
>>>
>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> First let me thank you for commenting.
>>>>
>>>> As far as I understood, what you would like to see from the proposed
>>>> tool is to view set of messages that are exchanged in reponse to a
>>>> particular input message. With the understanding that I am having at
>>>> the momnet, one way to do it is to filter out the central repository
>>>> of messages based on 'To' , 'From' headers and try to contruct the
>>>> message chain from it. We can allow the client GUI wich connects to
>>>> the central repository to provide the paramenters (For instance the
>>>> value of 'To' header) from which an intelligent filtering can be done
>>>> for the set of messages avialable at the central repository.
>>>>
>>>> Perhaps someone has an idea of a better way of doing it and is willing
>>>> to share it with us. It would be really nice to hear from them.
>>>>
>>>> Thanks in advance ..!!
>>>>
>>>> Best Regards,
>>>> Nilupa
>>>>
>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>> <an...@gmail.com> wrote:
>>>>> Personally, I think that the added value of extending the SOAP monitor
>>>>> module to collect messages in a central place is too limited to
>>>>> attract enough interest from the user community, so that it will be
>>>>> difficult to ensure the evolution of such a project in the future.
>>>>> However, what many people are looking for is a tool that allows to
>>>>> track the flow of a message through a distributed system, or more
>>>>> generally to track the sequence of events triggered by a given input
>>>>> message (sort of end-to-end transaction monitoring).
>>>>>
>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>>> proposing a GSoC project this summer. I would like to take project
>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>>> services middleware. I will submit more detailed proposal later. I
>>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>>> most welcome.
>>>>>>
>>>>>> Best Regards,
>>>>>> Nilupa Bandara
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>> Distributed TCP monitor
>>>>>>
>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>>> pull those messages. Also, we can do something to turn the module on
>>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>>> on only when they want to debug the system.
>>>>>>
>>>>>> Then we can take this to next level by adding this module to all
>>>>>> services in a system , configuring modules to send all collected
>>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>>> TCPMon for a whole system.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> ============================
>>> Srinath Perera, Ph.D.
>>>   WSO2 Inc. http://wso2.com
>>>   Blog: http://srinathsview.blogspot.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

Hello Everyone,

First let me thank all of you for your valuable inputs.

Now let me ask few questions to make sure that I understood the
suggested approach correctly.

- What are the components that you meant by 'Components who call
RemoteLogger.log()'  ? A Custom Message Receiver which logs the
messages that are going in (and out) to a Service Impl perhaps ?

- Does Service Impl needs check the availability of RemoteLogger (via
a thread local variable) and log the message it sends ( and receive)
using Service Clients to other applications ? I mean is it the case
where service author needs to insert 'RemoteLogger.Log()' in
appropriate places to ensure complete traceability ?

Using AOP - I just read few documentation on AspectJ and something
like the following could be done (But I'm not absolutely sure though
!!)

We could use some AOP code (package in a jar) which will, just before
each Service Client invocation insert the appropriate debug info
headers to outgoing payload , plus logs the response it gets.

This AOP could be instrumented (or weaved) into all the service
archive files in a given repository.

We could provide a script (and AOP jar) to automate this. For instance
./our_script AOP.jar <path-to-repository> will make all the services
that are available in the repository traceable.

Any suggestions ?

Nilupa


On Tue, Mar 30, 2010 at 3:11 PM, Glen Daniels <gl...@thoughtcraft.com> wrote:
> I'm a really terribly inconsistent blogger.  However, my last actual entry,
> from 2009, was about exactly this:
>
> http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html
>
> I like Andreas' suggestion below, but I think it might be a bit heavyweight
> for a GSoC project(?).  Maybe we could lower the bar to something like this:
>
> * Build a distributed logging web service which can accept log messages and
> correlationIDs, logging to a file per correlationID.
>
> * Build a Module that processes incoming <debug:logInfo> headers and makes an
> appropriate RemoteLogger (or whatever) available.  This could, at first, go
> in one of Axis2's contexts (OperationContext?) as a demonstration.
> Components who call RemoteLogger.log() send messages (hopefully on another
> thread) to the central log service.
>
> * The Module could automatically put the right metadata on outgoing messages,
> but only the responses, so you still have the problem Andreas points out
> about needing the app to transfer the logging info over to new
> ServiceClients.  I was imagining that could be achieved by also setting a
> ThreadLocal - so if your clients run on the same thread you're all set, and
> if there are new threads it's up to the framework to copy that ThreadLocal over.
>
> * Regardless of the discovery mechanism, you'd want something static like
> RemoteLogger.getLogger() to hide it.
>
> This approach is definitely a bit more intrusive, as it relies on the
> components themselves to be looking for a RemoteLogger, but it seems like a
> reasonable starting point which could be evolved.  However, maybe the AOP
> stuff isn't as challenging as I'm imagining....
>
> Thoughts?
>
> --Glen
>
> On 3/30/2010 4:57 AM, Andreas Veithen wrote:
>> I was actually referring to scenarios where a service may call other
>> services, which in turn may call again other services. In these
>> scenarios, it is not sufficient to simply collect received/sent
>> messages on different hosts, unless the system is isolated
>> (development environment) or the request rate is sufficiently low so
>> that messages can be correlated based on time. Here is a concrete
>> scenario from a project in my company:
>>
>> - Request comes in on an ESB that does security and validation.
>> - Request is processed by an application server which persists the
>> received information and publishes a JMS message (pub/sub events).
>> - The event is consumed by one or more components that may in turn
>> interact with other services.
>>
>> If you want track this flow, it is not sufficient to intercept the
>> messages on the different hosts: in addition, you need to instrument
>> the services so that the outgoing requests are correlated with the
>> incoming requests (by adding SOAP and/or transport headers). What I
>> would like to be able to do in the above scenario is to enable
>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> HTTP header to the initial request to enable logging and this
>> information is propagated along the chain. Every service/component
>> then sends a copy of the incoming/outgoing requests/responses to a
>> central place where the sequence of events is reconstructed.
>>
>> One way this could be achieved is with a tool that postprocesses the
>> artifacts from the build process. For each artifact, the tool would
>> disassemble the artifact, instrument the code using a set of AspectJ
>> aspects and reassemble the artifact for deployment. The responsibility
>> of the aspects would be to intercept messages, log them and modify
>> them to ensure proper correlation. Of course this only works if the
>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> use AspectJ for a similar use case (although at a much smaller scale)
>> in the transport test suites [1]. Interestingly, this approach would
>> allow to cover interactions that are not SOAP based, e.g. one could
>> even intercept the database queries executed during the flow.
>>
>> Andreas
>>
>> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>
>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>> Hi Nilupa;
>>>
>>> When we collect messages from a one location by installing proper
>>> handler that will intercept and send messages to to that one location
>>> (this one location can be a single server, pub/sub channel etc). There
>>> is many ways to make sense of those collected messages. What Andreas
>>> mentioned (following complete transaction) is a one possibility.
>>>
>>> I think you should come up with few scenarios on how you would make
>>> sense of the message.
>>>
>>> Thanks
>>> Srinath
>>>
>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> First let me thank you for commenting.
>>>>
>>>> As far as I understood, what you would like to see from the proposed
>>>> tool is to view set of messages that are exchanged in reponse to a
>>>> particular input message. With the understanding that I am having at
>>>> the momnet, one way to do it is to filter out the central repository
>>>> of messages based on 'To' , 'From' headers and try to contruct the
>>>> message chain from it. We can allow the client GUI wich connects to
>>>> the central repository to provide the paramenters (For instance the
>>>> value of 'To' header) from which an intelligent filtering can be done
>>>> for the set of messages avialable at the central repository.
>>>>
>>>> Perhaps someone has an idea of a better way of doing it and is willing
>>>> to share it with us. It would be really nice to hear from them.
>>>>
>>>> Thanks in advance ..!!
>>>>
>>>> Best Regards,
>>>> Nilupa
>>>>
>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>> <an...@gmail.com> wrote:
>>>>> Personally, I think that the added value of extending the SOAP monitor
>>>>> module to collect messages in a central place is too limited to
>>>>> attract enough interest from the user community, so that it will be
>>>>> difficult to ensure the evolution of such a project in the future.
>>>>> However, what many people are looking for is a tool that allows to
>>>>> track the flow of a message through a distributed system, or more
>>>>> generally to track the sequence of events triggered by a given input
>>>>> message (sort of end-to-end transaction monitoring).
>>>>>
>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>>> proposing a GSoC project this summer. I would like to take project
>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>>> services middleware. I will submit more detailed proposal later. I
>>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>>> most welcome.
>>>>>>
>>>>>> Best Regards,
>>>>>> Nilupa Bandara
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>> Distributed TCP monitor
>>>>>>
>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>>> pull those messages. Also, we can do something to turn the module on
>>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>>> on only when they want to debug the system.
>>>>>>
>>>>>> Then we can take this to next level by adding this module to all
>>>>>> services in a system , configuring modules to send all collected
>>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>>> TCPMon for a whole system.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> ============================
>>> Srinath Perera, Ph.D.
>>>   WSO2 Inc. http://wso2.com
>>>   Blog: http://srinathsview.blogspot.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

Hello Everyone,

First let me thank all of you for your valuable inputs.

Now let me ask few questions to make sure that I understood the
suggested approach correctly.

- What are the components that you meant by 'Components who call
RemoteLogger.log()'  ? A Custom Message Receiver which logs the
messages that are going in (and out) to a Service Impl perhaps ?

- Does Service Impl needs check the availability of RemoteLogger (via
a thread local variable) and log the message it sends ( and receive)
using Service Clients to other applications ? I mean is it the case
where service author needs to insert 'RemoteLogger.Log()' in
appropriate places to ensure complete traceability ?

Using AOP - I just read few documentation on AspectJ and something
like the following could be done (But I'm not absolutely sure though
!!)

We could use some AOP code (package in a jar) which will, just before
each Service Client invocation insert the appropriate debug info
headers to outgoing payload , plus logs the response it gets.

This AOP could be instrumented (or weaved) into all the service
archive files in a given repository.

We could provide a script (and AOP jar) to automate this. For instance
./our_script AOP.jar <path-to-repository> will make all the services
that are available in the repository traceable.

Any suggestions ?

Nilupa


On Tue, Mar 30, 2010 at 3:11 PM, Glen Daniels <gl...@thoughtcraft.com> wrote:
> I'm a really terribly inconsistent blogger.  However, my last actual entry,
> from 2009, was about exactly this:
>
> http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html
>
> I like Andreas' suggestion below, but I think it might be a bit heavyweight
> for a GSoC project(?).  Maybe we could lower the bar to something like this:
>
> * Build a distributed logging web service which can accept log messages and
> correlationIDs, logging to a file per correlationID.
>
> * Build a Module that processes incoming <debug:logInfo> headers and makes an
> appropriate RemoteLogger (or whatever) available.  This could, at first, go
> in one of Axis2's contexts (OperationContext?) as a demonstration.
> Components who call RemoteLogger.log() send messages (hopefully on another
> thread) to the central log service.
>
> * The Module could automatically put the right metadata on outgoing messages,
> but only the responses, so you still have the problem Andreas points out
> about needing the app to transfer the logging info over to new
> ServiceClients.  I was imagining that could be achieved by also setting a
> ThreadLocal - so if your clients run on the same thread you're all set, and
> if there are new threads it's up to the framework to copy that ThreadLocal over.
>
> * Regardless of the discovery mechanism, you'd want something static like
> RemoteLogger.getLogger() to hide it.
>
> This approach is definitely a bit more intrusive, as it relies on the
> components themselves to be looking for a RemoteLogger, but it seems like a
> reasonable starting point which could be evolved.  However, maybe the AOP
> stuff isn't as challenging as I'm imagining....
>
> Thoughts?
>
> --Glen
>
> On 3/30/2010 4:57 AM, Andreas Veithen wrote:
>> I was actually referring to scenarios where a service may call other
>> services, which in turn may call again other services. In these
>> scenarios, it is not sufficient to simply collect received/sent
>> messages on different hosts, unless the system is isolated
>> (development environment) or the request rate is sufficiently low so
>> that messages can be correlated based on time. Here is a concrete
>> scenario from a project in my company:
>>
>> - Request comes in on an ESB that does security and validation.
>> - Request is processed by an application server which persists the
>> received information and publishes a JMS message (pub/sub events).
>> - The event is consumed by one or more components that may in turn
>> interact with other services.
>>
>> If you want track this flow, it is not sufficient to intercept the
>> messages on the different hosts: in addition, you need to instrument
>> the services so that the outgoing requests are correlated with the
>> incoming requests (by adding SOAP and/or transport headers). What I
>> would like to be able to do in the above scenario is to enable
>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> HTTP header to the initial request to enable logging and this
>> information is propagated along the chain. Every service/component
>> then sends a copy of the incoming/outgoing requests/responses to a
>> central place where the sequence of events is reconstructed.
>>
>> One way this could be achieved is with a tool that postprocesses the
>> artifacts from the build process. For each artifact, the tool would
>> disassemble the artifact, instrument the code using a set of AspectJ
>> aspects and reassemble the artifact for deployment. The responsibility
>> of the aspects would be to intercept messages, log them and modify
>> them to ensure proper correlation. Of course this only works if the
>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> use AspectJ for a similar use case (although at a much smaller scale)
>> in the transport test suites [1]. Interestingly, this approach would
>> allow to cover interactions that are not SOAP based, e.g. one could
>> even intercept the database queries executed during the flow.
>>
>> Andreas
>>
>> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>
>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>> Hi Nilupa;
>>>
>>> When we collect messages from a one location by installing proper
>>> handler that will intercept and send messages to to that one location
>>> (this one location can be a single server, pub/sub channel etc). There
>>> is many ways to make sense of those collected messages. What Andreas
>>> mentioned (following complete transaction) is a one possibility.
>>>
>>> I think you should come up with few scenarios on how you would make
>>> sense of the message.
>>>
>>> Thanks
>>> Srinath
>>>
>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> First let me thank you for commenting.
>>>>
>>>> As far as I understood, what you would like to see from the proposed
>>>> tool is to view set of messages that are exchanged in reponse to a
>>>> particular input message. With the understanding that I am having at
>>>> the momnet, one way to do it is to filter out the central repository
>>>> of messages based on 'To' , 'From' headers and try to contruct the
>>>> message chain from it. We can allow the client GUI wich connects to
>>>> the central repository to provide the paramenters (For instance the
>>>> value of 'To' header) from which an intelligent filtering can be done
>>>> for the set of messages avialable at the central repository.
>>>>
>>>> Perhaps someone has an idea of a better way of doing it and is willing
>>>> to share it with us. It would be really nice to hear from them.
>>>>
>>>> Thanks in advance ..!!
>>>>
>>>> Best Regards,
>>>> Nilupa
>>>>
>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>> <an...@gmail.com> wrote:
>>>>> Personally, I think that the added value of extending the SOAP monitor
>>>>> module to collect messages in a central place is too limited to
>>>>> attract enough interest from the user community, so that it will be
>>>>> difficult to ensure the evolution of such a project in the future.
>>>>> However, what many people are looking for is a tool that allows to
>>>>> track the flow of a message through a distributed system, or more
>>>>> generally to track the sequence of events triggered by a given input
>>>>> message (sort of end-to-end transaction monitoring).
>>>>>
>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>>> proposing a GSoC project this summer. I would like to take project
>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>>> services middleware. I will submit more detailed proposal later. I
>>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>>> most welcome.
>>>>>>
>>>>>> Best Regards,
>>>>>> Nilupa Bandara
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>> Distributed TCP monitor
>>>>>>
>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>>> pull those messages. Also, we can do something to turn the module on
>>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>>> on only when they want to debug the system.
>>>>>>
>>>>>> Then we can take this to next level by adding this module to all
>>>>>> services in a system , configuring modules to send all collected
>>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>>> TCPMon for a whole system.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> ============================
>>> Srinath Perera, Ph.D.
>>>   WSO2 Inc. http://wso2.com
>>>   Blog: http://srinathsview.blogspot.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

On Tue, Mar 30, 2010 at 15:11, Glen Daniels <gl...@thoughtcraft.com> wrote:
> I'm a really terribly inconsistent blogger.  However, my last actual entry,
> from 2009, was about exactly this:
>
> http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html

That comes pretty damn close to what I was thinking about.

> I like Andreas' suggestion below, but I think it might be a bit heavyweight
> for a GSoC project(?).  Maybe we could lower the bar to something like this:

Hey, they get 5000$, so they should sweat a bit ;-)

> * Build a distributed logging web service which can accept log messages and
> correlationIDs, logging to a file per correlationID.
>
> * Build a Module that processes incoming <debug:logInfo> headers and makes an
> appropriate RemoteLogger (or whatever) available.  This could, at first, go
> in one of Axis2's contexts (OperationContext?) as a demonstration.
> Components who call RemoteLogger.log() send messages (hopefully on another
> thread) to the central log service.
>
> * The Module could automatically put the right metadata on outgoing messages,
> but only the responses, so you still have the problem Andreas points out
> about needing the app to transfer the logging info over to new
> ServiceClients.  I was imagining that could be achieved by also setting a
> ThreadLocal - so if your clients run on the same thread you're all set, and
> if there are new threads it's up to the framework to copy that ThreadLocal over.

Exactly. A simple ThreadLocal will probably cover the vast majority of
use cases, and for asynchronous executions, one can probably implement
some magic that propagates the context from one thread to the other.

> * Regardless of the discovery mechanism, you'd want something static like
> RemoteLogger.getLogger() to hide it.
>
> This approach is definitely a bit more intrusive, as it relies on the
> components themselves to be looking for a RemoteLogger, but it seems like a
> reasonable starting point which could be evolved.  However, maybe the AOP
> stuff isn't as challenging as I'm imagining....

It all depends on the layer in which you want to intercept the
messages (one extreme would be an around advice on
GenericServlet#service) and the protocols you want to cover (think
about non-SOAP JMS messages, remote EJB calls, etc.).

> Thoughts?
>
> --Glen
>
> On 3/30/2010 4:57 AM, Andreas Veithen wrote:
>> I was actually referring to scenarios where a service may call other
>> services, which in turn may call again other services. In these
>> scenarios, it is not sufficient to simply collect received/sent
>> messages on different hosts, unless the system is isolated
>> (development environment) or the request rate is sufficiently low so
>> that messages can be correlated based on time. Here is a concrete
>> scenario from a project in my company:
>>
>> - Request comes in on an ESB that does security and validation.
>> - Request is processed by an application server which persists the
>> received information and publishes a JMS message (pub/sub events).
>> - The event is consumed by one or more components that may in turn
>> interact with other services.
>>
>> If you want track this flow, it is not sufficient to intercept the
>> messages on the different hosts: in addition, you need to instrument
>> the services so that the outgoing requests are correlated with the
>> incoming requests (by adding SOAP and/or transport headers). What I
>> would like to be able to do in the above scenario is to enable
>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> HTTP header to the initial request to enable logging and this
>> information is propagated along the chain. Every service/component
>> then sends a copy of the incoming/outgoing requests/responses to a
>> central place where the sequence of events is reconstructed.
>>
>> One way this could be achieved is with a tool that postprocesses the
>> artifacts from the build process. For each artifact, the tool would
>> disassemble the artifact, instrument the code using a set of AspectJ
>> aspects and reassemble the artifact for deployment. The responsibility
>> of the aspects would be to intercept messages, log them and modify
>> them to ensure proper correlation. Of course this only works if the
>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> use AspectJ for a similar use case (although at a much smaller scale)
>> in the transport test suites [1]. Interestingly, this approach would
>> allow to cover interactions that are not SOAP based, e.g. one could
>> even intercept the database queries executed during the flow.
>>
>> Andreas
>>
>> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>
>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>> Hi Nilupa;
>>>
>>> When we collect messages from a one location by installing proper
>>> handler that will intercept and send messages to to that one location
>>> (this one location can be a single server, pub/sub channel etc). There
>>> is many ways to make sense of those collected messages. What Andreas
>>> mentioned (following complete transaction) is a one possibility.
>>>
>>> I think you should come up with few scenarios on how you would make
>>> sense of the message.
>>>
>>> Thanks
>>> Srinath
>>>
>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> First let me thank you for commenting.
>>>>
>>>> As far as I understood, what you would like to see from the proposed
>>>> tool is to view set of messages that are exchanged in reponse to a
>>>> particular input message. With the understanding that I am having at
>>>> the momnet, one way to do it is to filter out the central repository
>>>> of messages based on 'To' , 'From' headers and try to contruct the
>>>> message chain from it. We can allow the client GUI wich connects to
>>>> the central repository to provide the paramenters (For instance the
>>>> value of 'To' header) from which an intelligent filtering can be done
>>>> for the set of messages avialable at the central repository.
>>>>
>>>> Perhaps someone has an idea of a better way of doing it and is willing
>>>> to share it with us. It would be really nice to hear from them.
>>>>
>>>> Thanks in advance ..!!
>>>>
>>>> Best Regards,
>>>> Nilupa
>>>>
>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>> <an...@gmail.com> wrote:
>>>>> Personally, I think that the added value of extending the SOAP monitor
>>>>> module to collect messages in a central place is too limited to
>>>>> attract enough interest from the user community, so that it will be
>>>>> difficult to ensure the evolution of such a project in the future.
>>>>> However, what many people are looking for is a tool that allows to
>>>>> track the flow of a message through a distributed system, or more
>>>>> generally to track the sequence of events triggered by a given input
>>>>> message (sort of end-to-end transaction monitoring).
>>>>>
>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>>> proposing a GSoC project this summer. I would like to take project
>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>>> services middleware. I will submit more detailed proposal later. I
>>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>>> most welcome.
>>>>>>
>>>>>> Best Regards,
>>>>>> Nilupa Bandara
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>> Distributed TCP monitor
>>>>>>
>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>>> pull those messages. Also, we can do something to turn the module on
>>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>>> on only when they want to debug the system.
>>>>>>
>>>>>> Then we can take this to next level by adding this module to all
>>>>>> services in a system , configuring modules to send all collected
>>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>>> TCPMon for a whole system.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> ============================
>>> Srinath Perera, Ph.D.
>>>   WSO2 Inc. http://wso2.com
>>>   Blog: http://srinathsview.blogspot.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Glen Daniels <gl...@thoughtcraft.com>.

I'm a really terribly inconsistent blogger.  However, my last actual entry,
from 2009, was about exactly this:

http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html

I like Andreas' suggestion below, but I think it might be a bit heavyweight
for a GSoC project(?).  Maybe we could lower the bar to something like this:

* Build a distributed logging web service which can accept log messages and
correlationIDs, logging to a file per correlationID.

* Build a Module that processes incoming <debug:logInfo> headers and makes an
appropriate RemoteLogger (or whatever) available.  This could, at first, go
in one of Axis2's contexts (OperationContext?) as a demonstration.
Components who call RemoteLogger.log() send messages (hopefully on another
thread) to the central log service.

* The Module could automatically put the right metadata on outgoing messages,
but only the responses, so you still have the problem Andreas points out
about needing the app to transfer the logging info over to new
ServiceClients.  I was imagining that could be achieved by also setting a
ThreadLocal - so if your clients run on the same thread you're all set, and
if there are new threads it's up to the framework to copy that ThreadLocal over.

* Regardless of the discovery mechanism, you'd want something static like
RemoteLogger.getLogger() to hide it.

This approach is definitely a bit more intrusive, as it relies on the
components themselves to be looking for a RemoteLogger, but it seems like a
reasonable starting point which could be evolved.  However, maybe the AOP
stuff isn't as challenging as I'm imagining....

Thoughts?

--Glen

On 3/30/2010 4:57 AM, Andreas Veithen wrote:
> I was actually referring to scenarios where a service may call other
> services, which in turn may call again other services. In these
> scenarios, it is not sufficient to simply collect received/sent
> messages on different hosts, unless the system is isolated
> (development environment) or the request rate is sufficiently low so
> that messages can be correlated based on time. Here is a concrete
> scenario from a project in my company:
> 
> - Request comes in on an ESB that does security and validation.
> - Request is processed by an application server which persists the
> received information and publishes a JMS message (pub/sub events).
> - The event is consumed by one or more components that may in turn
> interact with other services.
> 
> If you want track this flow, it is not sufficient to intercept the
> messages on the different hosts: in addition, you need to instrument
> the services so that the outgoing requests are correlated with the
> incoming requests (by adding SOAP and/or transport headers). What I
> would like to be able to do in the above scenario is to enable
> end-to-end monitoring on a per-message basis: I add a special SOAP or
> HTTP header to the initial request to enable logging and this
> information is propagated along the chain. Every service/component
> then sends a copy of the incoming/outgoing requests/responses to a
> central place where the sequence of events is reconstructed.
> 
> One way this could be achieved is with a tool that postprocesses the
> artifacts from the build process. For each artifact, the tool would
> disassemble the artifact, instrument the code using a set of AspectJ
> aspects and reassemble the artifact for deployment. The responsibility
> of the aspects would be to intercept messages, log them and modify
> them to ensure proper correlation. Of course this only works if the
> artifacts are J2EE deployables (WARs or EARs). Note that we already
> use AspectJ for a similar use case (although at a much smaller scale)
> in the transport test suites [1]. Interestingly, this approach would
> allow to cover interactions that are not SOAP based, e.g. one could
> even intercept the database queries executed during the flow.
> 
> Andreas
> 
> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> 
> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>> Hi Nilupa;
>>
>> When we collect messages from a one location by installing proper
>> handler that will intercept and send messages to to that one location
>> (this one location can be a single server, pub/sub channel etc). There
>> is many ways to make sense of those collected messages. What Andreas
>> mentioned (following complete transaction) is a one possibility.
>>
>> I think you should come up with few scenarios on how you would make
>> sense of the message.
>>
>> Thanks
>> Srinath
>>
>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>> Hi,
>>>
>>> First let me thank you for commenting.
>>>
>>> As far as I understood, what you would like to see from the proposed
>>> tool is to view set of messages that are exchanged in reponse to a
>>> particular input message. With the understanding that I am having at
>>> the momnet, one way to do it is to filter out the central repository
>>> of messages based on 'To' , 'From' headers and try to contruct the
>>> message chain from it. We can allow the client GUI wich connects to
>>> the central repository to provide the paramenters (For instance the
>>> value of 'To' header) from which an intelligent filtering can be done
>>> for the set of messages avialable at the central repository.
>>>
>>> Perhaps someone has an idea of a better way of doing it and is willing
>>> to share it with us. It would be really nice to hear from them.
>>>
>>> Thanks in advance ..!!
>>>
>>> Best Regards,
>>> Nilupa
>>>
>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> <an...@gmail.com> wrote:
>>>> Personally, I think that the added value of extending the SOAP monitor
>>>> module to collect messages in a central place is too limited to
>>>> attract enough interest from the user community, so that it will be
>>>> difficult to ensure the evolution of such a project in the future.
>>>> However, what many people are looking for is a tool that allows to
>>>> track the flow of a message through a distributed system, or more
>>>> generally to track the sequence of events triggered by a given input
>>>> message (sort of end-to-end transaction monitoring).
>>>>
>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>> Hello,
>>>>>
>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>> proposing a GSoC project this summer. I would like to take project
>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>> services middleware. I will submit more detailed proposal later. I
>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>> most welcome.
>>>>>
>>>>> Best Regards,
>>>>> Nilupa Bandara
>>>>>
>>>>> [1]
>>>>>
>>>>> Distributed TCP monitor
>>>>>
>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>> pull those messages. Also, we can do something to turn the module on
>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>> on only when they want to debug the system.
>>>>>
>>>>> Then we can take this to next level by adding this module to all
>>>>> services in a system , configuring modules to send all collected
>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>> TCPMon for a whole system.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>>
>>
>> --
>> ============================
>> Srinath Perera, Ph.D.
>>   WSO2 Inc. http://wso2.com
>>   Blog: http://srinathsview.blogspot.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Thu, Apr 1, 2010 at 7:04 AM, Sanka Samaranayake <ss...@gmail.com> wrote:

> Samisa,
>
> Quick question, does WSO2 BAM project provide such a debugging
> facility. I mean can one monitor what is going on for a specific input
> in SOA senario ?
>

Yes it can.

However, as I mentioned in an earlier explanation, that is not the main
purpose of BAM.

Samisa...



> Sanka
>
>
>
> On Thu, Apr 1, 2010 at 3:28 AM, Sanka Samaranayake <ss...@gmail.com>
> wrote:
> > Hi Samisa,
> >
> > I think the this scope of this project would be to provide a
> > light-weight tool which allows to monitor the message flows, execution
> > in each node in response to those message flows to a given input
> > message primarily for debugging purposes . (Just like the way tcpmon
> > is used in developing Web services)
> >
> > And I guess now the question is whether such a tool is useful ..
> >
> > Anyone cares to share their thoughts ?
> >
> > Nilupa
> >
> > On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
> > <sa...@gmail.com> wrote:
> >>
> >>
> >> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com>
> wrote:
> >>>
> >>> I was thinking about Java Swing based UI
> >>
> >> Swing is a good start. But if you look at some of the requirements like
> >> monitoring multiple instances and correlate them, discussed in this
> thread,
> >> it sounds as if we would be better off having a Web interface, rather
> than a
> >> desktop client.
> >>
> >>>
> >>> because the client may have
> >>> to cache whatever messages it already has and will receive,
> >>
> >> If you are looking to support message-correlation, as discussed in some
> >> replies in this thread above, you need to have a persistence storage (a
> DB
> >> or something like that) rather than a memory cache.
> >> Because, some message sequences, like those involved in an async
> >> invocations, need to have a way to keep the messages, in there, longer,
> and
> >> may be there is a server re-start, in the middle.
> >> Of course, this requirements spend on the scope of this effort.
> >> Plus, if you want to deal with historic data, and look at what was
> happening
> >> to message sequences over time, you got to use a persistence layer.
> Again
> >> this might be out of scope for this project, but if this is to be
> extended
> >> to support these, you better think of this right now.
> >>
> >>>
> >>> build the
> >>> message flow and update the UI accordingly. Perhaps there is a better
> >>> way ?
> >>
> >> We already do what you are trying to do, and much more with WSO2 BAM. I
> am
> >> tempted to say that Shinding based gadget dashboard is really a
> compelling
> >> UI for this.
> >> I am not trying to take away you project or trying to hinder this effort
> at
> >> all.
> >> I think it will be a good idea for you to have a look at BAM and see
> what
> >> the missing pieces are and figure out the right model for this tool.
> >> Also, having had worked on WSO2 BAM project, I also have some
> understanding
> >> on sort of problems that you might run into, when doing this. For e.g.,
> if
> >> you want to correlate message sequences in a high volume setup, you will
> be
> >> swamped by the volume of messages, sooner than later. So correlation for
> the
> >> purpose of monitoring is not an easy thing to do. Correlation, for the
> >> purpose of debugging will not run into this problem, if you run this on
> a
> >> controlled environment. However, what you really want to debug is the
> deal
> >> clustered setup, and not a staged dev setup, the problem again surfaces.
> >> You might want to list the objectives and then break them down
> >> into monitoring and debugging spaces.
> >> Samisa...
> >>
> >>>
> >>> Thanks
> >>> Nilupa.
> >>>
> >>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
> >>> <sa...@gmail.com> wrote:
> >>> > What UI technology would be used for this?
> >>> >
> >>> > Samisa...
> >>> >
> >>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
> >>> > <an...@gmail.com> wrote:
> >>> >> I was thinking more about something like ITCAM for SOA.
> >>> >>
> >>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
> >>> >> <sa...@opensource.lk> wrote:
> >>> >>> That runtime monitoring / correlation is what the WSO2 Business
> >>> >>> Activity
> >>> >>> Monitor does / is for ...
> >>> >>> See: http://wso2.com/products/business-activity-monitor/
> >>> >>> Sanjiva.
> >>> >>>
> >>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
> >>> >>> <an...@gmail.com>
> >>> >>> wrote:
> >>> >>>>
> >>> >>>> I was actually referring to scenarios where a service may call
> other
> >>> >>>> services, which in turn may call again other services. In these
> >>> >>>> scenarios, it is not sufficient to simply collect received/sent
> >>> >>>> messages on different hosts, unless the system is isolated
> >>> >>>> (development environment) or the request rate is sufficiently low
> so
> >>> >>>> that messages can be correlated based on time. Here is a concrete
> >>> >>>> scenario from a project in my company:
> >>> >>>>
> >>> >>>> - Request comes in on an ESB that does security and validation.
> >>> >>>> - Request is processed by an application server which persists the
> >>> >>>> received information and publishes a JMS message (pub/sub events).
> >>> >>>> - The event is consumed by one or more components that may in turn
> >>> >>>> interact with other services.
> >>> >>>>
> >>> >>>> If you want track this flow, it is not sufficient to intercept the
> >>> >>>> messages on the different hosts: in addition, you need to
> instrument
> >>> >>>> the services so that the outgoing requests are correlated with the
> >>> >>>> incoming requests (by adding SOAP and/or transport headers). What
> I
> >>> >>>> would like to be able to do in the above scenario is to enable
> >>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP
> or
> >>> >>>> HTTP header to the initial request to enable logging and this
> >>> >>>> information is propagated along the chain. Every service/component
> >>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
> >>> >>>> central place where the sequence of events is reconstructed.
> >>> >>>>
> >>> >>>> One way this could be achieved is with a tool that postprocesses
> the
> >>> >>>> artifacts from the build process. For each artifact, the tool
> would
> >>> >>>> disassemble the artifact, instrument the code using a set of
> AspectJ
> >>> >>>> aspects and reassemble the artifact for deployment. The
> >>> >>>> responsibility
> >>> >>>> of the aspects would be to intercept messages, log them and modify
> >>> >>>> them to ensure proper correlation. Of course this only works if
> the
> >>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we
> already
> >>> >>>> use AspectJ for a similar use case (although at a much smaller
> scale)
> >>> >>>> in the transport test suites [1]. Interestingly, this approach
> would
> >>> >>>> allow to cover interactions that are not SOAP based, e.g. one
> could
> >>> >>>> even intercept the database queries executed during the flow.
> >>> >>>>
> >>> >>>> Andreas
> >>> >>>>
> >>> >>>> [1]
> >>> >>>>
> >>> >>>>
> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> >>> >>>>
> >>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <hemapani@gmail.com
> >
> >>> >>>> wrote:
> >>> >>>> > Hi Nilupa;
> >>> >>>> >
> >>> >>>> > When we collect messages from a one location by installing
> proper
> >>> >>>> > handler that will intercept and send messages to to that one
> >>> >>>> > location
> >>> >>>> > (this one location can be a single server, pub/sub channel etc).
> >>> >>>> > There
> >>> >>>> > is many ways to make sense of those collected messages. What
> >>> >>>> > Andreas
> >>> >>>> > mentioned (following complete transaction) is a one possibility.
> >>> >>>> >
> >>> >>>> > I think you should come up with few scenarios on how you would
> make
> >>> >>>> > sense of the message.
> >>> >>>> >
> >>> >>>> > Thanks
> >>> >>>> > Srinath
> >>> >>>> >
> >>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <
> nilupas@gmail.com>
> >>> >>>> > wrote:
> >>> >>>> >> Hi,
> >>> >>>> >>
> >>> >>>> >> First let me thank you for commenting.
> >>> >>>> >>
> >>> >>>> >> As far as I understood, what you would like to see from the
> >>> >>>> >> proposed
> >>> >>>> >> tool is to view set of messages that are exchanged in reponse
> to a
> >>> >>>> >> particular input message. With the understanding that I am
> having
> >>> >>>> >> at
> >>> >>>> >> the momnet, one way to do it is to filter out the central
> >>> >>>> >> repository
> >>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct
> the
> >>> >>>> >> message chain from it. We can allow the client GUI wich
> connects
> >>> >>>> >> to
> >>> >>>> >> the central repository to provide the paramenters (For instance
> >>> >>>> >> the
> >>> >>>> >> value of 'To' header) from which an intelligent filtering can
> be
> >>> >>>> >> done
> >>> >>>> >> for the set of messages avialable at the central repository.
> >>> >>>> >>
> >>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
> >>> >>>> >> willing
> >>> >>>> >> to share it with us. It would be really nice to hear from them.
> >>> >>>> >>
> >>> >>>> >> Thanks in advance ..!!
> >>> >>>> >>
> >>> >>>> >> Best Regards,
> >>> >>>> >> Nilupa
> >>> >>>> >>
> >>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> >>> >>>> >> <an...@gmail.com> wrote:
> >>> >>>> >>> Personally, I think that the added value of extending the SOAP
> >>> >>>> >>> monitor
> >>> >>>> >>> module to collect messages in a central place is too limited
> to
> >>> >>>> >>> attract enough interest from the user community, so that it
> will
> >>> >>>> >>> be
> >>> >>>> >>> difficult to ensure the evolution of such a project in the
> >>> >>>> >>> future.
> >>> >>>> >>> However, what many people are looking for is a tool that
> allows
> >>> >>>> >>> to
> >>> >>>> >>> track the flow of a message through a distributed system, or
> more
> >>> >>>> >>> generally to track the sequence of events triggered by a given
> >>> >>>> >>> input
> >>> >>>> >>> message (sort of end-to-end transaction monitoring).
> >>> >>>> >>>
> >>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <
> nilupas@gmail.com>
> >>> >>>> >>> wrote:
> >>> >>>> >>>> Hello,
> >>> >>>> >>>>
> >>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am
> thinking
> >>> >>>> >>>> of
> >>> >>>> >>>> proposing a GSoC project this summer. I would like to take
> >>> >>>> >>>> project
> >>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very
> interesting
> >>> >>>> >>>> and
> >>> >>>> >>>> also should be helpful to any Java developer using Apache
> Axis2
> >>> >>>> >>>> Web
> >>> >>>> >>>> services middleware. I will submit more detailed proposal
> later.
> >>> >>>> >>>> I
> >>> >>>> >>>> would really like to hear any feedback from you. Any
> suggestions
> >>> >>>> >>>> or
> >>> >>>> >>>> features that you would like to see in a "Distributed TCP
> >>> >>>> >>>> Monitor"
> >>> >>>> >>>> are
> >>> >>>> >>>> most welcome.
> >>> >>>> >>>>
> >>> >>>> >>>> Best Regards,
> >>> >>>> >>>> Nilupa Bandara
> >>> >>>> >>>>
> >>> >>>> >>>> [1]
> >>> >>>> >>>>
> >>> >>>> >>>> Distributed TCP monitor
> >>> >>>> >>>>
> >>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is
> bit
> >>> >>>> >>>> hard
> >>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can
> solve
> >>> >>>> >>>> the
> >>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
> >>> >>>> >>>> messages
> >>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via
> a
> >>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so
> user
> >>> >>>> >>>> can
> >>> >>>> >>>> pull those messages. Also, we can do something to turn the
> >>> >>>> >>>> module on
> >>> >>>> >>>> and off remotely, so users can have his module deployed, but
> >>> >>>> >>>> turn it
> >>> >>>> >>>> on only when they want to debug the system.
> >>> >>>> >>>>
> >>> >>>> >>>> Then we can take this to next level by adding this module to
> all
> >>> >>>> >>>> services in a system , configuring modules to send all
> collected
> >>> >>>> >>>> messages to a pub/sub channel, and subscribing to those
> messages
> >>> >>>> >>>> via
> >>> >>>> >>>> a
> >>> >>>> >>>> UI, and depicting those messages through a UI. Then users
> have a
> >>> >>>> >>>> TCPMon for a whole system.
> >>> >>>> >>>>
> >>> >>>> >>>>
> >>> >>>> >>>>
> ---------------------------------------------------------------------
> >>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >>>> For additional commands, e-mail:
> java-dev-help@axis.apache.org
> >>> >>>> >>>>
> >>> >>>> >>>>
> >>> >>>> >>>
> >>> >>>> >>>
> >>> >>>> >>>
> ---------------------------------------------------------------------
> >>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >>> For additional commands, e-mail:
> java-dev-help@axis.apache.org
> >>> >>>> >>>
> >>> >>>> >>>
> >>> >>>> >>
> >>> >>>> >>
> >>> >>>> >>
> ---------------------------------------------------------------------
> >>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>> >>
> >>> >>>> >>
> >>> >>>> >
> >>> >>>> >
> >>> >>>> >
> >>> >>>> > --
> >>> >>>> > ============================
> >>> >>>> > Srinath Perera, Ph.D.
> >>> >>>> >   WSO2 Inc. http://wso2.com
> >>> >>>> >   Blog: http://srinathsview.blogspot.com/
> >>> >>>> >
> >>> >>>> >
> >>> >>>> >
> ---------------------------------------------------------------------
> >>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>> >
> >>> >>>> >
> >>> >>>>
> >>> >>>>
> ---------------------------------------------------------------------
> >>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>>
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> --
> >>> >>> Sanjiva Weerawarana, Ph.D.
> >>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
> >>> >>> http://www.opensource.lk/
> >>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> >>> >>> Member; Apache Software Foundation; http://www.apache.org/
> >>> >>> Director; Sahana Software Foundation;
> http://www.sahanafoundation.org/
> >>> >>> Visiting Lecturer; University of Moratuwa;
> http://www.cse.mrt.ac.lk/
> >>> >>>
> >>> >>> Blog: http://sanjiva.weerawarana.org/
> >>> >>>
> >>> >>
> >>> >>
> >>> >>
> ---------------------------------------------------------------------
> >>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>
> >>> >
> >>> > ---------------------------------------------------------------------
> >>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >
> >>> >
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>
> >> Samisa...
> >> --
> >> blog: http://samisa-abeysinghe.blogspot.com/
> >>
> >
> >
> >
> > --
> > Sanka Samaranayake
> >
> > http://sankas.blogspot.com/
> >
>
>
>
> --
> Sanka Samaranayake
>
> http://sankas.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Thu, Apr 1, 2010 at 7:04 AM, Sanka Samaranayake <ss...@gmail.com> wrote:

> Samisa,
>
> Quick question, does WSO2 BAM project provide such a debugging
> facility. I mean can one monitor what is going on for a specific input
> in SOA senario ?
>

Yes it can.

However, as I mentioned in an earlier explanation, that is not the main
purpose of BAM.

Samisa...



> Sanka
>
>
>
> On Thu, Apr 1, 2010 at 3:28 AM, Sanka Samaranayake <ss...@gmail.com>
> wrote:
> > Hi Samisa,
> >
> > I think the this scope of this project would be to provide a
> > light-weight tool which allows to monitor the message flows, execution
> > in each node in response to those message flows to a given input
> > message primarily for debugging purposes . (Just like the way tcpmon
> > is used in developing Web services)
> >
> > And I guess now the question is whether such a tool is useful ..
> >
> > Anyone cares to share their thoughts ?
> >
> > Nilupa
> >
> > On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
> > <sa...@gmail.com> wrote:
> >>
> >>
> >> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com>
> wrote:
> >>>
> >>> I was thinking about Java Swing based UI
> >>
> >> Swing is a good start. But if you look at some of the requirements like
> >> monitoring multiple instances and correlate them, discussed in this
> thread,
> >> it sounds as if we would be better off having a Web interface, rather
> than a
> >> desktop client.
> >>
> >>>
> >>> because the client may have
> >>> to cache whatever messages it already has and will receive,
> >>
> >> If you are looking to support message-correlation, as discussed in some
> >> replies in this thread above, you need to have a persistence storage (a
> DB
> >> or something like that) rather than a memory cache.
> >> Because, some message sequences, like those involved in an async
> >> invocations, need to have a way to keep the messages, in there, longer,
> and
> >> may be there is a server re-start, in the middle.
> >> Of course, this requirements spend on the scope of this effort.
> >> Plus, if you want to deal with historic data, and look at what was
> happening
> >> to message sequences over time, you got to use a persistence layer.
> Again
> >> this might be out of scope for this project, but if this is to be
> extended
> >> to support these, you better think of this right now.
> >>
> >>>
> >>> build the
> >>> message flow and update the UI accordingly. Perhaps there is a better
> >>> way ?
> >>
> >> We already do what you are trying to do, and much more with WSO2 BAM. I
> am
> >> tempted to say that Shinding based gadget dashboard is really a
> compelling
> >> UI for this.
> >> I am not trying to take away you project or trying to hinder this effort
> at
> >> all.
> >> I think it will be a good idea for you to have a look at BAM and see
> what
> >> the missing pieces are and figure out the right model for this tool.
> >> Also, having had worked on WSO2 BAM project, I also have some
> understanding
> >> on sort of problems that you might run into, when doing this. For e.g.,
> if
> >> you want to correlate message sequences in a high volume setup, you will
> be
> >> swamped by the volume of messages, sooner than later. So correlation for
> the
> >> purpose of monitoring is not an easy thing to do. Correlation, for the
> >> purpose of debugging will not run into this problem, if you run this on
> a
> >> controlled environment. However, what you really want to debug is the
> deal
> >> clustered setup, and not a staged dev setup, the problem again surfaces.
> >> You might want to list the objectives and then break them down
> >> into monitoring and debugging spaces.
> >> Samisa...
> >>
> >>>
> >>> Thanks
> >>> Nilupa.
> >>>
> >>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
> >>> <sa...@gmail.com> wrote:
> >>> > What UI technology would be used for this?
> >>> >
> >>> > Samisa...
> >>> >
> >>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
> >>> > <an...@gmail.com> wrote:
> >>> >> I was thinking more about something like ITCAM for SOA.
> >>> >>
> >>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
> >>> >> <sa...@opensource.lk> wrote:
> >>> >>> That runtime monitoring / correlation is what the WSO2 Business
> >>> >>> Activity
> >>> >>> Monitor does / is for ...
> >>> >>> See: http://wso2.com/products/business-activity-monitor/
> >>> >>> Sanjiva.
> >>> >>>
> >>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
> >>> >>> <an...@gmail.com>
> >>> >>> wrote:
> >>> >>>>
> >>> >>>> I was actually referring to scenarios where a service may call
> other
> >>> >>>> services, which in turn may call again other services. In these
> >>> >>>> scenarios, it is not sufficient to simply collect received/sent
> >>> >>>> messages on different hosts, unless the system is isolated
> >>> >>>> (development environment) or the request rate is sufficiently low
> so
> >>> >>>> that messages can be correlated based on time. Here is a concrete
> >>> >>>> scenario from a project in my company:
> >>> >>>>
> >>> >>>> - Request comes in on an ESB that does security and validation.
> >>> >>>> - Request is processed by an application server which persists the
> >>> >>>> received information and publishes a JMS message (pub/sub events).
> >>> >>>> - The event is consumed by one or more components that may in turn
> >>> >>>> interact with other services.
> >>> >>>>
> >>> >>>> If you want track this flow, it is not sufficient to intercept the
> >>> >>>> messages on the different hosts: in addition, you need to
> instrument
> >>> >>>> the services so that the outgoing requests are correlated with the
> >>> >>>> incoming requests (by adding SOAP and/or transport headers). What
> I
> >>> >>>> would like to be able to do in the above scenario is to enable
> >>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP
> or
> >>> >>>> HTTP header to the initial request to enable logging and this
> >>> >>>> information is propagated along the chain. Every service/component
> >>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
> >>> >>>> central place where the sequence of events is reconstructed.
> >>> >>>>
> >>> >>>> One way this could be achieved is with a tool that postprocesses
> the
> >>> >>>> artifacts from the build process. For each artifact, the tool
> would
> >>> >>>> disassemble the artifact, instrument the code using a set of
> AspectJ
> >>> >>>> aspects and reassemble the artifact for deployment. The
> >>> >>>> responsibility
> >>> >>>> of the aspects would be to intercept messages, log them and modify
> >>> >>>> them to ensure proper correlation. Of course this only works if
> the
> >>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we
> already
> >>> >>>> use AspectJ for a similar use case (although at a much smaller
> scale)
> >>> >>>> in the transport test suites [1]. Interestingly, this approach
> would
> >>> >>>> allow to cover interactions that are not SOAP based, e.g. one
> could
> >>> >>>> even intercept the database queries executed during the flow.
> >>> >>>>
> >>> >>>> Andreas
> >>> >>>>
> >>> >>>> [1]
> >>> >>>>
> >>> >>>>
> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> >>> >>>>
> >>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <hemapani@gmail.com
> >
> >>> >>>> wrote:
> >>> >>>> > Hi Nilupa;
> >>> >>>> >
> >>> >>>> > When we collect messages from a one location by installing
> proper
> >>> >>>> > handler that will intercept and send messages to to that one
> >>> >>>> > location
> >>> >>>> > (this one location can be a single server, pub/sub channel etc).
> >>> >>>> > There
> >>> >>>> > is many ways to make sense of those collected messages. What
> >>> >>>> > Andreas
> >>> >>>> > mentioned (following complete transaction) is a one possibility.
> >>> >>>> >
> >>> >>>> > I think you should come up with few scenarios on how you would
> make
> >>> >>>> > sense of the message.
> >>> >>>> >
> >>> >>>> > Thanks
> >>> >>>> > Srinath
> >>> >>>> >
> >>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <
> nilupas@gmail.com>
> >>> >>>> > wrote:
> >>> >>>> >> Hi,
> >>> >>>> >>
> >>> >>>> >> First let me thank you for commenting.
> >>> >>>> >>
> >>> >>>> >> As far as I understood, what you would like to see from the
> >>> >>>> >> proposed
> >>> >>>> >> tool is to view set of messages that are exchanged in reponse
> to a
> >>> >>>> >> particular input message. With the understanding that I am
> having
> >>> >>>> >> at
> >>> >>>> >> the momnet, one way to do it is to filter out the central
> >>> >>>> >> repository
> >>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct
> the
> >>> >>>> >> message chain from it. We can allow the client GUI wich
> connects
> >>> >>>> >> to
> >>> >>>> >> the central repository to provide the paramenters (For instance
> >>> >>>> >> the
> >>> >>>> >> value of 'To' header) from which an intelligent filtering can
> be
> >>> >>>> >> done
> >>> >>>> >> for the set of messages avialable at the central repository.
> >>> >>>> >>
> >>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
> >>> >>>> >> willing
> >>> >>>> >> to share it with us. It would be really nice to hear from them.
> >>> >>>> >>
> >>> >>>> >> Thanks in advance ..!!
> >>> >>>> >>
> >>> >>>> >> Best Regards,
> >>> >>>> >> Nilupa
> >>> >>>> >>
> >>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> >>> >>>> >> <an...@gmail.com> wrote:
> >>> >>>> >>> Personally, I think that the added value of extending the SOAP
> >>> >>>> >>> monitor
> >>> >>>> >>> module to collect messages in a central place is too limited
> to
> >>> >>>> >>> attract enough interest from the user community, so that it
> will
> >>> >>>> >>> be
> >>> >>>> >>> difficult to ensure the evolution of such a project in the
> >>> >>>> >>> future.
> >>> >>>> >>> However, what many people are looking for is a tool that
> allows
> >>> >>>> >>> to
> >>> >>>> >>> track the flow of a message through a distributed system, or
> more
> >>> >>>> >>> generally to track the sequence of events triggered by a given
> >>> >>>> >>> input
> >>> >>>> >>> message (sort of end-to-end transaction monitoring).
> >>> >>>> >>>
> >>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <
> nilupas@gmail.com>
> >>> >>>> >>> wrote:
> >>> >>>> >>>> Hello,
> >>> >>>> >>>>
> >>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am
> thinking
> >>> >>>> >>>> of
> >>> >>>> >>>> proposing a GSoC project this summer. I would like to take
> >>> >>>> >>>> project
> >>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very
> interesting
> >>> >>>> >>>> and
> >>> >>>> >>>> also should be helpful to any Java developer using Apache
> Axis2
> >>> >>>> >>>> Web
> >>> >>>> >>>> services middleware. I will submit more detailed proposal
> later.
> >>> >>>> >>>> I
> >>> >>>> >>>> would really like to hear any feedback from you. Any
> suggestions
> >>> >>>> >>>> or
> >>> >>>> >>>> features that you would like to see in a "Distributed TCP
> >>> >>>> >>>> Monitor"
> >>> >>>> >>>> are
> >>> >>>> >>>> most welcome.
> >>> >>>> >>>>
> >>> >>>> >>>> Best Regards,
> >>> >>>> >>>> Nilupa Bandara
> >>> >>>> >>>>
> >>> >>>> >>>> [1]
> >>> >>>> >>>>
> >>> >>>> >>>> Distributed TCP monitor
> >>> >>>> >>>>
> >>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is
> bit
> >>> >>>> >>>> hard
> >>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can
> solve
> >>> >>>> >>>> the
> >>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
> >>> >>>> >>>> messages
> >>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via
> a
> >>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so
> user
> >>> >>>> >>>> can
> >>> >>>> >>>> pull those messages. Also, we can do something to turn the
> >>> >>>> >>>> module on
> >>> >>>> >>>> and off remotely, so users can have his module deployed, but
> >>> >>>> >>>> turn it
> >>> >>>> >>>> on only when they want to debug the system.
> >>> >>>> >>>>
> >>> >>>> >>>> Then we can take this to next level by adding this module to
> all
> >>> >>>> >>>> services in a system , configuring modules to send all
> collected
> >>> >>>> >>>> messages to a pub/sub channel, and subscribing to those
> messages
> >>> >>>> >>>> via
> >>> >>>> >>>> a
> >>> >>>> >>>> UI, and depicting those messages through a UI. Then users
> have a
> >>> >>>> >>>> TCPMon for a whole system.
> >>> >>>> >>>>
> >>> >>>> >>>>
> >>> >>>> >>>>
> ---------------------------------------------------------------------
> >>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >>>> For additional commands, e-mail:
> java-dev-help@axis.apache.org
> >>> >>>> >>>>
> >>> >>>> >>>>
> >>> >>>> >>>
> >>> >>>> >>>
> >>> >>>> >>>
> ---------------------------------------------------------------------
> >>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >>> For additional commands, e-mail:
> java-dev-help@axis.apache.org
> >>> >>>> >>>
> >>> >>>> >>>
> >>> >>>> >>
> >>> >>>> >>
> >>> >>>> >>
> ---------------------------------------------------------------------
> >>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>> >>
> >>> >>>> >>
> >>> >>>> >
> >>> >>>> >
> >>> >>>> >
> >>> >>>> > --
> >>> >>>> > ============================
> >>> >>>> > Srinath Perera, Ph.D.
> >>> >>>> >   WSO2 Inc. http://wso2.com
> >>> >>>> >   Blog: http://srinathsview.blogspot.com/
> >>> >>>> >
> >>> >>>> >
> >>> >>>> >
> ---------------------------------------------------------------------
> >>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>> >
> >>> >>>> >
> >>> >>>>
> >>> >>>>
> ---------------------------------------------------------------------
> >>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>>
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> --
> >>> >>> Sanjiva Weerawarana, Ph.D.
> >>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
> >>> >>> http://www.opensource.lk/
> >>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> >>> >>> Member; Apache Software Foundation; http://www.apache.org/
> >>> >>> Director; Sahana Software Foundation;
> http://www.sahanafoundation.org/
> >>> >>> Visiting Lecturer; University of Moratuwa;
> http://www.cse.mrt.ac.lk/
> >>> >>>
> >>> >>> Blog: http://sanjiva.weerawarana.org/
> >>> >>>
> >>> >>
> >>> >>
> >>> >>
> ---------------------------------------------------------------------
> >>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>
> >>> >
> >>> > ---------------------------------------------------------------------
> >>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >
> >>> >
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>
> >> Samisa...
> >> --
> >> blog: http://samisa-abeysinghe.blogspot.com/
> >>
> >
> >
> >
> > --
> > Sanka Samaranayake
> >
> > http://sankas.blogspot.com/
> >
>
>
>
> --
> Sanka Samaranayake
>
> http://sankas.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Thu, Apr 1, 2010 at 7:04 AM, Sanka Samaranayake <ss...@gmail.com> wrote:

> Samisa,
>
> Quick question, does WSO2 BAM project provide such a debugging
> facility. I mean can one monitor what is going on for a specific input
> in SOA senario ?
>

Yes it can.

However, as I mentioned in an earlier explanation, that is not the main
purpose of BAM.

Samisa...



> Sanka
>
>
>
> On Thu, Apr 1, 2010 at 3:28 AM, Sanka Samaranayake <ss...@gmail.com>
> wrote:
> > Hi Samisa,
> >
> > I think the this scope of this project would be to provide a
> > light-weight tool which allows to monitor the message flows, execution
> > in each node in response to those message flows to a given input
> > message primarily for debugging purposes . (Just like the way tcpmon
> > is used in developing Web services)
> >
> > And I guess now the question is whether such a tool is useful ..
> >
> > Anyone cares to share their thoughts ?
> >
> > Nilupa
> >
> > On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
> > <sa...@gmail.com> wrote:
> >>
> >>
> >> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com>
> wrote:
> >>>
> >>> I was thinking about Java Swing based UI
> >>
> >> Swing is a good start. But if you look at some of the requirements like
> >> monitoring multiple instances and correlate them, discussed in this
> thread,
> >> it sounds as if we would be better off having a Web interface, rather
> than a
> >> desktop client.
> >>
> >>>
> >>> because the client may have
> >>> to cache whatever messages it already has and will receive,
> >>
> >> If you are looking to support message-correlation, as discussed in some
> >> replies in this thread above, you need to have a persistence storage (a
> DB
> >> or something like that) rather than a memory cache.
> >> Because, some message sequences, like those involved in an async
> >> invocations, need to have a way to keep the messages, in there, longer,
> and
> >> may be there is a server re-start, in the middle.
> >> Of course, this requirements spend on the scope of this effort.
> >> Plus, if you want to deal with historic data, and look at what was
> happening
> >> to message sequences over time, you got to use a persistence layer.
> Again
> >> this might be out of scope for this project, but if this is to be
> extended
> >> to support these, you better think of this right now.
> >>
> >>>
> >>> build the
> >>> message flow and update the UI accordingly. Perhaps there is a better
> >>> way ?
> >>
> >> We already do what you are trying to do, and much more with WSO2 BAM. I
> am
> >> tempted to say that Shinding based gadget dashboard is really a
> compelling
> >> UI for this.
> >> I am not trying to take away you project or trying to hinder this effort
> at
> >> all.
> >> I think it will be a good idea for you to have a look at BAM and see
> what
> >> the missing pieces are and figure out the right model for this tool.
> >> Also, having had worked on WSO2 BAM project, I also have some
> understanding
> >> on sort of problems that you might run into, when doing this. For e.g.,
> if
> >> you want to correlate message sequences in a high volume setup, you will
> be
> >> swamped by the volume of messages, sooner than later. So correlation for
> the
> >> purpose of monitoring is not an easy thing to do. Correlation, for the
> >> purpose of debugging will not run into this problem, if you run this on
> a
> >> controlled environment. However, what you really want to debug is the
> deal
> >> clustered setup, and not a staged dev setup, the problem again surfaces.
> >> You might want to list the objectives and then break them down
> >> into monitoring and debugging spaces.
> >> Samisa...
> >>
> >>>
> >>> Thanks
> >>> Nilupa.
> >>>
> >>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
> >>> <sa...@gmail.com> wrote:
> >>> > What UI technology would be used for this?
> >>> >
> >>> > Samisa...
> >>> >
> >>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
> >>> > <an...@gmail.com> wrote:
> >>> >> I was thinking more about something like ITCAM for SOA.
> >>> >>
> >>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
> >>> >> <sa...@opensource.lk> wrote:
> >>> >>> That runtime monitoring / correlation is what the WSO2 Business
> >>> >>> Activity
> >>> >>> Monitor does / is for ...
> >>> >>> See: http://wso2.com/products/business-activity-monitor/
> >>> >>> Sanjiva.
> >>> >>>
> >>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
> >>> >>> <an...@gmail.com>
> >>> >>> wrote:
> >>> >>>>
> >>> >>>> I was actually referring to scenarios where a service may call
> other
> >>> >>>> services, which in turn may call again other services. In these
> >>> >>>> scenarios, it is not sufficient to simply collect received/sent
> >>> >>>> messages on different hosts, unless the system is isolated
> >>> >>>> (development environment) or the request rate is sufficiently low
> so
> >>> >>>> that messages can be correlated based on time. Here is a concrete
> >>> >>>> scenario from a project in my company:
> >>> >>>>
> >>> >>>> - Request comes in on an ESB that does security and validation.
> >>> >>>> - Request is processed by an application server which persists the
> >>> >>>> received information and publishes a JMS message (pub/sub events).
> >>> >>>> - The event is consumed by one or more components that may in turn
> >>> >>>> interact with other services.
> >>> >>>>
> >>> >>>> If you want track this flow, it is not sufficient to intercept the
> >>> >>>> messages on the different hosts: in addition, you need to
> instrument
> >>> >>>> the services so that the outgoing requests are correlated with the
> >>> >>>> incoming requests (by adding SOAP and/or transport headers). What
> I
> >>> >>>> would like to be able to do in the above scenario is to enable
> >>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP
> or
> >>> >>>> HTTP header to the initial request to enable logging and this
> >>> >>>> information is propagated along the chain. Every service/component
> >>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
> >>> >>>> central place where the sequence of events is reconstructed.
> >>> >>>>
> >>> >>>> One way this could be achieved is with a tool that postprocesses
> the
> >>> >>>> artifacts from the build process. For each artifact, the tool
> would
> >>> >>>> disassemble the artifact, instrument the code using a set of
> AspectJ
> >>> >>>> aspects and reassemble the artifact for deployment. The
> >>> >>>> responsibility
> >>> >>>> of the aspects would be to intercept messages, log them and modify
> >>> >>>> them to ensure proper correlation. Of course this only works if
> the
> >>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we
> already
> >>> >>>> use AspectJ for a similar use case (although at a much smaller
> scale)
> >>> >>>> in the transport test suites [1]. Interestingly, this approach
> would
> >>> >>>> allow to cover interactions that are not SOAP based, e.g. one
> could
> >>> >>>> even intercept the database queries executed during the flow.
> >>> >>>>
> >>> >>>> Andreas
> >>> >>>>
> >>> >>>> [1]
> >>> >>>>
> >>> >>>>
> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> >>> >>>>
> >>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <hemapani@gmail.com
> >
> >>> >>>> wrote:
> >>> >>>> > Hi Nilupa;
> >>> >>>> >
> >>> >>>> > When we collect messages from a one location by installing
> proper
> >>> >>>> > handler that will intercept and send messages to to that one
> >>> >>>> > location
> >>> >>>> > (this one location can be a single server, pub/sub channel etc).
> >>> >>>> > There
> >>> >>>> > is many ways to make sense of those collected messages. What
> >>> >>>> > Andreas
> >>> >>>> > mentioned (following complete transaction) is a one possibility.
> >>> >>>> >
> >>> >>>> > I think you should come up with few scenarios on how you would
> make
> >>> >>>> > sense of the message.
> >>> >>>> >
> >>> >>>> > Thanks
> >>> >>>> > Srinath
> >>> >>>> >
> >>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <
> nilupas@gmail.com>
> >>> >>>> > wrote:
> >>> >>>> >> Hi,
> >>> >>>> >>
> >>> >>>> >> First let me thank you for commenting.
> >>> >>>> >>
> >>> >>>> >> As far as I understood, what you would like to see from the
> >>> >>>> >> proposed
> >>> >>>> >> tool is to view set of messages that are exchanged in reponse
> to a
> >>> >>>> >> particular input message. With the understanding that I am
> having
> >>> >>>> >> at
> >>> >>>> >> the momnet, one way to do it is to filter out the central
> >>> >>>> >> repository
> >>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct
> the
> >>> >>>> >> message chain from it. We can allow the client GUI wich
> connects
> >>> >>>> >> to
> >>> >>>> >> the central repository to provide the paramenters (For instance
> >>> >>>> >> the
> >>> >>>> >> value of 'To' header) from which an intelligent filtering can
> be
> >>> >>>> >> done
> >>> >>>> >> for the set of messages avialable at the central repository.
> >>> >>>> >>
> >>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
> >>> >>>> >> willing
> >>> >>>> >> to share it with us. It would be really nice to hear from them.
> >>> >>>> >>
> >>> >>>> >> Thanks in advance ..!!
> >>> >>>> >>
> >>> >>>> >> Best Regards,
> >>> >>>> >> Nilupa
> >>> >>>> >>
> >>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> >>> >>>> >> <an...@gmail.com> wrote:
> >>> >>>> >>> Personally, I think that the added value of extending the SOAP
> >>> >>>> >>> monitor
> >>> >>>> >>> module to collect messages in a central place is too limited
> to
> >>> >>>> >>> attract enough interest from the user community, so that it
> will
> >>> >>>> >>> be
> >>> >>>> >>> difficult to ensure the evolution of such a project in the
> >>> >>>> >>> future.
> >>> >>>> >>> However, what many people are looking for is a tool that
> allows
> >>> >>>> >>> to
> >>> >>>> >>> track the flow of a message through a distributed system, or
> more
> >>> >>>> >>> generally to track the sequence of events triggered by a given
> >>> >>>> >>> input
> >>> >>>> >>> message (sort of end-to-end transaction monitoring).
> >>> >>>> >>>
> >>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <
> nilupas@gmail.com>
> >>> >>>> >>> wrote:
> >>> >>>> >>>> Hello,
> >>> >>>> >>>>
> >>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am
> thinking
> >>> >>>> >>>> of
> >>> >>>> >>>> proposing a GSoC project this summer. I would like to take
> >>> >>>> >>>> project
> >>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very
> interesting
> >>> >>>> >>>> and
> >>> >>>> >>>> also should be helpful to any Java developer using Apache
> Axis2
> >>> >>>> >>>> Web
> >>> >>>> >>>> services middleware. I will submit more detailed proposal
> later.
> >>> >>>> >>>> I
> >>> >>>> >>>> would really like to hear any feedback from you. Any
> suggestions
> >>> >>>> >>>> or
> >>> >>>> >>>> features that you would like to see in a "Distributed TCP
> >>> >>>> >>>> Monitor"
> >>> >>>> >>>> are
> >>> >>>> >>>> most welcome.
> >>> >>>> >>>>
> >>> >>>> >>>> Best Regards,
> >>> >>>> >>>> Nilupa Bandara
> >>> >>>> >>>>
> >>> >>>> >>>> [1]
> >>> >>>> >>>>
> >>> >>>> >>>> Distributed TCP monitor
> >>> >>>> >>>>
> >>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is
> bit
> >>> >>>> >>>> hard
> >>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can
> solve
> >>> >>>> >>>> the
> >>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
> >>> >>>> >>>> messages
> >>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via
> a
> >>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so
> user
> >>> >>>> >>>> can
> >>> >>>> >>>> pull those messages. Also, we can do something to turn the
> >>> >>>> >>>> module on
> >>> >>>> >>>> and off remotely, so users can have his module deployed, but
> >>> >>>> >>>> turn it
> >>> >>>> >>>> on only when they want to debug the system.
> >>> >>>> >>>>
> >>> >>>> >>>> Then we can take this to next level by adding this module to
> all
> >>> >>>> >>>> services in a system , configuring modules to send all
> collected
> >>> >>>> >>>> messages to a pub/sub channel, and subscribing to those
> messages
> >>> >>>> >>>> via
> >>> >>>> >>>> a
> >>> >>>> >>>> UI, and depicting those messages through a UI. Then users
> have a
> >>> >>>> >>>> TCPMon for a whole system.
> >>> >>>> >>>>
> >>> >>>> >>>>
> >>> >>>> >>>>
> ---------------------------------------------------------------------
> >>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >>>> For additional commands, e-mail:
> java-dev-help@axis.apache.org
> >>> >>>> >>>>
> >>> >>>> >>>>
> >>> >>>> >>>
> >>> >>>> >>>
> >>> >>>> >>>
> ---------------------------------------------------------------------
> >>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >>> For additional commands, e-mail:
> java-dev-help@axis.apache.org
> >>> >>>> >>>
> >>> >>>> >>>
> >>> >>>> >>
> >>> >>>> >>
> >>> >>>> >>
> ---------------------------------------------------------------------
> >>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>> >>
> >>> >>>> >>
> >>> >>>> >
> >>> >>>> >
> >>> >>>> >
> >>> >>>> > --
> >>> >>>> > ============================
> >>> >>>> > Srinath Perera, Ph.D.
> >>> >>>> >   WSO2 Inc. http://wso2.com
> >>> >>>> >   Blog: http://srinathsview.blogspot.com/
> >>> >>>> >
> >>> >>>> >
> >>> >>>> >
> ---------------------------------------------------------------------
> >>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>> >
> >>> >>>> >
> >>> >>>>
> >>> >>>>
> ---------------------------------------------------------------------
> >>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>>
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> --
> >>> >>> Sanjiva Weerawarana, Ph.D.
> >>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
> >>> >>> http://www.opensource.lk/
> >>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> >>> >>> Member; Apache Software Foundation; http://www.apache.org/
> >>> >>> Director; Sahana Software Foundation;
> http://www.sahanafoundation.org/
> >>> >>> Visiting Lecturer; University of Moratuwa;
> http://www.cse.mrt.ac.lk/
> >>> >>>
> >>> >>> Blog: http://sanjiva.weerawarana.org/
> >>> >>>
> >>> >>
> >>> >>
> >>> >>
> ---------------------------------------------------------------------
> >>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>
> >>> >
> >>> > ---------------------------------------------------------------------
> >>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >
> >>> >
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>
> >> Samisa...
> >> --
> >> blog: http://samisa-abeysinghe.blogspot.com/
> >>
> >
> >
> >
> > --
> > Sanka Samaranayake
> >
> > http://sankas.blogspot.com/
> >
>
>
>
> --
> Sanka Samaranayake
>
> http://sankas.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Thu, Apr 1, 2010 at 7:04 AM, Sanka Samaranayake <ss...@gmail.com> wrote:

> Samisa,
>
> Quick question, does WSO2 BAM project provide such a debugging
> facility. I mean can one monitor what is going on for a specific input
> in SOA senario ?
>

Yes it can.

However, as I mentioned in an earlier explanation, that is not the main
purpose of BAM.

Samisa...



> Sanka
>
>
>
> On Thu, Apr 1, 2010 at 3:28 AM, Sanka Samaranayake <ss...@gmail.com>
> wrote:
> > Hi Samisa,
> >
> > I think the this scope of this project would be to provide a
> > light-weight tool which allows to monitor the message flows, execution
> > in each node in response to those message flows to a given input
> > message primarily for debugging purposes . (Just like the way tcpmon
> > is used in developing Web services)
> >
> > And I guess now the question is whether such a tool is useful ..
> >
> > Anyone cares to share their thoughts ?
> >
> > Nilupa
> >
> > On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
> > <sa...@gmail.com> wrote:
> >>
> >>
> >> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com>
> wrote:
> >>>
> >>> I was thinking about Java Swing based UI
> >>
> >> Swing is a good start. But if you look at some of the requirements like
> >> monitoring multiple instances and correlate them, discussed in this
> thread,
> >> it sounds as if we would be better off having a Web interface, rather
> than a
> >> desktop client.
> >>
> >>>
> >>> because the client may have
> >>> to cache whatever messages it already has and will receive,
> >>
> >> If you are looking to support message-correlation, as discussed in some
> >> replies in this thread above, you need to have a persistence storage (a
> DB
> >> or something like that) rather than a memory cache.
> >> Because, some message sequences, like those involved in an async
> >> invocations, need to have a way to keep the messages, in there, longer,
> and
> >> may be there is a server re-start, in the middle.
> >> Of course, this requirements spend on the scope of this effort.
> >> Plus, if you want to deal with historic data, and look at what was
> happening
> >> to message sequences over time, you got to use a persistence layer.
> Again
> >> this might be out of scope for this project, but if this is to be
> extended
> >> to support these, you better think of this right now.
> >>
> >>>
> >>> build the
> >>> message flow and update the UI accordingly. Perhaps there is a better
> >>> way ?
> >>
> >> We already do what you are trying to do, and much more with WSO2 BAM. I
> am
> >> tempted to say that Shinding based gadget dashboard is really a
> compelling
> >> UI for this.
> >> I am not trying to take away you project or trying to hinder this effort
> at
> >> all.
> >> I think it will be a good idea for you to have a look at BAM and see
> what
> >> the missing pieces are and figure out the right model for this tool.
> >> Also, having had worked on WSO2 BAM project, I also have some
> understanding
> >> on sort of problems that you might run into, when doing this. For e.g.,
> if
> >> you want to correlate message sequences in a high volume setup, you will
> be
> >> swamped by the volume of messages, sooner than later. So correlation for
> the
> >> purpose of monitoring is not an easy thing to do. Correlation, for the
> >> purpose of debugging will not run into this problem, if you run this on
> a
> >> controlled environment. However, what you really want to debug is the
> deal
> >> clustered setup, and not a staged dev setup, the problem again surfaces.
> >> You might want to list the objectives and then break them down
> >> into monitoring and debugging spaces.
> >> Samisa...
> >>
> >>>
> >>> Thanks
> >>> Nilupa.
> >>>
> >>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
> >>> <sa...@gmail.com> wrote:
> >>> > What UI technology would be used for this?
> >>> >
> >>> > Samisa...
> >>> >
> >>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
> >>> > <an...@gmail.com> wrote:
> >>> >> I was thinking more about something like ITCAM for SOA.
> >>> >>
> >>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
> >>> >> <sa...@opensource.lk> wrote:
> >>> >>> That runtime monitoring / correlation is what the WSO2 Business
> >>> >>> Activity
> >>> >>> Monitor does / is for ...
> >>> >>> See: http://wso2.com/products/business-activity-monitor/
> >>> >>> Sanjiva.
> >>> >>>
> >>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
> >>> >>> <an...@gmail.com>
> >>> >>> wrote:
> >>> >>>>
> >>> >>>> I was actually referring to scenarios where a service may call
> other
> >>> >>>> services, which in turn may call again other services. In these
> >>> >>>> scenarios, it is not sufficient to simply collect received/sent
> >>> >>>> messages on different hosts, unless the system is isolated
> >>> >>>> (development environment) or the request rate is sufficiently low
> so
> >>> >>>> that messages can be correlated based on time. Here is a concrete
> >>> >>>> scenario from a project in my company:
> >>> >>>>
> >>> >>>> - Request comes in on an ESB that does security and validation.
> >>> >>>> - Request is processed by an application server which persists the
> >>> >>>> received information and publishes a JMS message (pub/sub events).
> >>> >>>> - The event is consumed by one or more components that may in turn
> >>> >>>> interact with other services.
> >>> >>>>
> >>> >>>> If you want track this flow, it is not sufficient to intercept the
> >>> >>>> messages on the different hosts: in addition, you need to
> instrument
> >>> >>>> the services so that the outgoing requests are correlated with the
> >>> >>>> incoming requests (by adding SOAP and/or transport headers). What
> I
> >>> >>>> would like to be able to do in the above scenario is to enable
> >>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP
> or
> >>> >>>> HTTP header to the initial request to enable logging and this
> >>> >>>> information is propagated along the chain. Every service/component
> >>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
> >>> >>>> central place where the sequence of events is reconstructed.
> >>> >>>>
> >>> >>>> One way this could be achieved is with a tool that postprocesses
> the
> >>> >>>> artifacts from the build process. For each artifact, the tool
> would
> >>> >>>> disassemble the artifact, instrument the code using a set of
> AspectJ
> >>> >>>> aspects and reassemble the artifact for deployment. The
> >>> >>>> responsibility
> >>> >>>> of the aspects would be to intercept messages, log them and modify
> >>> >>>> them to ensure proper correlation. Of course this only works if
> the
> >>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we
> already
> >>> >>>> use AspectJ for a similar use case (although at a much smaller
> scale)
> >>> >>>> in the transport test suites [1]. Interestingly, this approach
> would
> >>> >>>> allow to cover interactions that are not SOAP based, e.g. one
> could
> >>> >>>> even intercept the database queries executed during the flow.
> >>> >>>>
> >>> >>>> Andreas
> >>> >>>>
> >>> >>>> [1]
> >>> >>>>
> >>> >>>>
> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> >>> >>>>
> >>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <hemapani@gmail.com
> >
> >>> >>>> wrote:
> >>> >>>> > Hi Nilupa;
> >>> >>>> >
> >>> >>>> > When we collect messages from a one location by installing
> proper
> >>> >>>> > handler that will intercept and send messages to to that one
> >>> >>>> > location
> >>> >>>> > (this one location can be a single server, pub/sub channel etc).
> >>> >>>> > There
> >>> >>>> > is many ways to make sense of those collected messages. What
> >>> >>>> > Andreas
> >>> >>>> > mentioned (following complete transaction) is a one possibility.
> >>> >>>> >
> >>> >>>> > I think you should come up with few scenarios on how you would
> make
> >>> >>>> > sense of the message.
> >>> >>>> >
> >>> >>>> > Thanks
> >>> >>>> > Srinath
> >>> >>>> >
> >>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <
> nilupas@gmail.com>
> >>> >>>> > wrote:
> >>> >>>> >> Hi,
> >>> >>>> >>
> >>> >>>> >> First let me thank you for commenting.
> >>> >>>> >>
> >>> >>>> >> As far as I understood, what you would like to see from the
> >>> >>>> >> proposed
> >>> >>>> >> tool is to view set of messages that are exchanged in reponse
> to a
> >>> >>>> >> particular input message. With the understanding that I am
> having
> >>> >>>> >> at
> >>> >>>> >> the momnet, one way to do it is to filter out the central
> >>> >>>> >> repository
> >>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct
> the
> >>> >>>> >> message chain from it. We can allow the client GUI wich
> connects
> >>> >>>> >> to
> >>> >>>> >> the central repository to provide the paramenters (For instance
> >>> >>>> >> the
> >>> >>>> >> value of 'To' header) from which an intelligent filtering can
> be
> >>> >>>> >> done
> >>> >>>> >> for the set of messages avialable at the central repository.
> >>> >>>> >>
> >>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
> >>> >>>> >> willing
> >>> >>>> >> to share it with us. It would be really nice to hear from them.
> >>> >>>> >>
> >>> >>>> >> Thanks in advance ..!!
> >>> >>>> >>
> >>> >>>> >> Best Regards,
> >>> >>>> >> Nilupa
> >>> >>>> >>
> >>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> >>> >>>> >> <an...@gmail.com> wrote:
> >>> >>>> >>> Personally, I think that the added value of extending the SOAP
> >>> >>>> >>> monitor
> >>> >>>> >>> module to collect messages in a central place is too limited
> to
> >>> >>>> >>> attract enough interest from the user community, so that it
> will
> >>> >>>> >>> be
> >>> >>>> >>> difficult to ensure the evolution of such a project in the
> >>> >>>> >>> future.
> >>> >>>> >>> However, what many people are looking for is a tool that
> allows
> >>> >>>> >>> to
> >>> >>>> >>> track the flow of a message through a distributed system, or
> more
> >>> >>>> >>> generally to track the sequence of events triggered by a given
> >>> >>>> >>> input
> >>> >>>> >>> message (sort of end-to-end transaction monitoring).
> >>> >>>> >>>
> >>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <
> nilupas@gmail.com>
> >>> >>>> >>> wrote:
> >>> >>>> >>>> Hello,
> >>> >>>> >>>>
> >>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am
> thinking
> >>> >>>> >>>> of
> >>> >>>> >>>> proposing a GSoC project this summer. I would like to take
> >>> >>>> >>>> project
> >>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very
> interesting
> >>> >>>> >>>> and
> >>> >>>> >>>> also should be helpful to any Java developer using Apache
> Axis2
> >>> >>>> >>>> Web
> >>> >>>> >>>> services middleware. I will submit more detailed proposal
> later.
> >>> >>>> >>>> I
> >>> >>>> >>>> would really like to hear any feedback from you. Any
> suggestions
> >>> >>>> >>>> or
> >>> >>>> >>>> features that you would like to see in a "Distributed TCP
> >>> >>>> >>>> Monitor"
> >>> >>>> >>>> are
> >>> >>>> >>>> most welcome.
> >>> >>>> >>>>
> >>> >>>> >>>> Best Regards,
> >>> >>>> >>>> Nilupa Bandara
> >>> >>>> >>>>
> >>> >>>> >>>> [1]
> >>> >>>> >>>>
> >>> >>>> >>>> Distributed TCP monitor
> >>> >>>> >>>>
> >>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is
> bit
> >>> >>>> >>>> hard
> >>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can
> solve
> >>> >>>> >>>> the
> >>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
> >>> >>>> >>>> messages
> >>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via
> a
> >>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so
> user
> >>> >>>> >>>> can
> >>> >>>> >>>> pull those messages. Also, we can do something to turn the
> >>> >>>> >>>> module on
> >>> >>>> >>>> and off remotely, so users can have his module deployed, but
> >>> >>>> >>>> turn it
> >>> >>>> >>>> on only when they want to debug the system.
> >>> >>>> >>>>
> >>> >>>> >>>> Then we can take this to next level by adding this module to
> all
> >>> >>>> >>>> services in a system , configuring modules to send all
> collected
> >>> >>>> >>>> messages to a pub/sub channel, and subscribing to those
> messages
> >>> >>>> >>>> via
> >>> >>>> >>>> a
> >>> >>>> >>>> UI, and depicting those messages through a UI. Then users
> have a
> >>> >>>> >>>> TCPMon for a whole system.
> >>> >>>> >>>>
> >>> >>>> >>>>
> >>> >>>> >>>>
> ---------------------------------------------------------------------
> >>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >>>> For additional commands, e-mail:
> java-dev-help@axis.apache.org
> >>> >>>> >>>>
> >>> >>>> >>>>
> >>> >>>> >>>
> >>> >>>> >>>
> >>> >>>> >>>
> ---------------------------------------------------------------------
> >>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >>> For additional commands, e-mail:
> java-dev-help@axis.apache.org
> >>> >>>> >>>
> >>> >>>> >>>
> >>> >>>> >>
> >>> >>>> >>
> >>> >>>> >>
> ---------------------------------------------------------------------
> >>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>> >>
> >>> >>>> >>
> >>> >>>> >
> >>> >>>> >
> >>> >>>> >
> >>> >>>> > --
> >>> >>>> > ============================
> >>> >>>> > Srinath Perera, Ph.D.
> >>> >>>> >   WSO2 Inc. http://wso2.com
> >>> >>>> >   Blog: http://srinathsview.blogspot.com/
> >>> >>>> >
> >>> >>>> >
> >>> >>>> >
> ---------------------------------------------------------------------
> >>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>> >
> >>> >>>> >
> >>> >>>>
> >>> >>>>
> ---------------------------------------------------------------------
> >>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>>
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> --
> >>> >>> Sanjiva Weerawarana, Ph.D.
> >>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
> >>> >>> http://www.opensource.lk/
> >>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> >>> >>> Member; Apache Software Foundation; http://www.apache.org/
> >>> >>> Director; Sahana Software Foundation;
> http://www.sahanafoundation.org/
> >>> >>> Visiting Lecturer; University of Moratuwa;
> http://www.cse.mrt.ac.lk/
> >>> >>>
> >>> >>> Blog: http://sanjiva.weerawarana.org/
> >>> >>>
> >>> >>
> >>> >>
> >>> >>
> ---------------------------------------------------------------------
> >>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>
> >>> >
> >>> > ---------------------------------------------------------------------
> >>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >
> >>> >
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>
> >> Samisa...
> >> --
> >> blog: http://samisa-abeysinghe.blogspot.com/
> >>
> >
> >
> >
> > --
> > Sanka Samaranayake
> >
> > http://sankas.blogspot.com/
> >
>
>
>
> --
> Sanka Samaranayake
>
> http://sankas.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Thu, Apr 1, 2010 at 7:04 AM, Sanka Samaranayake <ss...@gmail.com> wrote:

> Samisa,
>
> Quick question, does WSO2 BAM project provide such a debugging
> facility. I mean can one monitor what is going on for a specific input
> in SOA senario ?
>

Yes it can.

However, as I mentioned in an earlier explanation, that is not the main
purpose of BAM.

Samisa...



> Sanka
>
>
>
> On Thu, Apr 1, 2010 at 3:28 AM, Sanka Samaranayake <ss...@gmail.com>
> wrote:
> > Hi Samisa,
> >
> > I think the this scope of this project would be to provide a
> > light-weight tool which allows to monitor the message flows, execution
> > in each node in response to those message flows to a given input
> > message primarily for debugging purposes . (Just like the way tcpmon
> > is used in developing Web services)
> >
> > And I guess now the question is whether such a tool is useful ..
> >
> > Anyone cares to share their thoughts ?
> >
> > Nilupa
> >
> > On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
> > <sa...@gmail.com> wrote:
> >>
> >>
> >> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com>
> wrote:
> >>>
> >>> I was thinking about Java Swing based UI
> >>
> >> Swing is a good start. But if you look at some of the requirements like
> >> monitoring multiple instances and correlate them, discussed in this
> thread,
> >> it sounds as if we would be better off having a Web interface, rather
> than a
> >> desktop client.
> >>
> >>>
> >>> because the client may have
> >>> to cache whatever messages it already has and will receive,
> >>
> >> If you are looking to support message-correlation, as discussed in some
> >> replies in this thread above, you need to have a persistence storage (a
> DB
> >> or something like that) rather than a memory cache.
> >> Because, some message sequences, like those involved in an async
> >> invocations, need to have a way to keep the messages, in there, longer,
> and
> >> may be there is a server re-start, in the middle.
> >> Of course, this requirements spend on the scope of this effort.
> >> Plus, if you want to deal with historic data, and look at what was
> happening
> >> to message sequences over time, you got to use a persistence layer.
> Again
> >> this might be out of scope for this project, but if this is to be
> extended
> >> to support these, you better think of this right now.
> >>
> >>>
> >>> build the
> >>> message flow and update the UI accordingly. Perhaps there is a better
> >>> way ?
> >>
> >> We already do what you are trying to do, and much more with WSO2 BAM. I
> am
> >> tempted to say that Shinding based gadget dashboard is really a
> compelling
> >> UI for this.
> >> I am not trying to take away you project or trying to hinder this effort
> at
> >> all.
> >> I think it will be a good idea for you to have a look at BAM and see
> what
> >> the missing pieces are and figure out the right model for this tool.
> >> Also, having had worked on WSO2 BAM project, I also have some
> understanding
> >> on sort of problems that you might run into, when doing this. For e.g.,
> if
> >> you want to correlate message sequences in a high volume setup, you will
> be
> >> swamped by the volume of messages, sooner than later. So correlation for
> the
> >> purpose of monitoring is not an easy thing to do. Correlation, for the
> >> purpose of debugging will not run into this problem, if you run this on
> a
> >> controlled environment. However, what you really want to debug is the
> deal
> >> clustered setup, and not a staged dev setup, the problem again surfaces.
> >> You might want to list the objectives and then break them down
> >> into monitoring and debugging spaces.
> >> Samisa...
> >>
> >>>
> >>> Thanks
> >>> Nilupa.
> >>>
> >>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
> >>> <sa...@gmail.com> wrote:
> >>> > What UI technology would be used for this?
> >>> >
> >>> > Samisa...
> >>> >
> >>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
> >>> > <an...@gmail.com> wrote:
> >>> >> I was thinking more about something like ITCAM for SOA.
> >>> >>
> >>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
> >>> >> <sa...@opensource.lk> wrote:
> >>> >>> That runtime monitoring / correlation is what the WSO2 Business
> >>> >>> Activity
> >>> >>> Monitor does / is for ...
> >>> >>> See: http://wso2.com/products/business-activity-monitor/
> >>> >>> Sanjiva.
> >>> >>>
> >>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
> >>> >>> <an...@gmail.com>
> >>> >>> wrote:
> >>> >>>>
> >>> >>>> I was actually referring to scenarios where a service may call
> other
> >>> >>>> services, which in turn may call again other services. In these
> >>> >>>> scenarios, it is not sufficient to simply collect received/sent
> >>> >>>> messages on different hosts, unless the system is isolated
> >>> >>>> (development environment) or the request rate is sufficiently low
> so
> >>> >>>> that messages can be correlated based on time. Here is a concrete
> >>> >>>> scenario from a project in my company:
> >>> >>>>
> >>> >>>> - Request comes in on an ESB that does security and validation.
> >>> >>>> - Request is processed by an application server which persists the
> >>> >>>> received information and publishes a JMS message (pub/sub events).
> >>> >>>> - The event is consumed by one or more components that may in turn
> >>> >>>> interact with other services.
> >>> >>>>
> >>> >>>> If you want track this flow, it is not sufficient to intercept the
> >>> >>>> messages on the different hosts: in addition, you need to
> instrument
> >>> >>>> the services so that the outgoing requests are correlated with the
> >>> >>>> incoming requests (by adding SOAP and/or transport headers). What
> I
> >>> >>>> would like to be able to do in the above scenario is to enable
> >>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP
> or
> >>> >>>> HTTP header to the initial request to enable logging and this
> >>> >>>> information is propagated along the chain. Every service/component
> >>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
> >>> >>>> central place where the sequence of events is reconstructed.
> >>> >>>>
> >>> >>>> One way this could be achieved is with a tool that postprocesses
> the
> >>> >>>> artifacts from the build process. For each artifact, the tool
> would
> >>> >>>> disassemble the artifact, instrument the code using a set of
> AspectJ
> >>> >>>> aspects and reassemble the artifact for deployment. The
> >>> >>>> responsibility
> >>> >>>> of the aspects would be to intercept messages, log them and modify
> >>> >>>> them to ensure proper correlation. Of course this only works if
> the
> >>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we
> already
> >>> >>>> use AspectJ for a similar use case (although at a much smaller
> scale)
> >>> >>>> in the transport test suites [1]. Interestingly, this approach
> would
> >>> >>>> allow to cover interactions that are not SOAP based, e.g. one
> could
> >>> >>>> even intercept the database queries executed during the flow.
> >>> >>>>
> >>> >>>> Andreas
> >>> >>>>
> >>> >>>> [1]
> >>> >>>>
> >>> >>>>
> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> >>> >>>>
> >>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <hemapani@gmail.com
> >
> >>> >>>> wrote:
> >>> >>>> > Hi Nilupa;
> >>> >>>> >
> >>> >>>> > When we collect messages from a one location by installing
> proper
> >>> >>>> > handler that will intercept and send messages to to that one
> >>> >>>> > location
> >>> >>>> > (this one location can be a single server, pub/sub channel etc).
> >>> >>>> > There
> >>> >>>> > is many ways to make sense of those collected messages. What
> >>> >>>> > Andreas
> >>> >>>> > mentioned (following complete transaction) is a one possibility.
> >>> >>>> >
> >>> >>>> > I think you should come up with few scenarios on how you would
> make
> >>> >>>> > sense of the message.
> >>> >>>> >
> >>> >>>> > Thanks
> >>> >>>> > Srinath
> >>> >>>> >
> >>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <
> nilupas@gmail.com>
> >>> >>>> > wrote:
> >>> >>>> >> Hi,
> >>> >>>> >>
> >>> >>>> >> First let me thank you for commenting.
> >>> >>>> >>
> >>> >>>> >> As far as I understood, what you would like to see from the
> >>> >>>> >> proposed
> >>> >>>> >> tool is to view set of messages that are exchanged in reponse
> to a
> >>> >>>> >> particular input message. With the understanding that I am
> having
> >>> >>>> >> at
> >>> >>>> >> the momnet, one way to do it is to filter out the central
> >>> >>>> >> repository
> >>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct
> the
> >>> >>>> >> message chain from it. We can allow the client GUI wich
> connects
> >>> >>>> >> to
> >>> >>>> >> the central repository to provide the paramenters (For instance
> >>> >>>> >> the
> >>> >>>> >> value of 'To' header) from which an intelligent filtering can
> be
> >>> >>>> >> done
> >>> >>>> >> for the set of messages avialable at the central repository.
> >>> >>>> >>
> >>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
> >>> >>>> >> willing
> >>> >>>> >> to share it with us. It would be really nice to hear from them.
> >>> >>>> >>
> >>> >>>> >> Thanks in advance ..!!
> >>> >>>> >>
> >>> >>>> >> Best Regards,
> >>> >>>> >> Nilupa
> >>> >>>> >>
> >>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> >>> >>>> >> <an...@gmail.com> wrote:
> >>> >>>> >>> Personally, I think that the added value of extending the SOAP
> >>> >>>> >>> monitor
> >>> >>>> >>> module to collect messages in a central place is too limited
> to
> >>> >>>> >>> attract enough interest from the user community, so that it
> will
> >>> >>>> >>> be
> >>> >>>> >>> difficult to ensure the evolution of such a project in the
> >>> >>>> >>> future.
> >>> >>>> >>> However, what many people are looking for is a tool that
> allows
> >>> >>>> >>> to
> >>> >>>> >>> track the flow of a message through a distributed system, or
> more
> >>> >>>> >>> generally to track the sequence of events triggered by a given
> >>> >>>> >>> input
> >>> >>>> >>> message (sort of end-to-end transaction monitoring).
> >>> >>>> >>>
> >>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <
> nilupas@gmail.com>
> >>> >>>> >>> wrote:
> >>> >>>> >>>> Hello,
> >>> >>>> >>>>
> >>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am
> thinking
> >>> >>>> >>>> of
> >>> >>>> >>>> proposing a GSoC project this summer. I would like to take
> >>> >>>> >>>> project
> >>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very
> interesting
> >>> >>>> >>>> and
> >>> >>>> >>>> also should be helpful to any Java developer using Apache
> Axis2
> >>> >>>> >>>> Web
> >>> >>>> >>>> services middleware. I will submit more detailed proposal
> later.
> >>> >>>> >>>> I
> >>> >>>> >>>> would really like to hear any feedback from you. Any
> suggestions
> >>> >>>> >>>> or
> >>> >>>> >>>> features that you would like to see in a "Distributed TCP
> >>> >>>> >>>> Monitor"
> >>> >>>> >>>> are
> >>> >>>> >>>> most welcome.
> >>> >>>> >>>>
> >>> >>>> >>>> Best Regards,
> >>> >>>> >>>> Nilupa Bandara
> >>> >>>> >>>>
> >>> >>>> >>>> [1]
> >>> >>>> >>>>
> >>> >>>> >>>> Distributed TCP monitor
> >>> >>>> >>>>
> >>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is
> bit
> >>> >>>> >>>> hard
> >>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can
> solve
> >>> >>>> >>>> the
> >>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
> >>> >>>> >>>> messages
> >>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via
> a
> >>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so
> user
> >>> >>>> >>>> can
> >>> >>>> >>>> pull those messages. Also, we can do something to turn the
> >>> >>>> >>>> module on
> >>> >>>> >>>> and off remotely, so users can have his module deployed, but
> >>> >>>> >>>> turn it
> >>> >>>> >>>> on only when they want to debug the system.
> >>> >>>> >>>>
> >>> >>>> >>>> Then we can take this to next level by adding this module to
> all
> >>> >>>> >>>> services in a system , configuring modules to send all
> collected
> >>> >>>> >>>> messages to a pub/sub channel, and subscribing to those
> messages
> >>> >>>> >>>> via
> >>> >>>> >>>> a
> >>> >>>> >>>> UI, and depicting those messages through a UI. Then users
> have a
> >>> >>>> >>>> TCPMon for a whole system.
> >>> >>>> >>>>
> >>> >>>> >>>>
> >>> >>>> >>>>
> ---------------------------------------------------------------------
> >>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >>>> For additional commands, e-mail:
> java-dev-help@axis.apache.org
> >>> >>>> >>>>
> >>> >>>> >>>>
> >>> >>>> >>>
> >>> >>>> >>>
> >>> >>>> >>>
> ---------------------------------------------------------------------
> >>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >>> For additional commands, e-mail:
> java-dev-help@axis.apache.org
> >>> >>>> >>>
> >>> >>>> >>>
> >>> >>>> >>
> >>> >>>> >>
> >>> >>>> >>
> ---------------------------------------------------------------------
> >>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>> >>
> >>> >>>> >>
> >>> >>>> >
> >>> >>>> >
> >>> >>>> >
> >>> >>>> > --
> >>> >>>> > ============================
> >>> >>>> > Srinath Perera, Ph.D.
> >>> >>>> >   WSO2 Inc. http://wso2.com
> >>> >>>> >   Blog: http://srinathsview.blogspot.com/
> >>> >>>> >
> >>> >>>> >
> >>> >>>> >
> ---------------------------------------------------------------------
> >>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>> >
> >>> >>>> >
> >>> >>>>
> >>> >>>>
> ---------------------------------------------------------------------
> >>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>>
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> --
> >>> >>> Sanjiva Weerawarana, Ph.D.
> >>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
> >>> >>> http://www.opensource.lk/
> >>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> >>> >>> Member; Apache Software Foundation; http://www.apache.org/
> >>> >>> Director; Sahana Software Foundation;
> http://www.sahanafoundation.org/
> >>> >>> Visiting Lecturer; University of Moratuwa;
> http://www.cse.mrt.ac.lk/
> >>> >>>
> >>> >>> Blog: http://sanjiva.weerawarana.org/
> >>> >>>
> >>> >>
> >>> >>
> >>> >>
> ---------------------------------------------------------------------
> >>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>
> >>> >
> >>> > ---------------------------------------------------------------------
> >>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >
> >>> >
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>
> >> Samisa...
> >> --
> >> blog: http://samisa-abeysinghe.blogspot.com/
> >>
> >
> >
> >
> > --
> > Sanka Samaranayake
> >
> > http://sankas.blogspot.com/
> >
>
>
>
> --
> Sanka Samaranayake
>
> http://sankas.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanka Samaranayake <ss...@gmail.com>.

Samisa,

Quick question, does WSO2 BAM project provide such a debugging
facility. I mean can one monitor what is going on for a specific input
in SOA senario ?

Sanka



On Thu, Apr 1, 2010 at 3:28 AM, Sanka Samaranayake <ss...@gmail.com> wrote:
> Hi Samisa,
>
> I think the this scope of this project would be to provide a
> light-weight tool which allows to monitor the message flows, execution
> in each node in response to those message flows to a given input
> message primarily for debugging purposes . (Just like the way tcpmon
> is used in developing Web services)
>
> And I guess now the question is whether such a tool is useful ..
>
> Anyone cares to share their thoughts ?
>
> Nilupa
>
> On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>
>>
>> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:
>>>
>>> I was thinking about Java Swing based UI
>>
>> Swing is a good start. But if you look at some of the requirements like
>> monitoring multiple instances and correlate them, discussed in this thread,
>> it sounds as if we would be better off having a Web interface, rather than a
>> desktop client.
>>
>>>
>>> because the client may have
>>> to cache whatever messages it already has and will receive,
>>
>> If you are looking to support message-correlation, as discussed in some
>> replies in this thread above, you need to have a persistence storage (a DB
>> or something like that) rather than a memory cache.
>> Because, some message sequences, like those involved in an async
>> invocations, need to have a way to keep the messages, in there, longer, and
>> may be there is a server re-start, in the middle.
>> Of course, this requirements spend on the scope of this effort.
>> Plus, if you want to deal with historic data, and look at what was happening
>> to message sequences over time, you got to use a persistence layer. Again
>> this might be out of scope for this project, but if this is to be extended
>> to support these, you better think of this right now.
>>
>>>
>>> build the
>>> message flow and update the UI accordingly. Perhaps there is a better
>>> way ?
>>
>> We already do what you are trying to do, and much more with WSO2 BAM. I am
>> tempted to say that Shinding based gadget dashboard is really a compelling
>> UI for this.
>> I am not trying to take away you project or trying to hinder this effort at
>> all.
>> I think it will be a good idea for you to have a look at BAM and see what
>> the missing pieces are and figure out the right model for this tool.
>> Also, having had worked on WSO2 BAM project, I also have some understanding
>> on sort of problems that you might run into, when doing this. For e.g., if
>> you want to correlate message sequences in a high volume setup, you will be
>> swamped by the volume of messages, sooner than later. So correlation for the
>> purpose of monitoring is not an easy thing to do. Correlation, for the
>> purpose of debugging will not run into this problem, if you run this on a
>> controlled environment. However, what you really want to debug is the deal
>> clustered setup, and not a staged dev setup, the problem again surfaces.
>> You might want to list the objectives and then break them down
>> into monitoring and debugging spaces.
>> Samisa...
>>
>>>
>>> Thanks
>>> Nilupa.
>>>
>>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
>>> <sa...@gmail.com> wrote:
>>> > What UI technology would be used for this?
>>> >
>>> > Samisa...
>>> >
>>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
>>> > <an...@gmail.com> wrote:
>>> >> I was thinking more about something like ITCAM for SOA.
>>> >>
>>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>>> >> <sa...@opensource.lk> wrote:
>>> >>> That runtime monitoring / correlation is what the WSO2 Business
>>> >>> Activity
>>> >>> Monitor does / is for ...
>>> >>> See: http://wso2.com/products/business-activity-monitor/
>>> >>> Sanjiva.
>>> >>>
>>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
>>> >>> <an...@gmail.com>
>>> >>> wrote:
>>> >>>>
>>> >>>> I was actually referring to scenarios where a service may call other
>>> >>>> services, which in turn may call again other services. In these
>>> >>>> scenarios, it is not sufficient to simply collect received/sent
>>> >>>> messages on different hosts, unless the system is isolated
>>> >>>> (development environment) or the request rate is sufficiently low so
>>> >>>> that messages can be correlated based on time. Here is a concrete
>>> >>>> scenario from a project in my company:
>>> >>>>
>>> >>>> - Request comes in on an ESB that does security and validation.
>>> >>>> - Request is processed by an application server which persists the
>>> >>>> received information and publishes a JMS message (pub/sub events).
>>> >>>> - The event is consumed by one or more components that may in turn
>>> >>>> interact with other services.
>>> >>>>
>>> >>>> If you want track this flow, it is not sufficient to intercept the
>>> >>>> messages on the different hosts: in addition, you need to instrument
>>> >>>> the services so that the outgoing requests are correlated with the
>>> >>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> >>>> would like to be able to do in the above scenario is to enable
>>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> >>>> HTTP header to the initial request to enable logging and this
>>> >>>> information is propagated along the chain. Every service/component
>>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> >>>> central place where the sequence of events is reconstructed.
>>> >>>>
>>> >>>> One way this could be achieved is with a tool that postprocesses the
>>> >>>> artifacts from the build process. For each artifact, the tool would
>>> >>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> >>>> aspects and reassemble the artifact for deployment. The
>>> >>>> responsibility
>>> >>>> of the aspects would be to intercept messages, log them and modify
>>> >>>> them to ensure proper correlation. Of course this only works if the
>>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> >>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> >>>> in the transport test suites [1]. Interestingly, this approach would
>>> >>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> >>>> even intercept the database queries executed during the flow.
>>> >>>>
>>> >>>> Andreas
>>> >>>>
>>> >>>> [1]
>>> >>>>
>>> >>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>> >>>>
>>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
>>> >>>> wrote:
>>> >>>> > Hi Nilupa;
>>> >>>> >
>>> >>>> > When we collect messages from a one location by installing proper
>>> >>>> > handler that will intercept and send messages to to that one
>>> >>>> > location
>>> >>>> > (this one location can be a single server, pub/sub channel etc).
>>> >>>> > There
>>> >>>> > is many ways to make sense of those collected messages. What
>>> >>>> > Andreas
>>> >>>> > mentioned (following complete transaction) is a one possibility.
>>> >>>> >
>>> >>>> > I think you should come up with few scenarios on how you would make
>>> >>>> > sense of the message.
>>> >>>> >
>>> >>>> > Thanks
>>> >>>> > Srinath
>>> >>>> >
>>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>> >>>> > wrote:
>>> >>>> >> Hi,
>>> >>>> >>
>>> >>>> >> First let me thank you for commenting.
>>> >>>> >>
>>> >>>> >> As far as I understood, what you would like to see from the
>>> >>>> >> proposed
>>> >>>> >> tool is to view set of messages that are exchanged in reponse to a
>>> >>>> >> particular input message. With the understanding that I am having
>>> >>>> >> at
>>> >>>> >> the momnet, one way to do it is to filter out the central
>>> >>>> >> repository
>>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>> >>>> >> message chain from it. We can allow the client GUI wich connects
>>> >>>> >> to
>>> >>>> >> the central repository to provide the paramenters (For instance
>>> >>>> >> the
>>> >>>> >> value of 'To' header) from which an intelligent filtering can be
>>> >>>> >> done
>>> >>>> >> for the set of messages avialable at the central repository.
>>> >>>> >>
>>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
>>> >>>> >> willing
>>> >>>> >> to share it with us. It would be really nice to hear from them.
>>> >>>> >>
>>> >>>> >> Thanks in advance ..!!
>>> >>>> >>
>>> >>>> >> Best Regards,
>>> >>>> >> Nilupa
>>> >>>> >>
>>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> >>>> >> <an...@gmail.com> wrote:
>>> >>>> >>> Personally, I think that the added value of extending the SOAP
>>> >>>> >>> monitor
>>> >>>> >>> module to collect messages in a central place is too limited to
>>> >>>> >>> attract enough interest from the user community, so that it will
>>> >>>> >>> be
>>> >>>> >>> difficult to ensure the evolution of such a project in the
>>> >>>> >>> future.
>>> >>>> >>> However, what many people are looking for is a tool that allows
>>> >>>> >>> to
>>> >>>> >>> track the flow of a message through a distributed system, or more
>>> >>>> >>> generally to track the sequence of events triggered by a given
>>> >>>> >>> input
>>> >>>> >>> message (sort of end-to-end transaction monitoring).
>>> >>>> >>>
>>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>> >>>> >>> wrote:
>>> >>>> >>>> Hello,
>>> >>>> >>>>
>>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
>>> >>>> >>>> of
>>> >>>> >>>> proposing a GSoC project this summer. I would like to take
>>> >>>> >>>> project
>>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
>>> >>>> >>>> and
>>> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
>>> >>>> >>>> Web
>>> >>>> >>>> services middleware. I will submit more detailed proposal later.
>>> >>>> >>>> I
>>> >>>> >>>> would really like to hear any feedback from you. Any suggestions
>>> >>>> >>>> or
>>> >>>> >>>> features that you would like to see in a "Distributed TCP
>>> >>>> >>>> Monitor"
>>> >>>> >>>> are
>>> >>>> >>>> most welcome.
>>> >>>> >>>>
>>> >>>> >>>> Best Regards,
>>> >>>> >>>> Nilupa Bandara
>>> >>>> >>>>
>>> >>>> >>>> [1]
>>> >>>> >>>>
>>> >>>> >>>> Distributed TCP monitor
>>> >>>> >>>>
>>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
>>> >>>> >>>> hard
>>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
>>> >>>> >>>> the
>>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
>>> >>>> >>>> messages
>>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
>>> >>>> >>>> can
>>> >>>> >>>> pull those messages. Also, we can do something to turn the
>>> >>>> >>>> module on
>>> >>>> >>>> and off remotely, so users can have his module deployed, but
>>> >>>> >>>> turn it
>>> >>>> >>>> on only when they want to debug the system.
>>> >>>> >>>>
>>> >>>> >>>> Then we can take this to next level by adding this module to all
>>> >>>> >>>> services in a system , configuring modules to send all collected
>>> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
>>> >>>> >>>> via
>>> >>>> >>>> a
>>> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>> >>>> >>>> TCPMon for a whole system.
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>> ---------------------------------------------------------------------
>>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>> ---------------------------------------------------------------------
>>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>
>>> >>>> >>
>>> >>>> >> ---------------------------------------------------------------------
>>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>
>>> >>>> >>
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> > --
>>> >>>> > ============================
>>> >>>> > Srinath Perera, Ph.D.
>>> >>>> >   WSO2 Inc. http://wso2.com
>>> >>>> >   Blog: http://srinathsview.blogspot.com/
>>> >>>> >
>>> >>>> >
>>> >>>> > ---------------------------------------------------------------------
>>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >
>>> >>>> >
>>> >>>>
>>> >>>> ---------------------------------------------------------------------
>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Sanjiva Weerawarana, Ph.D.
>>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>>> >>> http://www.opensource.lk/
>>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>>> >>> Member; Apache Software Foundation; http://www.apache.org/
>>> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>>> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>> >>>
>>> >>> Blog: http://sanjiva.weerawarana.org/
>>> >>>
>>> >>
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>>
>
>
>
> --
> Sanka Samaranayake
>
> http://sankas.blogspot.com/
>



-- 
Sanka Samaranayake

http://sankas.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

On Thu, Apr 1, 2010 at 03:28, Sanka Samaranayake <ss...@gmail.com> wrote:
> Hi Samisa,
>
> I think the this scope of this project would be to provide a
> light-weight tool which allows to monitor the message flows, execution
> in each node in response to those message flows to a given input
> message primarily for debugging purposes . (Just like the way tcpmon
> is used in developing Web services)
>
> And I guess now the question is whether such a tool is useful ..

Yes, if at least two additional requirements are satisfied:

- It should offer the possibility to measure response times. This
should be an easy one (except for the usual issues with time
measurements in distributed systems): just log the relevant
timestamps.
- It should not be limited to SOAP and it should not be specifically
targeted at Axis2: it should e.g. allow to add JDBC to the picture and
it should be easy to add support for other Web service stacks.

BTW, it may be a good idea to present the idea to other Apache
projects (CXF, ServiceMix, etc.) to see if they are interested in
supporting this project as well.
>
> Anyone cares to share their thoughts ?
>
> Nilupa
>
> On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>
>>
>> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:
>>>
>>> I was thinking about Java Swing based UI
>>
>> Swing is a good start. But if you look at some of the requirements like
>> monitoring multiple instances and correlate them, discussed in this thread,
>> it sounds as if we would be better off having a Web interface, rather than a
>> desktop client.
>>
>>>
>>> because the client may have
>>> to cache whatever messages it already has and will receive,
>>
>> If you are looking to support message-correlation, as discussed in some
>> replies in this thread above, you need to have a persistence storage (a DB
>> or something like that) rather than a memory cache.
>> Because, some message sequences, like those involved in an async
>> invocations, need to have a way to keep the messages, in there, longer, and
>> may be there is a server re-start, in the middle.
>> Of course, this requirements spend on the scope of this effort.
>> Plus, if you want to deal with historic data, and look at what was happening
>> to message sequences over time, you got to use a persistence layer. Again
>> this might be out of scope for this project, but if this is to be extended
>> to support these, you better think of this right now.
>>
>>>
>>> build the
>>> message flow and update the UI accordingly. Perhaps there is a better
>>> way ?
>>
>> We already do what you are trying to do, and much more with WSO2 BAM. I am
>> tempted to say that Shinding based gadget dashboard is really a compelling
>> UI for this.
>> I am not trying to take away you project or trying to hinder this effort at
>> all.
>> I think it will be a good idea for you to have a look at BAM and see what
>> the missing pieces are and figure out the right model for this tool.
>> Also, having had worked on WSO2 BAM project, I also have some understanding
>> on sort of problems that you might run into, when doing this. For e.g., if
>> you want to correlate message sequences in a high volume setup, you will be
>> swamped by the volume of messages, sooner than later. So correlation for the
>> purpose of monitoring is not an easy thing to do. Correlation, for the
>> purpose of debugging will not run into this problem, if you run this on a
>> controlled environment. However, what you really want to debug is the deal
>> clustered setup, and not a staged dev setup, the problem again surfaces.
>> You might want to list the objectives and then break them down
>> into monitoring and debugging spaces.
>> Samisa...
>>
>>>
>>> Thanks
>>> Nilupa.
>>>
>>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
>>> <sa...@gmail.com> wrote:
>>> > What UI technology would be used for this?
>>> >
>>> > Samisa...
>>> >
>>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
>>> > <an...@gmail.com> wrote:
>>> >> I was thinking more about something like ITCAM for SOA.
>>> >>
>>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>>> >> <sa...@opensource.lk> wrote:
>>> >>> That runtime monitoring / correlation is what the WSO2 Business
>>> >>> Activity
>>> >>> Monitor does / is for ...
>>> >>> See: http://wso2.com/products/business-activity-monitor/
>>> >>> Sanjiva.
>>> >>>
>>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
>>> >>> <an...@gmail.com>
>>> >>> wrote:
>>> >>>>
>>> >>>> I was actually referring to scenarios where a service may call other
>>> >>>> services, which in turn may call again other services. In these
>>> >>>> scenarios, it is not sufficient to simply collect received/sent
>>> >>>> messages on different hosts, unless the system is isolated
>>> >>>> (development environment) or the request rate is sufficiently low so
>>> >>>> that messages can be correlated based on time. Here is a concrete
>>> >>>> scenario from a project in my company:
>>> >>>>
>>> >>>> - Request comes in on an ESB that does security and validation.
>>> >>>> - Request is processed by an application server which persists the
>>> >>>> received information and publishes a JMS message (pub/sub events).
>>> >>>> - The event is consumed by one or more components that may in turn
>>> >>>> interact with other services.
>>> >>>>
>>> >>>> If you want track this flow, it is not sufficient to intercept the
>>> >>>> messages on the different hosts: in addition, you need to instrument
>>> >>>> the services so that the outgoing requests are correlated with the
>>> >>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> >>>> would like to be able to do in the above scenario is to enable
>>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> >>>> HTTP header to the initial request to enable logging and this
>>> >>>> information is propagated along the chain. Every service/component
>>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> >>>> central place where the sequence of events is reconstructed.
>>> >>>>
>>> >>>> One way this could be achieved is with a tool that postprocesses the
>>> >>>> artifacts from the build process. For each artifact, the tool would
>>> >>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> >>>> aspects and reassemble the artifact for deployment. The
>>> >>>> responsibility
>>> >>>> of the aspects would be to intercept messages, log them and modify
>>> >>>> them to ensure proper correlation. Of course this only works if the
>>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> >>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> >>>> in the transport test suites [1]. Interestingly, this approach would
>>> >>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> >>>> even intercept the database queries executed during the flow.
>>> >>>>
>>> >>>> Andreas
>>> >>>>
>>> >>>> [1]
>>> >>>>
>>> >>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>> >>>>
>>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
>>> >>>> wrote:
>>> >>>> > Hi Nilupa;
>>> >>>> >
>>> >>>> > When we collect messages from a one location by installing proper
>>> >>>> > handler that will intercept and send messages to to that one
>>> >>>> > location
>>> >>>> > (this one location can be a single server, pub/sub channel etc).
>>> >>>> > There
>>> >>>> > is many ways to make sense of those collected messages. What
>>> >>>> > Andreas
>>> >>>> > mentioned (following complete transaction) is a one possibility.
>>> >>>> >
>>> >>>> > I think you should come up with few scenarios on how you would make
>>> >>>> > sense of the message.
>>> >>>> >
>>> >>>> > Thanks
>>> >>>> > Srinath
>>> >>>> >
>>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>> >>>> > wrote:
>>> >>>> >> Hi,
>>> >>>> >>
>>> >>>> >> First let me thank you for commenting.
>>> >>>> >>
>>> >>>> >> As far as I understood, what you would like to see from the
>>> >>>> >> proposed
>>> >>>> >> tool is to view set of messages that are exchanged in reponse to a
>>> >>>> >> particular input message. With the understanding that I am having
>>> >>>> >> at
>>> >>>> >> the momnet, one way to do it is to filter out the central
>>> >>>> >> repository
>>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>> >>>> >> message chain from it. We can allow the client GUI wich connects
>>> >>>> >> to
>>> >>>> >> the central repository to provide the paramenters (For instance
>>> >>>> >> the
>>> >>>> >> value of 'To' header) from which an intelligent filtering can be
>>> >>>> >> done
>>> >>>> >> for the set of messages avialable at the central repository.
>>> >>>> >>
>>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
>>> >>>> >> willing
>>> >>>> >> to share it with us. It would be really nice to hear from them.
>>> >>>> >>
>>> >>>> >> Thanks in advance ..!!
>>> >>>> >>
>>> >>>> >> Best Regards,
>>> >>>> >> Nilupa
>>> >>>> >>
>>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> >>>> >> <an...@gmail.com> wrote:
>>> >>>> >>> Personally, I think that the added value of extending the SOAP
>>> >>>> >>> monitor
>>> >>>> >>> module to collect messages in a central place is too limited to
>>> >>>> >>> attract enough interest from the user community, so that it will
>>> >>>> >>> be
>>> >>>> >>> difficult to ensure the evolution of such a project in the
>>> >>>> >>> future.
>>> >>>> >>> However, what many people are looking for is a tool that allows
>>> >>>> >>> to
>>> >>>> >>> track the flow of a message through a distributed system, or more
>>> >>>> >>> generally to track the sequence of events triggered by a given
>>> >>>> >>> input
>>> >>>> >>> message (sort of end-to-end transaction monitoring).
>>> >>>> >>>
>>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>> >>>> >>> wrote:
>>> >>>> >>>> Hello,
>>> >>>> >>>>
>>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
>>> >>>> >>>> of
>>> >>>> >>>> proposing a GSoC project this summer. I would like to take
>>> >>>> >>>> project
>>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
>>> >>>> >>>> and
>>> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
>>> >>>> >>>> Web
>>> >>>> >>>> services middleware. I will submit more detailed proposal later.
>>> >>>> >>>> I
>>> >>>> >>>> would really like to hear any feedback from you. Any suggestions
>>> >>>> >>>> or
>>> >>>> >>>> features that you would like to see in a "Distributed TCP
>>> >>>> >>>> Monitor"
>>> >>>> >>>> are
>>> >>>> >>>> most welcome.
>>> >>>> >>>>
>>> >>>> >>>> Best Regards,
>>> >>>> >>>> Nilupa Bandara
>>> >>>> >>>>
>>> >>>> >>>> [1]
>>> >>>> >>>>
>>> >>>> >>>> Distributed TCP monitor
>>> >>>> >>>>
>>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
>>> >>>> >>>> hard
>>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
>>> >>>> >>>> the
>>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
>>> >>>> >>>> messages
>>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
>>> >>>> >>>> can
>>> >>>> >>>> pull those messages. Also, we can do something to turn the
>>> >>>> >>>> module on
>>> >>>> >>>> and off remotely, so users can have his module deployed, but
>>> >>>> >>>> turn it
>>> >>>> >>>> on only when they want to debug the system.
>>> >>>> >>>>
>>> >>>> >>>> Then we can take this to next level by adding this module to all
>>> >>>> >>>> services in a system , configuring modules to send all collected
>>> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
>>> >>>> >>>> via
>>> >>>> >>>> a
>>> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>> >>>> >>>> TCPMon for a whole system.
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>> ---------------------------------------------------------------------
>>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>> ---------------------------------------------------------------------
>>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>
>>> >>>> >>
>>> >>>> >> ---------------------------------------------------------------------
>>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>
>>> >>>> >>
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> > --
>>> >>>> > ============================
>>> >>>> > Srinath Perera, Ph.D.
>>> >>>> >   WSO2 Inc. http://wso2.com
>>> >>>> >   Blog: http://srinathsview.blogspot.com/
>>> >>>> >
>>> >>>> >
>>> >>>> > ---------------------------------------------------------------------
>>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >
>>> >>>> >
>>> >>>>
>>> >>>> ---------------------------------------------------------------------
>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Sanjiva Weerawarana, Ph.D.
>>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>>> >>> http://www.opensource.lk/
>>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>>> >>> Member; Apache Software Foundation; http://www.apache.org/
>>> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>>> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>> >>>
>>> >>> Blog: http://sanjiva.weerawarana.org/
>>> >>>
>>> >>
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>>
>
>
>
> --
> Sanka Samaranayake
>
> http://sankas.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

On Thu, Apr 1, 2010 at 03:28, Sanka Samaranayake <ss...@gmail.com> wrote:
> Hi Samisa,
>
> I think the this scope of this project would be to provide a
> light-weight tool which allows to monitor the message flows, execution
> in each node in response to those message flows to a given input
> message primarily for debugging purposes . (Just like the way tcpmon
> is used in developing Web services)
>
> And I guess now the question is whether such a tool is useful ..

Yes, if at least two additional requirements are satisfied:

- It should offer the possibility to measure response times. This
should be an easy one (except for the usual issues with time
measurements in distributed systems): just log the relevant
timestamps.
- It should not be limited to SOAP and it should not be specifically
targeted at Axis2: it should e.g. allow to add JDBC to the picture and
it should be easy to add support for other Web service stacks.

BTW, it may be a good idea to present the idea to other Apache
projects (CXF, ServiceMix, etc.) to see if they are interested in
supporting this project as well.
>
> Anyone cares to share their thoughts ?
>
> Nilupa
>
> On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>
>>
>> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:
>>>
>>> I was thinking about Java Swing based UI
>>
>> Swing is a good start. But if you look at some of the requirements like
>> monitoring multiple instances and correlate them, discussed in this thread,
>> it sounds as if we would be better off having a Web interface, rather than a
>> desktop client.
>>
>>>
>>> because the client may have
>>> to cache whatever messages it already has and will receive,
>>
>> If you are looking to support message-correlation, as discussed in some
>> replies in this thread above, you need to have a persistence storage (a DB
>> or something like that) rather than a memory cache.
>> Because, some message sequences, like those involved in an async
>> invocations, need to have a way to keep the messages, in there, longer, and
>> may be there is a server re-start, in the middle.
>> Of course, this requirements spend on the scope of this effort.
>> Plus, if you want to deal with historic data, and look at what was happening
>> to message sequences over time, you got to use a persistence layer. Again
>> this might be out of scope for this project, but if this is to be extended
>> to support these, you better think of this right now.
>>
>>>
>>> build the
>>> message flow and update the UI accordingly. Perhaps there is a better
>>> way ?
>>
>> We already do what you are trying to do, and much more with WSO2 BAM. I am
>> tempted to say that Shinding based gadget dashboard is really a compelling
>> UI for this.
>> I am not trying to take away you project or trying to hinder this effort at
>> all.
>> I think it will be a good idea for you to have a look at BAM and see what
>> the missing pieces are and figure out the right model for this tool.
>> Also, having had worked on WSO2 BAM project, I also have some understanding
>> on sort of problems that you might run into, when doing this. For e.g., if
>> you want to correlate message sequences in a high volume setup, you will be
>> swamped by the volume of messages, sooner than later. So correlation for the
>> purpose of monitoring is not an easy thing to do. Correlation, for the
>> purpose of debugging will not run into this problem, if you run this on a
>> controlled environment. However, what you really want to debug is the deal
>> clustered setup, and not a staged dev setup, the problem again surfaces.
>> You might want to list the objectives and then break them down
>> into monitoring and debugging spaces.
>> Samisa...
>>
>>>
>>> Thanks
>>> Nilupa.
>>>
>>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
>>> <sa...@gmail.com> wrote:
>>> > What UI technology would be used for this?
>>> >
>>> > Samisa...
>>> >
>>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
>>> > <an...@gmail.com> wrote:
>>> >> I was thinking more about something like ITCAM for SOA.
>>> >>
>>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>>> >> <sa...@opensource.lk> wrote:
>>> >>> That runtime monitoring / correlation is what the WSO2 Business
>>> >>> Activity
>>> >>> Monitor does / is for ...
>>> >>> See: http://wso2.com/products/business-activity-monitor/
>>> >>> Sanjiva.
>>> >>>
>>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
>>> >>> <an...@gmail.com>
>>> >>> wrote:
>>> >>>>
>>> >>>> I was actually referring to scenarios where a service may call other
>>> >>>> services, which in turn may call again other services. In these
>>> >>>> scenarios, it is not sufficient to simply collect received/sent
>>> >>>> messages on different hosts, unless the system is isolated
>>> >>>> (development environment) or the request rate is sufficiently low so
>>> >>>> that messages can be correlated based on time. Here is a concrete
>>> >>>> scenario from a project in my company:
>>> >>>>
>>> >>>> - Request comes in on an ESB that does security and validation.
>>> >>>> - Request is processed by an application server which persists the
>>> >>>> received information and publishes a JMS message (pub/sub events).
>>> >>>> - The event is consumed by one or more components that may in turn
>>> >>>> interact with other services.
>>> >>>>
>>> >>>> If you want track this flow, it is not sufficient to intercept the
>>> >>>> messages on the different hosts: in addition, you need to instrument
>>> >>>> the services so that the outgoing requests are correlated with the
>>> >>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> >>>> would like to be able to do in the above scenario is to enable
>>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> >>>> HTTP header to the initial request to enable logging and this
>>> >>>> information is propagated along the chain. Every service/component
>>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> >>>> central place where the sequence of events is reconstructed.
>>> >>>>
>>> >>>> One way this could be achieved is with a tool that postprocesses the
>>> >>>> artifacts from the build process. For each artifact, the tool would
>>> >>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> >>>> aspects and reassemble the artifact for deployment. The
>>> >>>> responsibility
>>> >>>> of the aspects would be to intercept messages, log them and modify
>>> >>>> them to ensure proper correlation. Of course this only works if the
>>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> >>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> >>>> in the transport test suites [1]. Interestingly, this approach would
>>> >>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> >>>> even intercept the database queries executed during the flow.
>>> >>>>
>>> >>>> Andreas
>>> >>>>
>>> >>>> [1]
>>> >>>>
>>> >>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>> >>>>
>>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
>>> >>>> wrote:
>>> >>>> > Hi Nilupa;
>>> >>>> >
>>> >>>> > When we collect messages from a one location by installing proper
>>> >>>> > handler that will intercept and send messages to to that one
>>> >>>> > location
>>> >>>> > (this one location can be a single server, pub/sub channel etc).
>>> >>>> > There
>>> >>>> > is many ways to make sense of those collected messages. What
>>> >>>> > Andreas
>>> >>>> > mentioned (following complete transaction) is a one possibility.
>>> >>>> >
>>> >>>> > I think you should come up with few scenarios on how you would make
>>> >>>> > sense of the message.
>>> >>>> >
>>> >>>> > Thanks
>>> >>>> > Srinath
>>> >>>> >
>>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>> >>>> > wrote:
>>> >>>> >> Hi,
>>> >>>> >>
>>> >>>> >> First let me thank you for commenting.
>>> >>>> >>
>>> >>>> >> As far as I understood, what you would like to see from the
>>> >>>> >> proposed
>>> >>>> >> tool is to view set of messages that are exchanged in reponse to a
>>> >>>> >> particular input message. With the understanding that I am having
>>> >>>> >> at
>>> >>>> >> the momnet, one way to do it is to filter out the central
>>> >>>> >> repository
>>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>> >>>> >> message chain from it. We can allow the client GUI wich connects
>>> >>>> >> to
>>> >>>> >> the central repository to provide the paramenters (For instance
>>> >>>> >> the
>>> >>>> >> value of 'To' header) from which an intelligent filtering can be
>>> >>>> >> done
>>> >>>> >> for the set of messages avialable at the central repository.
>>> >>>> >>
>>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
>>> >>>> >> willing
>>> >>>> >> to share it with us. It would be really nice to hear from them.
>>> >>>> >>
>>> >>>> >> Thanks in advance ..!!
>>> >>>> >>
>>> >>>> >> Best Regards,
>>> >>>> >> Nilupa
>>> >>>> >>
>>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> >>>> >> <an...@gmail.com> wrote:
>>> >>>> >>> Personally, I think that the added value of extending the SOAP
>>> >>>> >>> monitor
>>> >>>> >>> module to collect messages in a central place is too limited to
>>> >>>> >>> attract enough interest from the user community, so that it will
>>> >>>> >>> be
>>> >>>> >>> difficult to ensure the evolution of such a project in the
>>> >>>> >>> future.
>>> >>>> >>> However, what many people are looking for is a tool that allows
>>> >>>> >>> to
>>> >>>> >>> track the flow of a message through a distributed system, or more
>>> >>>> >>> generally to track the sequence of events triggered by a given
>>> >>>> >>> input
>>> >>>> >>> message (sort of end-to-end transaction monitoring).
>>> >>>> >>>
>>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>> >>>> >>> wrote:
>>> >>>> >>>> Hello,
>>> >>>> >>>>
>>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
>>> >>>> >>>> of
>>> >>>> >>>> proposing a GSoC project this summer. I would like to take
>>> >>>> >>>> project
>>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
>>> >>>> >>>> and
>>> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
>>> >>>> >>>> Web
>>> >>>> >>>> services middleware. I will submit more detailed proposal later.
>>> >>>> >>>> I
>>> >>>> >>>> would really like to hear any feedback from you. Any suggestions
>>> >>>> >>>> or
>>> >>>> >>>> features that you would like to see in a "Distributed TCP
>>> >>>> >>>> Monitor"
>>> >>>> >>>> are
>>> >>>> >>>> most welcome.
>>> >>>> >>>>
>>> >>>> >>>> Best Regards,
>>> >>>> >>>> Nilupa Bandara
>>> >>>> >>>>
>>> >>>> >>>> [1]
>>> >>>> >>>>
>>> >>>> >>>> Distributed TCP monitor
>>> >>>> >>>>
>>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
>>> >>>> >>>> hard
>>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
>>> >>>> >>>> the
>>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
>>> >>>> >>>> messages
>>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
>>> >>>> >>>> can
>>> >>>> >>>> pull those messages. Also, we can do something to turn the
>>> >>>> >>>> module on
>>> >>>> >>>> and off remotely, so users can have his module deployed, but
>>> >>>> >>>> turn it
>>> >>>> >>>> on only when they want to debug the system.
>>> >>>> >>>>
>>> >>>> >>>> Then we can take this to next level by adding this module to all
>>> >>>> >>>> services in a system , configuring modules to send all collected
>>> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
>>> >>>> >>>> via
>>> >>>> >>>> a
>>> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>> >>>> >>>> TCPMon for a whole system.
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>> ---------------------------------------------------------------------
>>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>> ---------------------------------------------------------------------
>>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>
>>> >>>> >>
>>> >>>> >> ---------------------------------------------------------------------
>>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>
>>> >>>> >>
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> > --
>>> >>>> > ============================
>>> >>>> > Srinath Perera, Ph.D.
>>> >>>> >   WSO2 Inc. http://wso2.com
>>> >>>> >   Blog: http://srinathsview.blogspot.com/
>>> >>>> >
>>> >>>> >
>>> >>>> > ---------------------------------------------------------------------
>>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >
>>> >>>> >
>>> >>>>
>>> >>>> ---------------------------------------------------------------------
>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Sanjiva Weerawarana, Ph.D.
>>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>>> >>> http://www.opensource.lk/
>>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>>> >>> Member; Apache Software Foundation; http://www.apache.org/
>>> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>>> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>> >>>
>>> >>> Blog: http://sanjiva.weerawarana.org/
>>> >>>
>>> >>
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>>
>
>
>
> --
> Sanka Samaranayake
>
> http://sankas.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

On Thu, Apr 1, 2010 at 03:28, Sanka Samaranayake <ss...@gmail.com> wrote:
> Hi Samisa,
>
> I think the this scope of this project would be to provide a
> light-weight tool which allows to monitor the message flows, execution
> in each node in response to those message flows to a given input
> message primarily for debugging purposes . (Just like the way tcpmon
> is used in developing Web services)
>
> And I guess now the question is whether such a tool is useful ..

Yes, if at least two additional requirements are satisfied:

- It should offer the possibility to measure response times. This
should be an easy one (except for the usual issues with time
measurements in distributed systems): just log the relevant
timestamps.
- It should not be limited to SOAP and it should not be specifically
targeted at Axis2: it should e.g. allow to add JDBC to the picture and
it should be easy to add support for other Web service stacks.

BTW, it may be a good idea to present the idea to other Apache
projects (CXF, ServiceMix, etc.) to see if they are interested in
supporting this project as well.
>
> Anyone cares to share their thoughts ?
>
> Nilupa
>
> On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>
>>
>> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:
>>>
>>> I was thinking about Java Swing based UI
>>
>> Swing is a good start. But if you look at some of the requirements like
>> monitoring multiple instances and correlate them, discussed in this thread,
>> it sounds as if we would be better off having a Web interface, rather than a
>> desktop client.
>>
>>>
>>> because the client may have
>>> to cache whatever messages it already has and will receive,
>>
>> If you are looking to support message-correlation, as discussed in some
>> replies in this thread above, you need to have a persistence storage (a DB
>> or something like that) rather than a memory cache.
>> Because, some message sequences, like those involved in an async
>> invocations, need to have a way to keep the messages, in there, longer, and
>> may be there is a server re-start, in the middle.
>> Of course, this requirements spend on the scope of this effort.
>> Plus, if you want to deal with historic data, and look at what was happening
>> to message sequences over time, you got to use a persistence layer. Again
>> this might be out of scope for this project, but if this is to be extended
>> to support these, you better think of this right now.
>>
>>>
>>> build the
>>> message flow and update the UI accordingly. Perhaps there is a better
>>> way ?
>>
>> We already do what you are trying to do, and much more with WSO2 BAM. I am
>> tempted to say that Shinding based gadget dashboard is really a compelling
>> UI for this.
>> I am not trying to take away you project or trying to hinder this effort at
>> all.
>> I think it will be a good idea for you to have a look at BAM and see what
>> the missing pieces are and figure out the right model for this tool.
>> Also, having had worked on WSO2 BAM project, I also have some understanding
>> on sort of problems that you might run into, when doing this. For e.g., if
>> you want to correlate message sequences in a high volume setup, you will be
>> swamped by the volume of messages, sooner than later. So correlation for the
>> purpose of monitoring is not an easy thing to do. Correlation, for the
>> purpose of debugging will not run into this problem, if you run this on a
>> controlled environment. However, what you really want to debug is the deal
>> clustered setup, and not a staged dev setup, the problem again surfaces.
>> You might want to list the objectives and then break them down
>> into monitoring and debugging spaces.
>> Samisa...
>>
>>>
>>> Thanks
>>> Nilupa.
>>>
>>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
>>> <sa...@gmail.com> wrote:
>>> > What UI technology would be used for this?
>>> >
>>> > Samisa...
>>> >
>>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
>>> > <an...@gmail.com> wrote:
>>> >> I was thinking more about something like ITCAM for SOA.
>>> >>
>>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>>> >> <sa...@opensource.lk> wrote:
>>> >>> That runtime monitoring / correlation is what the WSO2 Business
>>> >>> Activity
>>> >>> Monitor does / is for ...
>>> >>> See: http://wso2.com/products/business-activity-monitor/
>>> >>> Sanjiva.
>>> >>>
>>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
>>> >>> <an...@gmail.com>
>>> >>> wrote:
>>> >>>>
>>> >>>> I was actually referring to scenarios where a service may call other
>>> >>>> services, which in turn may call again other services. In these
>>> >>>> scenarios, it is not sufficient to simply collect received/sent
>>> >>>> messages on different hosts, unless the system is isolated
>>> >>>> (development environment) or the request rate is sufficiently low so
>>> >>>> that messages can be correlated based on time. Here is a concrete
>>> >>>> scenario from a project in my company:
>>> >>>>
>>> >>>> - Request comes in on an ESB that does security and validation.
>>> >>>> - Request is processed by an application server which persists the
>>> >>>> received information and publishes a JMS message (pub/sub events).
>>> >>>> - The event is consumed by one or more components that may in turn
>>> >>>> interact with other services.
>>> >>>>
>>> >>>> If you want track this flow, it is not sufficient to intercept the
>>> >>>> messages on the different hosts: in addition, you need to instrument
>>> >>>> the services so that the outgoing requests are correlated with the
>>> >>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> >>>> would like to be able to do in the above scenario is to enable
>>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> >>>> HTTP header to the initial request to enable logging and this
>>> >>>> information is propagated along the chain. Every service/component
>>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> >>>> central place where the sequence of events is reconstructed.
>>> >>>>
>>> >>>> One way this could be achieved is with a tool that postprocesses the
>>> >>>> artifacts from the build process. For each artifact, the tool would
>>> >>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> >>>> aspects and reassemble the artifact for deployment. The
>>> >>>> responsibility
>>> >>>> of the aspects would be to intercept messages, log them and modify
>>> >>>> them to ensure proper correlation. Of course this only works if the
>>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> >>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> >>>> in the transport test suites [1]. Interestingly, this approach would
>>> >>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> >>>> even intercept the database queries executed during the flow.
>>> >>>>
>>> >>>> Andreas
>>> >>>>
>>> >>>> [1]
>>> >>>>
>>> >>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>> >>>>
>>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
>>> >>>> wrote:
>>> >>>> > Hi Nilupa;
>>> >>>> >
>>> >>>> > When we collect messages from a one location by installing proper
>>> >>>> > handler that will intercept and send messages to to that one
>>> >>>> > location
>>> >>>> > (this one location can be a single server, pub/sub channel etc).
>>> >>>> > There
>>> >>>> > is many ways to make sense of those collected messages. What
>>> >>>> > Andreas
>>> >>>> > mentioned (following complete transaction) is a one possibility.
>>> >>>> >
>>> >>>> > I think you should come up with few scenarios on how you would make
>>> >>>> > sense of the message.
>>> >>>> >
>>> >>>> > Thanks
>>> >>>> > Srinath
>>> >>>> >
>>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>> >>>> > wrote:
>>> >>>> >> Hi,
>>> >>>> >>
>>> >>>> >> First let me thank you for commenting.
>>> >>>> >>
>>> >>>> >> As far as I understood, what you would like to see from the
>>> >>>> >> proposed
>>> >>>> >> tool is to view set of messages that are exchanged in reponse to a
>>> >>>> >> particular input message. With the understanding that I am having
>>> >>>> >> at
>>> >>>> >> the momnet, one way to do it is to filter out the central
>>> >>>> >> repository
>>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>> >>>> >> message chain from it. We can allow the client GUI wich connects
>>> >>>> >> to
>>> >>>> >> the central repository to provide the paramenters (For instance
>>> >>>> >> the
>>> >>>> >> value of 'To' header) from which an intelligent filtering can be
>>> >>>> >> done
>>> >>>> >> for the set of messages avialable at the central repository.
>>> >>>> >>
>>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
>>> >>>> >> willing
>>> >>>> >> to share it with us. It would be really nice to hear from them.
>>> >>>> >>
>>> >>>> >> Thanks in advance ..!!
>>> >>>> >>
>>> >>>> >> Best Regards,
>>> >>>> >> Nilupa
>>> >>>> >>
>>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> >>>> >> <an...@gmail.com> wrote:
>>> >>>> >>> Personally, I think that the added value of extending the SOAP
>>> >>>> >>> monitor
>>> >>>> >>> module to collect messages in a central place is too limited to
>>> >>>> >>> attract enough interest from the user community, so that it will
>>> >>>> >>> be
>>> >>>> >>> difficult to ensure the evolution of such a project in the
>>> >>>> >>> future.
>>> >>>> >>> However, what many people are looking for is a tool that allows
>>> >>>> >>> to
>>> >>>> >>> track the flow of a message through a distributed system, or more
>>> >>>> >>> generally to track the sequence of events triggered by a given
>>> >>>> >>> input
>>> >>>> >>> message (sort of end-to-end transaction monitoring).
>>> >>>> >>>
>>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>> >>>> >>> wrote:
>>> >>>> >>>> Hello,
>>> >>>> >>>>
>>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
>>> >>>> >>>> of
>>> >>>> >>>> proposing a GSoC project this summer. I would like to take
>>> >>>> >>>> project
>>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
>>> >>>> >>>> and
>>> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
>>> >>>> >>>> Web
>>> >>>> >>>> services middleware. I will submit more detailed proposal later.
>>> >>>> >>>> I
>>> >>>> >>>> would really like to hear any feedback from you. Any suggestions
>>> >>>> >>>> or
>>> >>>> >>>> features that you would like to see in a "Distributed TCP
>>> >>>> >>>> Monitor"
>>> >>>> >>>> are
>>> >>>> >>>> most welcome.
>>> >>>> >>>>
>>> >>>> >>>> Best Regards,
>>> >>>> >>>> Nilupa Bandara
>>> >>>> >>>>
>>> >>>> >>>> [1]
>>> >>>> >>>>
>>> >>>> >>>> Distributed TCP monitor
>>> >>>> >>>>
>>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
>>> >>>> >>>> hard
>>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
>>> >>>> >>>> the
>>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
>>> >>>> >>>> messages
>>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
>>> >>>> >>>> can
>>> >>>> >>>> pull those messages. Also, we can do something to turn the
>>> >>>> >>>> module on
>>> >>>> >>>> and off remotely, so users can have his module deployed, but
>>> >>>> >>>> turn it
>>> >>>> >>>> on only when they want to debug the system.
>>> >>>> >>>>
>>> >>>> >>>> Then we can take this to next level by adding this module to all
>>> >>>> >>>> services in a system , configuring modules to send all collected
>>> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
>>> >>>> >>>> via
>>> >>>> >>>> a
>>> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>> >>>> >>>> TCPMon for a whole system.
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>> ---------------------------------------------------------------------
>>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>> ---------------------------------------------------------------------
>>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>
>>> >>>> >>
>>> >>>> >> ---------------------------------------------------------------------
>>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>
>>> >>>> >>
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> > --
>>> >>>> > ============================
>>> >>>> > Srinath Perera, Ph.D.
>>> >>>> >   WSO2 Inc. http://wso2.com
>>> >>>> >   Blog: http://srinathsview.blogspot.com/
>>> >>>> >
>>> >>>> >
>>> >>>> > ---------------------------------------------------------------------
>>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >
>>> >>>> >
>>> >>>>
>>> >>>> ---------------------------------------------------------------------
>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Sanjiva Weerawarana, Ph.D.
>>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>>> >>> http://www.opensource.lk/
>>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>>> >>> Member; Apache Software Foundation; http://www.apache.org/
>>> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>>> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>> >>>
>>> >>> Blog: http://sanjiva.weerawarana.org/
>>> >>>
>>> >>
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>>
>
>
>
> --
> Sanka Samaranayake
>
> http://sankas.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanka Samaranayake <ss...@gmail.com>.

Samisa,

Quick question, does WSO2 BAM project provide such a debugging
facility. I mean can one monitor what is going on for a specific input
in SOA senario ?

Sanka



On Thu, Apr 1, 2010 at 3:28 AM, Sanka Samaranayake <ss...@gmail.com> wrote:
> Hi Samisa,
>
> I think the this scope of this project would be to provide a
> light-weight tool which allows to monitor the message flows, execution
> in each node in response to those message flows to a given input
> message primarily for debugging purposes . (Just like the way tcpmon
> is used in developing Web services)
>
> And I guess now the question is whether such a tool is useful ..
>
> Anyone cares to share their thoughts ?
>
> Nilupa
>
> On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>
>>
>> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:
>>>
>>> I was thinking about Java Swing based UI
>>
>> Swing is a good start. But if you look at some of the requirements like
>> monitoring multiple instances and correlate them, discussed in this thread,
>> it sounds as if we would be better off having a Web interface, rather than a
>> desktop client.
>>
>>>
>>> because the client may have
>>> to cache whatever messages it already has and will receive,
>>
>> If you are looking to support message-correlation, as discussed in some
>> replies in this thread above, you need to have a persistence storage (a DB
>> or something like that) rather than a memory cache.
>> Because, some message sequences, like those involved in an async
>> invocations, need to have a way to keep the messages, in there, longer, and
>> may be there is a server re-start, in the middle.
>> Of course, this requirements spend on the scope of this effort.
>> Plus, if you want to deal with historic data, and look at what was happening
>> to message sequences over time, you got to use a persistence layer. Again
>> this might be out of scope for this project, but if this is to be extended
>> to support these, you better think of this right now.
>>
>>>
>>> build the
>>> message flow and update the UI accordingly. Perhaps there is a better
>>> way ?
>>
>> We already do what you are trying to do, and much more with WSO2 BAM. I am
>> tempted to say that Shinding based gadget dashboard is really a compelling
>> UI for this.
>> I am not trying to take away you project or trying to hinder this effort at
>> all.
>> I think it will be a good idea for you to have a look at BAM and see what
>> the missing pieces are and figure out the right model for this tool.
>> Also, having had worked on WSO2 BAM project, I also have some understanding
>> on sort of problems that you might run into, when doing this. For e.g., if
>> you want to correlate message sequences in a high volume setup, you will be
>> swamped by the volume of messages, sooner than later. So correlation for the
>> purpose of monitoring is not an easy thing to do. Correlation, for the
>> purpose of debugging will not run into this problem, if you run this on a
>> controlled environment. However, what you really want to debug is the deal
>> clustered setup, and not a staged dev setup, the problem again surfaces.
>> You might want to list the objectives and then break them down
>> into monitoring and debugging spaces.
>> Samisa...
>>
>>>
>>> Thanks
>>> Nilupa.
>>>
>>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
>>> <sa...@gmail.com> wrote:
>>> > What UI technology would be used for this?
>>> >
>>> > Samisa...
>>> >
>>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
>>> > <an...@gmail.com> wrote:
>>> >> I was thinking more about something like ITCAM for SOA.
>>> >>
>>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>>> >> <sa...@opensource.lk> wrote:
>>> >>> That runtime monitoring / correlation is what the WSO2 Business
>>> >>> Activity
>>> >>> Monitor does / is for ...
>>> >>> See: http://wso2.com/products/business-activity-monitor/
>>> >>> Sanjiva.
>>> >>>
>>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
>>> >>> <an...@gmail.com>
>>> >>> wrote:
>>> >>>>
>>> >>>> I was actually referring to scenarios where a service may call other
>>> >>>> services, which in turn may call again other services. In these
>>> >>>> scenarios, it is not sufficient to simply collect received/sent
>>> >>>> messages on different hosts, unless the system is isolated
>>> >>>> (development environment) or the request rate is sufficiently low so
>>> >>>> that messages can be correlated based on time. Here is a concrete
>>> >>>> scenario from a project in my company:
>>> >>>>
>>> >>>> - Request comes in on an ESB that does security and validation.
>>> >>>> - Request is processed by an application server which persists the
>>> >>>> received information and publishes a JMS message (pub/sub events).
>>> >>>> - The event is consumed by one or more components that may in turn
>>> >>>> interact with other services.
>>> >>>>
>>> >>>> If you want track this flow, it is not sufficient to intercept the
>>> >>>> messages on the different hosts: in addition, you need to instrument
>>> >>>> the services so that the outgoing requests are correlated with the
>>> >>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> >>>> would like to be able to do in the above scenario is to enable
>>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> >>>> HTTP header to the initial request to enable logging and this
>>> >>>> information is propagated along the chain. Every service/component
>>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> >>>> central place where the sequence of events is reconstructed.
>>> >>>>
>>> >>>> One way this could be achieved is with a tool that postprocesses the
>>> >>>> artifacts from the build process. For each artifact, the tool would
>>> >>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> >>>> aspects and reassemble the artifact for deployment. The
>>> >>>> responsibility
>>> >>>> of the aspects would be to intercept messages, log them and modify
>>> >>>> them to ensure proper correlation. Of course this only works if the
>>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> >>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> >>>> in the transport test suites [1]. Interestingly, this approach would
>>> >>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> >>>> even intercept the database queries executed during the flow.
>>> >>>>
>>> >>>> Andreas
>>> >>>>
>>> >>>> [1]
>>> >>>>
>>> >>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>> >>>>
>>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
>>> >>>> wrote:
>>> >>>> > Hi Nilupa;
>>> >>>> >
>>> >>>> > When we collect messages from a one location by installing proper
>>> >>>> > handler that will intercept and send messages to to that one
>>> >>>> > location
>>> >>>> > (this one location can be a single server, pub/sub channel etc).
>>> >>>> > There
>>> >>>> > is many ways to make sense of those collected messages. What
>>> >>>> > Andreas
>>> >>>> > mentioned (following complete transaction) is a one possibility.
>>> >>>> >
>>> >>>> > I think you should come up with few scenarios on how you would make
>>> >>>> > sense of the message.
>>> >>>> >
>>> >>>> > Thanks
>>> >>>> > Srinath
>>> >>>> >
>>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>> >>>> > wrote:
>>> >>>> >> Hi,
>>> >>>> >>
>>> >>>> >> First let me thank you for commenting.
>>> >>>> >>
>>> >>>> >> As far as I understood, what you would like to see from the
>>> >>>> >> proposed
>>> >>>> >> tool is to view set of messages that are exchanged in reponse to a
>>> >>>> >> particular input message. With the understanding that I am having
>>> >>>> >> at
>>> >>>> >> the momnet, one way to do it is to filter out the central
>>> >>>> >> repository
>>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>> >>>> >> message chain from it. We can allow the client GUI wich connects
>>> >>>> >> to
>>> >>>> >> the central repository to provide the paramenters (For instance
>>> >>>> >> the
>>> >>>> >> value of 'To' header) from which an intelligent filtering can be
>>> >>>> >> done
>>> >>>> >> for the set of messages avialable at the central repository.
>>> >>>> >>
>>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
>>> >>>> >> willing
>>> >>>> >> to share it with us. It would be really nice to hear from them.
>>> >>>> >>
>>> >>>> >> Thanks in advance ..!!
>>> >>>> >>
>>> >>>> >> Best Regards,
>>> >>>> >> Nilupa
>>> >>>> >>
>>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> >>>> >> <an...@gmail.com> wrote:
>>> >>>> >>> Personally, I think that the added value of extending the SOAP
>>> >>>> >>> monitor
>>> >>>> >>> module to collect messages in a central place is too limited to
>>> >>>> >>> attract enough interest from the user community, so that it will
>>> >>>> >>> be
>>> >>>> >>> difficult to ensure the evolution of such a project in the
>>> >>>> >>> future.
>>> >>>> >>> However, what many people are looking for is a tool that allows
>>> >>>> >>> to
>>> >>>> >>> track the flow of a message through a distributed system, or more
>>> >>>> >>> generally to track the sequence of events triggered by a given
>>> >>>> >>> input
>>> >>>> >>> message (sort of end-to-end transaction monitoring).
>>> >>>> >>>
>>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>> >>>> >>> wrote:
>>> >>>> >>>> Hello,
>>> >>>> >>>>
>>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
>>> >>>> >>>> of
>>> >>>> >>>> proposing a GSoC project this summer. I would like to take
>>> >>>> >>>> project
>>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
>>> >>>> >>>> and
>>> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
>>> >>>> >>>> Web
>>> >>>> >>>> services middleware. I will submit more detailed proposal later.
>>> >>>> >>>> I
>>> >>>> >>>> would really like to hear any feedback from you. Any suggestions
>>> >>>> >>>> or
>>> >>>> >>>> features that you would like to see in a "Distributed TCP
>>> >>>> >>>> Monitor"
>>> >>>> >>>> are
>>> >>>> >>>> most welcome.
>>> >>>> >>>>
>>> >>>> >>>> Best Regards,
>>> >>>> >>>> Nilupa Bandara
>>> >>>> >>>>
>>> >>>> >>>> [1]
>>> >>>> >>>>
>>> >>>> >>>> Distributed TCP monitor
>>> >>>> >>>>
>>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
>>> >>>> >>>> hard
>>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
>>> >>>> >>>> the
>>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
>>> >>>> >>>> messages
>>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
>>> >>>> >>>> can
>>> >>>> >>>> pull those messages. Also, we can do something to turn the
>>> >>>> >>>> module on
>>> >>>> >>>> and off remotely, so users can have his module deployed, but
>>> >>>> >>>> turn it
>>> >>>> >>>> on only when they want to debug the system.
>>> >>>> >>>>
>>> >>>> >>>> Then we can take this to next level by adding this module to all
>>> >>>> >>>> services in a system , configuring modules to send all collected
>>> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
>>> >>>> >>>> via
>>> >>>> >>>> a
>>> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>> >>>> >>>> TCPMon for a whole system.
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>> ---------------------------------------------------------------------
>>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>> ---------------------------------------------------------------------
>>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>
>>> >>>> >>
>>> >>>> >> ---------------------------------------------------------------------
>>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>
>>> >>>> >>
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> > --
>>> >>>> > ============================
>>> >>>> > Srinath Perera, Ph.D.
>>> >>>> >   WSO2 Inc. http://wso2.com
>>> >>>> >   Blog: http://srinathsview.blogspot.com/
>>> >>>> >
>>> >>>> >
>>> >>>> > ---------------------------------------------------------------------
>>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >
>>> >>>> >
>>> >>>>
>>> >>>> ---------------------------------------------------------------------
>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Sanjiva Weerawarana, Ph.D.
>>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>>> >>> http://www.opensource.lk/
>>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>>> >>> Member; Apache Software Foundation; http://www.apache.org/
>>> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>>> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>> >>>
>>> >>> Blog: http://sanjiva.weerawarana.org/
>>> >>>
>>> >>
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>>
>
>
>
> --
> Sanka Samaranayake
>
> http://sankas.blogspot.com/
>



-- 
Sanka Samaranayake

http://sankas.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanka Samaranayake <ss...@gmail.com>.

Samisa,

Quick question, does WSO2 BAM project provide such a debugging
facility. I mean can one monitor what is going on for a specific input
in SOA senario ?

Sanka



On Thu, Apr 1, 2010 at 3:28 AM, Sanka Samaranayake <ss...@gmail.com> wrote:
> Hi Samisa,
>
> I think the this scope of this project would be to provide a
> light-weight tool which allows to monitor the message flows, execution
> in each node in response to those message flows to a given input
> message primarily for debugging purposes . (Just like the way tcpmon
> is used in developing Web services)
>
> And I guess now the question is whether such a tool is useful ..
>
> Anyone cares to share their thoughts ?
>
> Nilupa
>
> On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>
>>
>> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:
>>>
>>> I was thinking about Java Swing based UI
>>
>> Swing is a good start. But if you look at some of the requirements like
>> monitoring multiple instances and correlate them, discussed in this thread,
>> it sounds as if we would be better off having a Web interface, rather than a
>> desktop client.
>>
>>>
>>> because the client may have
>>> to cache whatever messages it already has and will receive,
>>
>> If you are looking to support message-correlation, as discussed in some
>> replies in this thread above, you need to have a persistence storage (a DB
>> or something like that) rather than a memory cache.
>> Because, some message sequences, like those involved in an async
>> invocations, need to have a way to keep the messages, in there, longer, and
>> may be there is a server re-start, in the middle.
>> Of course, this requirements spend on the scope of this effort.
>> Plus, if you want to deal with historic data, and look at what was happening
>> to message sequences over time, you got to use a persistence layer. Again
>> this might be out of scope for this project, but if this is to be extended
>> to support these, you better think of this right now.
>>
>>>
>>> build the
>>> message flow and update the UI accordingly. Perhaps there is a better
>>> way ?
>>
>> We already do what you are trying to do, and much more with WSO2 BAM. I am
>> tempted to say that Shinding based gadget dashboard is really a compelling
>> UI for this.
>> I am not trying to take away you project or trying to hinder this effort at
>> all.
>> I think it will be a good idea for you to have a look at BAM and see what
>> the missing pieces are and figure out the right model for this tool.
>> Also, having had worked on WSO2 BAM project, I also have some understanding
>> on sort of problems that you might run into, when doing this. For e.g., if
>> you want to correlate message sequences in a high volume setup, you will be
>> swamped by the volume of messages, sooner than later. So correlation for the
>> purpose of monitoring is not an easy thing to do. Correlation, for the
>> purpose of debugging will not run into this problem, if you run this on a
>> controlled environment. However, what you really want to debug is the deal
>> clustered setup, and not a staged dev setup, the problem again surfaces.
>> You might want to list the objectives and then break them down
>> into monitoring and debugging spaces.
>> Samisa...
>>
>>>
>>> Thanks
>>> Nilupa.
>>>
>>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
>>> <sa...@gmail.com> wrote:
>>> > What UI technology would be used for this?
>>> >
>>> > Samisa...
>>> >
>>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
>>> > <an...@gmail.com> wrote:
>>> >> I was thinking more about something like ITCAM for SOA.
>>> >>
>>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>>> >> <sa...@opensource.lk> wrote:
>>> >>> That runtime monitoring / correlation is what the WSO2 Business
>>> >>> Activity
>>> >>> Monitor does / is for ...
>>> >>> See: http://wso2.com/products/business-activity-monitor/
>>> >>> Sanjiva.
>>> >>>
>>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
>>> >>> <an...@gmail.com>
>>> >>> wrote:
>>> >>>>
>>> >>>> I was actually referring to scenarios where a service may call other
>>> >>>> services, which in turn may call again other services. In these
>>> >>>> scenarios, it is not sufficient to simply collect received/sent
>>> >>>> messages on different hosts, unless the system is isolated
>>> >>>> (development environment) or the request rate is sufficiently low so
>>> >>>> that messages can be correlated based on time. Here is a concrete
>>> >>>> scenario from a project in my company:
>>> >>>>
>>> >>>> - Request comes in on an ESB that does security and validation.
>>> >>>> - Request is processed by an application server which persists the
>>> >>>> received information and publishes a JMS message (pub/sub events).
>>> >>>> - The event is consumed by one or more components that may in turn
>>> >>>> interact with other services.
>>> >>>>
>>> >>>> If you want track this flow, it is not sufficient to intercept the
>>> >>>> messages on the different hosts: in addition, you need to instrument
>>> >>>> the services so that the outgoing requests are correlated with the
>>> >>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> >>>> would like to be able to do in the above scenario is to enable
>>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> >>>> HTTP header to the initial request to enable logging and this
>>> >>>> information is propagated along the chain. Every service/component
>>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> >>>> central place where the sequence of events is reconstructed.
>>> >>>>
>>> >>>> One way this could be achieved is with a tool that postprocesses the
>>> >>>> artifacts from the build process. For each artifact, the tool would
>>> >>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> >>>> aspects and reassemble the artifact for deployment. The
>>> >>>> responsibility
>>> >>>> of the aspects would be to intercept messages, log them and modify
>>> >>>> them to ensure proper correlation. Of course this only works if the
>>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> >>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> >>>> in the transport test suites [1]. Interestingly, this approach would
>>> >>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> >>>> even intercept the database queries executed during the flow.
>>> >>>>
>>> >>>> Andreas
>>> >>>>
>>> >>>> [1]
>>> >>>>
>>> >>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>> >>>>
>>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
>>> >>>> wrote:
>>> >>>> > Hi Nilupa;
>>> >>>> >
>>> >>>> > When we collect messages from a one location by installing proper
>>> >>>> > handler that will intercept and send messages to to that one
>>> >>>> > location
>>> >>>> > (this one location can be a single server, pub/sub channel etc).
>>> >>>> > There
>>> >>>> > is many ways to make sense of those collected messages. What
>>> >>>> > Andreas
>>> >>>> > mentioned (following complete transaction) is a one possibility.
>>> >>>> >
>>> >>>> > I think you should come up with few scenarios on how you would make
>>> >>>> > sense of the message.
>>> >>>> >
>>> >>>> > Thanks
>>> >>>> > Srinath
>>> >>>> >
>>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>> >>>> > wrote:
>>> >>>> >> Hi,
>>> >>>> >>
>>> >>>> >> First let me thank you for commenting.
>>> >>>> >>
>>> >>>> >> As far as I understood, what you would like to see from the
>>> >>>> >> proposed
>>> >>>> >> tool is to view set of messages that are exchanged in reponse to a
>>> >>>> >> particular input message. With the understanding that I am having
>>> >>>> >> at
>>> >>>> >> the momnet, one way to do it is to filter out the central
>>> >>>> >> repository
>>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>> >>>> >> message chain from it. We can allow the client GUI wich connects
>>> >>>> >> to
>>> >>>> >> the central repository to provide the paramenters (For instance
>>> >>>> >> the
>>> >>>> >> value of 'To' header) from which an intelligent filtering can be
>>> >>>> >> done
>>> >>>> >> for the set of messages avialable at the central repository.
>>> >>>> >>
>>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
>>> >>>> >> willing
>>> >>>> >> to share it with us. It would be really nice to hear from them.
>>> >>>> >>
>>> >>>> >> Thanks in advance ..!!
>>> >>>> >>
>>> >>>> >> Best Regards,
>>> >>>> >> Nilupa
>>> >>>> >>
>>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> >>>> >> <an...@gmail.com> wrote:
>>> >>>> >>> Personally, I think that the added value of extending the SOAP
>>> >>>> >>> monitor
>>> >>>> >>> module to collect messages in a central place is too limited to
>>> >>>> >>> attract enough interest from the user community, so that it will
>>> >>>> >>> be
>>> >>>> >>> difficult to ensure the evolution of such a project in the
>>> >>>> >>> future.
>>> >>>> >>> However, what many people are looking for is a tool that allows
>>> >>>> >>> to
>>> >>>> >>> track the flow of a message through a distributed system, or more
>>> >>>> >>> generally to track the sequence of events triggered by a given
>>> >>>> >>> input
>>> >>>> >>> message (sort of end-to-end transaction monitoring).
>>> >>>> >>>
>>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>> >>>> >>> wrote:
>>> >>>> >>>> Hello,
>>> >>>> >>>>
>>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
>>> >>>> >>>> of
>>> >>>> >>>> proposing a GSoC project this summer. I would like to take
>>> >>>> >>>> project
>>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
>>> >>>> >>>> and
>>> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
>>> >>>> >>>> Web
>>> >>>> >>>> services middleware. I will submit more detailed proposal later.
>>> >>>> >>>> I
>>> >>>> >>>> would really like to hear any feedback from you. Any suggestions
>>> >>>> >>>> or
>>> >>>> >>>> features that you would like to see in a "Distributed TCP
>>> >>>> >>>> Monitor"
>>> >>>> >>>> are
>>> >>>> >>>> most welcome.
>>> >>>> >>>>
>>> >>>> >>>> Best Regards,
>>> >>>> >>>> Nilupa Bandara
>>> >>>> >>>>
>>> >>>> >>>> [1]
>>> >>>> >>>>
>>> >>>> >>>> Distributed TCP monitor
>>> >>>> >>>>
>>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
>>> >>>> >>>> hard
>>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
>>> >>>> >>>> the
>>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
>>> >>>> >>>> messages
>>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
>>> >>>> >>>> can
>>> >>>> >>>> pull those messages. Also, we can do something to turn the
>>> >>>> >>>> module on
>>> >>>> >>>> and off remotely, so users can have his module deployed, but
>>> >>>> >>>> turn it
>>> >>>> >>>> on only when they want to debug the system.
>>> >>>> >>>>
>>> >>>> >>>> Then we can take this to next level by adding this module to all
>>> >>>> >>>> services in a system , configuring modules to send all collected
>>> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
>>> >>>> >>>> via
>>> >>>> >>>> a
>>> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>> >>>> >>>> TCPMon for a whole system.
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>> ---------------------------------------------------------------------
>>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>> ---------------------------------------------------------------------
>>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>
>>> >>>> >>
>>> >>>> >> ---------------------------------------------------------------------
>>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>
>>> >>>> >>
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> > --
>>> >>>> > ============================
>>> >>>> > Srinath Perera, Ph.D.
>>> >>>> >   WSO2 Inc. http://wso2.com
>>> >>>> >   Blog: http://srinathsview.blogspot.com/
>>> >>>> >
>>> >>>> >
>>> >>>> > ---------------------------------------------------------------------
>>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >
>>> >>>> >
>>> >>>>
>>> >>>> ---------------------------------------------------------------------
>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Sanjiva Weerawarana, Ph.D.
>>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>>> >>> http://www.opensource.lk/
>>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>>> >>> Member; Apache Software Foundation; http://www.apache.org/
>>> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>>> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>> >>>
>>> >>> Blog: http://sanjiva.weerawarana.org/
>>> >>>
>>> >>
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>>
>
>
>
> --
> Sanka Samaranayake
>
> http://sankas.blogspot.com/
>



-- 
Sanka Samaranayake

http://sankas.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

On Thu, Apr 1, 2010 at 03:28, Sanka Samaranayake <ss...@gmail.com> wrote:
> Hi Samisa,
>
> I think the this scope of this project would be to provide a
> light-weight tool which allows to monitor the message flows, execution
> in each node in response to those message flows to a given input
> message primarily for debugging purposes . (Just like the way tcpmon
> is used in developing Web services)
>
> And I guess now the question is whether such a tool is useful ..

Yes, if at least two additional requirements are satisfied:

- It should offer the possibility to measure response times. This
should be an easy one (except for the usual issues with time
measurements in distributed systems): just log the relevant
timestamps.
- It should not be limited to SOAP and it should not be specifically
targeted at Axis2: it should e.g. allow to add JDBC to the picture and
it should be easy to add support for other Web service stacks.

BTW, it may be a good idea to present the idea to other Apache
projects (CXF, ServiceMix, etc.) to see if they are interested in
supporting this project as well.
>
> Anyone cares to share their thoughts ?
>
> Nilupa
>
> On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>
>>
>> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:
>>>
>>> I was thinking about Java Swing based UI
>>
>> Swing is a good start. But if you look at some of the requirements like
>> monitoring multiple instances and correlate them, discussed in this thread,
>> it sounds as if we would be better off having a Web interface, rather than a
>> desktop client.
>>
>>>
>>> because the client may have
>>> to cache whatever messages it already has and will receive,
>>
>> If you are looking to support message-correlation, as discussed in some
>> replies in this thread above, you need to have a persistence storage (a DB
>> or something like that) rather than a memory cache.
>> Because, some message sequences, like those involved in an async
>> invocations, need to have a way to keep the messages, in there, longer, and
>> may be there is a server re-start, in the middle.
>> Of course, this requirements spend on the scope of this effort.
>> Plus, if you want to deal with historic data, and look at what was happening
>> to message sequences over time, you got to use a persistence layer. Again
>> this might be out of scope for this project, but if this is to be extended
>> to support these, you better think of this right now.
>>
>>>
>>> build the
>>> message flow and update the UI accordingly. Perhaps there is a better
>>> way ?
>>
>> We already do what you are trying to do, and much more with WSO2 BAM. I am
>> tempted to say that Shinding based gadget dashboard is really a compelling
>> UI for this.
>> I am not trying to take away you project or trying to hinder this effort at
>> all.
>> I think it will be a good idea for you to have a look at BAM and see what
>> the missing pieces are and figure out the right model for this tool.
>> Also, having had worked on WSO2 BAM project, I also have some understanding
>> on sort of problems that you might run into, when doing this. For e.g., if
>> you want to correlate message sequences in a high volume setup, you will be
>> swamped by the volume of messages, sooner than later. So correlation for the
>> purpose of monitoring is not an easy thing to do. Correlation, for the
>> purpose of debugging will not run into this problem, if you run this on a
>> controlled environment. However, what you really want to debug is the deal
>> clustered setup, and not a staged dev setup, the problem again surfaces.
>> You might want to list the objectives and then break them down
>> into monitoring and debugging spaces.
>> Samisa...
>>
>>>
>>> Thanks
>>> Nilupa.
>>>
>>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
>>> <sa...@gmail.com> wrote:
>>> > What UI technology would be used for this?
>>> >
>>> > Samisa...
>>> >
>>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
>>> > <an...@gmail.com> wrote:
>>> >> I was thinking more about something like ITCAM for SOA.
>>> >>
>>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>>> >> <sa...@opensource.lk> wrote:
>>> >>> That runtime monitoring / correlation is what the WSO2 Business
>>> >>> Activity
>>> >>> Monitor does / is for ...
>>> >>> See: http://wso2.com/products/business-activity-monitor/
>>> >>> Sanjiva.
>>> >>>
>>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
>>> >>> <an...@gmail.com>
>>> >>> wrote:
>>> >>>>
>>> >>>> I was actually referring to scenarios where a service may call other
>>> >>>> services, which in turn may call again other services. In these
>>> >>>> scenarios, it is not sufficient to simply collect received/sent
>>> >>>> messages on different hosts, unless the system is isolated
>>> >>>> (development environment) or the request rate is sufficiently low so
>>> >>>> that messages can be correlated based on time. Here is a concrete
>>> >>>> scenario from a project in my company:
>>> >>>>
>>> >>>> - Request comes in on an ESB that does security and validation.
>>> >>>> - Request is processed by an application server which persists the
>>> >>>> received information and publishes a JMS message (pub/sub events).
>>> >>>> - The event is consumed by one or more components that may in turn
>>> >>>> interact with other services.
>>> >>>>
>>> >>>> If you want track this flow, it is not sufficient to intercept the
>>> >>>> messages on the different hosts: in addition, you need to instrument
>>> >>>> the services so that the outgoing requests are correlated with the
>>> >>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> >>>> would like to be able to do in the above scenario is to enable
>>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> >>>> HTTP header to the initial request to enable logging and this
>>> >>>> information is propagated along the chain. Every service/component
>>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> >>>> central place where the sequence of events is reconstructed.
>>> >>>>
>>> >>>> One way this could be achieved is with a tool that postprocesses the
>>> >>>> artifacts from the build process. For each artifact, the tool would
>>> >>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> >>>> aspects and reassemble the artifact for deployment. The
>>> >>>> responsibility
>>> >>>> of the aspects would be to intercept messages, log them and modify
>>> >>>> them to ensure proper correlation. Of course this only works if the
>>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> >>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> >>>> in the transport test suites [1]. Interestingly, this approach would
>>> >>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> >>>> even intercept the database queries executed during the flow.
>>> >>>>
>>> >>>> Andreas
>>> >>>>
>>> >>>> [1]
>>> >>>>
>>> >>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>> >>>>
>>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
>>> >>>> wrote:
>>> >>>> > Hi Nilupa;
>>> >>>> >
>>> >>>> > When we collect messages from a one location by installing proper
>>> >>>> > handler that will intercept and send messages to to that one
>>> >>>> > location
>>> >>>> > (this one location can be a single server, pub/sub channel etc).
>>> >>>> > There
>>> >>>> > is many ways to make sense of those collected messages. What
>>> >>>> > Andreas
>>> >>>> > mentioned (following complete transaction) is a one possibility.
>>> >>>> >
>>> >>>> > I think you should come up with few scenarios on how you would make
>>> >>>> > sense of the message.
>>> >>>> >
>>> >>>> > Thanks
>>> >>>> > Srinath
>>> >>>> >
>>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>> >>>> > wrote:
>>> >>>> >> Hi,
>>> >>>> >>
>>> >>>> >> First let me thank you for commenting.
>>> >>>> >>
>>> >>>> >> As far as I understood, what you would like to see from the
>>> >>>> >> proposed
>>> >>>> >> tool is to view set of messages that are exchanged in reponse to a
>>> >>>> >> particular input message. With the understanding that I am having
>>> >>>> >> at
>>> >>>> >> the momnet, one way to do it is to filter out the central
>>> >>>> >> repository
>>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>> >>>> >> message chain from it. We can allow the client GUI wich connects
>>> >>>> >> to
>>> >>>> >> the central repository to provide the paramenters (For instance
>>> >>>> >> the
>>> >>>> >> value of 'To' header) from which an intelligent filtering can be
>>> >>>> >> done
>>> >>>> >> for the set of messages avialable at the central repository.
>>> >>>> >>
>>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
>>> >>>> >> willing
>>> >>>> >> to share it with us. It would be really nice to hear from them.
>>> >>>> >>
>>> >>>> >> Thanks in advance ..!!
>>> >>>> >>
>>> >>>> >> Best Regards,
>>> >>>> >> Nilupa
>>> >>>> >>
>>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> >>>> >> <an...@gmail.com> wrote:
>>> >>>> >>> Personally, I think that the added value of extending the SOAP
>>> >>>> >>> monitor
>>> >>>> >>> module to collect messages in a central place is too limited to
>>> >>>> >>> attract enough interest from the user community, so that it will
>>> >>>> >>> be
>>> >>>> >>> difficult to ensure the evolution of such a project in the
>>> >>>> >>> future.
>>> >>>> >>> However, what many people are looking for is a tool that allows
>>> >>>> >>> to
>>> >>>> >>> track the flow of a message through a distributed system, or more
>>> >>>> >>> generally to track the sequence of events triggered by a given
>>> >>>> >>> input
>>> >>>> >>> message (sort of end-to-end transaction monitoring).
>>> >>>> >>>
>>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>> >>>> >>> wrote:
>>> >>>> >>>> Hello,
>>> >>>> >>>>
>>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
>>> >>>> >>>> of
>>> >>>> >>>> proposing a GSoC project this summer. I would like to take
>>> >>>> >>>> project
>>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
>>> >>>> >>>> and
>>> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
>>> >>>> >>>> Web
>>> >>>> >>>> services middleware. I will submit more detailed proposal later.
>>> >>>> >>>> I
>>> >>>> >>>> would really like to hear any feedback from you. Any suggestions
>>> >>>> >>>> or
>>> >>>> >>>> features that you would like to see in a "Distributed TCP
>>> >>>> >>>> Monitor"
>>> >>>> >>>> are
>>> >>>> >>>> most welcome.
>>> >>>> >>>>
>>> >>>> >>>> Best Regards,
>>> >>>> >>>> Nilupa Bandara
>>> >>>> >>>>
>>> >>>> >>>> [1]
>>> >>>> >>>>
>>> >>>> >>>> Distributed TCP monitor
>>> >>>> >>>>
>>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
>>> >>>> >>>> hard
>>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
>>> >>>> >>>> the
>>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
>>> >>>> >>>> messages
>>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
>>> >>>> >>>> can
>>> >>>> >>>> pull those messages. Also, we can do something to turn the
>>> >>>> >>>> module on
>>> >>>> >>>> and off remotely, so users can have his module deployed, but
>>> >>>> >>>> turn it
>>> >>>> >>>> on only when they want to debug the system.
>>> >>>> >>>>
>>> >>>> >>>> Then we can take this to next level by adding this module to all
>>> >>>> >>>> services in a system , configuring modules to send all collected
>>> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
>>> >>>> >>>> via
>>> >>>> >>>> a
>>> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>> >>>> >>>> TCPMon for a whole system.
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>> ---------------------------------------------------------------------
>>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>> ---------------------------------------------------------------------
>>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>
>>> >>>> >>
>>> >>>> >> ---------------------------------------------------------------------
>>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>
>>> >>>> >>
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> > --
>>> >>>> > ============================
>>> >>>> > Srinath Perera, Ph.D.
>>> >>>> >   WSO2 Inc. http://wso2.com
>>> >>>> >   Blog: http://srinathsview.blogspot.com/
>>> >>>> >
>>> >>>> >
>>> >>>> > ---------------------------------------------------------------------
>>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >
>>> >>>> >
>>> >>>>
>>> >>>> ---------------------------------------------------------------------
>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Sanjiva Weerawarana, Ph.D.
>>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>>> >>> http://www.opensource.lk/
>>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>>> >>> Member; Apache Software Foundation; http://www.apache.org/
>>> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>>> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>> >>>
>>> >>> Blog: http://sanjiva.weerawarana.org/
>>> >>>
>>> >>
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>>
>
>
>
> --
> Sanka Samaranayake
>
> http://sankas.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanka Samaranayake <ss...@gmail.com>.

Samisa,

Quick question, does WSO2 BAM project provide such a debugging
facility. I mean can one monitor what is going on for a specific input
in SOA senario ?

Sanka



On Thu, Apr 1, 2010 at 3:28 AM, Sanka Samaranayake <ss...@gmail.com> wrote:
> Hi Samisa,
>
> I think the this scope of this project would be to provide a
> light-weight tool which allows to monitor the message flows, execution
> in each node in response to those message flows to a given input
> message primarily for debugging purposes . (Just like the way tcpmon
> is used in developing Web services)
>
> And I guess now the question is whether such a tool is useful ..
>
> Anyone cares to share their thoughts ?
>
> Nilupa
>
> On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>
>>
>> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:
>>>
>>> I was thinking about Java Swing based UI
>>
>> Swing is a good start. But if you look at some of the requirements like
>> monitoring multiple instances and correlate them, discussed in this thread,
>> it sounds as if we would be better off having a Web interface, rather than a
>> desktop client.
>>
>>>
>>> because the client may have
>>> to cache whatever messages it already has and will receive,
>>
>> If you are looking to support message-correlation, as discussed in some
>> replies in this thread above, you need to have a persistence storage (a DB
>> or something like that) rather than a memory cache.
>> Because, some message sequences, like those involved in an async
>> invocations, need to have a way to keep the messages, in there, longer, and
>> may be there is a server re-start, in the middle.
>> Of course, this requirements spend on the scope of this effort.
>> Plus, if you want to deal with historic data, and look at what was happening
>> to message sequences over time, you got to use a persistence layer. Again
>> this might be out of scope for this project, but if this is to be extended
>> to support these, you better think of this right now.
>>
>>>
>>> build the
>>> message flow and update the UI accordingly. Perhaps there is a better
>>> way ?
>>
>> We already do what you are trying to do, and much more with WSO2 BAM. I am
>> tempted to say that Shinding based gadget dashboard is really a compelling
>> UI for this.
>> I am not trying to take away you project or trying to hinder this effort at
>> all.
>> I think it will be a good idea for you to have a look at BAM and see what
>> the missing pieces are and figure out the right model for this tool.
>> Also, having had worked on WSO2 BAM project, I also have some understanding
>> on sort of problems that you might run into, when doing this. For e.g., if
>> you want to correlate message sequences in a high volume setup, you will be
>> swamped by the volume of messages, sooner than later. So correlation for the
>> purpose of monitoring is not an easy thing to do. Correlation, for the
>> purpose of debugging will not run into this problem, if you run this on a
>> controlled environment. However, what you really want to debug is the deal
>> clustered setup, and not a staged dev setup, the problem again surfaces.
>> You might want to list the objectives and then break them down
>> into monitoring and debugging spaces.
>> Samisa...
>>
>>>
>>> Thanks
>>> Nilupa.
>>>
>>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
>>> <sa...@gmail.com> wrote:
>>> > What UI technology would be used for this?
>>> >
>>> > Samisa...
>>> >
>>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
>>> > <an...@gmail.com> wrote:
>>> >> I was thinking more about something like ITCAM for SOA.
>>> >>
>>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>>> >> <sa...@opensource.lk> wrote:
>>> >>> That runtime monitoring / correlation is what the WSO2 Business
>>> >>> Activity
>>> >>> Monitor does / is for ...
>>> >>> See: http://wso2.com/products/business-activity-monitor/
>>> >>> Sanjiva.
>>> >>>
>>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
>>> >>> <an...@gmail.com>
>>> >>> wrote:
>>> >>>>
>>> >>>> I was actually referring to scenarios where a service may call other
>>> >>>> services, which in turn may call again other services. In these
>>> >>>> scenarios, it is not sufficient to simply collect received/sent
>>> >>>> messages on different hosts, unless the system is isolated
>>> >>>> (development environment) or the request rate is sufficiently low so
>>> >>>> that messages can be correlated based on time. Here is a concrete
>>> >>>> scenario from a project in my company:
>>> >>>>
>>> >>>> - Request comes in on an ESB that does security and validation.
>>> >>>> - Request is processed by an application server which persists the
>>> >>>> received information and publishes a JMS message (pub/sub events).
>>> >>>> - The event is consumed by one or more components that may in turn
>>> >>>> interact with other services.
>>> >>>>
>>> >>>> If you want track this flow, it is not sufficient to intercept the
>>> >>>> messages on the different hosts: in addition, you need to instrument
>>> >>>> the services so that the outgoing requests are correlated with the
>>> >>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> >>>> would like to be able to do in the above scenario is to enable
>>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> >>>> HTTP header to the initial request to enable logging and this
>>> >>>> information is propagated along the chain. Every service/component
>>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> >>>> central place where the sequence of events is reconstructed.
>>> >>>>
>>> >>>> One way this could be achieved is with a tool that postprocesses the
>>> >>>> artifacts from the build process. For each artifact, the tool would
>>> >>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> >>>> aspects and reassemble the artifact for deployment. The
>>> >>>> responsibility
>>> >>>> of the aspects would be to intercept messages, log them and modify
>>> >>>> them to ensure proper correlation. Of course this only works if the
>>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> >>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> >>>> in the transport test suites [1]. Interestingly, this approach would
>>> >>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> >>>> even intercept the database queries executed during the flow.
>>> >>>>
>>> >>>> Andreas
>>> >>>>
>>> >>>> [1]
>>> >>>>
>>> >>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>> >>>>
>>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
>>> >>>> wrote:
>>> >>>> > Hi Nilupa;
>>> >>>> >
>>> >>>> > When we collect messages from a one location by installing proper
>>> >>>> > handler that will intercept and send messages to to that one
>>> >>>> > location
>>> >>>> > (this one location can be a single server, pub/sub channel etc).
>>> >>>> > There
>>> >>>> > is many ways to make sense of those collected messages. What
>>> >>>> > Andreas
>>> >>>> > mentioned (following complete transaction) is a one possibility.
>>> >>>> >
>>> >>>> > I think you should come up with few scenarios on how you would make
>>> >>>> > sense of the message.
>>> >>>> >
>>> >>>> > Thanks
>>> >>>> > Srinath
>>> >>>> >
>>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>> >>>> > wrote:
>>> >>>> >> Hi,
>>> >>>> >>
>>> >>>> >> First let me thank you for commenting.
>>> >>>> >>
>>> >>>> >> As far as I understood, what you would like to see from the
>>> >>>> >> proposed
>>> >>>> >> tool is to view set of messages that are exchanged in reponse to a
>>> >>>> >> particular input message. With the understanding that I am having
>>> >>>> >> at
>>> >>>> >> the momnet, one way to do it is to filter out the central
>>> >>>> >> repository
>>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>> >>>> >> message chain from it. We can allow the client GUI wich connects
>>> >>>> >> to
>>> >>>> >> the central repository to provide the paramenters (For instance
>>> >>>> >> the
>>> >>>> >> value of 'To' header) from which an intelligent filtering can be
>>> >>>> >> done
>>> >>>> >> for the set of messages avialable at the central repository.
>>> >>>> >>
>>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
>>> >>>> >> willing
>>> >>>> >> to share it with us. It would be really nice to hear from them.
>>> >>>> >>
>>> >>>> >> Thanks in advance ..!!
>>> >>>> >>
>>> >>>> >> Best Regards,
>>> >>>> >> Nilupa
>>> >>>> >>
>>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> >>>> >> <an...@gmail.com> wrote:
>>> >>>> >>> Personally, I think that the added value of extending the SOAP
>>> >>>> >>> monitor
>>> >>>> >>> module to collect messages in a central place is too limited to
>>> >>>> >>> attract enough interest from the user community, so that it will
>>> >>>> >>> be
>>> >>>> >>> difficult to ensure the evolution of such a project in the
>>> >>>> >>> future.
>>> >>>> >>> However, what many people are looking for is a tool that allows
>>> >>>> >>> to
>>> >>>> >>> track the flow of a message through a distributed system, or more
>>> >>>> >>> generally to track the sequence of events triggered by a given
>>> >>>> >>> input
>>> >>>> >>> message (sort of end-to-end transaction monitoring).
>>> >>>> >>>
>>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>> >>>> >>> wrote:
>>> >>>> >>>> Hello,
>>> >>>> >>>>
>>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
>>> >>>> >>>> of
>>> >>>> >>>> proposing a GSoC project this summer. I would like to take
>>> >>>> >>>> project
>>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
>>> >>>> >>>> and
>>> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
>>> >>>> >>>> Web
>>> >>>> >>>> services middleware. I will submit more detailed proposal later.
>>> >>>> >>>> I
>>> >>>> >>>> would really like to hear any feedback from you. Any suggestions
>>> >>>> >>>> or
>>> >>>> >>>> features that you would like to see in a "Distributed TCP
>>> >>>> >>>> Monitor"
>>> >>>> >>>> are
>>> >>>> >>>> most welcome.
>>> >>>> >>>>
>>> >>>> >>>> Best Regards,
>>> >>>> >>>> Nilupa Bandara
>>> >>>> >>>>
>>> >>>> >>>> [1]
>>> >>>> >>>>
>>> >>>> >>>> Distributed TCP monitor
>>> >>>> >>>>
>>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
>>> >>>> >>>> hard
>>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
>>> >>>> >>>> the
>>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
>>> >>>> >>>> messages
>>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
>>> >>>> >>>> can
>>> >>>> >>>> pull those messages. Also, we can do something to turn the
>>> >>>> >>>> module on
>>> >>>> >>>> and off remotely, so users can have his module deployed, but
>>> >>>> >>>> turn it
>>> >>>> >>>> on only when they want to debug the system.
>>> >>>> >>>>
>>> >>>> >>>> Then we can take this to next level by adding this module to all
>>> >>>> >>>> services in a system , configuring modules to send all collected
>>> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
>>> >>>> >>>> via
>>> >>>> >>>> a
>>> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>> >>>> >>>> TCPMon for a whole system.
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>> ---------------------------------------------------------------------
>>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>> ---------------------------------------------------------------------
>>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>
>>> >>>> >>
>>> >>>> >> ---------------------------------------------------------------------
>>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>
>>> >>>> >>
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> > --
>>> >>>> > ============================
>>> >>>> > Srinath Perera, Ph.D.
>>> >>>> >   WSO2 Inc. http://wso2.com
>>> >>>> >   Blog: http://srinathsview.blogspot.com/
>>> >>>> >
>>> >>>> >
>>> >>>> > ---------------------------------------------------------------------
>>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >
>>> >>>> >
>>> >>>>
>>> >>>> ---------------------------------------------------------------------
>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Sanjiva Weerawarana, Ph.D.
>>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>>> >>> http://www.opensource.lk/
>>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>>> >>> Member; Apache Software Foundation; http://www.apache.org/
>>> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>>> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>> >>>
>>> >>> Blog: http://sanjiva.weerawarana.org/
>>> >>>
>>> >>
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>>
>
>
>
> --
> Sanka Samaranayake
>
> http://sankas.blogspot.com/
>



-- 
Sanka Samaranayake

http://sankas.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

On Thu, Apr 1, 2010 at 03:28, Sanka Samaranayake <ss...@gmail.com> wrote:
> Hi Samisa,
>
> I think the this scope of this project would be to provide a
> light-weight tool which allows to monitor the message flows, execution
> in each node in response to those message flows to a given input
> message primarily for debugging purposes . (Just like the way tcpmon
> is used in developing Web services)
>
> And I guess now the question is whether such a tool is useful ..

Yes, if at least two additional requirements are satisfied:

- It should offer the possibility to measure response times. This
should be an easy one (except for the usual issues with time
measurements in distributed systems): just log the relevant
timestamps.
- It should not be limited to SOAP and it should not be specifically
targeted at Axis2: it should e.g. allow to add JDBC to the picture and
it should be easy to add support for other Web service stacks.

BTW, it may be a good idea to present the idea to other Apache
projects (CXF, ServiceMix, etc.) to see if they are interested in
supporting this project as well.
>
> Anyone cares to share their thoughts ?
>
> Nilupa
>
> On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>
>>
>> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:
>>>
>>> I was thinking about Java Swing based UI
>>
>> Swing is a good start. But if you look at some of the requirements like
>> monitoring multiple instances and correlate them, discussed in this thread,
>> it sounds as if we would be better off having a Web interface, rather than a
>> desktop client.
>>
>>>
>>> because the client may have
>>> to cache whatever messages it already has and will receive,
>>
>> If you are looking to support message-correlation, as discussed in some
>> replies in this thread above, you need to have a persistence storage (a DB
>> or something like that) rather than a memory cache.
>> Because, some message sequences, like those involved in an async
>> invocations, need to have a way to keep the messages, in there, longer, and
>> may be there is a server re-start, in the middle.
>> Of course, this requirements spend on the scope of this effort.
>> Plus, if you want to deal with historic data, and look at what was happening
>> to message sequences over time, you got to use a persistence layer. Again
>> this might be out of scope for this project, but if this is to be extended
>> to support these, you better think of this right now.
>>
>>>
>>> build the
>>> message flow and update the UI accordingly. Perhaps there is a better
>>> way ?
>>
>> We already do what you are trying to do, and much more with WSO2 BAM. I am
>> tempted to say that Shinding based gadget dashboard is really a compelling
>> UI for this.
>> I am not trying to take away you project or trying to hinder this effort at
>> all.
>> I think it will be a good idea for you to have a look at BAM and see what
>> the missing pieces are and figure out the right model for this tool.
>> Also, having had worked on WSO2 BAM project, I also have some understanding
>> on sort of problems that you might run into, when doing this. For e.g., if
>> you want to correlate message sequences in a high volume setup, you will be
>> swamped by the volume of messages, sooner than later. So correlation for the
>> purpose of monitoring is not an easy thing to do. Correlation, for the
>> purpose of debugging will not run into this problem, if you run this on a
>> controlled environment. However, what you really want to debug is the deal
>> clustered setup, and not a staged dev setup, the problem again surfaces.
>> You might want to list the objectives and then break them down
>> into monitoring and debugging spaces.
>> Samisa...
>>
>>>
>>> Thanks
>>> Nilupa.
>>>
>>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
>>> <sa...@gmail.com> wrote:
>>> > What UI technology would be used for this?
>>> >
>>> > Samisa...
>>> >
>>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
>>> > <an...@gmail.com> wrote:
>>> >> I was thinking more about something like ITCAM for SOA.
>>> >>
>>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>>> >> <sa...@opensource.lk> wrote:
>>> >>> That runtime monitoring / correlation is what the WSO2 Business
>>> >>> Activity
>>> >>> Monitor does / is for ...
>>> >>> See: http://wso2.com/products/business-activity-monitor/
>>> >>> Sanjiva.
>>> >>>
>>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
>>> >>> <an...@gmail.com>
>>> >>> wrote:
>>> >>>>
>>> >>>> I was actually referring to scenarios where a service may call other
>>> >>>> services, which in turn may call again other services. In these
>>> >>>> scenarios, it is not sufficient to simply collect received/sent
>>> >>>> messages on different hosts, unless the system is isolated
>>> >>>> (development environment) or the request rate is sufficiently low so
>>> >>>> that messages can be correlated based on time. Here is a concrete
>>> >>>> scenario from a project in my company:
>>> >>>>
>>> >>>> - Request comes in on an ESB that does security and validation.
>>> >>>> - Request is processed by an application server which persists the
>>> >>>> received information and publishes a JMS message (pub/sub events).
>>> >>>> - The event is consumed by one or more components that may in turn
>>> >>>> interact with other services.
>>> >>>>
>>> >>>> If you want track this flow, it is not sufficient to intercept the
>>> >>>> messages on the different hosts: in addition, you need to instrument
>>> >>>> the services so that the outgoing requests are correlated with the
>>> >>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> >>>> would like to be able to do in the above scenario is to enable
>>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> >>>> HTTP header to the initial request to enable logging and this
>>> >>>> information is propagated along the chain. Every service/component
>>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> >>>> central place where the sequence of events is reconstructed.
>>> >>>>
>>> >>>> One way this could be achieved is with a tool that postprocesses the
>>> >>>> artifacts from the build process. For each artifact, the tool would
>>> >>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> >>>> aspects and reassemble the artifact for deployment. The
>>> >>>> responsibility
>>> >>>> of the aspects would be to intercept messages, log them and modify
>>> >>>> them to ensure proper correlation. Of course this only works if the
>>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> >>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> >>>> in the transport test suites [1]. Interestingly, this approach would
>>> >>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> >>>> even intercept the database queries executed during the flow.
>>> >>>>
>>> >>>> Andreas
>>> >>>>
>>> >>>> [1]
>>> >>>>
>>> >>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>> >>>>
>>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
>>> >>>> wrote:
>>> >>>> > Hi Nilupa;
>>> >>>> >
>>> >>>> > When we collect messages from a one location by installing proper
>>> >>>> > handler that will intercept and send messages to to that one
>>> >>>> > location
>>> >>>> > (this one location can be a single server, pub/sub channel etc).
>>> >>>> > There
>>> >>>> > is many ways to make sense of those collected messages. What
>>> >>>> > Andreas
>>> >>>> > mentioned (following complete transaction) is a one possibility.
>>> >>>> >
>>> >>>> > I think you should come up with few scenarios on how you would make
>>> >>>> > sense of the message.
>>> >>>> >
>>> >>>> > Thanks
>>> >>>> > Srinath
>>> >>>> >
>>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>> >>>> > wrote:
>>> >>>> >> Hi,
>>> >>>> >>
>>> >>>> >> First let me thank you for commenting.
>>> >>>> >>
>>> >>>> >> As far as I understood, what you would like to see from the
>>> >>>> >> proposed
>>> >>>> >> tool is to view set of messages that are exchanged in reponse to a
>>> >>>> >> particular input message. With the understanding that I am having
>>> >>>> >> at
>>> >>>> >> the momnet, one way to do it is to filter out the central
>>> >>>> >> repository
>>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>> >>>> >> message chain from it. We can allow the client GUI wich connects
>>> >>>> >> to
>>> >>>> >> the central repository to provide the paramenters (For instance
>>> >>>> >> the
>>> >>>> >> value of 'To' header) from which an intelligent filtering can be
>>> >>>> >> done
>>> >>>> >> for the set of messages avialable at the central repository.
>>> >>>> >>
>>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
>>> >>>> >> willing
>>> >>>> >> to share it with us. It would be really nice to hear from them.
>>> >>>> >>
>>> >>>> >> Thanks in advance ..!!
>>> >>>> >>
>>> >>>> >> Best Regards,
>>> >>>> >> Nilupa
>>> >>>> >>
>>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> >>>> >> <an...@gmail.com> wrote:
>>> >>>> >>> Personally, I think that the added value of extending the SOAP
>>> >>>> >>> monitor
>>> >>>> >>> module to collect messages in a central place is too limited to
>>> >>>> >>> attract enough interest from the user community, so that it will
>>> >>>> >>> be
>>> >>>> >>> difficult to ensure the evolution of such a project in the
>>> >>>> >>> future.
>>> >>>> >>> However, what many people are looking for is a tool that allows
>>> >>>> >>> to
>>> >>>> >>> track the flow of a message through a distributed system, or more
>>> >>>> >>> generally to track the sequence of events triggered by a given
>>> >>>> >>> input
>>> >>>> >>> message (sort of end-to-end transaction monitoring).
>>> >>>> >>>
>>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>> >>>> >>> wrote:
>>> >>>> >>>> Hello,
>>> >>>> >>>>
>>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
>>> >>>> >>>> of
>>> >>>> >>>> proposing a GSoC project this summer. I would like to take
>>> >>>> >>>> project
>>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
>>> >>>> >>>> and
>>> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
>>> >>>> >>>> Web
>>> >>>> >>>> services middleware. I will submit more detailed proposal later.
>>> >>>> >>>> I
>>> >>>> >>>> would really like to hear any feedback from you. Any suggestions
>>> >>>> >>>> or
>>> >>>> >>>> features that you would like to see in a "Distributed TCP
>>> >>>> >>>> Monitor"
>>> >>>> >>>> are
>>> >>>> >>>> most welcome.
>>> >>>> >>>>
>>> >>>> >>>> Best Regards,
>>> >>>> >>>> Nilupa Bandara
>>> >>>> >>>>
>>> >>>> >>>> [1]
>>> >>>> >>>>
>>> >>>> >>>> Distributed TCP monitor
>>> >>>> >>>>
>>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
>>> >>>> >>>> hard
>>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
>>> >>>> >>>> the
>>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
>>> >>>> >>>> messages
>>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
>>> >>>> >>>> can
>>> >>>> >>>> pull those messages. Also, we can do something to turn the
>>> >>>> >>>> module on
>>> >>>> >>>> and off remotely, so users can have his module deployed, but
>>> >>>> >>>> turn it
>>> >>>> >>>> on only when they want to debug the system.
>>> >>>> >>>>
>>> >>>> >>>> Then we can take this to next level by adding this module to all
>>> >>>> >>>> services in a system , configuring modules to send all collected
>>> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
>>> >>>> >>>> via
>>> >>>> >>>> a
>>> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>> >>>> >>>> TCPMon for a whole system.
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>> ---------------------------------------------------------------------
>>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>> ---------------------------------------------------------------------
>>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>
>>> >>>> >>
>>> >>>> >> ---------------------------------------------------------------------
>>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>
>>> >>>> >>
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> > --
>>> >>>> > ============================
>>> >>>> > Srinath Perera, Ph.D.
>>> >>>> >   WSO2 Inc. http://wso2.com
>>> >>>> >   Blog: http://srinathsview.blogspot.com/
>>> >>>> >
>>> >>>> >
>>> >>>> > ---------------------------------------------------------------------
>>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >
>>> >>>> >
>>> >>>>
>>> >>>> ---------------------------------------------------------------------
>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Sanjiva Weerawarana, Ph.D.
>>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>>> >>> http://www.opensource.lk/
>>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>>> >>> Member; Apache Software Foundation; http://www.apache.org/
>>> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>>> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>> >>>
>>> >>> Blog: http://sanjiva.weerawarana.org/
>>> >>>
>>> >>
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>>
>
>
>
> --
> Sanka Samaranayake
>
> http://sankas.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanka Samaranayake <ss...@gmail.com>.

Samisa,

Quick question, does WSO2 BAM project provide such a debugging
facility. I mean can one monitor what is going on for a specific input
in SOA senario ?

Sanka



On Thu, Apr 1, 2010 at 3:28 AM, Sanka Samaranayake <ss...@gmail.com> wrote:
> Hi Samisa,
>
> I think the this scope of this project would be to provide a
> light-weight tool which allows to monitor the message flows, execution
> in each node in response to those message flows to a given input
> message primarily for debugging purposes . (Just like the way tcpmon
> is used in developing Web services)
>
> And I guess now the question is whether such a tool is useful ..
>
> Anyone cares to share their thoughts ?
>
> Nilupa
>
> On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
>>
>>
>> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:
>>>
>>> I was thinking about Java Swing based UI
>>
>> Swing is a good start. But if you look at some of the requirements like
>> monitoring multiple instances and correlate them, discussed in this thread,
>> it sounds as if we would be better off having a Web interface, rather than a
>> desktop client.
>>
>>>
>>> because the client may have
>>> to cache whatever messages it already has and will receive,
>>
>> If you are looking to support message-correlation, as discussed in some
>> replies in this thread above, you need to have a persistence storage (a DB
>> or something like that) rather than a memory cache.
>> Because, some message sequences, like those involved in an async
>> invocations, need to have a way to keep the messages, in there, longer, and
>> may be there is a server re-start, in the middle.
>> Of course, this requirements spend on the scope of this effort.
>> Plus, if you want to deal with historic data, and look at what was happening
>> to message sequences over time, you got to use a persistence layer. Again
>> this might be out of scope for this project, but if this is to be extended
>> to support these, you better think of this right now.
>>
>>>
>>> build the
>>> message flow and update the UI accordingly. Perhaps there is a better
>>> way ?
>>
>> We already do what you are trying to do, and much more with WSO2 BAM. I am
>> tempted to say that Shinding based gadget dashboard is really a compelling
>> UI for this.
>> I am not trying to take away you project or trying to hinder this effort at
>> all.
>> I think it will be a good idea for you to have a look at BAM and see what
>> the missing pieces are and figure out the right model for this tool.
>> Also, having had worked on WSO2 BAM project, I also have some understanding
>> on sort of problems that you might run into, when doing this. For e.g., if
>> you want to correlate message sequences in a high volume setup, you will be
>> swamped by the volume of messages, sooner than later. So correlation for the
>> purpose of monitoring is not an easy thing to do. Correlation, for the
>> purpose of debugging will not run into this problem, if you run this on a
>> controlled environment. However, what you really want to debug is the deal
>> clustered setup, and not a staged dev setup, the problem again surfaces.
>> You might want to list the objectives and then break them down
>> into monitoring and debugging spaces.
>> Samisa...
>>
>>>
>>> Thanks
>>> Nilupa.
>>>
>>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
>>> <sa...@gmail.com> wrote:
>>> > What UI technology would be used for this?
>>> >
>>> > Samisa...
>>> >
>>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
>>> > <an...@gmail.com> wrote:
>>> >> I was thinking more about something like ITCAM for SOA.
>>> >>
>>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>>> >> <sa...@opensource.lk> wrote:
>>> >>> That runtime monitoring / correlation is what the WSO2 Business
>>> >>> Activity
>>> >>> Monitor does / is for ...
>>> >>> See: http://wso2.com/products/business-activity-monitor/
>>> >>> Sanjiva.
>>> >>>
>>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
>>> >>> <an...@gmail.com>
>>> >>> wrote:
>>> >>>>
>>> >>>> I was actually referring to scenarios where a service may call other
>>> >>>> services, which in turn may call again other services. In these
>>> >>>> scenarios, it is not sufficient to simply collect received/sent
>>> >>>> messages on different hosts, unless the system is isolated
>>> >>>> (development environment) or the request rate is sufficiently low so
>>> >>>> that messages can be correlated based on time. Here is a concrete
>>> >>>> scenario from a project in my company:
>>> >>>>
>>> >>>> - Request comes in on an ESB that does security and validation.
>>> >>>> - Request is processed by an application server which persists the
>>> >>>> received information and publishes a JMS message (pub/sub events).
>>> >>>> - The event is consumed by one or more components that may in turn
>>> >>>> interact with other services.
>>> >>>>
>>> >>>> If you want track this flow, it is not sufficient to intercept the
>>> >>>> messages on the different hosts: in addition, you need to instrument
>>> >>>> the services so that the outgoing requests are correlated with the
>>> >>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> >>>> would like to be able to do in the above scenario is to enable
>>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> >>>> HTTP header to the initial request to enable logging and this
>>> >>>> information is propagated along the chain. Every service/component
>>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> >>>> central place where the sequence of events is reconstructed.
>>> >>>>
>>> >>>> One way this could be achieved is with a tool that postprocesses the
>>> >>>> artifacts from the build process. For each artifact, the tool would
>>> >>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> >>>> aspects and reassemble the artifact for deployment. The
>>> >>>> responsibility
>>> >>>> of the aspects would be to intercept messages, log them and modify
>>> >>>> them to ensure proper correlation. Of course this only works if the
>>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> >>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> >>>> in the transport test suites [1]. Interestingly, this approach would
>>> >>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> >>>> even intercept the database queries executed during the flow.
>>> >>>>
>>> >>>> Andreas
>>> >>>>
>>> >>>> [1]
>>> >>>>
>>> >>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>> >>>>
>>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
>>> >>>> wrote:
>>> >>>> > Hi Nilupa;
>>> >>>> >
>>> >>>> > When we collect messages from a one location by installing proper
>>> >>>> > handler that will intercept and send messages to to that one
>>> >>>> > location
>>> >>>> > (this one location can be a single server, pub/sub channel etc).
>>> >>>> > There
>>> >>>> > is many ways to make sense of those collected messages. What
>>> >>>> > Andreas
>>> >>>> > mentioned (following complete transaction) is a one possibility.
>>> >>>> >
>>> >>>> > I think you should come up with few scenarios on how you would make
>>> >>>> > sense of the message.
>>> >>>> >
>>> >>>> > Thanks
>>> >>>> > Srinath
>>> >>>> >
>>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>> >>>> > wrote:
>>> >>>> >> Hi,
>>> >>>> >>
>>> >>>> >> First let me thank you for commenting.
>>> >>>> >>
>>> >>>> >> As far as I understood, what you would like to see from the
>>> >>>> >> proposed
>>> >>>> >> tool is to view set of messages that are exchanged in reponse to a
>>> >>>> >> particular input message. With the understanding that I am having
>>> >>>> >> at
>>> >>>> >> the momnet, one way to do it is to filter out the central
>>> >>>> >> repository
>>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>> >>>> >> message chain from it. We can allow the client GUI wich connects
>>> >>>> >> to
>>> >>>> >> the central repository to provide the paramenters (For instance
>>> >>>> >> the
>>> >>>> >> value of 'To' header) from which an intelligent filtering can be
>>> >>>> >> done
>>> >>>> >> for the set of messages avialable at the central repository.
>>> >>>> >>
>>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
>>> >>>> >> willing
>>> >>>> >> to share it with us. It would be really nice to hear from them.
>>> >>>> >>
>>> >>>> >> Thanks in advance ..!!
>>> >>>> >>
>>> >>>> >> Best Regards,
>>> >>>> >> Nilupa
>>> >>>> >>
>>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> >>>> >> <an...@gmail.com> wrote:
>>> >>>> >>> Personally, I think that the added value of extending the SOAP
>>> >>>> >>> monitor
>>> >>>> >>> module to collect messages in a central place is too limited to
>>> >>>> >>> attract enough interest from the user community, so that it will
>>> >>>> >>> be
>>> >>>> >>> difficult to ensure the evolution of such a project in the
>>> >>>> >>> future.
>>> >>>> >>> However, what many people are looking for is a tool that allows
>>> >>>> >>> to
>>> >>>> >>> track the flow of a message through a distributed system, or more
>>> >>>> >>> generally to track the sequence of events triggered by a given
>>> >>>> >>> input
>>> >>>> >>> message (sort of end-to-end transaction monitoring).
>>> >>>> >>>
>>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>> >>>> >>> wrote:
>>> >>>> >>>> Hello,
>>> >>>> >>>>
>>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
>>> >>>> >>>> of
>>> >>>> >>>> proposing a GSoC project this summer. I would like to take
>>> >>>> >>>> project
>>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
>>> >>>> >>>> and
>>> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
>>> >>>> >>>> Web
>>> >>>> >>>> services middleware. I will submit more detailed proposal later.
>>> >>>> >>>> I
>>> >>>> >>>> would really like to hear any feedback from you. Any suggestions
>>> >>>> >>>> or
>>> >>>> >>>> features that you would like to see in a "Distributed TCP
>>> >>>> >>>> Monitor"
>>> >>>> >>>> are
>>> >>>> >>>> most welcome.
>>> >>>> >>>>
>>> >>>> >>>> Best Regards,
>>> >>>> >>>> Nilupa Bandara
>>> >>>> >>>>
>>> >>>> >>>> [1]
>>> >>>> >>>>
>>> >>>> >>>> Distributed TCP monitor
>>> >>>> >>>>
>>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
>>> >>>> >>>> hard
>>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
>>> >>>> >>>> the
>>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
>>> >>>> >>>> messages
>>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
>>> >>>> >>>> can
>>> >>>> >>>> pull those messages. Also, we can do something to turn the
>>> >>>> >>>> module on
>>> >>>> >>>> and off remotely, so users can have his module deployed, but
>>> >>>> >>>> turn it
>>> >>>> >>>> on only when they want to debug the system.
>>> >>>> >>>>
>>> >>>> >>>> Then we can take this to next level by adding this module to all
>>> >>>> >>>> services in a system , configuring modules to send all collected
>>> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
>>> >>>> >>>> via
>>> >>>> >>>> a
>>> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>> >>>> >>>> TCPMon for a whole system.
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>> ---------------------------------------------------------------------
>>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>> ---------------------------------------------------------------------
>>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>>
>>> >>>> >>>
>>> >>>> >>
>>> >>>> >>
>>> >>>> >> ---------------------------------------------------------------------
>>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >>
>>> >>>> >>
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> > --
>>> >>>> > ============================
>>> >>>> > Srinath Perera, Ph.D.
>>> >>>> >   WSO2 Inc. http://wso2.com
>>> >>>> >   Blog: http://srinathsview.blogspot.com/
>>> >>>> >
>>> >>>> >
>>> >>>> > ---------------------------------------------------------------------
>>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>> >
>>> >>>> >
>>> >>>>
>>> >>>> ---------------------------------------------------------------------
>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Sanjiva Weerawarana, Ph.D.
>>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>>> >>> http://www.opensource.lk/
>>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>>> >>> Member; Apache Software Foundation; http://www.apache.org/
>>> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>>> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>> >>>
>>> >>> Blog: http://sanjiva.weerawarana.org/
>>> >>>
>>> >>
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>> Samisa...
>> --
>> blog: http://samisa-abeysinghe.blogspot.com/
>>
>
>
>
> --
> Sanka Samaranayake
>
> http://sankas.blogspot.com/
>



-- 
Sanka Samaranayake

http://sankas.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanka Samaranayake <ss...@gmail.com>.

Hi Samisa,

I think the this scope of this project would be to provide a
light-weight tool which allows to monitor the message flows, execution
in each node in response to those message flows to a given input
message primarily for debugging purposes . (Just like the way tcpmon
is used in developing Web services)

And I guess now the question is whether such a tool is useful ..

Anyone cares to share their thoughts ?

Nilupa

On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>
>
> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:
>>
>> I was thinking about Java Swing based UI
>
> Swing is a good start. But if you look at some of the requirements like
> monitoring multiple instances and correlate them, discussed in this thread,
> it sounds as if we would be better off having a Web interface, rather than a
> desktop client.
>
>>
>> because the client may have
>> to cache whatever messages it already has and will receive,
>
> If you are looking to support message-correlation, as discussed in some
> replies in this thread above, you need to have a persistence storage (a DB
> or something like that) rather than a memory cache.
> Because, some message sequences, like those involved in an async
> invocations, need to have a way to keep the messages, in there, longer, and
> may be there is a server re-start, in the middle.
> Of course, this requirements spend on the scope of this effort.
> Plus, if you want to deal with historic data, and look at what was happening
> to message sequences over time, you got to use a persistence layer. Again
> this might be out of scope for this project, but if this is to be extended
> to support these, you better think of this right now.
>
>>
>> build the
>> message flow and update the UI accordingly. Perhaps there is a better
>> way ?
>
> We already do what you are trying to do, and much more with WSO2 BAM. I am
> tempted to say that Shinding based gadget dashboard is really a compelling
> UI for this.
> I am not trying to take away you project or trying to hinder this effort at
> all.
> I think it will be a good idea for you to have a look at BAM and see what
> the missing pieces are and figure out the right model for this tool.
> Also, having had worked on WSO2 BAM project, I also have some understanding
> on sort of problems that you might run into, when doing this. For e.g., if
> you want to correlate message sequences in a high volume setup, you will be
> swamped by the volume of messages, sooner than later. So correlation for the
> purpose of monitoring is not an easy thing to do. Correlation, for the
> purpose of debugging will not run into this problem, if you run this on a
> controlled environment. However, what you really want to debug is the deal
> clustered setup, and not a staged dev setup, the problem again surfaces.
> You might want to list the objectives and then break them down
> into monitoring and debugging spaces.
> Samisa...
>
>>
>> Thanks
>> Nilupa.
>>
>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
>> <sa...@gmail.com> wrote:
>> > What UI technology would be used for this?
>> >
>> > Samisa...
>> >
>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
>> > <an...@gmail.com> wrote:
>> >> I was thinking more about something like ITCAM for SOA.
>> >>
>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>> >> <sa...@opensource.lk> wrote:
>> >>> That runtime monitoring / correlation is what the WSO2 Business
>> >>> Activity
>> >>> Monitor does / is for ...
>> >>> See: http://wso2.com/products/business-activity-monitor/
>> >>> Sanjiva.
>> >>>
>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
>> >>> <an...@gmail.com>
>> >>> wrote:
>> >>>>
>> >>>> I was actually referring to scenarios where a service may call other
>> >>>> services, which in turn may call again other services. In these
>> >>>> scenarios, it is not sufficient to simply collect received/sent
>> >>>> messages on different hosts, unless the system is isolated
>> >>>> (development environment) or the request rate is sufficiently low so
>> >>>> that messages can be correlated based on time. Here is a concrete
>> >>>> scenario from a project in my company:
>> >>>>
>> >>>> - Request comes in on an ESB that does security and validation.
>> >>>> - Request is processed by an application server which persists the
>> >>>> received information and publishes a JMS message (pub/sub events).
>> >>>> - The event is consumed by one or more components that may in turn
>> >>>> interact with other services.
>> >>>>
>> >>>> If you want track this flow, it is not sufficient to intercept the
>> >>>> messages on the different hosts: in addition, you need to instrument
>> >>>> the services so that the outgoing requests are correlated with the
>> >>>> incoming requests (by adding SOAP and/or transport headers). What I
>> >>>> would like to be able to do in the above scenario is to enable
>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> >>>> HTTP header to the initial request to enable logging and this
>> >>>> information is propagated along the chain. Every service/component
>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
>> >>>> central place where the sequence of events is reconstructed.
>> >>>>
>> >>>> One way this could be achieved is with a tool that postprocesses the
>> >>>> artifacts from the build process. For each artifact, the tool would
>> >>>> disassemble the artifact, instrument the code using a set of AspectJ
>> >>>> aspects and reassemble the artifact for deployment. The
>> >>>> responsibility
>> >>>> of the aspects would be to intercept messages, log them and modify
>> >>>> them to ensure proper correlation. Of course this only works if the
>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> >>>> use AspectJ for a similar use case (although at a much smaller scale)
>> >>>> in the transport test suites [1]. Interestingly, this approach would
>> >>>> allow to cover interactions that are not SOAP based, e.g. one could
>> >>>> even intercept the database queries executed during the flow.
>> >>>>
>> >>>> Andreas
>> >>>>
>> >>>> [1]
>> >>>>
>> >>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>> >>>>
>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
>> >>>> wrote:
>> >>>> > Hi Nilupa;
>> >>>> >
>> >>>> > When we collect messages from a one location by installing proper
>> >>>> > handler that will intercept and send messages to to that one
>> >>>> > location
>> >>>> > (this one location can be a single server, pub/sub channel etc).
>> >>>> > There
>> >>>> > is many ways to make sense of those collected messages. What
>> >>>> > Andreas
>> >>>> > mentioned (following complete transaction) is a one possibility.
>> >>>> >
>> >>>> > I think you should come up with few scenarios on how you would make
>> >>>> > sense of the message.
>> >>>> >
>> >>>> > Thanks
>> >>>> > Srinath
>> >>>> >
>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>> >>>> > wrote:
>> >>>> >> Hi,
>> >>>> >>
>> >>>> >> First let me thank you for commenting.
>> >>>> >>
>> >>>> >> As far as I understood, what you would like to see from the
>> >>>> >> proposed
>> >>>> >> tool is to view set of messages that are exchanged in reponse to a
>> >>>> >> particular input message. With the understanding that I am having
>> >>>> >> at
>> >>>> >> the momnet, one way to do it is to filter out the central
>> >>>> >> repository
>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>> >>>> >> message chain from it. We can allow the client GUI wich connects
>> >>>> >> to
>> >>>> >> the central repository to provide the paramenters (For instance
>> >>>> >> the
>> >>>> >> value of 'To' header) from which an intelligent filtering can be
>> >>>> >> done
>> >>>> >> for the set of messages avialable at the central repository.
>> >>>> >>
>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
>> >>>> >> willing
>> >>>> >> to share it with us. It would be really nice to hear from them.
>> >>>> >>
>> >>>> >> Thanks in advance ..!!
>> >>>> >>
>> >>>> >> Best Regards,
>> >>>> >> Nilupa
>> >>>> >>
>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>> >>>> >> <an...@gmail.com> wrote:
>> >>>> >>> Personally, I think that the added value of extending the SOAP
>> >>>> >>> monitor
>> >>>> >>> module to collect messages in a central place is too limited to
>> >>>> >>> attract enough interest from the user community, so that it will
>> >>>> >>> be
>> >>>> >>> difficult to ensure the evolution of such a project in the
>> >>>> >>> future.
>> >>>> >>> However, what many people are looking for is a tool that allows
>> >>>> >>> to
>> >>>> >>> track the flow of a message through a distributed system, or more
>> >>>> >>> generally to track the sequence of events triggered by a given
>> >>>> >>> input
>> >>>> >>> message (sort of end-to-end transaction monitoring).
>> >>>> >>>
>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>> >>>> >>> wrote:
>> >>>> >>>> Hello,
>> >>>> >>>>
>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
>> >>>> >>>> of
>> >>>> >>>> proposing a GSoC project this summer. I would like to take
>> >>>> >>>> project
>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
>> >>>> >>>> and
>> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
>> >>>> >>>> Web
>> >>>> >>>> services middleware. I will submit more detailed proposal later.
>> >>>> >>>> I
>> >>>> >>>> would really like to hear any feedback from you. Any suggestions
>> >>>> >>>> or
>> >>>> >>>> features that you would like to see in a "Distributed TCP
>> >>>> >>>> Monitor"
>> >>>> >>>> are
>> >>>> >>>> most welcome.
>> >>>> >>>>
>> >>>> >>>> Best Regards,
>> >>>> >>>> Nilupa Bandara
>> >>>> >>>>
>> >>>> >>>> [1]
>> >>>> >>>>
>> >>>> >>>> Distributed TCP monitor
>> >>>> >>>>
>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
>> >>>> >>>> hard
>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
>> >>>> >>>> the
>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
>> >>>> >>>> messages
>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
>> >>>> >>>> can
>> >>>> >>>> pull those messages. Also, we can do something to turn the
>> >>>> >>>> module on
>> >>>> >>>> and off remotely, so users can have his module deployed, but
>> >>>> >>>> turn it
>> >>>> >>>> on only when they want to debug the system.
>> >>>> >>>>
>> >>>> >>>> Then we can take this to next level by adding this module to all
>> >>>> >>>> services in a system , configuring modules to send all collected
>> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
>> >>>> >>>> via
>> >>>> >>>> a
>> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
>> >>>> >>>> TCPMon for a whole system.
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>> ---------------------------------------------------------------------
>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>
>> >>>> >>>
>> >>>> >>> ---------------------------------------------------------------------
>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >>>
>> >>>> >>>
>> >>>> >>
>> >>>> >>
>> >>>> >> ---------------------------------------------------------------------
>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >>
>> >>>> >>
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > --
>> >>>> > ============================
>> >>>> > Srinath Perera, Ph.D.
>> >>>> >   WSO2 Inc. http://wso2.com
>> >>>> >   Blog: http://srinathsview.blogspot.com/
>> >>>> >
>> >>>> >
>> >>>> > ---------------------------------------------------------------------
>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >
>> >>>> >
>> >>>>
>> >>>> ---------------------------------------------------------------------
>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Sanjiva Weerawarana, Ph.D.
>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>> >>> http://www.opensource.lk/
>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>> >>> Member; Apache Software Foundation; http://www.apache.org/
>> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>> >>>
>> >>> Blog: http://sanjiva.weerawarana.org/
>> >>>
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/
>



-- 
Sanka Samaranayake

http://sankas.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanka Samaranayake <ss...@gmail.com>.

Hi Samisa,

I think the this scope of this project would be to provide a
light-weight tool which allows to monitor the message flows, execution
in each node in response to those message flows to a given input
message primarily for debugging purposes . (Just like the way tcpmon
is used in developing Web services)

And I guess now the question is whether such a tool is useful ..

Anyone cares to share their thoughts ?

Nilupa

On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>
>
> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:
>>
>> I was thinking about Java Swing based UI
>
> Swing is a good start. But if you look at some of the requirements like
> monitoring multiple instances and correlate them, discussed in this thread,
> it sounds as if we would be better off having a Web interface, rather than a
> desktop client.
>
>>
>> because the client may have
>> to cache whatever messages it already has and will receive,
>
> If you are looking to support message-correlation, as discussed in some
> replies in this thread above, you need to have a persistence storage (a DB
> or something like that) rather than a memory cache.
> Because, some message sequences, like those involved in an async
> invocations, need to have a way to keep the messages, in there, longer, and
> may be there is a server re-start, in the middle.
> Of course, this requirements spend on the scope of this effort.
> Plus, if you want to deal with historic data, and look at what was happening
> to message sequences over time, you got to use a persistence layer. Again
> this might be out of scope for this project, but if this is to be extended
> to support these, you better think of this right now.
>
>>
>> build the
>> message flow and update the UI accordingly. Perhaps there is a better
>> way ?
>
> We already do what you are trying to do, and much more with WSO2 BAM. I am
> tempted to say that Shinding based gadget dashboard is really a compelling
> UI for this.
> I am not trying to take away you project or trying to hinder this effort at
> all.
> I think it will be a good idea for you to have a look at BAM and see what
> the missing pieces are and figure out the right model for this tool.
> Also, having had worked on WSO2 BAM project, I also have some understanding
> on sort of problems that you might run into, when doing this. For e.g., if
> you want to correlate message sequences in a high volume setup, you will be
> swamped by the volume of messages, sooner than later. So correlation for the
> purpose of monitoring is not an easy thing to do. Correlation, for the
> purpose of debugging will not run into this problem, if you run this on a
> controlled environment. However, what you really want to debug is the deal
> clustered setup, and not a staged dev setup, the problem again surfaces.
> You might want to list the objectives and then break them down
> into monitoring and debugging spaces.
> Samisa...
>
>>
>> Thanks
>> Nilupa.
>>
>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
>> <sa...@gmail.com> wrote:
>> > What UI technology would be used for this?
>> >
>> > Samisa...
>> >
>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
>> > <an...@gmail.com> wrote:
>> >> I was thinking more about something like ITCAM for SOA.
>> >>
>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>> >> <sa...@opensource.lk> wrote:
>> >>> That runtime monitoring / correlation is what the WSO2 Business
>> >>> Activity
>> >>> Monitor does / is for ...
>> >>> See: http://wso2.com/products/business-activity-monitor/
>> >>> Sanjiva.
>> >>>
>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
>> >>> <an...@gmail.com>
>> >>> wrote:
>> >>>>
>> >>>> I was actually referring to scenarios where a service may call other
>> >>>> services, which in turn may call again other services. In these
>> >>>> scenarios, it is not sufficient to simply collect received/sent
>> >>>> messages on different hosts, unless the system is isolated
>> >>>> (development environment) or the request rate is sufficiently low so
>> >>>> that messages can be correlated based on time. Here is a concrete
>> >>>> scenario from a project in my company:
>> >>>>
>> >>>> - Request comes in on an ESB that does security and validation.
>> >>>> - Request is processed by an application server which persists the
>> >>>> received information and publishes a JMS message (pub/sub events).
>> >>>> - The event is consumed by one or more components that may in turn
>> >>>> interact with other services.
>> >>>>
>> >>>> If you want track this flow, it is not sufficient to intercept the
>> >>>> messages on the different hosts: in addition, you need to instrument
>> >>>> the services so that the outgoing requests are correlated with the
>> >>>> incoming requests (by adding SOAP and/or transport headers). What I
>> >>>> would like to be able to do in the above scenario is to enable
>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> >>>> HTTP header to the initial request to enable logging and this
>> >>>> information is propagated along the chain. Every service/component
>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
>> >>>> central place where the sequence of events is reconstructed.
>> >>>>
>> >>>> One way this could be achieved is with a tool that postprocesses the
>> >>>> artifacts from the build process. For each artifact, the tool would
>> >>>> disassemble the artifact, instrument the code using a set of AspectJ
>> >>>> aspects and reassemble the artifact for deployment. The
>> >>>> responsibility
>> >>>> of the aspects would be to intercept messages, log them and modify
>> >>>> them to ensure proper correlation. Of course this only works if the
>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> >>>> use AspectJ for a similar use case (although at a much smaller scale)
>> >>>> in the transport test suites [1]. Interestingly, this approach would
>> >>>> allow to cover interactions that are not SOAP based, e.g. one could
>> >>>> even intercept the database queries executed during the flow.
>> >>>>
>> >>>> Andreas
>> >>>>
>> >>>> [1]
>> >>>>
>> >>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>> >>>>
>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
>> >>>> wrote:
>> >>>> > Hi Nilupa;
>> >>>> >
>> >>>> > When we collect messages from a one location by installing proper
>> >>>> > handler that will intercept and send messages to to that one
>> >>>> > location
>> >>>> > (this one location can be a single server, pub/sub channel etc).
>> >>>> > There
>> >>>> > is many ways to make sense of those collected messages. What
>> >>>> > Andreas
>> >>>> > mentioned (following complete transaction) is a one possibility.
>> >>>> >
>> >>>> > I think you should come up with few scenarios on how you would make
>> >>>> > sense of the message.
>> >>>> >
>> >>>> > Thanks
>> >>>> > Srinath
>> >>>> >
>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>> >>>> > wrote:
>> >>>> >> Hi,
>> >>>> >>
>> >>>> >> First let me thank you for commenting.
>> >>>> >>
>> >>>> >> As far as I understood, what you would like to see from the
>> >>>> >> proposed
>> >>>> >> tool is to view set of messages that are exchanged in reponse to a
>> >>>> >> particular input message. With the understanding that I am having
>> >>>> >> at
>> >>>> >> the momnet, one way to do it is to filter out the central
>> >>>> >> repository
>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>> >>>> >> message chain from it. We can allow the client GUI wich connects
>> >>>> >> to
>> >>>> >> the central repository to provide the paramenters (For instance
>> >>>> >> the
>> >>>> >> value of 'To' header) from which an intelligent filtering can be
>> >>>> >> done
>> >>>> >> for the set of messages avialable at the central repository.
>> >>>> >>
>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
>> >>>> >> willing
>> >>>> >> to share it with us. It would be really nice to hear from them.
>> >>>> >>
>> >>>> >> Thanks in advance ..!!
>> >>>> >>
>> >>>> >> Best Regards,
>> >>>> >> Nilupa
>> >>>> >>
>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>> >>>> >> <an...@gmail.com> wrote:
>> >>>> >>> Personally, I think that the added value of extending the SOAP
>> >>>> >>> monitor
>> >>>> >>> module to collect messages in a central place is too limited to
>> >>>> >>> attract enough interest from the user community, so that it will
>> >>>> >>> be
>> >>>> >>> difficult to ensure the evolution of such a project in the
>> >>>> >>> future.
>> >>>> >>> However, what many people are looking for is a tool that allows
>> >>>> >>> to
>> >>>> >>> track the flow of a message through a distributed system, or more
>> >>>> >>> generally to track the sequence of events triggered by a given
>> >>>> >>> input
>> >>>> >>> message (sort of end-to-end transaction monitoring).
>> >>>> >>>
>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>> >>>> >>> wrote:
>> >>>> >>>> Hello,
>> >>>> >>>>
>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
>> >>>> >>>> of
>> >>>> >>>> proposing a GSoC project this summer. I would like to take
>> >>>> >>>> project
>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
>> >>>> >>>> and
>> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
>> >>>> >>>> Web
>> >>>> >>>> services middleware. I will submit more detailed proposal later.
>> >>>> >>>> I
>> >>>> >>>> would really like to hear any feedback from you. Any suggestions
>> >>>> >>>> or
>> >>>> >>>> features that you would like to see in a "Distributed TCP
>> >>>> >>>> Monitor"
>> >>>> >>>> are
>> >>>> >>>> most welcome.
>> >>>> >>>>
>> >>>> >>>> Best Regards,
>> >>>> >>>> Nilupa Bandara
>> >>>> >>>>
>> >>>> >>>> [1]
>> >>>> >>>>
>> >>>> >>>> Distributed TCP monitor
>> >>>> >>>>
>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
>> >>>> >>>> hard
>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
>> >>>> >>>> the
>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
>> >>>> >>>> messages
>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
>> >>>> >>>> can
>> >>>> >>>> pull those messages. Also, we can do something to turn the
>> >>>> >>>> module on
>> >>>> >>>> and off remotely, so users can have his module deployed, but
>> >>>> >>>> turn it
>> >>>> >>>> on only when they want to debug the system.
>> >>>> >>>>
>> >>>> >>>> Then we can take this to next level by adding this module to all
>> >>>> >>>> services in a system , configuring modules to send all collected
>> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
>> >>>> >>>> via
>> >>>> >>>> a
>> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
>> >>>> >>>> TCPMon for a whole system.
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>> ---------------------------------------------------------------------
>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>
>> >>>> >>>
>> >>>> >>> ---------------------------------------------------------------------
>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >>>
>> >>>> >>>
>> >>>> >>
>> >>>> >>
>> >>>> >> ---------------------------------------------------------------------
>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >>
>> >>>> >>
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > --
>> >>>> > ============================
>> >>>> > Srinath Perera, Ph.D.
>> >>>> >   WSO2 Inc. http://wso2.com
>> >>>> >   Blog: http://srinathsview.blogspot.com/
>> >>>> >
>> >>>> >
>> >>>> > ---------------------------------------------------------------------
>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >
>> >>>> >
>> >>>>
>> >>>> ---------------------------------------------------------------------
>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Sanjiva Weerawarana, Ph.D.
>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>> >>> http://www.opensource.lk/
>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>> >>> Member; Apache Software Foundation; http://www.apache.org/
>> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>> >>>
>> >>> Blog: http://sanjiva.weerawarana.org/
>> >>>
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/
>



-- 
Sanka Samaranayake

http://sankas.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanka Samaranayake <ss...@gmail.com>.

Hi Samisa,

I think the this scope of this project would be to provide a
light-weight tool which allows to monitor the message flows, execution
in each node in response to those message flows to a given input
message primarily for debugging purposes . (Just like the way tcpmon
is used in developing Web services)

And I guess now the question is whether such a tool is useful ..

Anyone cares to share their thoughts ?

Nilupa

On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>
>
> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:
>>
>> I was thinking about Java Swing based UI
>
> Swing is a good start. But if you look at some of the requirements like
> monitoring multiple instances and correlate them, discussed in this thread,
> it sounds as if we would be better off having a Web interface, rather than a
> desktop client.
>
>>
>> because the client may have
>> to cache whatever messages it already has and will receive,
>
> If you are looking to support message-correlation, as discussed in some
> replies in this thread above, you need to have a persistence storage (a DB
> or something like that) rather than a memory cache.
> Because, some message sequences, like those involved in an async
> invocations, need to have a way to keep the messages, in there, longer, and
> may be there is a server re-start, in the middle.
> Of course, this requirements spend on the scope of this effort.
> Plus, if you want to deal with historic data, and look at what was happening
> to message sequences over time, you got to use a persistence layer. Again
> this might be out of scope for this project, but if this is to be extended
> to support these, you better think of this right now.
>
>>
>> build the
>> message flow and update the UI accordingly. Perhaps there is a better
>> way ?
>
> We already do what you are trying to do, and much more with WSO2 BAM. I am
> tempted to say that Shinding based gadget dashboard is really a compelling
> UI for this.
> I am not trying to take away you project or trying to hinder this effort at
> all.
> I think it will be a good idea for you to have a look at BAM and see what
> the missing pieces are and figure out the right model for this tool.
> Also, having had worked on WSO2 BAM project, I also have some understanding
> on sort of problems that you might run into, when doing this. For e.g., if
> you want to correlate message sequences in a high volume setup, you will be
> swamped by the volume of messages, sooner than later. So correlation for the
> purpose of monitoring is not an easy thing to do. Correlation, for the
> purpose of debugging will not run into this problem, if you run this on a
> controlled environment. However, what you really want to debug is the deal
> clustered setup, and not a staged dev setup, the problem again surfaces.
> You might want to list the objectives and then break them down
> into monitoring and debugging spaces.
> Samisa...
>
>>
>> Thanks
>> Nilupa.
>>
>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
>> <sa...@gmail.com> wrote:
>> > What UI technology would be used for this?
>> >
>> > Samisa...
>> >
>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
>> > <an...@gmail.com> wrote:
>> >> I was thinking more about something like ITCAM for SOA.
>> >>
>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>> >> <sa...@opensource.lk> wrote:
>> >>> That runtime monitoring / correlation is what the WSO2 Business
>> >>> Activity
>> >>> Monitor does / is for ...
>> >>> See: http://wso2.com/products/business-activity-monitor/
>> >>> Sanjiva.
>> >>>
>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
>> >>> <an...@gmail.com>
>> >>> wrote:
>> >>>>
>> >>>> I was actually referring to scenarios where a service may call other
>> >>>> services, which in turn may call again other services. In these
>> >>>> scenarios, it is not sufficient to simply collect received/sent
>> >>>> messages on different hosts, unless the system is isolated
>> >>>> (development environment) or the request rate is sufficiently low so
>> >>>> that messages can be correlated based on time. Here is a concrete
>> >>>> scenario from a project in my company:
>> >>>>
>> >>>> - Request comes in on an ESB that does security and validation.
>> >>>> - Request is processed by an application server which persists the
>> >>>> received information and publishes a JMS message (pub/sub events).
>> >>>> - The event is consumed by one or more components that may in turn
>> >>>> interact with other services.
>> >>>>
>> >>>> If you want track this flow, it is not sufficient to intercept the
>> >>>> messages on the different hosts: in addition, you need to instrument
>> >>>> the services so that the outgoing requests are correlated with the
>> >>>> incoming requests (by adding SOAP and/or transport headers). What I
>> >>>> would like to be able to do in the above scenario is to enable
>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> >>>> HTTP header to the initial request to enable logging and this
>> >>>> information is propagated along the chain. Every service/component
>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
>> >>>> central place where the sequence of events is reconstructed.
>> >>>>
>> >>>> One way this could be achieved is with a tool that postprocesses the
>> >>>> artifacts from the build process. For each artifact, the tool would
>> >>>> disassemble the artifact, instrument the code using a set of AspectJ
>> >>>> aspects and reassemble the artifact for deployment. The
>> >>>> responsibility
>> >>>> of the aspects would be to intercept messages, log them and modify
>> >>>> them to ensure proper correlation. Of course this only works if the
>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> >>>> use AspectJ for a similar use case (although at a much smaller scale)
>> >>>> in the transport test suites [1]. Interestingly, this approach would
>> >>>> allow to cover interactions that are not SOAP based, e.g. one could
>> >>>> even intercept the database queries executed during the flow.
>> >>>>
>> >>>> Andreas
>> >>>>
>> >>>> [1]
>> >>>>
>> >>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>> >>>>
>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
>> >>>> wrote:
>> >>>> > Hi Nilupa;
>> >>>> >
>> >>>> > When we collect messages from a one location by installing proper
>> >>>> > handler that will intercept and send messages to to that one
>> >>>> > location
>> >>>> > (this one location can be a single server, pub/sub channel etc).
>> >>>> > There
>> >>>> > is many ways to make sense of those collected messages. What
>> >>>> > Andreas
>> >>>> > mentioned (following complete transaction) is a one possibility.
>> >>>> >
>> >>>> > I think you should come up with few scenarios on how you would make
>> >>>> > sense of the message.
>> >>>> >
>> >>>> > Thanks
>> >>>> > Srinath
>> >>>> >
>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>> >>>> > wrote:
>> >>>> >> Hi,
>> >>>> >>
>> >>>> >> First let me thank you for commenting.
>> >>>> >>
>> >>>> >> As far as I understood, what you would like to see from the
>> >>>> >> proposed
>> >>>> >> tool is to view set of messages that are exchanged in reponse to a
>> >>>> >> particular input message. With the understanding that I am having
>> >>>> >> at
>> >>>> >> the momnet, one way to do it is to filter out the central
>> >>>> >> repository
>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>> >>>> >> message chain from it. We can allow the client GUI wich connects
>> >>>> >> to
>> >>>> >> the central repository to provide the paramenters (For instance
>> >>>> >> the
>> >>>> >> value of 'To' header) from which an intelligent filtering can be
>> >>>> >> done
>> >>>> >> for the set of messages avialable at the central repository.
>> >>>> >>
>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
>> >>>> >> willing
>> >>>> >> to share it with us. It would be really nice to hear from them.
>> >>>> >>
>> >>>> >> Thanks in advance ..!!
>> >>>> >>
>> >>>> >> Best Regards,
>> >>>> >> Nilupa
>> >>>> >>
>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>> >>>> >> <an...@gmail.com> wrote:
>> >>>> >>> Personally, I think that the added value of extending the SOAP
>> >>>> >>> monitor
>> >>>> >>> module to collect messages in a central place is too limited to
>> >>>> >>> attract enough interest from the user community, so that it will
>> >>>> >>> be
>> >>>> >>> difficult to ensure the evolution of such a project in the
>> >>>> >>> future.
>> >>>> >>> However, what many people are looking for is a tool that allows
>> >>>> >>> to
>> >>>> >>> track the flow of a message through a distributed system, or more
>> >>>> >>> generally to track the sequence of events triggered by a given
>> >>>> >>> input
>> >>>> >>> message (sort of end-to-end transaction monitoring).
>> >>>> >>>
>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>> >>>> >>> wrote:
>> >>>> >>>> Hello,
>> >>>> >>>>
>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
>> >>>> >>>> of
>> >>>> >>>> proposing a GSoC project this summer. I would like to take
>> >>>> >>>> project
>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
>> >>>> >>>> and
>> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
>> >>>> >>>> Web
>> >>>> >>>> services middleware. I will submit more detailed proposal later.
>> >>>> >>>> I
>> >>>> >>>> would really like to hear any feedback from you. Any suggestions
>> >>>> >>>> or
>> >>>> >>>> features that you would like to see in a "Distributed TCP
>> >>>> >>>> Monitor"
>> >>>> >>>> are
>> >>>> >>>> most welcome.
>> >>>> >>>>
>> >>>> >>>> Best Regards,
>> >>>> >>>> Nilupa Bandara
>> >>>> >>>>
>> >>>> >>>> [1]
>> >>>> >>>>
>> >>>> >>>> Distributed TCP monitor
>> >>>> >>>>
>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
>> >>>> >>>> hard
>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
>> >>>> >>>> the
>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
>> >>>> >>>> messages
>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
>> >>>> >>>> can
>> >>>> >>>> pull those messages. Also, we can do something to turn the
>> >>>> >>>> module on
>> >>>> >>>> and off remotely, so users can have his module deployed, but
>> >>>> >>>> turn it
>> >>>> >>>> on only when they want to debug the system.
>> >>>> >>>>
>> >>>> >>>> Then we can take this to next level by adding this module to all
>> >>>> >>>> services in a system , configuring modules to send all collected
>> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
>> >>>> >>>> via
>> >>>> >>>> a
>> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
>> >>>> >>>> TCPMon for a whole system.
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>> ---------------------------------------------------------------------
>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>
>> >>>> >>>
>> >>>> >>> ---------------------------------------------------------------------
>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >>>
>> >>>> >>>
>> >>>> >>
>> >>>> >>
>> >>>> >> ---------------------------------------------------------------------
>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >>
>> >>>> >>
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > --
>> >>>> > ============================
>> >>>> > Srinath Perera, Ph.D.
>> >>>> >   WSO2 Inc. http://wso2.com
>> >>>> >   Blog: http://srinathsview.blogspot.com/
>> >>>> >
>> >>>> >
>> >>>> > ---------------------------------------------------------------------
>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >
>> >>>> >
>> >>>>
>> >>>> ---------------------------------------------------------------------
>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Sanjiva Weerawarana, Ph.D.
>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>> >>> http://www.opensource.lk/
>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>> >>> Member; Apache Software Foundation; http://www.apache.org/
>> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>> >>>
>> >>> Blog: http://sanjiva.weerawarana.org/
>> >>>
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/
>



-- 
Sanka Samaranayake

http://sankas.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanka Samaranayake <ss...@gmail.com>.

Hi Samisa,

I think the this scope of this project would be to provide a
light-weight tool which allows to monitor the message flows, execution
in each node in response to those message flows to a given input
message primarily for debugging purposes . (Just like the way tcpmon
is used in developing Web services)

And I guess now the question is whether such a tool is useful ..

Anyone cares to share their thoughts ?

Nilupa

On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>
>
> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:
>>
>> I was thinking about Java Swing based UI
>
> Swing is a good start. But if you look at some of the requirements like
> monitoring multiple instances and correlate them, discussed in this thread,
> it sounds as if we would be better off having a Web interface, rather than a
> desktop client.
>
>>
>> because the client may have
>> to cache whatever messages it already has and will receive,
>
> If you are looking to support message-correlation, as discussed in some
> replies in this thread above, you need to have a persistence storage (a DB
> or something like that) rather than a memory cache.
> Because, some message sequences, like those involved in an async
> invocations, need to have a way to keep the messages, in there, longer, and
> may be there is a server re-start, in the middle.
> Of course, this requirements spend on the scope of this effort.
> Plus, if you want to deal with historic data, and look at what was happening
> to message sequences over time, you got to use a persistence layer. Again
> this might be out of scope for this project, but if this is to be extended
> to support these, you better think of this right now.
>
>>
>> build the
>> message flow and update the UI accordingly. Perhaps there is a better
>> way ?
>
> We already do what you are trying to do, and much more with WSO2 BAM. I am
> tempted to say that Shinding based gadget dashboard is really a compelling
> UI for this.
> I am not trying to take away you project or trying to hinder this effort at
> all.
> I think it will be a good idea for you to have a look at BAM and see what
> the missing pieces are and figure out the right model for this tool.
> Also, having had worked on WSO2 BAM project, I also have some understanding
> on sort of problems that you might run into, when doing this. For e.g., if
> you want to correlate message sequences in a high volume setup, you will be
> swamped by the volume of messages, sooner than later. So correlation for the
> purpose of monitoring is not an easy thing to do. Correlation, for the
> purpose of debugging will not run into this problem, if you run this on a
> controlled environment. However, what you really want to debug is the deal
> clustered setup, and not a staged dev setup, the problem again surfaces.
> You might want to list the objectives and then break them down
> into monitoring and debugging spaces.
> Samisa...
>
>>
>> Thanks
>> Nilupa.
>>
>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
>> <sa...@gmail.com> wrote:
>> > What UI technology would be used for this?
>> >
>> > Samisa...
>> >
>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
>> > <an...@gmail.com> wrote:
>> >> I was thinking more about something like ITCAM for SOA.
>> >>
>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>> >> <sa...@opensource.lk> wrote:
>> >>> That runtime monitoring / correlation is what the WSO2 Business
>> >>> Activity
>> >>> Monitor does / is for ...
>> >>> See: http://wso2.com/products/business-activity-monitor/
>> >>> Sanjiva.
>> >>>
>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
>> >>> <an...@gmail.com>
>> >>> wrote:
>> >>>>
>> >>>> I was actually referring to scenarios where a service may call other
>> >>>> services, which in turn may call again other services. In these
>> >>>> scenarios, it is not sufficient to simply collect received/sent
>> >>>> messages on different hosts, unless the system is isolated
>> >>>> (development environment) or the request rate is sufficiently low so
>> >>>> that messages can be correlated based on time. Here is a concrete
>> >>>> scenario from a project in my company:
>> >>>>
>> >>>> - Request comes in on an ESB that does security and validation.
>> >>>> - Request is processed by an application server which persists the
>> >>>> received information and publishes a JMS message (pub/sub events).
>> >>>> - The event is consumed by one or more components that may in turn
>> >>>> interact with other services.
>> >>>>
>> >>>> If you want track this flow, it is not sufficient to intercept the
>> >>>> messages on the different hosts: in addition, you need to instrument
>> >>>> the services so that the outgoing requests are correlated with the
>> >>>> incoming requests (by adding SOAP and/or transport headers). What I
>> >>>> would like to be able to do in the above scenario is to enable
>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> >>>> HTTP header to the initial request to enable logging and this
>> >>>> information is propagated along the chain. Every service/component
>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
>> >>>> central place where the sequence of events is reconstructed.
>> >>>>
>> >>>> One way this could be achieved is with a tool that postprocesses the
>> >>>> artifacts from the build process. For each artifact, the tool would
>> >>>> disassemble the artifact, instrument the code using a set of AspectJ
>> >>>> aspects and reassemble the artifact for deployment. The
>> >>>> responsibility
>> >>>> of the aspects would be to intercept messages, log them and modify
>> >>>> them to ensure proper correlation. Of course this only works if the
>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> >>>> use AspectJ for a similar use case (although at a much smaller scale)
>> >>>> in the transport test suites [1]. Interestingly, this approach would
>> >>>> allow to cover interactions that are not SOAP based, e.g. one could
>> >>>> even intercept the database queries executed during the flow.
>> >>>>
>> >>>> Andreas
>> >>>>
>> >>>> [1]
>> >>>>
>> >>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>> >>>>
>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
>> >>>> wrote:
>> >>>> > Hi Nilupa;
>> >>>> >
>> >>>> > When we collect messages from a one location by installing proper
>> >>>> > handler that will intercept and send messages to to that one
>> >>>> > location
>> >>>> > (this one location can be a single server, pub/sub channel etc).
>> >>>> > There
>> >>>> > is many ways to make sense of those collected messages. What
>> >>>> > Andreas
>> >>>> > mentioned (following complete transaction) is a one possibility.
>> >>>> >
>> >>>> > I think you should come up with few scenarios on how you would make
>> >>>> > sense of the message.
>> >>>> >
>> >>>> > Thanks
>> >>>> > Srinath
>> >>>> >
>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>> >>>> > wrote:
>> >>>> >> Hi,
>> >>>> >>
>> >>>> >> First let me thank you for commenting.
>> >>>> >>
>> >>>> >> As far as I understood, what you would like to see from the
>> >>>> >> proposed
>> >>>> >> tool is to view set of messages that are exchanged in reponse to a
>> >>>> >> particular input message. With the understanding that I am having
>> >>>> >> at
>> >>>> >> the momnet, one way to do it is to filter out the central
>> >>>> >> repository
>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>> >>>> >> message chain from it. We can allow the client GUI wich connects
>> >>>> >> to
>> >>>> >> the central repository to provide the paramenters (For instance
>> >>>> >> the
>> >>>> >> value of 'To' header) from which an intelligent filtering can be
>> >>>> >> done
>> >>>> >> for the set of messages avialable at the central repository.
>> >>>> >>
>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
>> >>>> >> willing
>> >>>> >> to share it with us. It would be really nice to hear from them.
>> >>>> >>
>> >>>> >> Thanks in advance ..!!
>> >>>> >>
>> >>>> >> Best Regards,
>> >>>> >> Nilupa
>> >>>> >>
>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>> >>>> >> <an...@gmail.com> wrote:
>> >>>> >>> Personally, I think that the added value of extending the SOAP
>> >>>> >>> monitor
>> >>>> >>> module to collect messages in a central place is too limited to
>> >>>> >>> attract enough interest from the user community, so that it will
>> >>>> >>> be
>> >>>> >>> difficult to ensure the evolution of such a project in the
>> >>>> >>> future.
>> >>>> >>> However, what many people are looking for is a tool that allows
>> >>>> >>> to
>> >>>> >>> track the flow of a message through a distributed system, or more
>> >>>> >>> generally to track the sequence of events triggered by a given
>> >>>> >>> input
>> >>>> >>> message (sort of end-to-end transaction monitoring).
>> >>>> >>>
>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>> >>>> >>> wrote:
>> >>>> >>>> Hello,
>> >>>> >>>>
>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
>> >>>> >>>> of
>> >>>> >>>> proposing a GSoC project this summer. I would like to take
>> >>>> >>>> project
>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
>> >>>> >>>> and
>> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
>> >>>> >>>> Web
>> >>>> >>>> services middleware. I will submit more detailed proposal later.
>> >>>> >>>> I
>> >>>> >>>> would really like to hear any feedback from you. Any suggestions
>> >>>> >>>> or
>> >>>> >>>> features that you would like to see in a "Distributed TCP
>> >>>> >>>> Monitor"
>> >>>> >>>> are
>> >>>> >>>> most welcome.
>> >>>> >>>>
>> >>>> >>>> Best Regards,
>> >>>> >>>> Nilupa Bandara
>> >>>> >>>>
>> >>>> >>>> [1]
>> >>>> >>>>
>> >>>> >>>> Distributed TCP monitor
>> >>>> >>>>
>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
>> >>>> >>>> hard
>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
>> >>>> >>>> the
>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
>> >>>> >>>> messages
>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
>> >>>> >>>> can
>> >>>> >>>> pull those messages. Also, we can do something to turn the
>> >>>> >>>> module on
>> >>>> >>>> and off remotely, so users can have his module deployed, but
>> >>>> >>>> turn it
>> >>>> >>>> on only when they want to debug the system.
>> >>>> >>>>
>> >>>> >>>> Then we can take this to next level by adding this module to all
>> >>>> >>>> services in a system , configuring modules to send all collected
>> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
>> >>>> >>>> via
>> >>>> >>>> a
>> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
>> >>>> >>>> TCPMon for a whole system.
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>> ---------------------------------------------------------------------
>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>
>> >>>> >>>
>> >>>> >>> ---------------------------------------------------------------------
>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >>>
>> >>>> >>>
>> >>>> >>
>> >>>> >>
>> >>>> >> ---------------------------------------------------------------------
>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >>
>> >>>> >>
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > --
>> >>>> > ============================
>> >>>> > Srinath Perera, Ph.D.
>> >>>> >   WSO2 Inc. http://wso2.com
>> >>>> >   Blog: http://srinathsview.blogspot.com/
>> >>>> >
>> >>>> >
>> >>>> > ---------------------------------------------------------------------
>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >
>> >>>> >
>> >>>>
>> >>>> ---------------------------------------------------------------------
>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Sanjiva Weerawarana, Ph.D.
>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>> >>> http://www.opensource.lk/
>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>> >>> Member; Apache Software Foundation; http://www.apache.org/
>> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>> >>>
>> >>> Blog: http://sanjiva.weerawarana.org/
>> >>>
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/
>



-- 
Sanka Samaranayake

http://sankas.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanka Samaranayake <ss...@gmail.com>.

Hi Samisa,

I think the this scope of this project would be to provide a
light-weight tool which allows to monitor the message flows, execution
in each node in response to those message flows to a given input
message primarily for debugging purposes . (Just like the way tcpmon
is used in developing Web services)

And I guess now the question is whether such a tool is useful ..

Anyone cares to share their thoughts ?

Nilupa

On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
>
>
> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:
>>
>> I was thinking about Java Swing based UI
>
> Swing is a good start. But if you look at some of the requirements like
> monitoring multiple instances and correlate them, discussed in this thread,
> it sounds as if we would be better off having a Web interface, rather than a
> desktop client.
>
>>
>> because the client may have
>> to cache whatever messages it already has and will receive,
>
> If you are looking to support message-correlation, as discussed in some
> replies in this thread above, you need to have a persistence storage (a DB
> or something like that) rather than a memory cache.
> Because, some message sequences, like those involved in an async
> invocations, need to have a way to keep the messages, in there, longer, and
> may be there is a server re-start, in the middle.
> Of course, this requirements spend on the scope of this effort.
> Plus, if you want to deal with historic data, and look at what was happening
> to message sequences over time, you got to use a persistence layer. Again
> this might be out of scope for this project, but if this is to be extended
> to support these, you better think of this right now.
>
>>
>> build the
>> message flow and update the UI accordingly. Perhaps there is a better
>> way ?
>
> We already do what you are trying to do, and much more with WSO2 BAM. I am
> tempted to say that Shinding based gadget dashboard is really a compelling
> UI for this.
> I am not trying to take away you project or trying to hinder this effort at
> all.
> I think it will be a good idea for you to have a look at BAM and see what
> the missing pieces are and figure out the right model for this tool.
> Also, having had worked on WSO2 BAM project, I also have some understanding
> on sort of problems that you might run into, when doing this. For e.g., if
> you want to correlate message sequences in a high volume setup, you will be
> swamped by the volume of messages, sooner than later. So correlation for the
> purpose of monitoring is not an easy thing to do. Correlation, for the
> purpose of debugging will not run into this problem, if you run this on a
> controlled environment. However, what you really want to debug is the deal
> clustered setup, and not a staged dev setup, the problem again surfaces.
> You might want to list the objectives and then break them down
> into monitoring and debugging spaces.
> Samisa...
>
>>
>> Thanks
>> Nilupa.
>>
>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
>> <sa...@gmail.com> wrote:
>> > What UI technology would be used for this?
>> >
>> > Samisa...
>> >
>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
>> > <an...@gmail.com> wrote:
>> >> I was thinking more about something like ITCAM for SOA.
>> >>
>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>> >> <sa...@opensource.lk> wrote:
>> >>> That runtime monitoring / correlation is what the WSO2 Business
>> >>> Activity
>> >>> Monitor does / is for ...
>> >>> See: http://wso2.com/products/business-activity-monitor/
>> >>> Sanjiva.
>> >>>
>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
>> >>> <an...@gmail.com>
>> >>> wrote:
>> >>>>
>> >>>> I was actually referring to scenarios where a service may call other
>> >>>> services, which in turn may call again other services. In these
>> >>>> scenarios, it is not sufficient to simply collect received/sent
>> >>>> messages on different hosts, unless the system is isolated
>> >>>> (development environment) or the request rate is sufficiently low so
>> >>>> that messages can be correlated based on time. Here is a concrete
>> >>>> scenario from a project in my company:
>> >>>>
>> >>>> - Request comes in on an ESB that does security and validation.
>> >>>> - Request is processed by an application server which persists the
>> >>>> received information and publishes a JMS message (pub/sub events).
>> >>>> - The event is consumed by one or more components that may in turn
>> >>>> interact with other services.
>> >>>>
>> >>>> If you want track this flow, it is not sufficient to intercept the
>> >>>> messages on the different hosts: in addition, you need to instrument
>> >>>> the services so that the outgoing requests are correlated with the
>> >>>> incoming requests (by adding SOAP and/or transport headers). What I
>> >>>> would like to be able to do in the above scenario is to enable
>> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> >>>> HTTP header to the initial request to enable logging and this
>> >>>> information is propagated along the chain. Every service/component
>> >>>> then sends a copy of the incoming/outgoing requests/responses to a
>> >>>> central place where the sequence of events is reconstructed.
>> >>>>
>> >>>> One way this could be achieved is with a tool that postprocesses the
>> >>>> artifacts from the build process. For each artifact, the tool would
>> >>>> disassemble the artifact, instrument the code using a set of AspectJ
>> >>>> aspects and reassemble the artifact for deployment. The
>> >>>> responsibility
>> >>>> of the aspects would be to intercept messages, log them and modify
>> >>>> them to ensure proper correlation. Of course this only works if the
>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> >>>> use AspectJ for a similar use case (although at a much smaller scale)
>> >>>> in the transport test suites [1]. Interestingly, this approach would
>> >>>> allow to cover interactions that are not SOAP based, e.g. one could
>> >>>> even intercept the database queries executed during the flow.
>> >>>>
>> >>>> Andreas
>> >>>>
>> >>>> [1]
>> >>>>
>> >>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>> >>>>
>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
>> >>>> wrote:
>> >>>> > Hi Nilupa;
>> >>>> >
>> >>>> > When we collect messages from a one location by installing proper
>> >>>> > handler that will intercept and send messages to to that one
>> >>>> > location
>> >>>> > (this one location can be a single server, pub/sub channel etc).
>> >>>> > There
>> >>>> > is many ways to make sense of those collected messages. What
>> >>>> > Andreas
>> >>>> > mentioned (following complete transaction) is a one possibility.
>> >>>> >
>> >>>> > I think you should come up with few scenarios on how you would make
>> >>>> > sense of the message.
>> >>>> >
>> >>>> > Thanks
>> >>>> > Srinath
>> >>>> >
>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>> >>>> > wrote:
>> >>>> >> Hi,
>> >>>> >>
>> >>>> >> First let me thank you for commenting.
>> >>>> >>
>> >>>> >> As far as I understood, what you would like to see from the
>> >>>> >> proposed
>> >>>> >> tool is to view set of messages that are exchanged in reponse to a
>> >>>> >> particular input message. With the understanding that I am having
>> >>>> >> at
>> >>>> >> the momnet, one way to do it is to filter out the central
>> >>>> >> repository
>> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>> >>>> >> message chain from it. We can allow the client GUI wich connects
>> >>>> >> to
>> >>>> >> the central repository to provide the paramenters (For instance
>> >>>> >> the
>> >>>> >> value of 'To' header) from which an intelligent filtering can be
>> >>>> >> done
>> >>>> >> for the set of messages avialable at the central repository.
>> >>>> >>
>> >>>> >> Perhaps someone has an idea of a better way of doing it and is
>> >>>> >> willing
>> >>>> >> to share it with us. It would be really nice to hear from them.
>> >>>> >>
>> >>>> >> Thanks in advance ..!!
>> >>>> >>
>> >>>> >> Best Regards,
>> >>>> >> Nilupa
>> >>>> >>
>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>> >>>> >> <an...@gmail.com> wrote:
>> >>>> >>> Personally, I think that the added value of extending the SOAP
>> >>>> >>> monitor
>> >>>> >>> module to collect messages in a central place is too limited to
>> >>>> >>> attract enough interest from the user community, so that it will
>> >>>> >>> be
>> >>>> >>> difficult to ensure the evolution of such a project in the
>> >>>> >>> future.
>> >>>> >>> However, what many people are looking for is a tool that allows
>> >>>> >>> to
>> >>>> >>> track the flow of a message through a distributed system, or more
>> >>>> >>> generally to track the sequence of events triggered by a given
>> >>>> >>> input
>> >>>> >>> message (sort of end-to-end transaction monitoring).
>> >>>> >>>
>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>> >>>> >>> wrote:
>> >>>> >>>> Hello,
>> >>>> >>>>
>> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
>> >>>> >>>> of
>> >>>> >>>> proposing a GSoC project this summer. I would like to take
>> >>>> >>>> project
>> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
>> >>>> >>>> and
>> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
>> >>>> >>>> Web
>> >>>> >>>> services middleware. I will submit more detailed proposal later.
>> >>>> >>>> I
>> >>>> >>>> would really like to hear any feedback from you. Any suggestions
>> >>>> >>>> or
>> >>>> >>>> features that you would like to see in a "Distributed TCP
>> >>>> >>>> Monitor"
>> >>>> >>>> are
>> >>>> >>>> most welcome.
>> >>>> >>>>
>> >>>> >>>> Best Regards,
>> >>>> >>>> Nilupa Bandara
>> >>>> >>>>
>> >>>> >>>> [1]
>> >>>> >>>>
>> >>>> >>>> Distributed TCP monitor
>> >>>> >>>>
>> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
>> >>>> >>>> hard
>> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
>> >>>> >>>> the
>> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
>> >>>> >>>> messages
>> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
>> >>>> >>>> can
>> >>>> >>>> pull those messages. Also, we can do something to turn the
>> >>>> >>>> module on
>> >>>> >>>> and off remotely, so users can have his module deployed, but
>> >>>> >>>> turn it
>> >>>> >>>> on only when they want to debug the system.
>> >>>> >>>>
>> >>>> >>>> Then we can take this to next level by adding this module to all
>> >>>> >>>> services in a system , configuring modules to send all collected
>> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
>> >>>> >>>> via
>> >>>> >>>> a
>> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
>> >>>> >>>> TCPMon for a whole system.
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>> ---------------------------------------------------------------------
>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>
>> >>>> >>>
>> >>>> >>> ---------------------------------------------------------------------
>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >>>
>> >>>> >>>
>> >>>> >>
>> >>>> >>
>> >>>> >> ---------------------------------------------------------------------
>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >>
>> >>>> >>
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > --
>> >>>> > ============================
>> >>>> > Srinath Perera, Ph.D.
>> >>>> >   WSO2 Inc. http://wso2.com
>> >>>> >   Blog: http://srinathsview.blogspot.com/
>> >>>> >
>> >>>> >
>> >>>> > ---------------------------------------------------------------------
>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>> >
>> >>>> >
>> >>>>
>> >>>> ---------------------------------------------------------------------
>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Sanjiva Weerawarana, Ph.D.
>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>> >>> http://www.opensource.lk/
>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>> >>> Member; Apache Software Foundation; http://www.apache.org/
>> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>> >>>
>> >>> Blog: http://sanjiva.weerawarana.org/
>> >>>
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
> Samisa...
> --
> blog: http://samisa-abeysinghe.blogspot.com/
>



-- 
Sanka Samaranayake

http://sankas.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:

> I was thinking about Java Swing based UI


Swing is a good start. But if you look at some of the requirements like
monitoring multiple instances and correlate them, discussed in this thread,
it sounds as if we would be better off having a Web interface, rather than a
desktop client.


> because the client may have
> to cache whatever messages it already has and will receive,


If you are looking to support message-correlation, as discussed in some
replies in this thread above, you need to have a persistence storage (a DB
or something like that) rather than a memory cache.
Because, some message sequences, like those involved in an async
invocations, need to have a way to keep the messages, in there, longer, and
may be there is a server re-start, in the middle.
Of course, this requirements spend on the scope of this effort.

Plus, if you want to deal with historic data, and look at what was happening
to message sequences over time, you got to use a persistence layer. Again
this might be out of scope for this project, but if this is to be extended
to support these, you better think of this right now.


> build the
> message flow and update the UI accordingly. Perhaps there is a better
> way ?


We already do what you are trying to do, and much more with WSO2 BAM. I am
tempted to say that Shinding based gadget dashboard is really a compelling
UI for this.

I am not trying to take away you project or trying to hinder this effort at
all.
I think it will be a good idea for you to have a look at BAM and see what
the missing pieces are and figure out the right model for this tool.

Also, having had worked on WSO2 BAM project, I also have some understanding
on sort of problems that you might run into, when doing this. For e.g., if
you want to correlate message sequences in a high volume setup, you will be
swamped by the volume of messages, sooner than later. So correlation for the
purpose of monitoring is not an easy thing to do. Correlation, for the
purpose of debugging will not run into this problem, if you run this on a
controlled environment. However, what you really want to debug is the deal
clustered setup, and not a staged dev setup, the problem again surfaces.

You might want to list the objectives and then break them down
into monitoring and debugging spaces.

Samisa...


>
> Thanks
> Nilupa.
>
> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> > What UI technology would be used for this?
> >
> > Samisa...
> >
> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
> > <an...@gmail.com> wrote:
> >> I was thinking more about something like ITCAM for SOA.
> >>
> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
> >> <sa...@opensource.lk> wrote:
> >>> That runtime monitoring / correlation is what the WSO2 Business
> Activity
> >>> Monitor does / is for ...
> >>> See: http://wso2.com/products/business-activity-monitor/
> >>> Sanjiva.
> >>>
> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <
> andreas.veithen@gmail.com>
> >>> wrote:
> >>>>
> >>>> I was actually referring to scenarios where a service may call other
> >>>> services, which in turn may call again other services. In these
> >>>> scenarios, it is not sufficient to simply collect received/sent
> >>>> messages on different hosts, unless the system is isolated
> >>>> (development environment) or the request rate is sufficiently low so
> >>>> that messages can be correlated based on time. Here is a concrete
> >>>> scenario from a project in my company:
> >>>>
> >>>> - Request comes in on an ESB that does security and validation.
> >>>> - Request is processed by an application server which persists the
> >>>> received information and publishes a JMS message (pub/sub events).
> >>>> - The event is consumed by one or more components that may in turn
> >>>> interact with other services.
> >>>>
> >>>> If you want track this flow, it is not sufficient to intercept the
> >>>> messages on the different hosts: in addition, you need to instrument
> >>>> the services so that the outgoing requests are correlated with the
> >>>> incoming requests (by adding SOAP and/or transport headers). What I
> >>>> would like to be able to do in the above scenario is to enable
> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
> >>>> HTTP header to the initial request to enable logging and this
> >>>> information is propagated along the chain. Every service/component
> >>>> then sends a copy of the incoming/outgoing requests/responses to a
> >>>> central place where the sequence of events is reconstructed.
> >>>>
> >>>> One way this could be achieved is with a tool that postprocesses the
> >>>> artifacts from the build process. For each artifact, the tool would
> >>>> disassemble the artifact, instrument the code using a set of AspectJ
> >>>> aspects and reassemble the artifact for deployment. The responsibility
> >>>> of the aspects would be to intercept messages, log them and modify
> >>>> them to ensure proper correlation. Of course this only works if the
> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
> >>>> use AspectJ for a similar use case (although at a much smaller scale)
> >>>> in the transport test suites [1]. Interestingly, this approach would
> >>>> allow to cover interactions that are not SOAP based, e.g. one could
> >>>> even intercept the database queries executed during the flow.
> >>>>
> >>>> Andreas
> >>>>
> >>>> [1]
> >>>>
> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> >>>>
> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
> wrote:
> >>>> > Hi Nilupa;
> >>>> >
> >>>> > When we collect messages from a one location by installing proper
> >>>> > handler that will intercept and send messages to to that one
> location
> >>>> > (this one location can be a single server, pub/sub channel etc).
> There
> >>>> > is many ways to make sense of those collected messages. What Andreas
> >>>> > mentioned (following complete transaction) is a one possibility.
> >>>> >
> >>>> > I think you should come up with few scenarios on how you would make
> >>>> > sense of the message.
> >>>> >
> >>>> > Thanks
> >>>> > Srinath
> >>>> >
> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
> >>>> > wrote:
> >>>> >> Hi,
> >>>> >>
> >>>> >> First let me thank you for commenting.
> >>>> >>
> >>>> >> As far as I understood, what you would like to see from the
> proposed
> >>>> >> tool is to view set of messages that are exchanged in reponse to a
> >>>> >> particular input message. With the understanding that I am having
> at
> >>>> >> the momnet, one way to do it is to filter out the central
> repository
> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
> >>>> >> message chain from it. We can allow the client GUI wich connects to
> >>>> >> the central repository to provide the paramenters (For instance the
> >>>> >> value of 'To' header) from which an intelligent filtering can be
> done
> >>>> >> for the set of messages avialable at the central repository.
> >>>> >>
> >>>> >> Perhaps someone has an idea of a better way of doing it and is
> willing
> >>>> >> to share it with us. It would be really nice to hear from them.
> >>>> >>
> >>>> >> Thanks in advance ..!!
> >>>> >>
> >>>> >> Best Regards,
> >>>> >> Nilupa
> >>>> >>
> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> >>>> >> <an...@gmail.com> wrote:
> >>>> >>> Personally, I think that the added value of extending the SOAP
> monitor
> >>>> >>> module to collect messages in a central place is too limited to
> >>>> >>> attract enough interest from the user community, so that it will
> be
> >>>> >>> difficult to ensure the evolution of such a project in the future.
> >>>> >>> However, what many people are looking for is a tool that allows to
> >>>> >>> track the flow of a message through a distributed system, or more
> >>>> >>> generally to track the sequence of events triggered by a given
> input
> >>>> >>> message (sort of end-to-end transaction monitoring).
> >>>> >>>
> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
> >>>> >>> wrote:
> >>>> >>>> Hello,
> >>>> >>>>
> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
> of
> >>>> >>>> proposing a GSoC project this summer. I would like to take
> project
> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
> and
> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
> Web
> >>>> >>>> services middleware. I will submit more detailed proposal later.
> I
> >>>> >>>> would really like to hear any feedback from you. Any suggestions
> or
> >>>> >>>> features that you would like to see in a "Distributed TCP
> Monitor"
> >>>> >>>> are
> >>>> >>>> most welcome.
> >>>> >>>>
> >>>> >>>> Best Regards,
> >>>> >>>> Nilupa Bandara
> >>>> >>>>
> >>>> >>>> [1]
> >>>> >>>>
> >>>> >>>> Distributed TCP monitor
> >>>> >>>>
> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
> hard
> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
> the
> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
> messages
> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
> can
> >>>> >>>> pull those messages. Also, we can do something to turn the module
> on
> >>>> >>>> and off remotely, so users can have his module deployed, but turn
> it
> >>>> >>>> on only when they want to debug the system.
> >>>> >>>>
> >>>> >>>> Then we can take this to next level by adding this module to all
> >>>> >>>> services in a system , configuring modules to send all collected
> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
> via
> >>>> >>>> a
> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
> >>>> >>>> TCPMon for a whole system.
> >>>> >>>>
> >>>> >>>>
> ---------------------------------------------------------------------
> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>>>
> >>>> >>>>
> >>>> >>>
> >>>> >>>
> ---------------------------------------------------------------------
> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>>
> >>>> >>>
> >>>> >>
> >>>> >>
> ---------------------------------------------------------------------
> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>
> >>>> >>
> >>>> >
> >>>> >
> >>>> >
> >>>> > --
> >>>> > ============================
> >>>> > Srinath Perera, Ph.D.
> >>>> >   WSO2 Inc. http://wso2.com
> >>>> >   Blog: http://srinathsview.blogspot.com/
> >>>> >
> >>>> >
> ---------------------------------------------------------------------
> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >
> >>>> >
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Sanjiva Weerawarana, Ph.D.
> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
> >>> http://www.opensource.lk/
> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> >>> Member; Apache Software Foundation; http://www.apache.org/
> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
> >>>
> >>> Blog: http://sanjiva.weerawarana.org/
> >>>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:

> I was thinking about Java Swing based UI


Swing is a good start. But if you look at some of the requirements like
monitoring multiple instances and correlate them, discussed in this thread,
it sounds as if we would be better off having a Web interface, rather than a
desktop client.


> because the client may have
> to cache whatever messages it already has and will receive,


If you are looking to support message-correlation, as discussed in some
replies in this thread above, you need to have a persistence storage (a DB
or something like that) rather than a memory cache.
Because, some message sequences, like those involved in an async
invocations, need to have a way to keep the messages, in there, longer, and
may be there is a server re-start, in the middle.
Of course, this requirements spend on the scope of this effort.

Plus, if you want to deal with historic data, and look at what was happening
to message sequences over time, you got to use a persistence layer. Again
this might be out of scope for this project, but if this is to be extended
to support these, you better think of this right now.


> build the
> message flow and update the UI accordingly. Perhaps there is a better
> way ?


We already do what you are trying to do, and much more with WSO2 BAM. I am
tempted to say that Shinding based gadget dashboard is really a compelling
UI for this.

I am not trying to take away you project or trying to hinder this effort at
all.
I think it will be a good idea for you to have a look at BAM and see what
the missing pieces are and figure out the right model for this tool.

Also, having had worked on WSO2 BAM project, I also have some understanding
on sort of problems that you might run into, when doing this. For e.g., if
you want to correlate message sequences in a high volume setup, you will be
swamped by the volume of messages, sooner than later. So correlation for the
purpose of monitoring is not an easy thing to do. Correlation, for the
purpose of debugging will not run into this problem, if you run this on a
controlled environment. However, what you really want to debug is the deal
clustered setup, and not a staged dev setup, the problem again surfaces.

You might want to list the objectives and then break them down
into monitoring and debugging spaces.

Samisa...


>
> Thanks
> Nilupa.
>
> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> > What UI technology would be used for this?
> >
> > Samisa...
> >
> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
> > <an...@gmail.com> wrote:
> >> I was thinking more about something like ITCAM for SOA.
> >>
> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
> >> <sa...@opensource.lk> wrote:
> >>> That runtime monitoring / correlation is what the WSO2 Business
> Activity
> >>> Monitor does / is for ...
> >>> See: http://wso2.com/products/business-activity-monitor/
> >>> Sanjiva.
> >>>
> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <
> andreas.veithen@gmail.com>
> >>> wrote:
> >>>>
> >>>> I was actually referring to scenarios where a service may call other
> >>>> services, which in turn may call again other services. In these
> >>>> scenarios, it is not sufficient to simply collect received/sent
> >>>> messages on different hosts, unless the system is isolated
> >>>> (development environment) or the request rate is sufficiently low so
> >>>> that messages can be correlated based on time. Here is a concrete
> >>>> scenario from a project in my company:
> >>>>
> >>>> - Request comes in on an ESB that does security and validation.
> >>>> - Request is processed by an application server which persists the
> >>>> received information and publishes a JMS message (pub/sub events).
> >>>> - The event is consumed by one or more components that may in turn
> >>>> interact with other services.
> >>>>
> >>>> If you want track this flow, it is not sufficient to intercept the
> >>>> messages on the different hosts: in addition, you need to instrument
> >>>> the services so that the outgoing requests are correlated with the
> >>>> incoming requests (by adding SOAP and/or transport headers). What I
> >>>> would like to be able to do in the above scenario is to enable
> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
> >>>> HTTP header to the initial request to enable logging and this
> >>>> information is propagated along the chain. Every service/component
> >>>> then sends a copy of the incoming/outgoing requests/responses to a
> >>>> central place where the sequence of events is reconstructed.
> >>>>
> >>>> One way this could be achieved is with a tool that postprocesses the
> >>>> artifacts from the build process. For each artifact, the tool would
> >>>> disassemble the artifact, instrument the code using a set of AspectJ
> >>>> aspects and reassemble the artifact for deployment. The responsibility
> >>>> of the aspects would be to intercept messages, log them and modify
> >>>> them to ensure proper correlation. Of course this only works if the
> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
> >>>> use AspectJ for a similar use case (although at a much smaller scale)
> >>>> in the transport test suites [1]. Interestingly, this approach would
> >>>> allow to cover interactions that are not SOAP based, e.g. one could
> >>>> even intercept the database queries executed during the flow.
> >>>>
> >>>> Andreas
> >>>>
> >>>> [1]
> >>>>
> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> >>>>
> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
> wrote:
> >>>> > Hi Nilupa;
> >>>> >
> >>>> > When we collect messages from a one location by installing proper
> >>>> > handler that will intercept and send messages to to that one
> location
> >>>> > (this one location can be a single server, pub/sub channel etc).
> There
> >>>> > is many ways to make sense of those collected messages. What Andreas
> >>>> > mentioned (following complete transaction) is a one possibility.
> >>>> >
> >>>> > I think you should come up with few scenarios on how you would make
> >>>> > sense of the message.
> >>>> >
> >>>> > Thanks
> >>>> > Srinath
> >>>> >
> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
> >>>> > wrote:
> >>>> >> Hi,
> >>>> >>
> >>>> >> First let me thank you for commenting.
> >>>> >>
> >>>> >> As far as I understood, what you would like to see from the
> proposed
> >>>> >> tool is to view set of messages that are exchanged in reponse to a
> >>>> >> particular input message. With the understanding that I am having
> at
> >>>> >> the momnet, one way to do it is to filter out the central
> repository
> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
> >>>> >> message chain from it. We can allow the client GUI wich connects to
> >>>> >> the central repository to provide the paramenters (For instance the
> >>>> >> value of 'To' header) from which an intelligent filtering can be
> done
> >>>> >> for the set of messages avialable at the central repository.
> >>>> >>
> >>>> >> Perhaps someone has an idea of a better way of doing it and is
> willing
> >>>> >> to share it with us. It would be really nice to hear from them.
> >>>> >>
> >>>> >> Thanks in advance ..!!
> >>>> >>
> >>>> >> Best Regards,
> >>>> >> Nilupa
> >>>> >>
> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> >>>> >> <an...@gmail.com> wrote:
> >>>> >>> Personally, I think that the added value of extending the SOAP
> monitor
> >>>> >>> module to collect messages in a central place is too limited to
> >>>> >>> attract enough interest from the user community, so that it will
> be
> >>>> >>> difficult to ensure the evolution of such a project in the future.
> >>>> >>> However, what many people are looking for is a tool that allows to
> >>>> >>> track the flow of a message through a distributed system, or more
> >>>> >>> generally to track the sequence of events triggered by a given
> input
> >>>> >>> message (sort of end-to-end transaction monitoring).
> >>>> >>>
> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
> >>>> >>> wrote:
> >>>> >>>> Hello,
> >>>> >>>>
> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
> of
> >>>> >>>> proposing a GSoC project this summer. I would like to take
> project
> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
> and
> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
> Web
> >>>> >>>> services middleware. I will submit more detailed proposal later.
> I
> >>>> >>>> would really like to hear any feedback from you. Any suggestions
> or
> >>>> >>>> features that you would like to see in a "Distributed TCP
> Monitor"
> >>>> >>>> are
> >>>> >>>> most welcome.
> >>>> >>>>
> >>>> >>>> Best Regards,
> >>>> >>>> Nilupa Bandara
> >>>> >>>>
> >>>> >>>> [1]
> >>>> >>>>
> >>>> >>>> Distributed TCP monitor
> >>>> >>>>
> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
> hard
> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
> the
> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
> messages
> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
> can
> >>>> >>>> pull those messages. Also, we can do something to turn the module
> on
> >>>> >>>> and off remotely, so users can have his module deployed, but turn
> it
> >>>> >>>> on only when they want to debug the system.
> >>>> >>>>
> >>>> >>>> Then we can take this to next level by adding this module to all
> >>>> >>>> services in a system , configuring modules to send all collected
> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
> via
> >>>> >>>> a
> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
> >>>> >>>> TCPMon for a whole system.
> >>>> >>>>
> >>>> >>>>
> ---------------------------------------------------------------------
> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>>>
> >>>> >>>>
> >>>> >>>
> >>>> >>>
> ---------------------------------------------------------------------
> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>>
> >>>> >>>
> >>>> >>
> >>>> >>
> ---------------------------------------------------------------------
> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>
> >>>> >>
> >>>> >
> >>>> >
> >>>> >
> >>>> > --
> >>>> > ============================
> >>>> > Srinath Perera, Ph.D.
> >>>> >   WSO2 Inc. http://wso2.com
> >>>> >   Blog: http://srinathsview.blogspot.com/
> >>>> >
> >>>> >
> ---------------------------------------------------------------------
> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >
> >>>> >
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Sanjiva Weerawarana, Ph.D.
> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
> >>> http://www.opensource.lk/
> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> >>> Member; Apache Software Foundation; http://www.apache.org/
> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
> >>>
> >>> Blog: http://sanjiva.weerawarana.org/
> >>>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:

> I was thinking about Java Swing based UI


Swing is a good start. But if you look at some of the requirements like
monitoring multiple instances and correlate them, discussed in this thread,
it sounds as if we would be better off having a Web interface, rather than a
desktop client.


> because the client may have
> to cache whatever messages it already has and will receive,


If you are looking to support message-correlation, as discussed in some
replies in this thread above, you need to have a persistence storage (a DB
or something like that) rather than a memory cache.
Because, some message sequences, like those involved in an async
invocations, need to have a way to keep the messages, in there, longer, and
may be there is a server re-start, in the middle.
Of course, this requirements spend on the scope of this effort.

Plus, if you want to deal with historic data, and look at what was happening
to message sequences over time, you got to use a persistence layer. Again
this might be out of scope for this project, but if this is to be extended
to support these, you better think of this right now.


> build the
> message flow and update the UI accordingly. Perhaps there is a better
> way ?


We already do what you are trying to do, and much more with WSO2 BAM. I am
tempted to say that Shinding based gadget dashboard is really a compelling
UI for this.

I am not trying to take away you project or trying to hinder this effort at
all.
I think it will be a good idea for you to have a look at BAM and see what
the missing pieces are and figure out the right model for this tool.

Also, having had worked on WSO2 BAM project, I also have some understanding
on sort of problems that you might run into, when doing this. For e.g., if
you want to correlate message sequences in a high volume setup, you will be
swamped by the volume of messages, sooner than later. So correlation for the
purpose of monitoring is not an easy thing to do. Correlation, for the
purpose of debugging will not run into this problem, if you run this on a
controlled environment. However, what you really want to debug is the deal
clustered setup, and not a staged dev setup, the problem again surfaces.

You might want to list the objectives and then break them down
into monitoring and debugging spaces.

Samisa...


>
> Thanks
> Nilupa.
>
> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> > What UI technology would be used for this?
> >
> > Samisa...
> >
> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
> > <an...@gmail.com> wrote:
> >> I was thinking more about something like ITCAM for SOA.
> >>
> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
> >> <sa...@opensource.lk> wrote:
> >>> That runtime monitoring / correlation is what the WSO2 Business
> Activity
> >>> Monitor does / is for ...
> >>> See: http://wso2.com/products/business-activity-monitor/
> >>> Sanjiva.
> >>>
> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <
> andreas.veithen@gmail.com>
> >>> wrote:
> >>>>
> >>>> I was actually referring to scenarios where a service may call other
> >>>> services, which in turn may call again other services. In these
> >>>> scenarios, it is not sufficient to simply collect received/sent
> >>>> messages on different hosts, unless the system is isolated
> >>>> (development environment) or the request rate is sufficiently low so
> >>>> that messages can be correlated based on time. Here is a concrete
> >>>> scenario from a project in my company:
> >>>>
> >>>> - Request comes in on an ESB that does security and validation.
> >>>> - Request is processed by an application server which persists the
> >>>> received information and publishes a JMS message (pub/sub events).
> >>>> - The event is consumed by one or more components that may in turn
> >>>> interact with other services.
> >>>>
> >>>> If you want track this flow, it is not sufficient to intercept the
> >>>> messages on the different hosts: in addition, you need to instrument
> >>>> the services so that the outgoing requests are correlated with the
> >>>> incoming requests (by adding SOAP and/or transport headers). What I
> >>>> would like to be able to do in the above scenario is to enable
> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
> >>>> HTTP header to the initial request to enable logging and this
> >>>> information is propagated along the chain. Every service/component
> >>>> then sends a copy of the incoming/outgoing requests/responses to a
> >>>> central place where the sequence of events is reconstructed.
> >>>>
> >>>> One way this could be achieved is with a tool that postprocesses the
> >>>> artifacts from the build process. For each artifact, the tool would
> >>>> disassemble the artifact, instrument the code using a set of AspectJ
> >>>> aspects and reassemble the artifact for deployment. The responsibility
> >>>> of the aspects would be to intercept messages, log them and modify
> >>>> them to ensure proper correlation. Of course this only works if the
> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
> >>>> use AspectJ for a similar use case (although at a much smaller scale)
> >>>> in the transport test suites [1]. Interestingly, this approach would
> >>>> allow to cover interactions that are not SOAP based, e.g. one could
> >>>> even intercept the database queries executed during the flow.
> >>>>
> >>>> Andreas
> >>>>
> >>>> [1]
> >>>>
> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> >>>>
> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
> wrote:
> >>>> > Hi Nilupa;
> >>>> >
> >>>> > When we collect messages from a one location by installing proper
> >>>> > handler that will intercept and send messages to to that one
> location
> >>>> > (this one location can be a single server, pub/sub channel etc).
> There
> >>>> > is many ways to make sense of those collected messages. What Andreas
> >>>> > mentioned (following complete transaction) is a one possibility.
> >>>> >
> >>>> > I think you should come up with few scenarios on how you would make
> >>>> > sense of the message.
> >>>> >
> >>>> > Thanks
> >>>> > Srinath
> >>>> >
> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
> >>>> > wrote:
> >>>> >> Hi,
> >>>> >>
> >>>> >> First let me thank you for commenting.
> >>>> >>
> >>>> >> As far as I understood, what you would like to see from the
> proposed
> >>>> >> tool is to view set of messages that are exchanged in reponse to a
> >>>> >> particular input message. With the understanding that I am having
> at
> >>>> >> the momnet, one way to do it is to filter out the central
> repository
> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
> >>>> >> message chain from it. We can allow the client GUI wich connects to
> >>>> >> the central repository to provide the paramenters (For instance the
> >>>> >> value of 'To' header) from which an intelligent filtering can be
> done
> >>>> >> for the set of messages avialable at the central repository.
> >>>> >>
> >>>> >> Perhaps someone has an idea of a better way of doing it and is
> willing
> >>>> >> to share it with us. It would be really nice to hear from them.
> >>>> >>
> >>>> >> Thanks in advance ..!!
> >>>> >>
> >>>> >> Best Regards,
> >>>> >> Nilupa
> >>>> >>
> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> >>>> >> <an...@gmail.com> wrote:
> >>>> >>> Personally, I think that the added value of extending the SOAP
> monitor
> >>>> >>> module to collect messages in a central place is too limited to
> >>>> >>> attract enough interest from the user community, so that it will
> be
> >>>> >>> difficult to ensure the evolution of such a project in the future.
> >>>> >>> However, what many people are looking for is a tool that allows to
> >>>> >>> track the flow of a message through a distributed system, or more
> >>>> >>> generally to track the sequence of events triggered by a given
> input
> >>>> >>> message (sort of end-to-end transaction monitoring).
> >>>> >>>
> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
> >>>> >>> wrote:
> >>>> >>>> Hello,
> >>>> >>>>
> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
> of
> >>>> >>>> proposing a GSoC project this summer. I would like to take
> project
> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
> and
> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
> Web
> >>>> >>>> services middleware. I will submit more detailed proposal later.
> I
> >>>> >>>> would really like to hear any feedback from you. Any suggestions
> or
> >>>> >>>> features that you would like to see in a "Distributed TCP
> Monitor"
> >>>> >>>> are
> >>>> >>>> most welcome.
> >>>> >>>>
> >>>> >>>> Best Regards,
> >>>> >>>> Nilupa Bandara
> >>>> >>>>
> >>>> >>>> [1]
> >>>> >>>>
> >>>> >>>> Distributed TCP monitor
> >>>> >>>>
> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
> hard
> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
> the
> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
> messages
> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
> can
> >>>> >>>> pull those messages. Also, we can do something to turn the module
> on
> >>>> >>>> and off remotely, so users can have his module deployed, but turn
> it
> >>>> >>>> on only when they want to debug the system.
> >>>> >>>>
> >>>> >>>> Then we can take this to next level by adding this module to all
> >>>> >>>> services in a system , configuring modules to send all collected
> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
> via
> >>>> >>>> a
> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
> >>>> >>>> TCPMon for a whole system.
> >>>> >>>>
> >>>> >>>>
> ---------------------------------------------------------------------
> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>>>
> >>>> >>>>
> >>>> >>>
> >>>> >>>
> ---------------------------------------------------------------------
> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>>
> >>>> >>>
> >>>> >>
> >>>> >>
> ---------------------------------------------------------------------
> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>
> >>>> >>
> >>>> >
> >>>> >
> >>>> >
> >>>> > --
> >>>> > ============================
> >>>> > Srinath Perera, Ph.D.
> >>>> >   WSO2 Inc. http://wso2.com
> >>>> >   Blog: http://srinathsview.blogspot.com/
> >>>> >
> >>>> >
> ---------------------------------------------------------------------
> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >
> >>>> >
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Sanjiva Weerawarana, Ph.D.
> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
> >>> http://www.opensource.lk/
> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> >>> Member; Apache Software Foundation; http://www.apache.org/
> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
> >>>
> >>> Blog: http://sanjiva.weerawarana.org/
> >>>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:

> I was thinking about Java Swing based UI


Swing is a good start. But if you look at some of the requirements like
monitoring multiple instances and correlate them, discussed in this thread,
it sounds as if we would be better off having a Web interface, rather than a
desktop client.


> because the client may have
> to cache whatever messages it already has and will receive,


If you are looking to support message-correlation, as discussed in some
replies in this thread above, you need to have a persistence storage (a DB
or something like that) rather than a memory cache.
Because, some message sequences, like those involved in an async
invocations, need to have a way to keep the messages, in there, longer, and
may be there is a server re-start, in the middle.
Of course, this requirements spend on the scope of this effort.

Plus, if you want to deal with historic data, and look at what was happening
to message sequences over time, you got to use a persistence layer. Again
this might be out of scope for this project, but if this is to be extended
to support these, you better think of this right now.


> build the
> message flow and update the UI accordingly. Perhaps there is a better
> way ?


We already do what you are trying to do, and much more with WSO2 BAM. I am
tempted to say that Shinding based gadget dashboard is really a compelling
UI for this.

I am not trying to take away you project or trying to hinder this effort at
all.
I think it will be a good idea for you to have a look at BAM and see what
the missing pieces are and figure out the right model for this tool.

Also, having had worked on WSO2 BAM project, I also have some understanding
on sort of problems that you might run into, when doing this. For e.g., if
you want to correlate message sequences in a high volume setup, you will be
swamped by the volume of messages, sooner than later. So correlation for the
purpose of monitoring is not an easy thing to do. Correlation, for the
purpose of debugging will not run into this problem, if you run this on a
controlled environment. However, what you really want to debug is the deal
clustered setup, and not a staged dev setup, the problem again surfaces.

You might want to list the objectives and then break them down
into monitoring and debugging spaces.

Samisa...


>
> Thanks
> Nilupa.
>
> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> > What UI technology would be used for this?
> >
> > Samisa...
> >
> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
> > <an...@gmail.com> wrote:
> >> I was thinking more about something like ITCAM for SOA.
> >>
> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
> >> <sa...@opensource.lk> wrote:
> >>> That runtime monitoring / correlation is what the WSO2 Business
> Activity
> >>> Monitor does / is for ...
> >>> See: http://wso2.com/products/business-activity-monitor/
> >>> Sanjiva.
> >>>
> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <
> andreas.veithen@gmail.com>
> >>> wrote:
> >>>>
> >>>> I was actually referring to scenarios where a service may call other
> >>>> services, which in turn may call again other services. In these
> >>>> scenarios, it is not sufficient to simply collect received/sent
> >>>> messages on different hosts, unless the system is isolated
> >>>> (development environment) or the request rate is sufficiently low so
> >>>> that messages can be correlated based on time. Here is a concrete
> >>>> scenario from a project in my company:
> >>>>
> >>>> - Request comes in on an ESB that does security and validation.
> >>>> - Request is processed by an application server which persists the
> >>>> received information and publishes a JMS message (pub/sub events).
> >>>> - The event is consumed by one or more components that may in turn
> >>>> interact with other services.
> >>>>
> >>>> If you want track this flow, it is not sufficient to intercept the
> >>>> messages on the different hosts: in addition, you need to instrument
> >>>> the services so that the outgoing requests are correlated with the
> >>>> incoming requests (by adding SOAP and/or transport headers). What I
> >>>> would like to be able to do in the above scenario is to enable
> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
> >>>> HTTP header to the initial request to enable logging and this
> >>>> information is propagated along the chain. Every service/component
> >>>> then sends a copy of the incoming/outgoing requests/responses to a
> >>>> central place where the sequence of events is reconstructed.
> >>>>
> >>>> One way this could be achieved is with a tool that postprocesses the
> >>>> artifacts from the build process. For each artifact, the tool would
> >>>> disassemble the artifact, instrument the code using a set of AspectJ
> >>>> aspects and reassemble the artifact for deployment. The responsibility
> >>>> of the aspects would be to intercept messages, log them and modify
> >>>> them to ensure proper correlation. Of course this only works if the
> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
> >>>> use AspectJ for a similar use case (although at a much smaller scale)
> >>>> in the transport test suites [1]. Interestingly, this approach would
> >>>> allow to cover interactions that are not SOAP based, e.g. one could
> >>>> even intercept the database queries executed during the flow.
> >>>>
> >>>> Andreas
> >>>>
> >>>> [1]
> >>>>
> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> >>>>
> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
> wrote:
> >>>> > Hi Nilupa;
> >>>> >
> >>>> > When we collect messages from a one location by installing proper
> >>>> > handler that will intercept and send messages to to that one
> location
> >>>> > (this one location can be a single server, pub/sub channel etc).
> There
> >>>> > is many ways to make sense of those collected messages. What Andreas
> >>>> > mentioned (following complete transaction) is a one possibility.
> >>>> >
> >>>> > I think you should come up with few scenarios on how you would make
> >>>> > sense of the message.
> >>>> >
> >>>> > Thanks
> >>>> > Srinath
> >>>> >
> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
> >>>> > wrote:
> >>>> >> Hi,
> >>>> >>
> >>>> >> First let me thank you for commenting.
> >>>> >>
> >>>> >> As far as I understood, what you would like to see from the
> proposed
> >>>> >> tool is to view set of messages that are exchanged in reponse to a
> >>>> >> particular input message. With the understanding that I am having
> at
> >>>> >> the momnet, one way to do it is to filter out the central
> repository
> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
> >>>> >> message chain from it. We can allow the client GUI wich connects to
> >>>> >> the central repository to provide the paramenters (For instance the
> >>>> >> value of 'To' header) from which an intelligent filtering can be
> done
> >>>> >> for the set of messages avialable at the central repository.
> >>>> >>
> >>>> >> Perhaps someone has an idea of a better way of doing it and is
> willing
> >>>> >> to share it with us. It would be really nice to hear from them.
> >>>> >>
> >>>> >> Thanks in advance ..!!
> >>>> >>
> >>>> >> Best Regards,
> >>>> >> Nilupa
> >>>> >>
> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> >>>> >> <an...@gmail.com> wrote:
> >>>> >>> Personally, I think that the added value of extending the SOAP
> monitor
> >>>> >>> module to collect messages in a central place is too limited to
> >>>> >>> attract enough interest from the user community, so that it will
> be
> >>>> >>> difficult to ensure the evolution of such a project in the future.
> >>>> >>> However, what many people are looking for is a tool that allows to
> >>>> >>> track the flow of a message through a distributed system, or more
> >>>> >>> generally to track the sequence of events triggered by a given
> input
> >>>> >>> message (sort of end-to-end transaction monitoring).
> >>>> >>>
> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
> >>>> >>> wrote:
> >>>> >>>> Hello,
> >>>> >>>>
> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
> of
> >>>> >>>> proposing a GSoC project this summer. I would like to take
> project
> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
> and
> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
> Web
> >>>> >>>> services middleware. I will submit more detailed proposal later.
> I
> >>>> >>>> would really like to hear any feedback from you. Any suggestions
> or
> >>>> >>>> features that you would like to see in a "Distributed TCP
> Monitor"
> >>>> >>>> are
> >>>> >>>> most welcome.
> >>>> >>>>
> >>>> >>>> Best Regards,
> >>>> >>>> Nilupa Bandara
> >>>> >>>>
> >>>> >>>> [1]
> >>>> >>>>
> >>>> >>>> Distributed TCP monitor
> >>>> >>>>
> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
> hard
> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
> the
> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
> messages
> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
> can
> >>>> >>>> pull those messages. Also, we can do something to turn the module
> on
> >>>> >>>> and off remotely, so users can have his module deployed, but turn
> it
> >>>> >>>> on only when they want to debug the system.
> >>>> >>>>
> >>>> >>>> Then we can take this to next level by adding this module to all
> >>>> >>>> services in a system , configuring modules to send all collected
> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
> via
> >>>> >>>> a
> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
> >>>> >>>> TCPMon for a whole system.
> >>>> >>>>
> >>>> >>>>
> ---------------------------------------------------------------------
> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>>>
> >>>> >>>>
> >>>> >>>
> >>>> >>>
> ---------------------------------------------------------------------
> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>>
> >>>> >>>
> >>>> >>
> >>>> >>
> ---------------------------------------------------------------------
> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>
> >>>> >>
> >>>> >
> >>>> >
> >>>> >
> >>>> > --
> >>>> > ============================
> >>>> > Srinath Perera, Ph.D.
> >>>> >   WSO2 Inc. http://wso2.com
> >>>> >   Blog: http://srinathsview.blogspot.com/
> >>>> >
> >>>> >
> ---------------------------------------------------------------------
> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >
> >>>> >
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Sanjiva Weerawarana, Ph.D.
> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
> >>> http://www.opensource.lk/
> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> >>> Member; Apache Software Foundation; http://www.apache.org/
> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
> >>>
> >>> Blog: http://sanjiva.weerawarana.org/
> >>>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <ni...@gmail.com> wrote:

> I was thinking about Java Swing based UI


Swing is a good start. But if you look at some of the requirements like
monitoring multiple instances and correlate them, discussed in this thread,
it sounds as if we would be better off having a Web interface, rather than a
desktop client.


> because the client may have
> to cache whatever messages it already has and will receive,


If you are looking to support message-correlation, as discussed in some
replies in this thread above, you need to have a persistence storage (a DB
or something like that) rather than a memory cache.
Because, some message sequences, like those involved in an async
invocations, need to have a way to keep the messages, in there, longer, and
may be there is a server re-start, in the middle.
Of course, this requirements spend on the scope of this effort.

Plus, if you want to deal with historic data, and look at what was happening
to message sequences over time, you got to use a persistence layer. Again
this might be out of scope for this project, but if this is to be extended
to support these, you better think of this right now.


> build the
> message flow and update the UI accordingly. Perhaps there is a better
> way ?


We already do what you are trying to do, and much more with WSO2 BAM. I am
tempted to say that Shinding based gadget dashboard is really a compelling
UI for this.

I am not trying to take away you project or trying to hinder this effort at
all.
I think it will be a good idea for you to have a look at BAM and see what
the missing pieces are and figure out the right model for this tool.

Also, having had worked on WSO2 BAM project, I also have some understanding
on sort of problems that you might run into, when doing this. For e.g., if
you want to correlate message sequences in a high volume setup, you will be
swamped by the volume of messages, sooner than later. So correlation for the
purpose of monitoring is not an easy thing to do. Correlation, for the
purpose of debugging will not run into this problem, if you run this on a
controlled environment. However, what you really want to debug is the deal
clustered setup, and not a staged dev setup, the problem again surfaces.

You might want to list the objectives and then break them down
into monitoring and debugging spaces.

Samisa...


>
> Thanks
> Nilupa.
>
> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
> <sa...@gmail.com> wrote:
> > What UI technology would be used for this?
> >
> > Samisa...
> >
> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
> > <an...@gmail.com> wrote:
> >> I was thinking more about something like ITCAM for SOA.
> >>
> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
> >> <sa...@opensource.lk> wrote:
> >>> That runtime monitoring / correlation is what the WSO2 Business
> Activity
> >>> Monitor does / is for ...
> >>> See: http://wso2.com/products/business-activity-monitor/
> >>> Sanjiva.
> >>>
> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <
> andreas.veithen@gmail.com>
> >>> wrote:
> >>>>
> >>>> I was actually referring to scenarios where a service may call other
> >>>> services, which in turn may call again other services. In these
> >>>> scenarios, it is not sufficient to simply collect received/sent
> >>>> messages on different hosts, unless the system is isolated
> >>>> (development environment) or the request rate is sufficiently low so
> >>>> that messages can be correlated based on time. Here is a concrete
> >>>> scenario from a project in my company:
> >>>>
> >>>> - Request comes in on an ESB that does security and validation.
> >>>> - Request is processed by an application server which persists the
> >>>> received information and publishes a JMS message (pub/sub events).
> >>>> - The event is consumed by one or more components that may in turn
> >>>> interact with other services.
> >>>>
> >>>> If you want track this flow, it is not sufficient to intercept the
> >>>> messages on the different hosts: in addition, you need to instrument
> >>>> the services so that the outgoing requests are correlated with the
> >>>> incoming requests (by adding SOAP and/or transport headers). What I
> >>>> would like to be able to do in the above scenario is to enable
> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
> >>>> HTTP header to the initial request to enable logging and this
> >>>> information is propagated along the chain. Every service/component
> >>>> then sends a copy of the incoming/outgoing requests/responses to a
> >>>> central place where the sequence of events is reconstructed.
> >>>>
> >>>> One way this could be achieved is with a tool that postprocesses the
> >>>> artifacts from the build process. For each artifact, the tool would
> >>>> disassemble the artifact, instrument the code using a set of AspectJ
> >>>> aspects and reassemble the artifact for deployment. The responsibility
> >>>> of the aspects would be to intercept messages, log them and modify
> >>>> them to ensure proper correlation. Of course this only works if the
> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
> >>>> use AspectJ for a similar use case (although at a much smaller scale)
> >>>> in the transport test suites [1]. Interestingly, this approach would
> >>>> allow to cover interactions that are not SOAP based, e.g. one could
> >>>> even intercept the database queries executed during the flow.
> >>>>
> >>>> Andreas
> >>>>
> >>>> [1]
> >>>>
> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> >>>>
> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com>
> wrote:
> >>>> > Hi Nilupa;
> >>>> >
> >>>> > When we collect messages from a one location by installing proper
> >>>> > handler that will intercept and send messages to to that one
> location
> >>>> > (this one location can be a single server, pub/sub channel etc).
> There
> >>>> > is many ways to make sense of those collected messages. What Andreas
> >>>> > mentioned (following complete transaction) is a one possibility.
> >>>> >
> >>>> > I think you should come up with few scenarios on how you would make
> >>>> > sense of the message.
> >>>> >
> >>>> > Thanks
> >>>> > Srinath
> >>>> >
> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
> >>>> > wrote:
> >>>> >> Hi,
> >>>> >>
> >>>> >> First let me thank you for commenting.
> >>>> >>
> >>>> >> As far as I understood, what you would like to see from the
> proposed
> >>>> >> tool is to view set of messages that are exchanged in reponse to a
> >>>> >> particular input message. With the understanding that I am having
> at
> >>>> >> the momnet, one way to do it is to filter out the central
> repository
> >>>> >> of messages based on 'To' , 'From' headers and try to contruct the
> >>>> >> message chain from it. We can allow the client GUI wich connects to
> >>>> >> the central repository to provide the paramenters (For instance the
> >>>> >> value of 'To' header) from which an intelligent filtering can be
> done
> >>>> >> for the set of messages avialable at the central repository.
> >>>> >>
> >>>> >> Perhaps someone has an idea of a better way of doing it and is
> willing
> >>>> >> to share it with us. It would be really nice to hear from them.
> >>>> >>
> >>>> >> Thanks in advance ..!!
> >>>> >>
> >>>> >> Best Regards,
> >>>> >> Nilupa
> >>>> >>
> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> >>>> >> <an...@gmail.com> wrote:
> >>>> >>> Personally, I think that the added value of extending the SOAP
> monitor
> >>>> >>> module to collect messages in a central place is too limited to
> >>>> >>> attract enough interest from the user community, so that it will
> be
> >>>> >>> difficult to ensure the evolution of such a project in the future.
> >>>> >>> However, what many people are looking for is a tool that allows to
> >>>> >>> track the flow of a message through a distributed system, or more
> >>>> >>> generally to track the sequence of events triggered by a given
> input
> >>>> >>> message (sort of end-to-end transaction monitoring).
> >>>> >>>
> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
> >>>> >>> wrote:
> >>>> >>>> Hello,
> >>>> >>>>
> >>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking
> of
> >>>> >>>> proposing a GSoC project this summer. I would like to take
> project
> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting
> and
> >>>> >>>> also should be helpful to any Java developer using Apache Axis2
> Web
> >>>> >>>> services middleware. I will submit more detailed proposal later.
> I
> >>>> >>>> would really like to hear any feedback from you. Any suggestions
> or
> >>>> >>>> features that you would like to see in a "Distributed TCP
> Monitor"
> >>>> >>>> are
> >>>> >>>> most welcome.
> >>>> >>>>
> >>>> >>>> Best Regards,
> >>>> >>>> Nilupa Bandara
> >>>> >>>>
> >>>> >>>> [1]
> >>>> >>>>
> >>>> >>>> Distributed TCP monitor
> >>>> >>>>
> >>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit
> hard
> >>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve
> the
> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
> messages
> >>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user
> can
> >>>> >>>> pull those messages. Also, we can do something to turn the module
> on
> >>>> >>>> and off remotely, so users can have his module deployed, but turn
> it
> >>>> >>>> on only when they want to debug the system.
> >>>> >>>>
> >>>> >>>> Then we can take this to next level by adding this module to all
> >>>> >>>> services in a system , configuring modules to send all collected
> >>>> >>>> messages to a pub/sub channel, and subscribing to those messages
> via
> >>>> >>>> a
> >>>> >>>> UI, and depicting those messages through a UI. Then users have a
> >>>> >>>> TCPMon for a whole system.
> >>>> >>>>
> >>>> >>>>
> ---------------------------------------------------------------------
> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>>>
> >>>> >>>>
> >>>> >>>
> >>>> >>>
> ---------------------------------------------------------------------
> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>>
> >>>> >>>
> >>>> >>
> >>>> >>
> ---------------------------------------------------------------------
> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>
> >>>> >>
> >>>> >
> >>>> >
> >>>> >
> >>>> > --
> >>>> > ============================
> >>>> > Srinath Perera, Ph.D.
> >>>> >   WSO2 Inc. http://wso2.com
> >>>> >   Blog: http://srinathsview.blogspot.com/
> >>>> >
> >>>> >
> ---------------------------------------------------------------------
> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >
> >>>> >
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Sanjiva Weerawarana, Ph.D.
> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
> >>> http://www.opensource.lk/
> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> >>> Member; Apache Software Foundation; http://www.apache.org/
> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
> >>>
> >>> Blog: http://sanjiva.weerawarana.org/
> >>>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

I was thinking about Java Swing based UI because the client may have
to cache whatever messages it already has and will receive, build the
message flow and update the UI accordingly. Perhaps there is a better
way ?

Thanks
Nilupa.

On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
> What UI technology would be used for this?
>
> Samisa...
>
> On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
> <an...@gmail.com> wrote:
>> I was thinking more about something like ITCAM for SOA.
>>
>> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>> <sa...@opensource.lk> wrote:
>>> That runtime monitoring / correlation is what the WSO2 Business Activity
>>> Monitor does / is for ...
>>> See: http://wso2.com/products/business-activity-monitor/
>>> Sanjiva.
>>>
>>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <an...@gmail.com>
>>> wrote:
>>>>
>>>> I was actually referring to scenarios where a service may call other
>>>> services, which in turn may call again other services. In these
>>>> scenarios, it is not sufficient to simply collect received/sent
>>>> messages on different hosts, unless the system is isolated
>>>> (development environment) or the request rate is sufficiently low so
>>>> that messages can be correlated based on time. Here is a concrete
>>>> scenario from a project in my company:
>>>>
>>>> - Request comes in on an ESB that does security and validation.
>>>> - Request is processed by an application server which persists the
>>>> received information and publishes a JMS message (pub/sub events).
>>>> - The event is consumed by one or more components that may in turn
>>>> interact with other services.
>>>>
>>>> If you want track this flow, it is not sufficient to intercept the
>>>> messages on the different hosts: in addition, you need to instrument
>>>> the services so that the outgoing requests are correlated with the
>>>> incoming requests (by adding SOAP and/or transport headers). What I
>>>> would like to be able to do in the above scenario is to enable
>>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>>> HTTP header to the initial request to enable logging and this
>>>> information is propagated along the chain. Every service/component
>>>> then sends a copy of the incoming/outgoing requests/responses to a
>>>> central place where the sequence of events is reconstructed.
>>>>
>>>> One way this could be achieved is with a tool that postprocesses the
>>>> artifacts from the build process. For each artifact, the tool would
>>>> disassemble the artifact, instrument the code using a set of AspectJ
>>>> aspects and reassemble the artifact for deployment. The responsibility
>>>> of the aspects would be to intercept messages, log them and modify
>>>> them to ensure proper correlation. Of course this only works if the
>>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>>> use AspectJ for a similar use case (although at a much smaller scale)
>>>> in the transport test suites [1]. Interestingly, this approach would
>>>> allow to cover interactions that are not SOAP based, e.g. one could
>>>> even intercept the database queries executed during the flow.
>>>>
>>>> Andreas
>>>>
>>>> [1]
>>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>>>
>>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>>> > Hi Nilupa;
>>>> >
>>>> > When we collect messages from a one location by installing proper
>>>> > handler that will intercept and send messages to to that one location
>>>> > (this one location can be a single server, pub/sub channel etc). There
>>>> > is many ways to make sense of those collected messages. What Andreas
>>>> > mentioned (following complete transaction) is a one possibility.
>>>> >
>>>> > I think you should come up with few scenarios on how you would make
>>>> > sense of the message.
>>>> >
>>>> > Thanks
>>>> > Srinath
>>>> >
>>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>>> > wrote:
>>>> >> Hi,
>>>> >>
>>>> >> First let me thank you for commenting.
>>>> >>
>>>> >> As far as I understood, what you would like to see from the proposed
>>>> >> tool is to view set of messages that are exchanged in reponse to a
>>>> >> particular input message. With the understanding that I am having at
>>>> >> the momnet, one way to do it is to filter out the central repository
>>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>>> >> message chain from it. We can allow the client GUI wich connects to
>>>> >> the central repository to provide the paramenters (For instance the
>>>> >> value of 'To' header) from which an intelligent filtering can be done
>>>> >> for the set of messages avialable at the central repository.
>>>> >>
>>>> >> Perhaps someone has an idea of a better way of doing it and is willing
>>>> >> to share it with us. It would be really nice to hear from them.
>>>> >>
>>>> >> Thanks in advance ..!!
>>>> >>
>>>> >> Best Regards,
>>>> >> Nilupa
>>>> >>
>>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>> >> <an...@gmail.com> wrote:
>>>> >>> Personally, I think that the added value of extending the SOAP monitor
>>>> >>> module to collect messages in a central place is too limited to
>>>> >>> attract enough interest from the user community, so that it will be
>>>> >>> difficult to ensure the evolution of such a project in the future.
>>>> >>> However, what many people are looking for is a tool that allows to
>>>> >>> track the flow of a message through a distributed system, or more
>>>> >>> generally to track the sequence of events triggered by a given input
>>>> >>> message (sort of end-to-end transaction monitoring).
>>>> >>>
>>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>>> >>> wrote:
>>>> >>>> Hello,
>>>> >>>>
>>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>> >>>> proposing a GSoC project this summer. I would like to take project
>>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>> >>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>> >>>> services middleware. I will submit more detailed proposal later. I
>>>> >>>> would really like to hear any feedback from you. Any suggestions or
>>>> >>>> features that you would like to see in a "Distributed TCP Monitor"
>>>> >>>> are
>>>> >>>> most welcome.
>>>> >>>>
>>>> >>>> Best Regards,
>>>> >>>> Nilupa Bandara
>>>> >>>>
>>>> >>>> [1]
>>>> >>>>
>>>> >>>> Distributed TCP monitor
>>>> >>>>
>>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>> >>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>> >>>> pull those messages. Also, we can do something to turn the module on
>>>> >>>> and off remotely, so users can have his module deployed, but turn it
>>>> >>>> on only when they want to debug the system.
>>>> >>>>
>>>> >>>> Then we can take this to next level by adding this module to all
>>>> >>>> services in a system , configuring modules to send all collected
>>>> >>>> messages to a pub/sub channel, and subscribing to those messages via
>>>> >>>> a
>>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>>> >>>> TCPMon for a whole system.
>>>> >>>>
>>>> >>>> ---------------------------------------------------------------------
>>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >>>>
>>>> >>>>
>>>> >>>
>>>> >>> ---------------------------------------------------------------------
>>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >>>
>>>> >>>
>>>> >>
>>>> >> ---------------------------------------------------------------------
>>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >>
>>>> >>
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > ============================
>>>> > Srinath Perera, Ph.D.
>>>> >   WSO2 Inc. http://wso2.com
>>>> >   Blog: http://srinathsview.blogspot.com/
>>>> >
>>>> > ---------------------------------------------------------------------
>>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >
>>>> >
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>
>>>
>>>
>>> --
>>> Sanjiva Weerawarana, Ph.D.
>>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>>> http://www.opensource.lk/
>>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>>> Member; Apache Software Foundation; http://www.apache.org/
>>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>>
>>> Blog: http://sanjiva.weerawarana.org/
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

I was thinking about Java Swing based UI because the client may have
to cache whatever messages it already has and will receive, build the
message flow and update the UI accordingly. Perhaps there is a better
way ?

Thanks
Nilupa.

On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
> What UI technology would be used for this?
>
> Samisa...
>
> On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
> <an...@gmail.com> wrote:
>> I was thinking more about something like ITCAM for SOA.
>>
>> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>> <sa...@opensource.lk> wrote:
>>> That runtime monitoring / correlation is what the WSO2 Business Activity
>>> Monitor does / is for ...
>>> See: http://wso2.com/products/business-activity-monitor/
>>> Sanjiva.
>>>
>>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <an...@gmail.com>
>>> wrote:
>>>>
>>>> I was actually referring to scenarios where a service may call other
>>>> services, which in turn may call again other services. In these
>>>> scenarios, it is not sufficient to simply collect received/sent
>>>> messages on different hosts, unless the system is isolated
>>>> (development environment) or the request rate is sufficiently low so
>>>> that messages can be correlated based on time. Here is a concrete
>>>> scenario from a project in my company:
>>>>
>>>> - Request comes in on an ESB that does security and validation.
>>>> - Request is processed by an application server which persists the
>>>> received information and publishes a JMS message (pub/sub events).
>>>> - The event is consumed by one or more components that may in turn
>>>> interact with other services.
>>>>
>>>> If you want track this flow, it is not sufficient to intercept the
>>>> messages on the different hosts: in addition, you need to instrument
>>>> the services so that the outgoing requests are correlated with the
>>>> incoming requests (by adding SOAP and/or transport headers). What I
>>>> would like to be able to do in the above scenario is to enable
>>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>>> HTTP header to the initial request to enable logging and this
>>>> information is propagated along the chain. Every service/component
>>>> then sends a copy of the incoming/outgoing requests/responses to a
>>>> central place where the sequence of events is reconstructed.
>>>>
>>>> One way this could be achieved is with a tool that postprocesses the
>>>> artifacts from the build process. For each artifact, the tool would
>>>> disassemble the artifact, instrument the code using a set of AspectJ
>>>> aspects and reassemble the artifact for deployment. The responsibility
>>>> of the aspects would be to intercept messages, log them and modify
>>>> them to ensure proper correlation. Of course this only works if the
>>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>>> use AspectJ for a similar use case (although at a much smaller scale)
>>>> in the transport test suites [1]. Interestingly, this approach would
>>>> allow to cover interactions that are not SOAP based, e.g. one could
>>>> even intercept the database queries executed during the flow.
>>>>
>>>> Andreas
>>>>
>>>> [1]
>>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>>>
>>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>>> > Hi Nilupa;
>>>> >
>>>> > When we collect messages from a one location by installing proper
>>>> > handler that will intercept and send messages to to that one location
>>>> > (this one location can be a single server, pub/sub channel etc). There
>>>> > is many ways to make sense of those collected messages. What Andreas
>>>> > mentioned (following complete transaction) is a one possibility.
>>>> >
>>>> > I think you should come up with few scenarios on how you would make
>>>> > sense of the message.
>>>> >
>>>> > Thanks
>>>> > Srinath
>>>> >
>>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>>> > wrote:
>>>> >> Hi,
>>>> >>
>>>> >> First let me thank you for commenting.
>>>> >>
>>>> >> As far as I understood, what you would like to see from the proposed
>>>> >> tool is to view set of messages that are exchanged in reponse to a
>>>> >> particular input message. With the understanding that I am having at
>>>> >> the momnet, one way to do it is to filter out the central repository
>>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>>> >> message chain from it. We can allow the client GUI wich connects to
>>>> >> the central repository to provide the paramenters (For instance the
>>>> >> value of 'To' header) from which an intelligent filtering can be done
>>>> >> for the set of messages avialable at the central repository.
>>>> >>
>>>> >> Perhaps someone has an idea of a better way of doing it and is willing
>>>> >> to share it with us. It would be really nice to hear from them.
>>>> >>
>>>> >> Thanks in advance ..!!
>>>> >>
>>>> >> Best Regards,
>>>> >> Nilupa
>>>> >>
>>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>> >> <an...@gmail.com> wrote:
>>>> >>> Personally, I think that the added value of extending the SOAP monitor
>>>> >>> module to collect messages in a central place is too limited to
>>>> >>> attract enough interest from the user community, so that it will be
>>>> >>> difficult to ensure the evolution of such a project in the future.
>>>> >>> However, what many people are looking for is a tool that allows to
>>>> >>> track the flow of a message through a distributed system, or more
>>>> >>> generally to track the sequence of events triggered by a given input
>>>> >>> message (sort of end-to-end transaction monitoring).
>>>> >>>
>>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>>> >>> wrote:
>>>> >>>> Hello,
>>>> >>>>
>>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>> >>>> proposing a GSoC project this summer. I would like to take project
>>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>> >>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>> >>>> services middleware. I will submit more detailed proposal later. I
>>>> >>>> would really like to hear any feedback from you. Any suggestions or
>>>> >>>> features that you would like to see in a "Distributed TCP Monitor"
>>>> >>>> are
>>>> >>>> most welcome.
>>>> >>>>
>>>> >>>> Best Regards,
>>>> >>>> Nilupa Bandara
>>>> >>>>
>>>> >>>> [1]
>>>> >>>>
>>>> >>>> Distributed TCP monitor
>>>> >>>>
>>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>> >>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>> >>>> pull those messages. Also, we can do something to turn the module on
>>>> >>>> and off remotely, so users can have his module deployed, but turn it
>>>> >>>> on only when they want to debug the system.
>>>> >>>>
>>>> >>>> Then we can take this to next level by adding this module to all
>>>> >>>> services in a system , configuring modules to send all collected
>>>> >>>> messages to a pub/sub channel, and subscribing to those messages via
>>>> >>>> a
>>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>>> >>>> TCPMon for a whole system.
>>>> >>>>
>>>> >>>> ---------------------------------------------------------------------
>>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >>>>
>>>> >>>>
>>>> >>>
>>>> >>> ---------------------------------------------------------------------
>>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >>>
>>>> >>>
>>>> >>
>>>> >> ---------------------------------------------------------------------
>>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >>
>>>> >>
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > ============================
>>>> > Srinath Perera, Ph.D.
>>>> >   WSO2 Inc. http://wso2.com
>>>> >   Blog: http://srinathsview.blogspot.com/
>>>> >
>>>> > ---------------------------------------------------------------------
>>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >
>>>> >
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>
>>>
>>>
>>> --
>>> Sanjiva Weerawarana, Ph.D.
>>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>>> http://www.opensource.lk/
>>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>>> Member; Apache Software Foundation; http://www.apache.org/
>>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>>
>>> Blog: http://sanjiva.weerawarana.org/
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

I was thinking about Java Swing based UI because the client may have
to cache whatever messages it already has and will receive, build the
message flow and update the UI accordingly. Perhaps there is a better
way ?

Thanks
Nilupa.

On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
> What UI technology would be used for this?
>
> Samisa...
>
> On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
> <an...@gmail.com> wrote:
>> I was thinking more about something like ITCAM for SOA.
>>
>> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>> <sa...@opensource.lk> wrote:
>>> That runtime monitoring / correlation is what the WSO2 Business Activity
>>> Monitor does / is for ...
>>> See: http://wso2.com/products/business-activity-monitor/
>>> Sanjiva.
>>>
>>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <an...@gmail.com>
>>> wrote:
>>>>
>>>> I was actually referring to scenarios where a service may call other
>>>> services, which in turn may call again other services. In these
>>>> scenarios, it is not sufficient to simply collect received/sent
>>>> messages on different hosts, unless the system is isolated
>>>> (development environment) or the request rate is sufficiently low so
>>>> that messages can be correlated based on time. Here is a concrete
>>>> scenario from a project in my company:
>>>>
>>>> - Request comes in on an ESB that does security and validation.
>>>> - Request is processed by an application server which persists the
>>>> received information and publishes a JMS message (pub/sub events).
>>>> - The event is consumed by one or more components that may in turn
>>>> interact with other services.
>>>>
>>>> If you want track this flow, it is not sufficient to intercept the
>>>> messages on the different hosts: in addition, you need to instrument
>>>> the services so that the outgoing requests are correlated with the
>>>> incoming requests (by adding SOAP and/or transport headers). What I
>>>> would like to be able to do in the above scenario is to enable
>>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>>> HTTP header to the initial request to enable logging and this
>>>> information is propagated along the chain. Every service/component
>>>> then sends a copy of the incoming/outgoing requests/responses to a
>>>> central place where the sequence of events is reconstructed.
>>>>
>>>> One way this could be achieved is with a tool that postprocesses the
>>>> artifacts from the build process. For each artifact, the tool would
>>>> disassemble the artifact, instrument the code using a set of AspectJ
>>>> aspects and reassemble the artifact for deployment. The responsibility
>>>> of the aspects would be to intercept messages, log them and modify
>>>> them to ensure proper correlation. Of course this only works if the
>>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>>> use AspectJ for a similar use case (although at a much smaller scale)
>>>> in the transport test suites [1]. Interestingly, this approach would
>>>> allow to cover interactions that are not SOAP based, e.g. one could
>>>> even intercept the database queries executed during the flow.
>>>>
>>>> Andreas
>>>>
>>>> [1]
>>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>>>
>>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>>> > Hi Nilupa;
>>>> >
>>>> > When we collect messages from a one location by installing proper
>>>> > handler that will intercept and send messages to to that one location
>>>> > (this one location can be a single server, pub/sub channel etc). There
>>>> > is many ways to make sense of those collected messages. What Andreas
>>>> > mentioned (following complete transaction) is a one possibility.
>>>> >
>>>> > I think you should come up with few scenarios on how you would make
>>>> > sense of the message.
>>>> >
>>>> > Thanks
>>>> > Srinath
>>>> >
>>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>>> > wrote:
>>>> >> Hi,
>>>> >>
>>>> >> First let me thank you for commenting.
>>>> >>
>>>> >> As far as I understood, what you would like to see from the proposed
>>>> >> tool is to view set of messages that are exchanged in reponse to a
>>>> >> particular input message. With the understanding that I am having at
>>>> >> the momnet, one way to do it is to filter out the central repository
>>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>>> >> message chain from it. We can allow the client GUI wich connects to
>>>> >> the central repository to provide the paramenters (For instance the
>>>> >> value of 'To' header) from which an intelligent filtering can be done
>>>> >> for the set of messages avialable at the central repository.
>>>> >>
>>>> >> Perhaps someone has an idea of a better way of doing it and is willing
>>>> >> to share it with us. It would be really nice to hear from them.
>>>> >>
>>>> >> Thanks in advance ..!!
>>>> >>
>>>> >> Best Regards,
>>>> >> Nilupa
>>>> >>
>>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>> >> <an...@gmail.com> wrote:
>>>> >>> Personally, I think that the added value of extending the SOAP monitor
>>>> >>> module to collect messages in a central place is too limited to
>>>> >>> attract enough interest from the user community, so that it will be
>>>> >>> difficult to ensure the evolution of such a project in the future.
>>>> >>> However, what many people are looking for is a tool that allows to
>>>> >>> track the flow of a message through a distributed system, or more
>>>> >>> generally to track the sequence of events triggered by a given input
>>>> >>> message (sort of end-to-end transaction monitoring).
>>>> >>>
>>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>>> >>> wrote:
>>>> >>>> Hello,
>>>> >>>>
>>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>> >>>> proposing a GSoC project this summer. I would like to take project
>>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>> >>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>> >>>> services middleware. I will submit more detailed proposal later. I
>>>> >>>> would really like to hear any feedback from you. Any suggestions or
>>>> >>>> features that you would like to see in a "Distributed TCP Monitor"
>>>> >>>> are
>>>> >>>> most welcome.
>>>> >>>>
>>>> >>>> Best Regards,
>>>> >>>> Nilupa Bandara
>>>> >>>>
>>>> >>>> [1]
>>>> >>>>
>>>> >>>> Distributed TCP monitor
>>>> >>>>
>>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>> >>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>> >>>> pull those messages. Also, we can do something to turn the module on
>>>> >>>> and off remotely, so users can have his module deployed, but turn it
>>>> >>>> on only when they want to debug the system.
>>>> >>>>
>>>> >>>> Then we can take this to next level by adding this module to all
>>>> >>>> services in a system , configuring modules to send all collected
>>>> >>>> messages to a pub/sub channel, and subscribing to those messages via
>>>> >>>> a
>>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>>> >>>> TCPMon for a whole system.
>>>> >>>>
>>>> >>>> ---------------------------------------------------------------------
>>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >>>>
>>>> >>>>
>>>> >>>
>>>> >>> ---------------------------------------------------------------------
>>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >>>
>>>> >>>
>>>> >>
>>>> >> ---------------------------------------------------------------------
>>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >>
>>>> >>
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > ============================
>>>> > Srinath Perera, Ph.D.
>>>> >   WSO2 Inc. http://wso2.com
>>>> >   Blog: http://srinathsview.blogspot.com/
>>>> >
>>>> > ---------------------------------------------------------------------
>>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >
>>>> >
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>
>>>
>>>
>>> --
>>> Sanjiva Weerawarana, Ph.D.
>>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>>> http://www.opensource.lk/
>>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>>> Member; Apache Software Foundation; http://www.apache.org/
>>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>>
>>> Blog: http://sanjiva.weerawarana.org/
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

I was thinking about Java Swing based UI because the client may have
to cache whatever messages it already has and will receive, build the
message flow and update the UI accordingly. Perhaps there is a better
way ?

Thanks
Nilupa.

On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
> What UI technology would be used for this?
>
> Samisa...
>
> On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
> <an...@gmail.com> wrote:
>> I was thinking more about something like ITCAM for SOA.
>>
>> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>> <sa...@opensource.lk> wrote:
>>> That runtime monitoring / correlation is what the WSO2 Business Activity
>>> Monitor does / is for ...
>>> See: http://wso2.com/products/business-activity-monitor/
>>> Sanjiva.
>>>
>>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <an...@gmail.com>
>>> wrote:
>>>>
>>>> I was actually referring to scenarios where a service may call other
>>>> services, which in turn may call again other services. In these
>>>> scenarios, it is not sufficient to simply collect received/sent
>>>> messages on different hosts, unless the system is isolated
>>>> (development environment) or the request rate is sufficiently low so
>>>> that messages can be correlated based on time. Here is a concrete
>>>> scenario from a project in my company:
>>>>
>>>> - Request comes in on an ESB that does security and validation.
>>>> - Request is processed by an application server which persists the
>>>> received information and publishes a JMS message (pub/sub events).
>>>> - The event is consumed by one or more components that may in turn
>>>> interact with other services.
>>>>
>>>> If you want track this flow, it is not sufficient to intercept the
>>>> messages on the different hosts: in addition, you need to instrument
>>>> the services so that the outgoing requests are correlated with the
>>>> incoming requests (by adding SOAP and/or transport headers). What I
>>>> would like to be able to do in the above scenario is to enable
>>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>>> HTTP header to the initial request to enable logging and this
>>>> information is propagated along the chain. Every service/component
>>>> then sends a copy of the incoming/outgoing requests/responses to a
>>>> central place where the sequence of events is reconstructed.
>>>>
>>>> One way this could be achieved is with a tool that postprocesses the
>>>> artifacts from the build process. For each artifact, the tool would
>>>> disassemble the artifact, instrument the code using a set of AspectJ
>>>> aspects and reassemble the artifact for deployment. The responsibility
>>>> of the aspects would be to intercept messages, log them and modify
>>>> them to ensure proper correlation. Of course this only works if the
>>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>>> use AspectJ for a similar use case (although at a much smaller scale)
>>>> in the transport test suites [1]. Interestingly, this approach would
>>>> allow to cover interactions that are not SOAP based, e.g. one could
>>>> even intercept the database queries executed during the flow.
>>>>
>>>> Andreas
>>>>
>>>> [1]
>>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>>>
>>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>>> > Hi Nilupa;
>>>> >
>>>> > When we collect messages from a one location by installing proper
>>>> > handler that will intercept and send messages to to that one location
>>>> > (this one location can be a single server, pub/sub channel etc). There
>>>> > is many ways to make sense of those collected messages. What Andreas
>>>> > mentioned (following complete transaction) is a one possibility.
>>>> >
>>>> > I think you should come up with few scenarios on how you would make
>>>> > sense of the message.
>>>> >
>>>> > Thanks
>>>> > Srinath
>>>> >
>>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>>> > wrote:
>>>> >> Hi,
>>>> >>
>>>> >> First let me thank you for commenting.
>>>> >>
>>>> >> As far as I understood, what you would like to see from the proposed
>>>> >> tool is to view set of messages that are exchanged in reponse to a
>>>> >> particular input message. With the understanding that I am having at
>>>> >> the momnet, one way to do it is to filter out the central repository
>>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>>> >> message chain from it. We can allow the client GUI wich connects to
>>>> >> the central repository to provide the paramenters (For instance the
>>>> >> value of 'To' header) from which an intelligent filtering can be done
>>>> >> for the set of messages avialable at the central repository.
>>>> >>
>>>> >> Perhaps someone has an idea of a better way of doing it and is willing
>>>> >> to share it with us. It would be really nice to hear from them.
>>>> >>
>>>> >> Thanks in advance ..!!
>>>> >>
>>>> >> Best Regards,
>>>> >> Nilupa
>>>> >>
>>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>> >> <an...@gmail.com> wrote:
>>>> >>> Personally, I think that the added value of extending the SOAP monitor
>>>> >>> module to collect messages in a central place is too limited to
>>>> >>> attract enough interest from the user community, so that it will be
>>>> >>> difficult to ensure the evolution of such a project in the future.
>>>> >>> However, what many people are looking for is a tool that allows to
>>>> >>> track the flow of a message through a distributed system, or more
>>>> >>> generally to track the sequence of events triggered by a given input
>>>> >>> message (sort of end-to-end transaction monitoring).
>>>> >>>
>>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>>> >>> wrote:
>>>> >>>> Hello,
>>>> >>>>
>>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>> >>>> proposing a GSoC project this summer. I would like to take project
>>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>> >>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>> >>>> services middleware. I will submit more detailed proposal later. I
>>>> >>>> would really like to hear any feedback from you. Any suggestions or
>>>> >>>> features that you would like to see in a "Distributed TCP Monitor"
>>>> >>>> are
>>>> >>>> most welcome.
>>>> >>>>
>>>> >>>> Best Regards,
>>>> >>>> Nilupa Bandara
>>>> >>>>
>>>> >>>> [1]
>>>> >>>>
>>>> >>>> Distributed TCP monitor
>>>> >>>>
>>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>> >>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>> >>>> pull those messages. Also, we can do something to turn the module on
>>>> >>>> and off remotely, so users can have his module deployed, but turn it
>>>> >>>> on only when they want to debug the system.
>>>> >>>>
>>>> >>>> Then we can take this to next level by adding this module to all
>>>> >>>> services in a system , configuring modules to send all collected
>>>> >>>> messages to a pub/sub channel, and subscribing to those messages via
>>>> >>>> a
>>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>>> >>>> TCPMon for a whole system.
>>>> >>>>
>>>> >>>> ---------------------------------------------------------------------
>>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >>>>
>>>> >>>>
>>>> >>>
>>>> >>> ---------------------------------------------------------------------
>>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >>>
>>>> >>>
>>>> >>
>>>> >> ---------------------------------------------------------------------
>>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >>
>>>> >>
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > ============================
>>>> > Srinath Perera, Ph.D.
>>>> >   WSO2 Inc. http://wso2.com
>>>> >   Blog: http://srinathsview.blogspot.com/
>>>> >
>>>> > ---------------------------------------------------------------------
>>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >
>>>> >
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>
>>>
>>>
>>> --
>>> Sanjiva Weerawarana, Ph.D.
>>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>>> http://www.opensource.lk/
>>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>>> Member; Apache Software Foundation; http://www.apache.org/
>>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>>
>>> Blog: http://sanjiva.weerawarana.org/
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

I was thinking about Java Swing based UI because the client may have
to cache whatever messages it already has and will receive, build the
message flow and update the UI accordingly. Perhaps there is a better
way ?

Thanks
Nilupa.

On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
<sa...@gmail.com> wrote:
> What UI technology would be used for this?
>
> Samisa...
>
> On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
> <an...@gmail.com> wrote:
>> I was thinking more about something like ITCAM for SOA.
>>
>> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
>> <sa...@opensource.lk> wrote:
>>> That runtime monitoring / correlation is what the WSO2 Business Activity
>>> Monitor does / is for ...
>>> See: http://wso2.com/products/business-activity-monitor/
>>> Sanjiva.
>>>
>>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <an...@gmail.com>
>>> wrote:
>>>>
>>>> I was actually referring to scenarios where a service may call other
>>>> services, which in turn may call again other services. In these
>>>> scenarios, it is not sufficient to simply collect received/sent
>>>> messages on different hosts, unless the system is isolated
>>>> (development environment) or the request rate is sufficiently low so
>>>> that messages can be correlated based on time. Here is a concrete
>>>> scenario from a project in my company:
>>>>
>>>> - Request comes in on an ESB that does security and validation.
>>>> - Request is processed by an application server which persists the
>>>> received information and publishes a JMS message (pub/sub events).
>>>> - The event is consumed by one or more components that may in turn
>>>> interact with other services.
>>>>
>>>> If you want track this flow, it is not sufficient to intercept the
>>>> messages on the different hosts: in addition, you need to instrument
>>>> the services so that the outgoing requests are correlated with the
>>>> incoming requests (by adding SOAP and/or transport headers). What I
>>>> would like to be able to do in the above scenario is to enable
>>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>>> HTTP header to the initial request to enable logging and this
>>>> information is propagated along the chain. Every service/component
>>>> then sends a copy of the incoming/outgoing requests/responses to a
>>>> central place where the sequence of events is reconstructed.
>>>>
>>>> One way this could be achieved is with a tool that postprocesses the
>>>> artifacts from the build process. For each artifact, the tool would
>>>> disassemble the artifact, instrument the code using a set of AspectJ
>>>> aspects and reassemble the artifact for deployment. The responsibility
>>>> of the aspects would be to intercept messages, log them and modify
>>>> them to ensure proper correlation. Of course this only works if the
>>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>>> use AspectJ for a similar use case (although at a much smaller scale)
>>>> in the transport test suites [1]. Interestingly, this approach would
>>>> allow to cover interactions that are not SOAP based, e.g. one could
>>>> even intercept the database queries executed during the flow.
>>>>
>>>> Andreas
>>>>
>>>> [1]
>>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>>>
>>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>>> > Hi Nilupa;
>>>> >
>>>> > When we collect messages from a one location by installing proper
>>>> > handler that will intercept and send messages to to that one location
>>>> > (this one location can be a single server, pub/sub channel etc). There
>>>> > is many ways to make sense of those collected messages. What Andreas
>>>> > mentioned (following complete transaction) is a one possibility.
>>>> >
>>>> > I think you should come up with few scenarios on how you would make
>>>> > sense of the message.
>>>> >
>>>> > Thanks
>>>> > Srinath
>>>> >
>>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>>> > wrote:
>>>> >> Hi,
>>>> >>
>>>> >> First let me thank you for commenting.
>>>> >>
>>>> >> As far as I understood, what you would like to see from the proposed
>>>> >> tool is to view set of messages that are exchanged in reponse to a
>>>> >> particular input message. With the understanding that I am having at
>>>> >> the momnet, one way to do it is to filter out the central repository
>>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>>> >> message chain from it. We can allow the client GUI wich connects to
>>>> >> the central repository to provide the paramenters (For instance the
>>>> >> value of 'To' header) from which an intelligent filtering can be done
>>>> >> for the set of messages avialable at the central repository.
>>>> >>
>>>> >> Perhaps someone has an idea of a better way of doing it and is willing
>>>> >> to share it with us. It would be really nice to hear from them.
>>>> >>
>>>> >> Thanks in advance ..!!
>>>> >>
>>>> >> Best Regards,
>>>> >> Nilupa
>>>> >>
>>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>> >> <an...@gmail.com> wrote:
>>>> >>> Personally, I think that the added value of extending the SOAP monitor
>>>> >>> module to collect messages in a central place is too limited to
>>>> >>> attract enough interest from the user community, so that it will be
>>>> >>> difficult to ensure the evolution of such a project in the future.
>>>> >>> However, what many people are looking for is a tool that allows to
>>>> >>> track the flow of a message through a distributed system, or more
>>>> >>> generally to track the sequence of events triggered by a given input
>>>> >>> message (sort of end-to-end transaction monitoring).
>>>> >>>
>>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>>> >>> wrote:
>>>> >>>> Hello,
>>>> >>>>
>>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>> >>>> proposing a GSoC project this summer. I would like to take project
>>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>> >>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>> >>>> services middleware. I will submit more detailed proposal later. I
>>>> >>>> would really like to hear any feedback from you. Any suggestions or
>>>> >>>> features that you would like to see in a "Distributed TCP Monitor"
>>>> >>>> are
>>>> >>>> most welcome.
>>>> >>>>
>>>> >>>> Best Regards,
>>>> >>>> Nilupa Bandara
>>>> >>>>
>>>> >>>> [1]
>>>> >>>>
>>>> >>>> Distributed TCP monitor
>>>> >>>>
>>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>> >>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>> >>>> pull those messages. Also, we can do something to turn the module on
>>>> >>>> and off remotely, so users can have his module deployed, but turn it
>>>> >>>> on only when they want to debug the system.
>>>> >>>>
>>>> >>>> Then we can take this to next level by adding this module to all
>>>> >>>> services in a system , configuring modules to send all collected
>>>> >>>> messages to a pub/sub channel, and subscribing to those messages via
>>>> >>>> a
>>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>>> >>>> TCPMon for a whole system.
>>>> >>>>
>>>> >>>> ---------------------------------------------------------------------
>>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >>>>
>>>> >>>>
>>>> >>>
>>>> >>> ---------------------------------------------------------------------
>>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >>>
>>>> >>>
>>>> >>
>>>> >> ---------------------------------------------------------------------
>>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >>
>>>> >>
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > ============================
>>>> > Srinath Perera, Ph.D.
>>>> >   WSO2 Inc. http://wso2.com
>>>> >   Blog: http://srinathsview.blogspot.com/
>>>> >
>>>> > ---------------------------------------------------------------------
>>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>>> >
>>>> >
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>
>>>
>>>
>>> --
>>> Sanjiva Weerawarana, Ph.D.
>>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>>> http://www.opensource.lk/
>>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>>> Member; Apache Software Foundation; http://www.apache.org/
>>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>>
>>> Blog: http://sanjiva.weerawarana.org/
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

What UI technology would be used for this?

Samisa...

On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
<an...@gmail.com> wrote:
> I was thinking more about something like ITCAM for SOA.
>
> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
> <sa...@opensource.lk> wrote:
>> That runtime monitoring / correlation is what the WSO2 Business Activity
>> Monitor does / is for ...
>> See: http://wso2.com/products/business-activity-monitor/
>> Sanjiva.
>>
>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <an...@gmail.com>
>> wrote:
>>>
>>> I was actually referring to scenarios where a service may call other
>>> services, which in turn may call again other services. In these
>>> scenarios, it is not sufficient to simply collect received/sent
>>> messages on different hosts, unless the system is isolated
>>> (development environment) or the request rate is sufficiently low so
>>> that messages can be correlated based on time. Here is a concrete
>>> scenario from a project in my company:
>>>
>>> - Request comes in on an ESB that does security and validation.
>>> - Request is processed by an application server which persists the
>>> received information and publishes a JMS message (pub/sub events).
>>> - The event is consumed by one or more components that may in turn
>>> interact with other services.
>>>
>>> If you want track this flow, it is not sufficient to intercept the
>>> messages on the different hosts: in addition, you need to instrument
>>> the services so that the outgoing requests are correlated with the
>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> would like to be able to do in the above scenario is to enable
>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> HTTP header to the initial request to enable logging and this
>>> information is propagated along the chain. Every service/component
>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> central place where the sequence of events is reconstructed.
>>>
>>> One way this could be achieved is with a tool that postprocesses the
>>> artifacts from the build process. For each artifact, the tool would
>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> aspects and reassemble the artifact for deployment. The responsibility
>>> of the aspects would be to intercept messages, log them and modify
>>> them to ensure proper correlation. Of course this only works if the
>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> in the transport test suites [1]. Interestingly, this approach would
>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> even intercept the database queries executed during the flow.
>>>
>>> Andreas
>>>
>>> [1]
>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>>
>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>> > Hi Nilupa;
>>> >
>>> > When we collect messages from a one location by installing proper
>>> > handler that will intercept and send messages to to that one location
>>> > (this one location can be a single server, pub/sub channel etc). There
>>> > is many ways to make sense of those collected messages. What Andreas
>>> > mentioned (following complete transaction) is a one possibility.
>>> >
>>> > I think you should come up with few scenarios on how you would make
>>> > sense of the message.
>>> >
>>> > Thanks
>>> > Srinath
>>> >
>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>> > wrote:
>>> >> Hi,
>>> >>
>>> >> First let me thank you for commenting.
>>> >>
>>> >> As far as I understood, what you would like to see from the proposed
>>> >> tool is to view set of messages that are exchanged in reponse to a
>>> >> particular input message. With the understanding that I am having at
>>> >> the momnet, one way to do it is to filter out the central repository
>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>> >> message chain from it. We can allow the client GUI wich connects to
>>> >> the central repository to provide the paramenters (For instance the
>>> >> value of 'To' header) from which an intelligent filtering can be done
>>> >> for the set of messages avialable at the central repository.
>>> >>
>>> >> Perhaps someone has an idea of a better way of doing it and is willing
>>> >> to share it with us. It would be really nice to hear from them.
>>> >>
>>> >> Thanks in advance ..!!
>>> >>
>>> >> Best Regards,
>>> >> Nilupa
>>> >>
>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> >> <an...@gmail.com> wrote:
>>> >>> Personally, I think that the added value of extending the SOAP monitor
>>> >>> module to collect messages in a central place is too limited to
>>> >>> attract enough interest from the user community, so that it will be
>>> >>> difficult to ensure the evolution of such a project in the future.
>>> >>> However, what many people are looking for is a tool that allows to
>>> >>> track the flow of a message through a distributed system, or more
>>> >>> generally to track the sequence of events triggered by a given input
>>> >>> message (sort of end-to-end transaction monitoring).
>>> >>>
>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>> >>> wrote:
>>> >>>> Hello,
>>> >>>>
>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>> >>>> proposing a GSoC project this summer. I would like to take project
>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>> >>>> also should be helpful to any Java developer using Apache Axis2 Web
>>> >>>> services middleware. I will submit more detailed proposal later. I
>>> >>>> would really like to hear any feedback from you. Any suggestions or
>>> >>>> features that you would like to see in a "Distributed TCP Monitor"
>>> >>>> are
>>> >>>> most welcome.
>>> >>>>
>>> >>>> Best Regards,
>>> >>>> Nilupa Bandara
>>> >>>>
>>> >>>> [1]
>>> >>>>
>>> >>>> Distributed TCP monitor
>>> >>>>
>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>> >>>> problem by writing a Axis2 module (Handler) that intercept messages
>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>> >>>> pull those messages. Also, we can do something to turn the module on
>>> >>>> and off remotely, so users can have his module deployed, but turn it
>>> >>>> on only when they want to debug the system.
>>> >>>>
>>> >>>> Then we can take this to next level by adding this module to all
>>> >>>> services in a system , configuring modules to send all collected
>>> >>>> messages to a pub/sub channel, and subscribing to those messages via
>>> >>>> a
>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>> >>>> TCPMon for a whole system.
>>> >>>>
>>> >>>> ---------------------------------------------------------------------
>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>>
>>> >>>>
>>> >>>
>>> >>> ---------------------------------------------------------------------
>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>
>>> >>>
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > ============================
>>> > Srinath Perera, Ph.D.
>>> >   WSO2 Inc. http://wso2.com
>>> >   Blog: http://srinathsview.blogspot.com/
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>
>>
>>
>> --
>> Sanjiva Weerawarana, Ph.D.
>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>> http://www.opensource.lk/
>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>> Member; Apache Software Foundation; http://www.apache.org/
>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>
>> Blog: http://sanjiva.weerawarana.org/
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

What UI technology would be used for this?

Samisa...

On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
<an...@gmail.com> wrote:
> I was thinking more about something like ITCAM for SOA.
>
> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
> <sa...@opensource.lk> wrote:
>> That runtime monitoring / correlation is what the WSO2 Business Activity
>> Monitor does / is for ...
>> See: http://wso2.com/products/business-activity-monitor/
>> Sanjiva.
>>
>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <an...@gmail.com>
>> wrote:
>>>
>>> I was actually referring to scenarios where a service may call other
>>> services, which in turn may call again other services. In these
>>> scenarios, it is not sufficient to simply collect received/sent
>>> messages on different hosts, unless the system is isolated
>>> (development environment) or the request rate is sufficiently low so
>>> that messages can be correlated based on time. Here is a concrete
>>> scenario from a project in my company:
>>>
>>> - Request comes in on an ESB that does security and validation.
>>> - Request is processed by an application server which persists the
>>> received information and publishes a JMS message (pub/sub events).
>>> - The event is consumed by one or more components that may in turn
>>> interact with other services.
>>>
>>> If you want track this flow, it is not sufficient to intercept the
>>> messages on the different hosts: in addition, you need to instrument
>>> the services so that the outgoing requests are correlated with the
>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> would like to be able to do in the above scenario is to enable
>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> HTTP header to the initial request to enable logging and this
>>> information is propagated along the chain. Every service/component
>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> central place where the sequence of events is reconstructed.
>>>
>>> One way this could be achieved is with a tool that postprocesses the
>>> artifacts from the build process. For each artifact, the tool would
>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> aspects and reassemble the artifact for deployment. The responsibility
>>> of the aspects would be to intercept messages, log them and modify
>>> them to ensure proper correlation. Of course this only works if the
>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> in the transport test suites [1]. Interestingly, this approach would
>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> even intercept the database queries executed during the flow.
>>>
>>> Andreas
>>>
>>> [1]
>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>>
>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>> > Hi Nilupa;
>>> >
>>> > When we collect messages from a one location by installing proper
>>> > handler that will intercept and send messages to to that one location
>>> > (this one location can be a single server, pub/sub channel etc). There
>>> > is many ways to make sense of those collected messages. What Andreas
>>> > mentioned (following complete transaction) is a one possibility.
>>> >
>>> > I think you should come up with few scenarios on how you would make
>>> > sense of the message.
>>> >
>>> > Thanks
>>> > Srinath
>>> >
>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>> > wrote:
>>> >> Hi,
>>> >>
>>> >> First let me thank you for commenting.
>>> >>
>>> >> As far as I understood, what you would like to see from the proposed
>>> >> tool is to view set of messages that are exchanged in reponse to a
>>> >> particular input message. With the understanding that I am having at
>>> >> the momnet, one way to do it is to filter out the central repository
>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>> >> message chain from it. We can allow the client GUI wich connects to
>>> >> the central repository to provide the paramenters (For instance the
>>> >> value of 'To' header) from which an intelligent filtering can be done
>>> >> for the set of messages avialable at the central repository.
>>> >>
>>> >> Perhaps someone has an idea of a better way of doing it and is willing
>>> >> to share it with us. It would be really nice to hear from them.
>>> >>
>>> >> Thanks in advance ..!!
>>> >>
>>> >> Best Regards,
>>> >> Nilupa
>>> >>
>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> >> <an...@gmail.com> wrote:
>>> >>> Personally, I think that the added value of extending the SOAP monitor
>>> >>> module to collect messages in a central place is too limited to
>>> >>> attract enough interest from the user community, so that it will be
>>> >>> difficult to ensure the evolution of such a project in the future.
>>> >>> However, what many people are looking for is a tool that allows to
>>> >>> track the flow of a message through a distributed system, or more
>>> >>> generally to track the sequence of events triggered by a given input
>>> >>> message (sort of end-to-end transaction monitoring).
>>> >>>
>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>> >>> wrote:
>>> >>>> Hello,
>>> >>>>
>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>> >>>> proposing a GSoC project this summer. I would like to take project
>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>> >>>> also should be helpful to any Java developer using Apache Axis2 Web
>>> >>>> services middleware. I will submit more detailed proposal later. I
>>> >>>> would really like to hear any feedback from you. Any suggestions or
>>> >>>> features that you would like to see in a "Distributed TCP Monitor"
>>> >>>> are
>>> >>>> most welcome.
>>> >>>>
>>> >>>> Best Regards,
>>> >>>> Nilupa Bandara
>>> >>>>
>>> >>>> [1]
>>> >>>>
>>> >>>> Distributed TCP monitor
>>> >>>>
>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>> >>>> problem by writing a Axis2 module (Handler) that intercept messages
>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>> >>>> pull those messages. Also, we can do something to turn the module on
>>> >>>> and off remotely, so users can have his module deployed, but turn it
>>> >>>> on only when they want to debug the system.
>>> >>>>
>>> >>>> Then we can take this to next level by adding this module to all
>>> >>>> services in a system , configuring modules to send all collected
>>> >>>> messages to a pub/sub channel, and subscribing to those messages via
>>> >>>> a
>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>> >>>> TCPMon for a whole system.
>>> >>>>
>>> >>>> ---------------------------------------------------------------------
>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>>
>>> >>>>
>>> >>>
>>> >>> ---------------------------------------------------------------------
>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>
>>> >>>
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > ============================
>>> > Srinath Perera, Ph.D.
>>> >   WSO2 Inc. http://wso2.com
>>> >   Blog: http://srinathsview.blogspot.com/
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>
>>
>>
>> --
>> Sanjiva Weerawarana, Ph.D.
>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>> http://www.opensource.lk/
>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>> Member; Apache Software Foundation; http://www.apache.org/
>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>
>> Blog: http://sanjiva.weerawarana.org/
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

What UI technology would be used for this?

Samisa...

On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
<an...@gmail.com> wrote:
> I was thinking more about something like ITCAM for SOA.
>
> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
> <sa...@opensource.lk> wrote:
>> That runtime monitoring / correlation is what the WSO2 Business Activity
>> Monitor does / is for ...
>> See: http://wso2.com/products/business-activity-monitor/
>> Sanjiva.
>>
>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <an...@gmail.com>
>> wrote:
>>>
>>> I was actually referring to scenarios where a service may call other
>>> services, which in turn may call again other services. In these
>>> scenarios, it is not sufficient to simply collect received/sent
>>> messages on different hosts, unless the system is isolated
>>> (development environment) or the request rate is sufficiently low so
>>> that messages can be correlated based on time. Here is a concrete
>>> scenario from a project in my company:
>>>
>>> - Request comes in on an ESB that does security and validation.
>>> - Request is processed by an application server which persists the
>>> received information and publishes a JMS message (pub/sub events).
>>> - The event is consumed by one or more components that may in turn
>>> interact with other services.
>>>
>>> If you want track this flow, it is not sufficient to intercept the
>>> messages on the different hosts: in addition, you need to instrument
>>> the services so that the outgoing requests are correlated with the
>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> would like to be able to do in the above scenario is to enable
>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> HTTP header to the initial request to enable logging and this
>>> information is propagated along the chain. Every service/component
>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> central place where the sequence of events is reconstructed.
>>>
>>> One way this could be achieved is with a tool that postprocesses the
>>> artifacts from the build process. For each artifact, the tool would
>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> aspects and reassemble the artifact for deployment. The responsibility
>>> of the aspects would be to intercept messages, log them and modify
>>> them to ensure proper correlation. Of course this only works if the
>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> in the transport test suites [1]. Interestingly, this approach would
>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> even intercept the database queries executed during the flow.
>>>
>>> Andreas
>>>
>>> [1]
>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>>
>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>> > Hi Nilupa;
>>> >
>>> > When we collect messages from a one location by installing proper
>>> > handler that will intercept and send messages to to that one location
>>> > (this one location can be a single server, pub/sub channel etc). There
>>> > is many ways to make sense of those collected messages. What Andreas
>>> > mentioned (following complete transaction) is a one possibility.
>>> >
>>> > I think you should come up with few scenarios on how you would make
>>> > sense of the message.
>>> >
>>> > Thanks
>>> > Srinath
>>> >
>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>> > wrote:
>>> >> Hi,
>>> >>
>>> >> First let me thank you for commenting.
>>> >>
>>> >> As far as I understood, what you would like to see from the proposed
>>> >> tool is to view set of messages that are exchanged in reponse to a
>>> >> particular input message. With the understanding that I am having at
>>> >> the momnet, one way to do it is to filter out the central repository
>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>> >> message chain from it. We can allow the client GUI wich connects to
>>> >> the central repository to provide the paramenters (For instance the
>>> >> value of 'To' header) from which an intelligent filtering can be done
>>> >> for the set of messages avialable at the central repository.
>>> >>
>>> >> Perhaps someone has an idea of a better way of doing it and is willing
>>> >> to share it with us. It would be really nice to hear from them.
>>> >>
>>> >> Thanks in advance ..!!
>>> >>
>>> >> Best Regards,
>>> >> Nilupa
>>> >>
>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> >> <an...@gmail.com> wrote:
>>> >>> Personally, I think that the added value of extending the SOAP monitor
>>> >>> module to collect messages in a central place is too limited to
>>> >>> attract enough interest from the user community, so that it will be
>>> >>> difficult to ensure the evolution of such a project in the future.
>>> >>> However, what many people are looking for is a tool that allows to
>>> >>> track the flow of a message through a distributed system, or more
>>> >>> generally to track the sequence of events triggered by a given input
>>> >>> message (sort of end-to-end transaction monitoring).
>>> >>>
>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>> >>> wrote:
>>> >>>> Hello,
>>> >>>>
>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>> >>>> proposing a GSoC project this summer. I would like to take project
>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>> >>>> also should be helpful to any Java developer using Apache Axis2 Web
>>> >>>> services middleware. I will submit more detailed proposal later. I
>>> >>>> would really like to hear any feedback from you. Any suggestions or
>>> >>>> features that you would like to see in a "Distributed TCP Monitor"
>>> >>>> are
>>> >>>> most welcome.
>>> >>>>
>>> >>>> Best Regards,
>>> >>>> Nilupa Bandara
>>> >>>>
>>> >>>> [1]
>>> >>>>
>>> >>>> Distributed TCP monitor
>>> >>>>
>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>> >>>> problem by writing a Axis2 module (Handler) that intercept messages
>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>> >>>> pull those messages. Also, we can do something to turn the module on
>>> >>>> and off remotely, so users can have his module deployed, but turn it
>>> >>>> on only when they want to debug the system.
>>> >>>>
>>> >>>> Then we can take this to next level by adding this module to all
>>> >>>> services in a system , configuring modules to send all collected
>>> >>>> messages to a pub/sub channel, and subscribing to those messages via
>>> >>>> a
>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>> >>>> TCPMon for a whole system.
>>> >>>>
>>> >>>> ---------------------------------------------------------------------
>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>>
>>> >>>>
>>> >>>
>>> >>> ---------------------------------------------------------------------
>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>
>>> >>>
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > ============================
>>> > Srinath Perera, Ph.D.
>>> >   WSO2 Inc. http://wso2.com
>>> >   Blog: http://srinathsview.blogspot.com/
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>
>>
>>
>> --
>> Sanjiva Weerawarana, Ph.D.
>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>> http://www.opensource.lk/
>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>> Member; Apache Software Foundation; http://www.apache.org/
>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>
>> Blog: http://sanjiva.weerawarana.org/
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

What UI technology would be used for this?

Samisa...

On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
<an...@gmail.com> wrote:
> I was thinking more about something like ITCAM for SOA.
>
> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
> <sa...@opensource.lk> wrote:
>> That runtime monitoring / correlation is what the WSO2 Business Activity
>> Monitor does / is for ...
>> See: http://wso2.com/products/business-activity-monitor/
>> Sanjiva.
>>
>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <an...@gmail.com>
>> wrote:
>>>
>>> I was actually referring to scenarios where a service may call other
>>> services, which in turn may call again other services. In these
>>> scenarios, it is not sufficient to simply collect received/sent
>>> messages on different hosts, unless the system is isolated
>>> (development environment) or the request rate is sufficiently low so
>>> that messages can be correlated based on time. Here is a concrete
>>> scenario from a project in my company:
>>>
>>> - Request comes in on an ESB that does security and validation.
>>> - Request is processed by an application server which persists the
>>> received information and publishes a JMS message (pub/sub events).
>>> - The event is consumed by one or more components that may in turn
>>> interact with other services.
>>>
>>> If you want track this flow, it is not sufficient to intercept the
>>> messages on the different hosts: in addition, you need to instrument
>>> the services so that the outgoing requests are correlated with the
>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> would like to be able to do in the above scenario is to enable
>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> HTTP header to the initial request to enable logging and this
>>> information is propagated along the chain. Every service/component
>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> central place where the sequence of events is reconstructed.
>>>
>>> One way this could be achieved is with a tool that postprocesses the
>>> artifacts from the build process. For each artifact, the tool would
>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> aspects and reassemble the artifact for deployment. The responsibility
>>> of the aspects would be to intercept messages, log them and modify
>>> them to ensure proper correlation. Of course this only works if the
>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> in the transport test suites [1]. Interestingly, this approach would
>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> even intercept the database queries executed during the flow.
>>>
>>> Andreas
>>>
>>> [1]
>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>>
>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>> > Hi Nilupa;
>>> >
>>> > When we collect messages from a one location by installing proper
>>> > handler that will intercept and send messages to to that one location
>>> > (this one location can be a single server, pub/sub channel etc). There
>>> > is many ways to make sense of those collected messages. What Andreas
>>> > mentioned (following complete transaction) is a one possibility.
>>> >
>>> > I think you should come up with few scenarios on how you would make
>>> > sense of the message.
>>> >
>>> > Thanks
>>> > Srinath
>>> >
>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>> > wrote:
>>> >> Hi,
>>> >>
>>> >> First let me thank you for commenting.
>>> >>
>>> >> As far as I understood, what you would like to see from the proposed
>>> >> tool is to view set of messages that are exchanged in reponse to a
>>> >> particular input message. With the understanding that I am having at
>>> >> the momnet, one way to do it is to filter out the central repository
>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>> >> message chain from it. We can allow the client GUI wich connects to
>>> >> the central repository to provide the paramenters (For instance the
>>> >> value of 'To' header) from which an intelligent filtering can be done
>>> >> for the set of messages avialable at the central repository.
>>> >>
>>> >> Perhaps someone has an idea of a better way of doing it and is willing
>>> >> to share it with us. It would be really nice to hear from them.
>>> >>
>>> >> Thanks in advance ..!!
>>> >>
>>> >> Best Regards,
>>> >> Nilupa
>>> >>
>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> >> <an...@gmail.com> wrote:
>>> >>> Personally, I think that the added value of extending the SOAP monitor
>>> >>> module to collect messages in a central place is too limited to
>>> >>> attract enough interest from the user community, so that it will be
>>> >>> difficult to ensure the evolution of such a project in the future.
>>> >>> However, what many people are looking for is a tool that allows to
>>> >>> track the flow of a message through a distributed system, or more
>>> >>> generally to track the sequence of events triggered by a given input
>>> >>> message (sort of end-to-end transaction monitoring).
>>> >>>
>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>> >>> wrote:
>>> >>>> Hello,
>>> >>>>
>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>> >>>> proposing a GSoC project this summer. I would like to take project
>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>> >>>> also should be helpful to any Java developer using Apache Axis2 Web
>>> >>>> services middleware. I will submit more detailed proposal later. I
>>> >>>> would really like to hear any feedback from you. Any suggestions or
>>> >>>> features that you would like to see in a "Distributed TCP Monitor"
>>> >>>> are
>>> >>>> most welcome.
>>> >>>>
>>> >>>> Best Regards,
>>> >>>> Nilupa Bandara
>>> >>>>
>>> >>>> [1]
>>> >>>>
>>> >>>> Distributed TCP monitor
>>> >>>>
>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>> >>>> problem by writing a Axis2 module (Handler) that intercept messages
>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>> >>>> pull those messages. Also, we can do something to turn the module on
>>> >>>> and off remotely, so users can have his module deployed, but turn it
>>> >>>> on only when they want to debug the system.
>>> >>>>
>>> >>>> Then we can take this to next level by adding this module to all
>>> >>>> services in a system , configuring modules to send all collected
>>> >>>> messages to a pub/sub channel, and subscribing to those messages via
>>> >>>> a
>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>> >>>> TCPMon for a whole system.
>>> >>>>
>>> >>>> ---------------------------------------------------------------------
>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>>
>>> >>>>
>>> >>>
>>> >>> ---------------------------------------------------------------------
>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>
>>> >>>
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > ============================
>>> > Srinath Perera, Ph.D.
>>> >   WSO2 Inc. http://wso2.com
>>> >   Blog: http://srinathsview.blogspot.com/
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>
>>
>>
>> --
>> Sanjiva Weerawarana, Ph.D.
>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>> http://www.opensource.lk/
>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>> Member; Apache Software Foundation; http://www.apache.org/
>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>
>> Blog: http://sanjiva.weerawarana.org/
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

What UI technology would be used for this?

Samisa...

On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
<an...@gmail.com> wrote:
> I was thinking more about something like ITCAM for SOA.
>
> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
> <sa...@opensource.lk> wrote:
>> That runtime monitoring / correlation is what the WSO2 Business Activity
>> Monitor does / is for ...
>> See: http://wso2.com/products/business-activity-monitor/
>> Sanjiva.
>>
>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <an...@gmail.com>
>> wrote:
>>>
>>> I was actually referring to scenarios where a service may call other
>>> services, which in turn may call again other services. In these
>>> scenarios, it is not sufficient to simply collect received/sent
>>> messages on different hosts, unless the system is isolated
>>> (development environment) or the request rate is sufficiently low so
>>> that messages can be correlated based on time. Here is a concrete
>>> scenario from a project in my company:
>>>
>>> - Request comes in on an ESB that does security and validation.
>>> - Request is processed by an application server which persists the
>>> received information and publishes a JMS message (pub/sub events).
>>> - The event is consumed by one or more components that may in turn
>>> interact with other services.
>>>
>>> If you want track this flow, it is not sufficient to intercept the
>>> messages on the different hosts: in addition, you need to instrument
>>> the services so that the outgoing requests are correlated with the
>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> would like to be able to do in the above scenario is to enable
>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> HTTP header to the initial request to enable logging and this
>>> information is propagated along the chain. Every service/component
>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> central place where the sequence of events is reconstructed.
>>>
>>> One way this could be achieved is with a tool that postprocesses the
>>> artifacts from the build process. For each artifact, the tool would
>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> aspects and reassemble the artifact for deployment. The responsibility
>>> of the aspects would be to intercept messages, log them and modify
>>> them to ensure proper correlation. Of course this only works if the
>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> in the transport test suites [1]. Interestingly, this approach would
>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> even intercept the database queries executed during the flow.
>>>
>>> Andreas
>>>
>>> [1]
>>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>>
>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>>> > Hi Nilupa;
>>> >
>>> > When we collect messages from a one location by installing proper
>>> > handler that will intercept and send messages to to that one location
>>> > (this one location can be a single server, pub/sub channel etc). There
>>> > is many ways to make sense of those collected messages. What Andreas
>>> > mentioned (following complete transaction) is a one possibility.
>>> >
>>> > I think you should come up with few scenarios on how you would make
>>> > sense of the message.
>>> >
>>> > Thanks
>>> > Srinath
>>> >
>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>>> > wrote:
>>> >> Hi,
>>> >>
>>> >> First let me thank you for commenting.
>>> >>
>>> >> As far as I understood, what you would like to see from the proposed
>>> >> tool is to view set of messages that are exchanged in reponse to a
>>> >> particular input message. With the understanding that I am having at
>>> >> the momnet, one way to do it is to filter out the central repository
>>> >> of messages based on 'To' , 'From' headers and try to contruct the
>>> >> message chain from it. We can allow the client GUI wich connects to
>>> >> the central repository to provide the paramenters (For instance the
>>> >> value of 'To' header) from which an intelligent filtering can be done
>>> >> for the set of messages avialable at the central repository.
>>> >>
>>> >> Perhaps someone has an idea of a better way of doing it and is willing
>>> >> to share it with us. It would be really nice to hear from them.
>>> >>
>>> >> Thanks in advance ..!!
>>> >>
>>> >> Best Regards,
>>> >> Nilupa
>>> >>
>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> >> <an...@gmail.com> wrote:
>>> >>> Personally, I think that the added value of extending the SOAP monitor
>>> >>> module to collect messages in a central place is too limited to
>>> >>> attract enough interest from the user community, so that it will be
>>> >>> difficult to ensure the evolution of such a project in the future.
>>> >>> However, what many people are looking for is a tool that allows to
>>> >>> track the flow of a message through a distributed system, or more
>>> >>> generally to track the sequence of events triggered by a given input
>>> >>> message (sort of end-to-end transaction monitoring).
>>> >>>
>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>>> >>> wrote:
>>> >>>> Hello,
>>> >>>>
>>> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>> >>>> proposing a GSoC project this summer. I would like to take project
>>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>> >>>> also should be helpful to any Java developer using Apache Axis2 Web
>>> >>>> services middleware. I will submit more detailed proposal later. I
>>> >>>> would really like to hear any feedback from you. Any suggestions or
>>> >>>> features that you would like to see in a "Distributed TCP Monitor"
>>> >>>> are
>>> >>>> most welcome.
>>> >>>>
>>> >>>> Best Regards,
>>> >>>> Nilupa Bandara
>>> >>>>
>>> >>>> [1]
>>> >>>>
>>> >>>> Distributed TCP monitor
>>> >>>>
>>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>> >>>> problem by writing a Axis2 module (Handler) that intercept messages
>>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>> >>>> pull those messages. Also, we can do something to turn the module on
>>> >>>> and off remotely, so users can have his module deployed, but turn it
>>> >>>> on only when they want to debug the system.
>>> >>>>
>>> >>>> Then we can take this to next level by adding this module to all
>>> >>>> services in a system , configuring modules to send all collected
>>> >>>> messages to a pub/sub channel, and subscribing to those messages via
>>> >>>> a
>>> >>>> UI, and depicting those messages through a UI. Then users have a
>>> >>>> TCPMon for a whole system.
>>> >>>>
>>> >>>> ---------------------------------------------------------------------
>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>>
>>> >>>>
>>> >>>
>>> >>> ---------------------------------------------------------------------
>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>>
>>> >>>
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >>
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > ============================
>>> > Srinath Perera, Ph.D.
>>> >   WSO2 Inc. http://wso2.com
>>> >   Blog: http://srinathsview.blogspot.com/
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>
>>
>>
>> --
>> Sanjiva Weerawarana, Ph.D.
>> Founder, Director & Chief Scientist; Lanka Software Foundation;
>> http://www.opensource.lk/
>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
>> Member; Apache Software Foundation; http://www.apache.org/
>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>>
>> Blog: http://sanjiva.weerawarana.org/
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

I was thinking more about something like ITCAM for SOA.

On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
<sa...@opensource.lk> wrote:
> That runtime monitoring / correlation is what the WSO2 Business Activity
> Monitor does / is for ...
> See: http://wso2.com/products/business-activity-monitor/
> Sanjiva.
>
> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <an...@gmail.com>
> wrote:
>>
>> I was actually referring to scenarios where a service may call other
>> services, which in turn may call again other services. In these
>> scenarios, it is not sufficient to simply collect received/sent
>> messages on different hosts, unless the system is isolated
>> (development environment) or the request rate is sufficiently low so
>> that messages can be correlated based on time. Here is a concrete
>> scenario from a project in my company:
>>
>> - Request comes in on an ESB that does security and validation.
>> - Request is processed by an application server which persists the
>> received information and publishes a JMS message (pub/sub events).
>> - The event is consumed by one or more components that may in turn
>> interact with other services.
>>
>> If you want track this flow, it is not sufficient to intercept the
>> messages on the different hosts: in addition, you need to instrument
>> the services so that the outgoing requests are correlated with the
>> incoming requests (by adding SOAP and/or transport headers). What I
>> would like to be able to do in the above scenario is to enable
>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> HTTP header to the initial request to enable logging and this
>> information is propagated along the chain. Every service/component
>> then sends a copy of the incoming/outgoing requests/responses to a
>> central place where the sequence of events is reconstructed.
>>
>> One way this could be achieved is with a tool that postprocesses the
>> artifacts from the build process. For each artifact, the tool would
>> disassemble the artifact, instrument the code using a set of AspectJ
>> aspects and reassemble the artifact for deployment. The responsibility
>> of the aspects would be to intercept messages, log them and modify
>> them to ensure proper correlation. Of course this only works if the
>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> use AspectJ for a similar use case (although at a much smaller scale)
>> in the transport test suites [1]. Interestingly, this approach would
>> allow to cover interactions that are not SOAP based, e.g. one could
>> even intercept the database queries executed during the flow.
>>
>> Andreas
>>
>> [1]
>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>
>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>> > Hi Nilupa;
>> >
>> > When we collect messages from a one location by installing proper
>> > handler that will intercept and send messages to to that one location
>> > (this one location can be a single server, pub/sub channel etc). There
>> > is many ways to make sense of those collected messages. What Andreas
>> > mentioned (following complete transaction) is a one possibility.
>> >
>> > I think you should come up with few scenarios on how you would make
>> > sense of the message.
>> >
>> > Thanks
>> > Srinath
>> >
>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>> > wrote:
>> >> Hi,
>> >>
>> >> First let me thank you for commenting.
>> >>
>> >> As far as I understood, what you would like to see from the proposed
>> >> tool is to view set of messages that are exchanged in reponse to a
>> >> particular input message. With the understanding that I am having at
>> >> the momnet, one way to do it is to filter out the central repository
>> >> of messages based on 'To' , 'From' headers and try to contruct the
>> >> message chain from it. We can allow the client GUI wich connects to
>> >> the central repository to provide the paramenters (For instance the
>> >> value of 'To' header) from which an intelligent filtering can be done
>> >> for the set of messages avialable at the central repository.
>> >>
>> >> Perhaps someone has an idea of a better way of doing it and is willing
>> >> to share it with us. It would be really nice to hear from them.
>> >>
>> >> Thanks in advance ..!!
>> >>
>> >> Best Regards,
>> >> Nilupa
>> >>
>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>> >> <an...@gmail.com> wrote:
>> >>> Personally, I think that the added value of extending the SOAP monitor
>> >>> module to collect messages in a central place is too limited to
>> >>> attract enough interest from the user community, so that it will be
>> >>> difficult to ensure the evolution of such a project in the future.
>> >>> However, what many people are looking for is a tool that allows to
>> >>> track the flow of a message through a distributed system, or more
>> >>> generally to track the sequence of events triggered by a given input
>> >>> message (sort of end-to-end transaction monitoring).
>> >>>
>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>> >>> wrote:
>> >>>> Hello,
>> >>>>
>> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
>> >>>> proposing a GSoC project this summer. I would like to take project
>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>> >>>> also should be helpful to any Java developer using Apache Axis2 Web
>> >>>> services middleware. I will submit more detailed proposal later. I
>> >>>> would really like to hear any feedback from you. Any suggestions or
>> >>>> features that you would like to see in a "Distributed TCP Monitor"
>> >>>> are
>> >>>> most welcome.
>> >>>>
>> >>>> Best Regards,
>> >>>> Nilupa Bandara
>> >>>>
>> >>>> [1]
>> >>>>
>> >>>> Distributed TCP monitor
>> >>>>
>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>> >>>> problem by writing a Axis2 module (Handler) that intercept messages
>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
>> >>>> pull those messages. Also, we can do something to turn the module on
>> >>>> and off remotely, so users can have his module deployed, but turn it
>> >>>> on only when they want to debug the system.
>> >>>>
>> >>>> Then we can take this to next level by adding this module to all
>> >>>> services in a system , configuring modules to send all collected
>> >>>> messages to a pub/sub channel, and subscribing to those messages via
>> >>>> a
>> >>>> UI, and depicting those messages through a UI. Then users have a
>> >>>> TCPMon for a whole system.
>> >>>>
>> >>>> ---------------------------------------------------------------------
>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>>
>> >>>>
>> >>>
>> >>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>
>> >>>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>
>> >>
>> >
>> >
>> >
>> > --
>> > ============================
>> > Srinath Perera, Ph.D.
>> >   WSO2 Inc. http://wso2.com
>> >   Blog: http://srinathsview.blogspot.com/
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
>
>
> --
> Sanjiva Weerawarana, Ph.D.
> Founder, Director & Chief Scientist; Lanka Software Foundation;
> http://www.opensource.lk/
> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> Member; Apache Software Foundation; http://www.apache.org/
> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>
> Blog: http://sanjiva.weerawarana.org/
>

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

I was thinking more about something like ITCAM for SOA.

On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
<sa...@opensource.lk> wrote:
> That runtime monitoring / correlation is what the WSO2 Business Activity
> Monitor does / is for ...
> See:�http://wso2.com/products/business-activity-monitor/
> Sanjiva.
>
> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <an...@gmail.com>
> wrote:
>>
>> I was actually referring to scenarios where a service may call other
>> services, which in turn may call again other services. In these
>> scenarios, it is not sufficient to simply collect received/sent
>> messages on different hosts, unless the system is isolated
>> (development environment) or the request rate is sufficiently low so
>> that messages can be correlated based on time. Here is a concrete
>> scenario from a project in my company:
>>
>> - Request comes in on an ESB that does security and validation.
>> - Request is processed by an application server which persists the
>> received information and publishes a JMS message (pub/sub events).
>> - The event is consumed by one or more components that may in turn
>> interact with other services.
>>
>> If you want track this flow, it is not sufficient to intercept the
>> messages on the different hosts: in addition, you need to instrument
>> the services so that the outgoing requests are correlated with the
>> incoming requests (by adding SOAP and/or transport headers). What I
>> would like to be able to do in the above scenario is to enable
>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> HTTP header to the initial request to enable logging and this
>> information is propagated along the chain. Every service/component
>> then sends a copy of the incoming/outgoing requests/responses to a
>> central place where the sequence of events is reconstructed.
>>
>> One way this could be achieved is with a tool that postprocesses the
>> artifacts from the build process. For each artifact, the tool would
>> disassemble the artifact, instrument the code using a set of AspectJ
>> aspects and reassemble the artifact for deployment. The responsibility
>> of the aspects would be to intercept messages, log them and modify
>> them to ensure proper correlation. Of course this only works if the
>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> use AspectJ for a similar use case (although at a much smaller scale)
>> in the transport test suites [1]. Interestingly, this approach would
>> allow to cover interactions that are not SOAP based, e.g. one could
>> even intercept the database queries executed during the flow.
>>
>> Andreas
>>
>> [1]
>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>
>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>> > Hi Nilupa;
>> >
>> > When we collect messages from a one location by installing proper
>> > handler that will intercept and send messages to to that one location
>> > (this one location can be a single server, pub/sub channel etc). There
>> > is many ways to make sense of those collected messages. What Andreas
>> > mentioned (following complete transaction) is a one possibility.
>> >
>> > I think you should come up with few scenarios on how you would make
>> > sense of the message.
>> >
>> > Thanks
>> > Srinath
>> >
>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>> > wrote:
>> >> Hi,
>> >>
>> >> First let me thank you for commenting.
>> >>
>> >> As far as I understood, what you would like to see from the proposed
>> >> tool is to view set of messages that are exchanged in reponse to a
>> >> particular input message. With the understanding that I am having at
>> >> the momnet, one way to do it is to filter out the central repository
>> >> of messages based on 'To' , 'From' headers and try to contruct the
>> >> message chain from it. We can allow the client GUI wich connects to
>> >> the central repository to provide the paramenters (For instance the
>> >> value of 'To' header) from which an intelligent filtering can be done
>> >> for the set of messages avialable at the central repository.
>> >>
>> >> Perhaps someone has an idea of a better way of doing it and is willing
>> >> to share it with us. It would be really nice to hear from them.
>> >>
>> >> Thanks in advance ..!!
>> >>
>> >> Best Regards,
>> >> Nilupa
>> >>
>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>> >> <an...@gmail.com> wrote:
>> >>> Personally, I think that the added value of extending the SOAP monitor
>> >>> module to collect messages in a central place is too limited to
>> >>> attract enough interest from the user community, so that it will be
>> >>> difficult to ensure the evolution of such a project in the future.
>> >>> However, what many people are looking for is a tool that allows to
>> >>> track the flow of a message through a distributed system, or more
>> >>> generally to track the sequence of events triggered by a given input
>> >>> message (sort of end-to-end transaction monitoring).
>> >>>
>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>> >>> wrote:
>> >>>> Hello,
>> >>>>
>> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
>> >>>> proposing a GSoC project this summer. I would like to take project
>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>> >>>> also should be helpful to any Java developer using Apache Axis2 Web
>> >>>> services middleware. I will submit more detailed proposal later. I
>> >>>> would really like to hear any feedback from you. Any suggestions or
>> >>>> features that you would like to see in a "Distributed TCP Monitor"
>> >>>> are
>> >>>> most welcome.
>> >>>>
>> >>>> Best Regards,
>> >>>> Nilupa Bandara
>> >>>>
>> >>>> [1]
>> >>>>
>> >>>> Distributed TCP monitor
>> >>>>
>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>> >>>> problem by writing a Axis2 module (Handler) that intercept messages
>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
>> >>>> pull those messages. Also, we can do something to turn the module on
>> >>>> and off remotely, so users can have his module deployed, but turn it
>> >>>> on only when they want to debug the system.
>> >>>>
>> >>>> Then we can take this to next level by adding this module to all
>> >>>> services in a system , configuring modules to send all collected
>> >>>> messages to a pub/sub channel, and subscribing to those messages via
>> >>>> a
>> >>>> UI, and depicting those messages through a UI. Then users have a
>> >>>> TCPMon for a whole system.
>> >>>>
>> >>>> ---------------------------------------------------------------------
>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>>
>> >>>>
>> >>>
>> >>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>
>> >>>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>
>> >>
>> >
>> >
>> >
>> > --
>> > ============================
>> > Srinath Perera, Ph.D.
>> > � WSO2 Inc. http://wso2.com
>> > � Blog: http://srinathsview.blogspot.com/
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
>
>
> --
> Sanjiva Weerawarana, Ph.D.
> Founder, Director & Chief Scientist; Lanka Software Foundation;
> http://www.opensource.lk/
> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> Member; Apache Software Foundation; http://www.apache.org/
> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>
> Blog: http://sanjiva.weerawarana.org/
>

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

I was thinking more about something like ITCAM for SOA.

On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
<sa...@opensource.lk> wrote:
> That runtime monitoring / correlation is what the WSO2 Business Activity
> Monitor does / is for ...
> See: http://wso2.com/products/business-activity-monitor/
> Sanjiva.
>
> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <an...@gmail.com>
> wrote:
>>
>> I was actually referring to scenarios where a service may call other
>> services, which in turn may call again other services. In these
>> scenarios, it is not sufficient to simply collect received/sent
>> messages on different hosts, unless the system is isolated
>> (development environment) or the request rate is sufficiently low so
>> that messages can be correlated based on time. Here is a concrete
>> scenario from a project in my company:
>>
>> - Request comes in on an ESB that does security and validation.
>> - Request is processed by an application server which persists the
>> received information and publishes a JMS message (pub/sub events).
>> - The event is consumed by one or more components that may in turn
>> interact with other services.
>>
>> If you want track this flow, it is not sufficient to intercept the
>> messages on the different hosts: in addition, you need to instrument
>> the services so that the outgoing requests are correlated with the
>> incoming requests (by adding SOAP and/or transport headers). What I
>> would like to be able to do in the above scenario is to enable
>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> HTTP header to the initial request to enable logging and this
>> information is propagated along the chain. Every service/component
>> then sends a copy of the incoming/outgoing requests/responses to a
>> central place where the sequence of events is reconstructed.
>>
>> One way this could be achieved is with a tool that postprocesses the
>> artifacts from the build process. For each artifact, the tool would
>> disassemble the artifact, instrument the code using a set of AspectJ
>> aspects and reassemble the artifact for deployment. The responsibility
>> of the aspects would be to intercept messages, log them and modify
>> them to ensure proper correlation. Of course this only works if the
>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> use AspectJ for a similar use case (although at a much smaller scale)
>> in the transport test suites [1]. Interestingly, this approach would
>> allow to cover interactions that are not SOAP based, e.g. one could
>> even intercept the database queries executed during the flow.
>>
>> Andreas
>>
>> [1]
>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>
>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>> > Hi Nilupa;
>> >
>> > When we collect messages from a one location by installing proper
>> > handler that will intercept and send messages to to that one location
>> > (this one location can be a single server, pub/sub channel etc). There
>> > is many ways to make sense of those collected messages. What Andreas
>> > mentioned (following complete transaction) is a one possibility.
>> >
>> > I think you should come up with few scenarios on how you would make
>> > sense of the message.
>> >
>> > Thanks
>> > Srinath
>> >
>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>> > wrote:
>> >> Hi,
>> >>
>> >> First let me thank you for commenting.
>> >>
>> >> As far as I understood, what you would like to see from the proposed
>> >> tool is to view set of messages that are exchanged in reponse to a
>> >> particular input message. With the understanding that I am having at
>> >> the momnet, one way to do it is to filter out the central repository
>> >> of messages based on 'To' , 'From' headers and try to contruct the
>> >> message chain from it. We can allow the client GUI wich connects to
>> >> the central repository to provide the paramenters (For instance the
>> >> value of 'To' header) from which an intelligent filtering can be done
>> >> for the set of messages avialable at the central repository.
>> >>
>> >> Perhaps someone has an idea of a better way of doing it and is willing
>> >> to share it with us. It would be really nice to hear from them.
>> >>
>> >> Thanks in advance ..!!
>> >>
>> >> Best Regards,
>> >> Nilupa
>> >>
>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>> >> <an...@gmail.com> wrote:
>> >>> Personally, I think that the added value of extending the SOAP monitor
>> >>> module to collect messages in a central place is too limited to
>> >>> attract enough interest from the user community, so that it will be
>> >>> difficult to ensure the evolution of such a project in the future.
>> >>> However, what many people are looking for is a tool that allows to
>> >>> track the flow of a message through a distributed system, or more
>> >>> generally to track the sequence of events triggered by a given input
>> >>> message (sort of end-to-end transaction monitoring).
>> >>>
>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>> >>> wrote:
>> >>>> Hello,
>> >>>>
>> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
>> >>>> proposing a GSoC project this summer. I would like to take project
>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>> >>>> also should be helpful to any Java developer using Apache Axis2 Web
>> >>>> services middleware. I will submit more detailed proposal later. I
>> >>>> would really like to hear any feedback from you. Any suggestions or
>> >>>> features that you would like to see in a "Distributed TCP Monitor"
>> >>>> are
>> >>>> most welcome.
>> >>>>
>> >>>> Best Regards,
>> >>>> Nilupa Bandara
>> >>>>
>> >>>> [1]
>> >>>>
>> >>>> Distributed TCP monitor
>> >>>>
>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>> >>>> problem by writing a Axis2 module (Handler) that intercept messages
>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
>> >>>> pull those messages. Also, we can do something to turn the module on
>> >>>> and off remotely, so users can have his module deployed, but turn it
>> >>>> on only when they want to debug the system.
>> >>>>
>> >>>> Then we can take this to next level by adding this module to all
>> >>>> services in a system , configuring modules to send all collected
>> >>>> messages to a pub/sub channel, and subscribing to those messages via
>> >>>> a
>> >>>> UI, and depicting those messages through a UI. Then users have a
>> >>>> TCPMon for a whole system.
>> >>>>
>> >>>> ---------------------------------------------------------------------
>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>>
>> >>>>
>> >>>
>> >>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>
>> >>>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>
>> >>
>> >
>> >
>> >
>> > --
>> > ============================
>> > Srinath Perera, Ph.D.
>> >   WSO2 Inc. http://wso2.com
>> >   Blog: http://srinathsview.blogspot.com/
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
>
>
> --
> Sanjiva Weerawarana, Ph.D.
> Founder, Director & Chief Scientist; Lanka Software Foundation;
> http://www.opensource.lk/
> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> Member; Apache Software Foundation; http://www.apache.org/
> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>
> Blog: http://sanjiva.weerawarana.org/
>

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

I was thinking more about something like ITCAM for SOA.

On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
<sa...@opensource.lk> wrote:
> That runtime monitoring / correlation is what the WSO2 Business Activity
> Monitor does / is for ...
> See:�http://wso2.com/products/business-activity-monitor/
> Sanjiva.
>
> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <an...@gmail.com>
> wrote:
>>
>> I was actually referring to scenarios where a service may call other
>> services, which in turn may call again other services. In these
>> scenarios, it is not sufficient to simply collect received/sent
>> messages on different hosts, unless the system is isolated
>> (development environment) or the request rate is sufficiently low so
>> that messages can be correlated based on time. Here is a concrete
>> scenario from a project in my company:
>>
>> - Request comes in on an ESB that does security and validation.
>> - Request is processed by an application server which persists the
>> received information and publishes a JMS message (pub/sub events).
>> - The event is consumed by one or more components that may in turn
>> interact with other services.
>>
>> If you want track this flow, it is not sufficient to intercept the
>> messages on the different hosts: in addition, you need to instrument
>> the services so that the outgoing requests are correlated with the
>> incoming requests (by adding SOAP and/or transport headers). What I
>> would like to be able to do in the above scenario is to enable
>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> HTTP header to the initial request to enable logging and this
>> information is propagated along the chain. Every service/component
>> then sends a copy of the incoming/outgoing requests/responses to a
>> central place where the sequence of events is reconstructed.
>>
>> One way this could be achieved is with a tool that postprocesses the
>> artifacts from the build process. For each artifact, the tool would
>> disassemble the artifact, instrument the code using a set of AspectJ
>> aspects and reassemble the artifact for deployment. The responsibility
>> of the aspects would be to intercept messages, log them and modify
>> them to ensure proper correlation. Of course this only works if the
>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> use AspectJ for a similar use case (although at a much smaller scale)
>> in the transport test suites [1]. Interestingly, this approach would
>> allow to cover interactions that are not SOAP based, e.g. one could
>> even intercept the database queries executed during the flow.
>>
>> Andreas
>>
>> [1]
>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>
>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>> > Hi Nilupa;
>> >
>> > When we collect messages from a one location by installing proper
>> > handler that will intercept and send messages to to that one location
>> > (this one location can be a single server, pub/sub channel etc). There
>> > is many ways to make sense of those collected messages. What Andreas
>> > mentioned (following complete transaction) is a one possibility.
>> >
>> > I think you should come up with few scenarios on how you would make
>> > sense of the message.
>> >
>> > Thanks
>> > Srinath
>> >
>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>> > wrote:
>> >> Hi,
>> >>
>> >> First let me thank you for commenting.
>> >>
>> >> As far as I understood, what you would like to see from the proposed
>> >> tool is to view set of messages that are exchanged in reponse to a
>> >> particular input message. With the understanding that I am having at
>> >> the momnet, one way to do it is to filter out the central repository
>> >> of messages based on 'To' , 'From' headers and try to contruct the
>> >> message chain from it. We can allow the client GUI wich connects to
>> >> the central repository to provide the paramenters (For instance the
>> >> value of 'To' header) from which an intelligent filtering can be done
>> >> for the set of messages avialable at the central repository.
>> >>
>> >> Perhaps someone has an idea of a better way of doing it and is willing
>> >> to share it with us. It would be really nice to hear from them.
>> >>
>> >> Thanks in advance ..!!
>> >>
>> >> Best Regards,
>> >> Nilupa
>> >>
>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>> >> <an...@gmail.com> wrote:
>> >>> Personally, I think that the added value of extending the SOAP monitor
>> >>> module to collect messages in a central place is too limited to
>> >>> attract enough interest from the user community, so that it will be
>> >>> difficult to ensure the evolution of such a project in the future.
>> >>> However, what many people are looking for is a tool that allows to
>> >>> track the flow of a message through a distributed system, or more
>> >>> generally to track the sequence of events triggered by a given input
>> >>> message (sort of end-to-end transaction monitoring).
>> >>>
>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>> >>> wrote:
>> >>>> Hello,
>> >>>>
>> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
>> >>>> proposing a GSoC project this summer. I would like to take project
>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>> >>>> also should be helpful to any Java developer using Apache Axis2 Web
>> >>>> services middleware. I will submit more detailed proposal later. I
>> >>>> would really like to hear any feedback from you. Any suggestions or
>> >>>> features that you would like to see in a "Distributed TCP Monitor"
>> >>>> are
>> >>>> most welcome.
>> >>>>
>> >>>> Best Regards,
>> >>>> Nilupa Bandara
>> >>>>
>> >>>> [1]
>> >>>>
>> >>>> Distributed TCP monitor
>> >>>>
>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>> >>>> problem by writing a Axis2 module (Handler) that intercept messages
>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
>> >>>> pull those messages. Also, we can do something to turn the module on
>> >>>> and off remotely, so users can have his module deployed, but turn it
>> >>>> on only when they want to debug the system.
>> >>>>
>> >>>> Then we can take this to next level by adding this module to all
>> >>>> services in a system , configuring modules to send all collected
>> >>>> messages to a pub/sub channel, and subscribing to those messages via
>> >>>> a
>> >>>> UI, and depicting those messages through a UI. Then users have a
>> >>>> TCPMon for a whole system.
>> >>>>
>> >>>> ---------------------------------------------------------------------
>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>>
>> >>>>
>> >>>
>> >>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>
>> >>>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>
>> >>
>> >
>> >
>> >
>> > --
>> > ============================
>> > Srinath Perera, Ph.D.
>> > � WSO2 Inc. http://wso2.com
>> > � Blog: http://srinathsview.blogspot.com/
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
>
>
> --
> Sanjiva Weerawarana, Ph.D.
> Founder, Director & Chief Scientist; Lanka Software Foundation;
> http://www.opensource.lk/
> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> Member; Apache Software Foundation; http://www.apache.org/
> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>
> Blog: http://sanjiva.weerawarana.org/
>

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

I was thinking more about something like ITCAM for SOA.

On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
<sa...@opensource.lk> wrote:
> That runtime monitoring / correlation is what the WSO2 Business Activity
> Monitor does / is for ...
> See: http://wso2.com/products/business-activity-monitor/
> Sanjiva.
>
> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <an...@gmail.com>
> wrote:
>>
>> I was actually referring to scenarios where a service may call other
>> services, which in turn may call again other services. In these
>> scenarios, it is not sufficient to simply collect received/sent
>> messages on different hosts, unless the system is isolated
>> (development environment) or the request rate is sufficiently low so
>> that messages can be correlated based on time. Here is a concrete
>> scenario from a project in my company:
>>
>> - Request comes in on an ESB that does security and validation.
>> - Request is processed by an application server which persists the
>> received information and publishes a JMS message (pub/sub events).
>> - The event is consumed by one or more components that may in turn
>> interact with other services.
>>
>> If you want track this flow, it is not sufficient to intercept the
>> messages on the different hosts: in addition, you need to instrument
>> the services so that the outgoing requests are correlated with the
>> incoming requests (by adding SOAP and/or transport headers). What I
>> would like to be able to do in the above scenario is to enable
>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> HTTP header to the initial request to enable logging and this
>> information is propagated along the chain. Every service/component
>> then sends a copy of the incoming/outgoing requests/responses to a
>> central place where the sequence of events is reconstructed.
>>
>> One way this could be achieved is with a tool that postprocesses the
>> artifacts from the build process. For each artifact, the tool would
>> disassemble the artifact, instrument the code using a set of AspectJ
>> aspects and reassemble the artifact for deployment. The responsibility
>> of the aspects would be to intercept messages, log them and modify
>> them to ensure proper correlation. Of course this only works if the
>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> use AspectJ for a similar use case (although at a much smaller scale)
>> in the transport test suites [1]. Interestingly, this approach would
>> allow to cover interactions that are not SOAP based, e.g. one could
>> even intercept the database queries executed during the flow.
>>
>> Andreas
>>
>> [1]
>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>
>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>> > Hi Nilupa;
>> >
>> > When we collect messages from a one location by installing proper
>> > handler that will intercept and send messages to to that one location
>> > (this one location can be a single server, pub/sub channel etc). There
>> > is many ways to make sense of those collected messages. What Andreas
>> > mentioned (following complete transaction) is a one possibility.
>> >
>> > I think you should come up with few scenarios on how you would make
>> > sense of the message.
>> >
>> > Thanks
>> > Srinath
>> >
>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
>> > wrote:
>> >> Hi,
>> >>
>> >> First let me thank you for commenting.
>> >>
>> >> As far as I understood, what you would like to see from the proposed
>> >> tool is to view set of messages that are exchanged in reponse to a
>> >> particular input message. With the understanding that I am having at
>> >> the momnet, one way to do it is to filter out the central repository
>> >> of messages based on 'To' , 'From' headers and try to contruct the
>> >> message chain from it. We can allow the client GUI wich connects to
>> >> the central repository to provide the paramenters (For instance the
>> >> value of 'To' header) from which an intelligent filtering can be done
>> >> for the set of messages avialable at the central repository.
>> >>
>> >> Perhaps someone has an idea of a better way of doing it and is willing
>> >> to share it with us. It would be really nice to hear from them.
>> >>
>> >> Thanks in advance ..!!
>> >>
>> >> Best Regards,
>> >> Nilupa
>> >>
>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>> >> <an...@gmail.com> wrote:
>> >>> Personally, I think that the added value of extending the SOAP monitor
>> >>> module to collect messages in a central place is too limited to
>> >>> attract enough interest from the user community, so that it will be
>> >>> difficult to ensure the evolution of such a project in the future.
>> >>> However, what many people are looking for is a tool that allows to
>> >>> track the flow of a message through a distributed system, or more
>> >>> generally to track the sequence of events triggered by a given input
>> >>> message (sort of end-to-end transaction monitoring).
>> >>>
>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
>> >>> wrote:
>> >>>> Hello,
>> >>>>
>> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
>> >>>> proposing a GSoC project this summer. I would like to take project
>> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>> >>>> also should be helpful to any Java developer using Apache Axis2 Web
>> >>>> services middleware. I will submit more detailed proposal later. I
>> >>>> would really like to hear any feedback from you. Any suggestions or
>> >>>> features that you would like to see in a "Distributed TCP Monitor"
>> >>>> are
>> >>>> most welcome.
>> >>>>
>> >>>> Best Regards,
>> >>>> Nilupa Bandara
>> >>>>
>> >>>> [1]
>> >>>>
>> >>>> Distributed TCP monitor
>> >>>>
>> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>> >>>> problem by writing a Axis2 module (Handler) that intercept messages
>> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
>> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
>> >>>> pull those messages. Also, we can do something to turn the module on
>> >>>> and off remotely, so users can have his module deployed, but turn it
>> >>>> on only when they want to debug the system.
>> >>>>
>> >>>> Then we can take this to next level by adding this module to all
>> >>>> services in a system , configuring modules to send all collected
>> >>>> messages to a pub/sub channel, and subscribing to those messages via
>> >>>> a
>> >>>> UI, and depicting those messages through a UI. Then users have a
>> >>>> TCPMon for a whole system.
>> >>>>
>> >>>> ---------------------------------------------------------------------
>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>>
>> >>>>
>> >>>
>> >>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>>
>> >>>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>> >>
>> >>
>> >
>> >
>> >
>> > --
>> > ============================
>> > Srinath Perera, Ph.D.
>> >   WSO2 Inc. http://wso2.com
>> >   Blog: http://srinathsview.blogspot.com/
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
>
>
> --
> Sanjiva Weerawarana, Ph.D.
> Founder, Director & Chief Scientist; Lanka Software Foundation;
> http://www.opensource.lk/
> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> Member; Apache Software Foundation; http://www.apache.org/
> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>
> Blog: http://sanjiva.weerawarana.org/
>

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanjiva Weerawarana <sa...@opensource.lk>.

That runtime monitoring / correlation is what the WSO2 Business Activity
Monitor does / is for ...

See: http://wso2.com/products/business-activity-monitor/

Sanjiva.

On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
<an...@gmail.com>wrote:

> I was actually referring to scenarios where a service may call other
> services, which in turn may call again other services. In these
> scenarios, it is not sufficient to simply collect received/sent
> messages on different hosts, unless the system is isolated
> (development environment) or the request rate is sufficiently low so
> that messages can be correlated based on time. Here is a concrete
> scenario from a project in my company:
>
> - Request comes in on an ESB that does security and validation.
> - Request is processed by an application server which persists the
> received information and publishes a JMS message (pub/sub events).
> - The event is consumed by one or more components that may in turn
> interact with other services.
>
> If you want track this flow, it is not sufficient to intercept the
> messages on the different hosts: in addition, you need to instrument
> the services so that the outgoing requests are correlated with the
> incoming requests (by adding SOAP and/or transport headers). What I
> would like to be able to do in the above scenario is to enable
> end-to-end monitoring on a per-message basis: I add a special SOAP or
> HTTP header to the initial request to enable logging and this
> information is propagated along the chain. Every service/component
> then sends a copy of the incoming/outgoing requests/responses to a
> central place where the sequence of events is reconstructed.
>
> One way this could be achieved is with a tool that postprocesses the
> artifacts from the build process. For each artifact, the tool would
> disassemble the artifact, instrument the code using a set of AspectJ
> aspects and reassemble the artifact for deployment. The responsibility
> of the aspects would be to intercept messages, log them and modify
> them to ensure proper correlation. Of course this only works if the
> artifacts are J2EE deployables (WARs or EARs). Note that we already
> use AspectJ for a similar use case (although at a much smaller scale)
> in the transport test suites [1]. Interestingly, this approach would
> allow to cover interactions that are not SOAP based, e.g. one could
> even intercept the database queries executed during the flow.
>
> Andreas
>
> [1]
> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>
> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
> > Hi Nilupa;
> >
> > When we collect messages from a one location by installing proper
> > handler that will intercept and send messages to to that one location
> > (this one location can be a single server, pub/sub channel etc). There
> > is many ways to make sense of those collected messages. What Andreas
> > mentioned (following complete transaction) is a one possibility.
> >
> > I think you should come up with few scenarios on how you would make
> > sense of the message.
> >
> > Thanks
> > Srinath
> >
> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
> wrote:
> >> Hi,
> >>
> >> First let me thank you for commenting.
> >>
> >> As far as I understood, what you would like to see from the proposed
> >> tool is to view set of messages that are exchanged in reponse to a
> >> particular input message. With the understanding that I am having at
> >> the momnet, one way to do it is to filter out the central repository
> >> of messages based on 'To' , 'From' headers and try to contruct the
> >> message chain from it. We can allow the client GUI wich connects to
> >> the central repository to provide the paramenters (For instance the
> >> value of 'To' header) from which an intelligent filtering can be done
> >> for the set of messages avialable at the central repository.
> >>
> >> Perhaps someone has an idea of a better way of doing it and is willing
> >> to share it with us. It would be really nice to hear from them.
> >>
> >> Thanks in advance ..!!
> >>
> >> Best Regards,
> >> Nilupa
> >>
> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> >> <an...@gmail.com> wrote:
> >>> Personally, I think that the added value of extending the SOAP monitor
> >>> module to collect messages in a central place is too limited to
> >>> attract enough interest from the user community, so that it will be
> >>> difficult to ensure the evolution of such a project in the future.
> >>> However, what many people are looking for is a tool that allows to
> >>> track the flow of a message through a distributed system, or more
> >>> generally to track the sequence of events triggered by a given input
> >>> message (sort of end-to-end transaction monitoring).
> >>>
> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
> wrote:
> >>>> Hello,
> >>>>
> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
> >>>> proposing a GSoC project this summer. I would like to take project
> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
> >>>> also should be helpful to any Java developer using Apache Axis2 Web
> >>>> services middleware. I will submit more detailed proposal later. I
> >>>> would really like to hear any feedback from you. Any suggestions or
> >>>> features that you would like to see in a "Distributed TCP Monitor" are
> >>>> most welcome.
> >>>>
> >>>> Best Regards,
> >>>> Nilupa Bandara
> >>>>
> >>>> [1]
> >>>>
> >>>> Distributed TCP monitor
> >>>>
> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
> >>>> problem by writing a Axis2 module (Handler) that intercept messages
> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
> >>>> pull those messages. Also, we can do something to turn the module on
> >>>> and off remotely, so users can have his module deployed, but turn it
> >>>> on only when they want to debug the system.
> >>>>
> >>>> Then we can take this to next level by adding this module to all
> >>>> services in a system , configuring modules to send all collected
> >>>> messages to a pub/sub channel, and subscribing to those messages via a
> >>>> UI, and depicting those messages through a UI. Then users have a
> >>>> TCPMon for a whole system.
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>
> >>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>
> >>
> >
> >
> >
> > --
> > ============================
> > Srinath Perera, Ph.D.
> >   WSO2 Inc. http://wso2.com
> >   Blog: http://srinathsview.blogspot.com/
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>


-- 
Sanjiva Weerawarana, Ph.D.
Founder, Director & Chief Scientist; Lanka Software Foundation;
http://www.opensource.lk/
Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
Member; Apache Software Foundation; http://www.apache.org/
Director; Sahana Software Foundation; http://www.sahanafoundation.org/
Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/

Blog: http://sanjiva.weerawarana.org/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanjiva Weerawarana <sa...@opensource.lk>.

That runtime monitoring / correlation is what the WSO2 Business Activity
Monitor does / is for ...

See: http://wso2.com/products/business-activity-monitor/

Sanjiva.

On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
<an...@gmail.com>wrote:

> I was actually referring to scenarios where a service may call other
> services, which in turn may call again other services. In these
> scenarios, it is not sufficient to simply collect received/sent
> messages on different hosts, unless the system is isolated
> (development environment) or the request rate is sufficiently low so
> that messages can be correlated based on time. Here is a concrete
> scenario from a project in my company:
>
> - Request comes in on an ESB that does security and validation.
> - Request is processed by an application server which persists the
> received information and publishes a JMS message (pub/sub events).
> - The event is consumed by one or more components that may in turn
> interact with other services.
>
> If you want track this flow, it is not sufficient to intercept the
> messages on the different hosts: in addition, you need to instrument
> the services so that the outgoing requests are correlated with the
> incoming requests (by adding SOAP and/or transport headers). What I
> would like to be able to do in the above scenario is to enable
> end-to-end monitoring on a per-message basis: I add a special SOAP or
> HTTP header to the initial request to enable logging and this
> information is propagated along the chain. Every service/component
> then sends a copy of the incoming/outgoing requests/responses to a
> central place where the sequence of events is reconstructed.
>
> One way this could be achieved is with a tool that postprocesses the
> artifacts from the build process. For each artifact, the tool would
> disassemble the artifact, instrument the code using a set of AspectJ
> aspects and reassemble the artifact for deployment. The responsibility
> of the aspects would be to intercept messages, log them and modify
> them to ensure proper correlation. Of course this only works if the
> artifacts are J2EE deployables (WARs or EARs). Note that we already
> use AspectJ for a similar use case (although at a much smaller scale)
> in the transport test suites [1]. Interestingly, this approach would
> allow to cover interactions that are not SOAP based, e.g. one could
> even intercept the database queries executed during the flow.
>
> Andreas
>
> [1]
> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>
> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
> > Hi Nilupa;
> >
> > When we collect messages from a one location by installing proper
> > handler that will intercept and send messages to to that one location
> > (this one location can be a single server, pub/sub channel etc). There
> > is many ways to make sense of those collected messages. What Andreas
> > mentioned (following complete transaction) is a one possibility.
> >
> > I think you should come up with few scenarios on how you would make
> > sense of the message.
> >
> > Thanks
> > Srinath
> >
> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
> wrote:
> >> Hi,
> >>
> >> First let me thank you for commenting.
> >>
> >> As far as I understood, what you would like to see from the proposed
> >> tool is to view set of messages that are exchanged in reponse to a
> >> particular input message. With the understanding that I am having at
> >> the momnet, one way to do it is to filter out the central repository
> >> of messages based on 'To' , 'From' headers and try to contruct the
> >> message chain from it. We can allow the client GUI wich connects to
> >> the central repository to provide the paramenters (For instance the
> >> value of 'To' header) from which an intelligent filtering can be done
> >> for the set of messages avialable at the central repository.
> >>
> >> Perhaps someone has an idea of a better way of doing it and is willing
> >> to share it with us. It would be really nice to hear from them.
> >>
> >> Thanks in advance ..!!
> >>
> >> Best Regards,
> >> Nilupa
> >>
> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> >> <an...@gmail.com> wrote:
> >>> Personally, I think that the added value of extending the SOAP monitor
> >>> module to collect messages in a central place is too limited to
> >>> attract enough interest from the user community, so that it will be
> >>> difficult to ensure the evolution of such a project in the future.
> >>> However, what many people are looking for is a tool that allows to
> >>> track the flow of a message through a distributed system, or more
> >>> generally to track the sequence of events triggered by a given input
> >>> message (sort of end-to-end transaction monitoring).
> >>>
> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
> wrote:
> >>>> Hello,
> >>>>
> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
> >>>> proposing a GSoC project this summer. I would like to take project
> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
> >>>> also should be helpful to any Java developer using Apache Axis2 Web
> >>>> services middleware. I will submit more detailed proposal later. I
> >>>> would really like to hear any feedback from you. Any suggestions or
> >>>> features that you would like to see in a "Distributed TCP Monitor" are
> >>>> most welcome.
> >>>>
> >>>> Best Regards,
> >>>> Nilupa Bandara
> >>>>
> >>>> [1]
> >>>>
> >>>> Distributed TCP monitor
> >>>>
> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
> >>>> problem by writing a Axis2 module (Handler) that intercept messages
> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
> >>>> pull those messages. Also, we can do something to turn the module on
> >>>> and off remotely, so users can have his module deployed, but turn it
> >>>> on only when they want to debug the system.
> >>>>
> >>>> Then we can take this to next level by adding this module to all
> >>>> services in a system , configuring modules to send all collected
> >>>> messages to a pub/sub channel, and subscribing to those messages via a
> >>>> UI, and depicting those messages through a UI. Then users have a
> >>>> TCPMon for a whole system.
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>
> >>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>
> >>
> >
> >
> >
> > --
> > ============================
> > Srinath Perera, Ph.D.
> >   WSO2 Inc. http://wso2.com
> >   Blog: http://srinathsview.blogspot.com/
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>


-- 
Sanjiva Weerawarana, Ph.D.
Founder, Director & Chief Scientist; Lanka Software Foundation;
http://www.opensource.lk/
Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
Member; Apache Software Foundation; http://www.apache.org/
Director; Sahana Software Foundation; http://www.sahanafoundation.org/
Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/

Blog: http://sanjiva.weerawarana.org/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Glen Daniels <gl...@thoughtcraft.com>.

I'm a really terribly inconsistent blogger.  However, my last actual entry,
from 2009, was about exactly this:

http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html

I like Andreas' suggestion below, but I think it might be a bit heavyweight
for a GSoC project(?).  Maybe we could lower the bar to something like this:

* Build a distributed logging web service which can accept log messages and
correlationIDs, logging to a file per correlationID.

* Build a Module that processes incoming <debug:logInfo> headers and makes an
appropriate RemoteLogger (or whatever) available.  This could, at first, go
in one of Axis2's contexts (OperationContext?) as a demonstration.
Components who call RemoteLogger.log() send messages (hopefully on another
thread) to the central log service.

* The Module could automatically put the right metadata on outgoing messages,
but only the responses, so you still have the problem Andreas points out
about needing the app to transfer the logging info over to new
ServiceClients.  I was imagining that could be achieved by also setting a
ThreadLocal - so if your clients run on the same thread you're all set, and
if there are new threads it's up to the framework to copy that ThreadLocal over.

* Regardless of the discovery mechanism, you'd want something static like
RemoteLogger.getLogger() to hide it.

This approach is definitely a bit more intrusive, as it relies on the
components themselves to be looking for a RemoteLogger, but it seems like a
reasonable starting point which could be evolved.  However, maybe the AOP
stuff isn't as challenging as I'm imagining....

Thoughts?

--Glen

On 3/30/2010 4:57 AM, Andreas Veithen wrote:
> I was actually referring to scenarios where a service may call other
> services, which in turn may call again other services. In these
> scenarios, it is not sufficient to simply collect received/sent
> messages on different hosts, unless the system is isolated
> (development environment) or the request rate is sufficiently low so
> that messages can be correlated based on time. Here is a concrete
> scenario from a project in my company:
> 
> - Request comes in on an ESB that does security and validation.
> - Request is processed by an application server which persists the
> received information and publishes a JMS message (pub/sub events).
> - The event is consumed by one or more components that may in turn
> interact with other services.
> 
> If you want track this flow, it is not sufficient to intercept the
> messages on the different hosts: in addition, you need to instrument
> the services so that the outgoing requests are correlated with the
> incoming requests (by adding SOAP and/or transport headers). What I
> would like to be able to do in the above scenario is to enable
> end-to-end monitoring on a per-message basis: I add a special SOAP or
> HTTP header to the initial request to enable logging and this
> information is propagated along the chain. Every service/component
> then sends a copy of the incoming/outgoing requests/responses to a
> central place where the sequence of events is reconstructed.
> 
> One way this could be achieved is with a tool that postprocesses the
> artifacts from the build process. For each artifact, the tool would
> disassemble the artifact, instrument the code using a set of AspectJ
> aspects and reassemble the artifact for deployment. The responsibility
> of the aspects would be to intercept messages, log them and modify
> them to ensure proper correlation. Of course this only works if the
> artifacts are J2EE deployables (WARs or EARs). Note that we already
> use AspectJ for a similar use case (although at a much smaller scale)
> in the transport test suites [1]. Interestingly, this approach would
> allow to cover interactions that are not SOAP based, e.g. one could
> even intercept the database queries executed during the flow.
> 
> Andreas
> 
> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> 
> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>> Hi Nilupa;
>>
>> When we collect messages from a one location by installing proper
>> handler that will intercept and send messages to to that one location
>> (this one location can be a single server, pub/sub channel etc). There
>> is many ways to make sense of those collected messages. What Andreas
>> mentioned (following complete transaction) is a one possibility.
>>
>> I think you should come up with few scenarios on how you would make
>> sense of the message.
>>
>> Thanks
>> Srinath
>>
>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>> Hi,
>>>
>>> First let me thank you for commenting.
>>>
>>> As far as I understood, what you would like to see from the proposed
>>> tool is to view set of messages that are exchanged in reponse to a
>>> particular input message. With the understanding that I am having at
>>> the momnet, one way to do it is to filter out the central repository
>>> of messages based on 'To' , 'From' headers and try to contruct the
>>> message chain from it. We can allow the client GUI wich connects to
>>> the central repository to provide the paramenters (For instance the
>>> value of 'To' header) from which an intelligent filtering can be done
>>> for the set of messages avialable at the central repository.
>>>
>>> Perhaps someone has an idea of a better way of doing it and is willing
>>> to share it with us. It would be really nice to hear from them.
>>>
>>> Thanks in advance ..!!
>>>
>>> Best Regards,
>>> Nilupa
>>>
>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> <an...@gmail.com> wrote:
>>>> Personally, I think that the added value of extending the SOAP monitor
>>>> module to collect messages in a central place is too limited to
>>>> attract enough interest from the user community, so that it will be
>>>> difficult to ensure the evolution of such a project in the future.
>>>> However, what many people are looking for is a tool that allows to
>>>> track the flow of a message through a distributed system, or more
>>>> generally to track the sequence of events triggered by a given input
>>>> message (sort of end-to-end transaction monitoring).
>>>>
>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>> Hello,
>>>>>
>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>> proposing a GSoC project this summer. I would like to take project
>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>> services middleware. I will submit more detailed proposal later. I
>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>> most welcome.
>>>>>
>>>>> Best Regards,
>>>>> Nilupa Bandara
>>>>>
>>>>> [1]
>>>>>
>>>>> Distributed TCP monitor
>>>>>
>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>> pull those messages. Also, we can do something to turn the module on
>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>> on only when they want to debug the system.
>>>>>
>>>>> Then we can take this to next level by adding this module to all
>>>>> services in a system , configuring modules to send all collected
>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>> TCPMon for a whole system.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>>
>>
>> --
>> ============================
>> Srinath Perera, Ph.D.
>>   WSO2 Inc. http://wso2.com
>>   Blog: http://srinathsview.blogspot.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Glen Daniels <gl...@thoughtcraft.com>.

I'm a really terribly inconsistent blogger.  However, my last actual entry,
from 2009, was about exactly this:

http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html

I like Andreas' suggestion below, but I think it might be a bit heavyweight
for a GSoC project(?).  Maybe we could lower the bar to something like this:

* Build a distributed logging web service which can accept log messages and
correlationIDs, logging to a file per correlationID.

* Build a Module that processes incoming <debug:logInfo> headers and makes an
appropriate RemoteLogger (or whatever) available.  This could, at first, go
in one of Axis2's contexts (OperationContext?) as a demonstration.
Components who call RemoteLogger.log() send messages (hopefully on another
thread) to the central log service.

* The Module could automatically put the right metadata on outgoing messages,
but only the responses, so you still have the problem Andreas points out
about needing the app to transfer the logging info over to new
ServiceClients.  I was imagining that could be achieved by also setting a
ThreadLocal - so if your clients run on the same thread you're all set, and
if there are new threads it's up to the framework to copy that ThreadLocal over.

* Regardless of the discovery mechanism, you'd want something static like
RemoteLogger.getLogger() to hide it.

This approach is definitely a bit more intrusive, as it relies on the
components themselves to be looking for a RemoteLogger, but it seems like a
reasonable starting point which could be evolved.  However, maybe the AOP
stuff isn't as challenging as I'm imagining....

Thoughts?

--Glen

On 3/30/2010 4:57 AM, Andreas Veithen wrote:
> I was actually referring to scenarios where a service may call other
> services, which in turn may call again other services. In these
> scenarios, it is not sufficient to simply collect received/sent
> messages on different hosts, unless the system is isolated
> (development environment) or the request rate is sufficiently low so
> that messages can be correlated based on time. Here is a concrete
> scenario from a project in my company:
> 
> - Request comes in on an ESB that does security and validation.
> - Request is processed by an application server which persists the
> received information and publishes a JMS message (pub/sub events).
> - The event is consumed by one or more components that may in turn
> interact with other services.
> 
> If you want track this flow, it is not sufficient to intercept the
> messages on the different hosts: in addition, you need to instrument
> the services so that the outgoing requests are correlated with the
> incoming requests (by adding SOAP and/or transport headers). What I
> would like to be able to do in the above scenario is to enable
> end-to-end monitoring on a per-message basis: I add a special SOAP or
> HTTP header to the initial request to enable logging and this
> information is propagated along the chain. Every service/component
> then sends a copy of the incoming/outgoing requests/responses to a
> central place where the sequence of events is reconstructed.
> 
> One way this could be achieved is with a tool that postprocesses the
> artifacts from the build process. For each artifact, the tool would
> disassemble the artifact, instrument the code using a set of AspectJ
> aspects and reassemble the artifact for deployment. The responsibility
> of the aspects would be to intercept messages, log them and modify
> them to ensure proper correlation. Of course this only works if the
> artifacts are J2EE deployables (WARs or EARs). Note that we already
> use AspectJ for a similar use case (although at a much smaller scale)
> in the transport test suites [1]. Interestingly, this approach would
> allow to cover interactions that are not SOAP based, e.g. one could
> even intercept the database queries executed during the flow.
> 
> Andreas
> 
> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> 
> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>> Hi Nilupa;
>>
>> When we collect messages from a one location by installing proper
>> handler that will intercept and send messages to to that one location
>> (this one location can be a single server, pub/sub channel etc). There
>> is many ways to make sense of those collected messages. What Andreas
>> mentioned (following complete transaction) is a one possibility.
>>
>> I think you should come up with few scenarios on how you would make
>> sense of the message.
>>
>> Thanks
>> Srinath
>>
>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>> Hi,
>>>
>>> First let me thank you for commenting.
>>>
>>> As far as I understood, what you would like to see from the proposed
>>> tool is to view set of messages that are exchanged in reponse to a
>>> particular input message. With the understanding that I am having at
>>> the momnet, one way to do it is to filter out the central repository
>>> of messages based on 'To' , 'From' headers and try to contruct the
>>> message chain from it. We can allow the client GUI wich connects to
>>> the central repository to provide the paramenters (For instance the
>>> value of 'To' header) from which an intelligent filtering can be done
>>> for the set of messages avialable at the central repository.
>>>
>>> Perhaps someone has an idea of a better way of doing it and is willing
>>> to share it with us. It would be really nice to hear from them.
>>>
>>> Thanks in advance ..!!
>>>
>>> Best Regards,
>>> Nilupa
>>>
>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> <an...@gmail.com> wrote:
>>>> Personally, I think that the added value of extending the SOAP monitor
>>>> module to collect messages in a central place is too limited to
>>>> attract enough interest from the user community, so that it will be
>>>> difficult to ensure the evolution of such a project in the future.
>>>> However, what many people are looking for is a tool that allows to
>>>> track the flow of a message through a distributed system, or more
>>>> generally to track the sequence of events triggered by a given input
>>>> message (sort of end-to-end transaction monitoring).
>>>>
>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>> Hello,
>>>>>
>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>> proposing a GSoC project this summer. I would like to take project
>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>> services middleware. I will submit more detailed proposal later. I
>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>> most welcome.
>>>>>
>>>>> Best Regards,
>>>>> Nilupa Bandara
>>>>>
>>>>> [1]
>>>>>
>>>>> Distributed TCP monitor
>>>>>
>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>> pull those messages. Also, we can do something to turn the module on
>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>> on only when they want to debug the system.
>>>>>
>>>>> Then we can take this to next level by adding this module to all
>>>>> services in a system , configuring modules to send all collected
>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>> TCPMon for a whole system.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>>
>>
>> --
>> ============================
>> Srinath Perera, Ph.D.
>>   WSO2 Inc. http://wso2.com
>>   Blog: http://srinathsview.blogspot.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Glen Daniels <gl...@thoughtcraft.com>.

I'm a really terribly inconsistent blogger.  However, my last actual entry,
from 2009, was about exactly this:

http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html

I like Andreas' suggestion below, but I think it might be a bit heavyweight
for a GSoC project(?).  Maybe we could lower the bar to something like this:

* Build a distributed logging web service which can accept log messages and
correlationIDs, logging to a file per correlationID.

* Build a Module that processes incoming <debug:logInfo> headers and makes an
appropriate RemoteLogger (or whatever) available.  This could, at first, go
in one of Axis2's contexts (OperationContext?) as a demonstration.
Components who call RemoteLogger.log() send messages (hopefully on another
thread) to the central log service.

* The Module could automatically put the right metadata on outgoing messages,
but only the responses, so you still have the problem Andreas points out
about needing the app to transfer the logging info over to new
ServiceClients.  I was imagining that could be achieved by also setting a
ThreadLocal - so if your clients run on the same thread you're all set, and
if there are new threads it's up to the framework to copy that ThreadLocal over.

* Regardless of the discovery mechanism, you'd want something static like
RemoteLogger.getLogger() to hide it.

This approach is definitely a bit more intrusive, as it relies on the
components themselves to be looking for a RemoteLogger, but it seems like a
reasonable starting point which could be evolved.  However, maybe the AOP
stuff isn't as challenging as I'm imagining....

Thoughts?

--Glen

On 3/30/2010 4:57 AM, Andreas Veithen wrote:
> I was actually referring to scenarios where a service may call other
> services, which in turn may call again other services. In these
> scenarios, it is not sufficient to simply collect received/sent
> messages on different hosts, unless the system is isolated
> (development environment) or the request rate is sufficiently low so
> that messages can be correlated based on time. Here is a concrete
> scenario from a project in my company:
> 
> - Request comes in on an ESB that does security and validation.
> - Request is processed by an application server which persists the
> received information and publishes a JMS message (pub/sub events).
> - The event is consumed by one or more components that may in turn
> interact with other services.
> 
> If you want track this flow, it is not sufficient to intercept the
> messages on the different hosts: in addition, you need to instrument
> the services so that the outgoing requests are correlated with the
> incoming requests (by adding SOAP and/or transport headers). What I
> would like to be able to do in the above scenario is to enable
> end-to-end monitoring on a per-message basis: I add a special SOAP or
> HTTP header to the initial request to enable logging and this
> information is propagated along the chain. Every service/component
> then sends a copy of the incoming/outgoing requests/responses to a
> central place where the sequence of events is reconstructed.
> 
> One way this could be achieved is with a tool that postprocesses the
> artifacts from the build process. For each artifact, the tool would
> disassemble the artifact, instrument the code using a set of AspectJ
> aspects and reassemble the artifact for deployment. The responsibility
> of the aspects would be to intercept messages, log them and modify
> them to ensure proper correlation. Of course this only works if the
> artifacts are J2EE deployables (WARs or EARs). Note that we already
> use AspectJ for a similar use case (although at a much smaller scale)
> in the transport test suites [1]. Interestingly, this approach would
> allow to cover interactions that are not SOAP based, e.g. one could
> even intercept the database queries executed during the flow.
> 
> Andreas
> 
> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> 
> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>> Hi Nilupa;
>>
>> When we collect messages from a one location by installing proper
>> handler that will intercept and send messages to to that one location
>> (this one location can be a single server, pub/sub channel etc). There
>> is many ways to make sense of those collected messages. What Andreas
>> mentioned (following complete transaction) is a one possibility.
>>
>> I think you should come up with few scenarios on how you would make
>> sense of the message.
>>
>> Thanks
>> Srinath
>>
>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>> Hi,
>>>
>>> First let me thank you for commenting.
>>>
>>> As far as I understood, what you would like to see from the proposed
>>> tool is to view set of messages that are exchanged in reponse to a
>>> particular input message. With the understanding that I am having at
>>> the momnet, one way to do it is to filter out the central repository
>>> of messages based on 'To' , 'From' headers and try to contruct the
>>> message chain from it. We can allow the client GUI wich connects to
>>> the central repository to provide the paramenters (For instance the
>>> value of 'To' header) from which an intelligent filtering can be done
>>> for the set of messages avialable at the central repository.
>>>
>>> Perhaps someone has an idea of a better way of doing it and is willing
>>> to share it with us. It would be really nice to hear from them.
>>>
>>> Thanks in advance ..!!
>>>
>>> Best Regards,
>>> Nilupa
>>>
>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> <an...@gmail.com> wrote:
>>>> Personally, I think that the added value of extending the SOAP monitor
>>>> module to collect messages in a central place is too limited to
>>>> attract enough interest from the user community, so that it will be
>>>> difficult to ensure the evolution of such a project in the future.
>>>> However, what many people are looking for is a tool that allows to
>>>> track the flow of a message through a distributed system, or more
>>>> generally to track the sequence of events triggered by a given input
>>>> message (sort of end-to-end transaction monitoring).
>>>>
>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>> Hello,
>>>>>
>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>> proposing a GSoC project this summer. I would like to take project
>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>> services middleware. I will submit more detailed proposal later. I
>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>> most welcome.
>>>>>
>>>>> Best Regards,
>>>>> Nilupa Bandara
>>>>>
>>>>> [1]
>>>>>
>>>>> Distributed TCP monitor
>>>>>
>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>> pull those messages. Also, we can do something to turn the module on
>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>> on only when they want to debug the system.
>>>>>
>>>>> Then we can take this to next level by adding this module to all
>>>>> services in a system , configuring modules to send all collected
>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>> TCPMon for a whole system.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>>
>>
>> --
>> ============================
>> Srinath Perera, Ph.D.
>>   WSO2 Inc. http://wso2.com
>>   Blog: http://srinathsview.blogspot.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanjiva Weerawarana <sa...@opensource.lk>.

That runtime monitoring / correlation is what the WSO2 Business Activity
Monitor does / is for ...

See: http://wso2.com/products/business-activity-monitor/

Sanjiva.

On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
<an...@gmail.com>wrote:

> I was actually referring to scenarios where a service may call other
> services, which in turn may call again other services. In these
> scenarios, it is not sufficient to simply collect received/sent
> messages on different hosts, unless the system is isolated
> (development environment) or the request rate is sufficiently low so
> that messages can be correlated based on time. Here is a concrete
> scenario from a project in my company:
>
> - Request comes in on an ESB that does security and validation.
> - Request is processed by an application server which persists the
> received information and publishes a JMS message (pub/sub events).
> - The event is consumed by one or more components that may in turn
> interact with other services.
>
> If you want track this flow, it is not sufficient to intercept the
> messages on the different hosts: in addition, you need to instrument
> the services so that the outgoing requests are correlated with the
> incoming requests (by adding SOAP and/or transport headers). What I
> would like to be able to do in the above scenario is to enable
> end-to-end monitoring on a per-message basis: I add a special SOAP or
> HTTP header to the initial request to enable logging and this
> information is propagated along the chain. Every service/component
> then sends a copy of the incoming/outgoing requests/responses to a
> central place where the sequence of events is reconstructed.
>
> One way this could be achieved is with a tool that postprocesses the
> artifacts from the build process. For each artifact, the tool would
> disassemble the artifact, instrument the code using a set of AspectJ
> aspects and reassemble the artifact for deployment. The responsibility
> of the aspects would be to intercept messages, log them and modify
> them to ensure proper correlation. Of course this only works if the
> artifacts are J2EE deployables (WARs or EARs). Note that we already
> use AspectJ for a similar use case (although at a much smaller scale)
> in the transport test suites [1]. Interestingly, this approach would
> allow to cover interactions that are not SOAP based, e.g. one could
> even intercept the database queries executed during the flow.
>
> Andreas
>
> [1]
> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>
> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
> > Hi Nilupa;
> >
> > When we collect messages from a one location by installing proper
> > handler that will intercept and send messages to to that one location
> > (this one location can be a single server, pub/sub channel etc). There
> > is many ways to make sense of those collected messages. What Andreas
> > mentioned (following complete transaction) is a one possibility.
> >
> > I think you should come up with few scenarios on how you would make
> > sense of the message.
> >
> > Thanks
> > Srinath
> >
> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
> wrote:
> >> Hi,
> >>
> >> First let me thank you for commenting.
> >>
> >> As far as I understood, what you would like to see from the proposed
> >> tool is to view set of messages that are exchanged in reponse to a
> >> particular input message. With the understanding that I am having at
> >> the momnet, one way to do it is to filter out the central repository
> >> of messages based on 'To' , 'From' headers and try to contruct the
> >> message chain from it. We can allow the client GUI wich connects to
> >> the central repository to provide the paramenters (For instance the
> >> value of 'To' header) from which an intelligent filtering can be done
> >> for the set of messages avialable at the central repository.
> >>
> >> Perhaps someone has an idea of a better way of doing it and is willing
> >> to share it with us. It would be really nice to hear from them.
> >>
> >> Thanks in advance ..!!
> >>
> >> Best Regards,
> >> Nilupa
> >>
> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> >> <an...@gmail.com> wrote:
> >>> Personally, I think that the added value of extending the SOAP monitor
> >>> module to collect messages in a central place is too limited to
> >>> attract enough interest from the user community, so that it will be
> >>> difficult to ensure the evolution of such a project in the future.
> >>> However, what many people are looking for is a tool that allows to
> >>> track the flow of a message through a distributed system, or more
> >>> generally to track the sequence of events triggered by a given input
> >>> message (sort of end-to-end transaction monitoring).
> >>>
> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
> wrote:
> >>>> Hello,
> >>>>
> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
> >>>> proposing a GSoC project this summer. I would like to take project
> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
> >>>> also should be helpful to any Java developer using Apache Axis2 Web
> >>>> services middleware. I will submit more detailed proposal later. I
> >>>> would really like to hear any feedback from you. Any suggestions or
> >>>> features that you would like to see in a "Distributed TCP Monitor" are
> >>>> most welcome.
> >>>>
> >>>> Best Regards,
> >>>> Nilupa Bandara
> >>>>
> >>>> [1]
> >>>>
> >>>> Distributed TCP monitor
> >>>>
> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
> >>>> problem by writing a Axis2 module (Handler) that intercept messages
> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
> >>>> pull those messages. Also, we can do something to turn the module on
> >>>> and off remotely, so users can have his module deployed, but turn it
> >>>> on only when they want to debug the system.
> >>>>
> >>>> Then we can take this to next level by adding this module to all
> >>>> services in a system , configuring modules to send all collected
> >>>> messages to a pub/sub channel, and subscribing to those messages via a
> >>>> UI, and depicting those messages through a UI. Then users have a
> >>>> TCPMon for a whole system.
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>
> >>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>
> >>
> >
> >
> >
> > --
> > ============================
> > Srinath Perera, Ph.D.
> >   WSO2 Inc. http://wso2.com
> >   Blog: http://srinathsview.blogspot.com/
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>


-- 
Sanjiva Weerawarana, Ph.D.
Founder, Director & Chief Scientist; Lanka Software Foundation;
http://www.opensource.lk/
Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
Member; Apache Software Foundation; http://www.apache.org/
Director; Sahana Software Foundation; http://www.sahanafoundation.org/
Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/

Blog: http://sanjiva.weerawarana.org/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanjiva Weerawarana <sa...@opensource.lk>.

That runtime monitoring / correlation is what the WSO2 Business Activity
Monitor does / is for ...

See: http://wso2.com/products/business-activity-monitor/

Sanjiva.

On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
<an...@gmail.com>wrote:

> I was actually referring to scenarios where a service may call other
> services, which in turn may call again other services. In these
> scenarios, it is not sufficient to simply collect received/sent
> messages on different hosts, unless the system is isolated
> (development environment) or the request rate is sufficiently low so
> that messages can be correlated based on time. Here is a concrete
> scenario from a project in my company:
>
> - Request comes in on an ESB that does security and validation.
> - Request is processed by an application server which persists the
> received information and publishes a JMS message (pub/sub events).
> - The event is consumed by one or more components that may in turn
> interact with other services.
>
> If you want track this flow, it is not sufficient to intercept the
> messages on the different hosts: in addition, you need to instrument
> the services so that the outgoing requests are correlated with the
> incoming requests (by adding SOAP and/or transport headers). What I
> would like to be able to do in the above scenario is to enable
> end-to-end monitoring on a per-message basis: I add a special SOAP or
> HTTP header to the initial request to enable logging and this
> information is propagated along the chain. Every service/component
> then sends a copy of the incoming/outgoing requests/responses to a
> central place where the sequence of events is reconstructed.
>
> One way this could be achieved is with a tool that postprocesses the
> artifacts from the build process. For each artifact, the tool would
> disassemble the artifact, instrument the code using a set of AspectJ
> aspects and reassemble the artifact for deployment. The responsibility
> of the aspects would be to intercept messages, log them and modify
> them to ensure proper correlation. Of course this only works if the
> artifacts are J2EE deployables (WARs or EARs). Note that we already
> use AspectJ for a similar use case (although at a much smaller scale)
> in the transport test suites [1]. Interestingly, this approach would
> allow to cover interactions that are not SOAP based, e.g. one could
> even intercept the database queries executed during the flow.
>
> Andreas
>
> [1]
> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>
> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
> > Hi Nilupa;
> >
> > When we collect messages from a one location by installing proper
> > handler that will intercept and send messages to to that one location
> > (this one location can be a single server, pub/sub channel etc). There
> > is many ways to make sense of those collected messages. What Andreas
> > mentioned (following complete transaction) is a one possibility.
> >
> > I think you should come up with few scenarios on how you would make
> > sense of the message.
> >
> > Thanks
> > Srinath
> >
> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
> wrote:
> >> Hi,
> >>
> >> First let me thank you for commenting.
> >>
> >> As far as I understood, what you would like to see from the proposed
> >> tool is to view set of messages that are exchanged in reponse to a
> >> particular input message. With the understanding that I am having at
> >> the momnet, one way to do it is to filter out the central repository
> >> of messages based on 'To' , 'From' headers and try to contruct the
> >> message chain from it. We can allow the client GUI wich connects to
> >> the central repository to provide the paramenters (For instance the
> >> value of 'To' header) from which an intelligent filtering can be done
> >> for the set of messages avialable at the central repository.
> >>
> >> Perhaps someone has an idea of a better way of doing it and is willing
> >> to share it with us. It would be really nice to hear from them.
> >>
> >> Thanks in advance ..!!
> >>
> >> Best Regards,
> >> Nilupa
> >>
> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> >> <an...@gmail.com> wrote:
> >>> Personally, I think that the added value of extending the SOAP monitor
> >>> module to collect messages in a central place is too limited to
> >>> attract enough interest from the user community, so that it will be
> >>> difficult to ensure the evolution of such a project in the future.
> >>> However, what many people are looking for is a tool that allows to
> >>> track the flow of a message through a distributed system, or more
> >>> generally to track the sequence of events triggered by a given input
> >>> message (sort of end-to-end transaction monitoring).
> >>>
> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
> wrote:
> >>>> Hello,
> >>>>
> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
> >>>> proposing a GSoC project this summer. I would like to take project
> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
> >>>> also should be helpful to any Java developer using Apache Axis2 Web
> >>>> services middleware. I will submit more detailed proposal later. I
> >>>> would really like to hear any feedback from you. Any suggestions or
> >>>> features that you would like to see in a "Distributed TCP Monitor" are
> >>>> most welcome.
> >>>>
> >>>> Best Regards,
> >>>> Nilupa Bandara
> >>>>
> >>>> [1]
> >>>>
> >>>> Distributed TCP monitor
> >>>>
> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
> >>>> problem by writing a Axis2 module (Handler) that intercept messages
> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
> >>>> pull those messages. Also, we can do something to turn the module on
> >>>> and off remotely, so users can have his module deployed, but turn it
> >>>> on only when they want to debug the system.
> >>>>
> >>>> Then we can take this to next level by adding this module to all
> >>>> services in a system , configuring modules to send all collected
> >>>> messages to a pub/sub channel, and subscribing to those messages via a
> >>>> UI, and depicting those messages through a UI. Then users have a
> >>>> TCPMon for a whole system.
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>
> >>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>
> >>
> >
> >
> >
> > --
> > ============================
> > Srinath Perera, Ph.D.
> >   WSO2 Inc. http://wso2.com
> >   Blog: http://srinathsview.blogspot.com/
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>


-- 
Sanjiva Weerawarana, Ph.D.
Founder, Director & Chief Scientist; Lanka Software Foundation;
http://www.opensource.lk/
Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
Member; Apache Software Foundation; http://www.apache.org/
Director; Sahana Software Foundation; http://www.sahanafoundation.org/
Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/

Blog: http://sanjiva.weerawarana.org/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Glen Daniels <gl...@thoughtcraft.com>.

I'm a really terribly inconsistent blogger.  However, my last actual entry,
from 2009, was about exactly this:

http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html

I like Andreas' suggestion below, but I think it might be a bit heavyweight
for a GSoC project(?).  Maybe we could lower the bar to something like this:

* Build a distributed logging web service which can accept log messages and
correlationIDs, logging to a file per correlationID.

* Build a Module that processes incoming <debug:logInfo> headers and makes an
appropriate RemoteLogger (or whatever) available.  This could, at first, go
in one of Axis2's contexts (OperationContext?) as a demonstration.
Components who call RemoteLogger.log() send messages (hopefully on another
thread) to the central log service.

* The Module could automatically put the right metadata on outgoing messages,
but only the responses, so you still have the problem Andreas points out
about needing the app to transfer the logging info over to new
ServiceClients.  I was imagining that could be achieved by also setting a
ThreadLocal - so if your clients run on the same thread you're all set, and
if there are new threads it's up to the framework to copy that ThreadLocal over.

* Regardless of the discovery mechanism, you'd want something static like
RemoteLogger.getLogger() to hide it.

This approach is definitely a bit more intrusive, as it relies on the
components themselves to be looking for a RemoteLogger, but it seems like a
reasonable starting point which could be evolved.  However, maybe the AOP
stuff isn't as challenging as I'm imagining....

Thoughts?

--Glen

On 3/30/2010 4:57 AM, Andreas Veithen wrote:
> I was actually referring to scenarios where a service may call other
> services, which in turn may call again other services. In these
> scenarios, it is not sufficient to simply collect received/sent
> messages on different hosts, unless the system is isolated
> (development environment) or the request rate is sufficiently low so
> that messages can be correlated based on time. Here is a concrete
> scenario from a project in my company:
> 
> - Request comes in on an ESB that does security and validation.
> - Request is processed by an application server which persists the
> received information and publishes a JMS message (pub/sub events).
> - The event is consumed by one or more components that may in turn
> interact with other services.
> 
> If you want track this flow, it is not sufficient to intercept the
> messages on the different hosts: in addition, you need to instrument
> the services so that the outgoing requests are correlated with the
> incoming requests (by adding SOAP and/or transport headers). What I
> would like to be able to do in the above scenario is to enable
> end-to-end monitoring on a per-message basis: I add a special SOAP or
> HTTP header to the initial request to enable logging and this
> information is propagated along the chain. Every service/component
> then sends a copy of the incoming/outgoing requests/responses to a
> central place where the sequence of events is reconstructed.
> 
> One way this could be achieved is with a tool that postprocesses the
> artifacts from the build process. For each artifact, the tool would
> disassemble the artifact, instrument the code using a set of AspectJ
> aspects and reassemble the artifact for deployment. The responsibility
> of the aspects would be to intercept messages, log them and modify
> them to ensure proper correlation. Of course this only works if the
> artifacts are J2EE deployables (WARs or EARs). Note that we already
> use AspectJ for a similar use case (although at a much smaller scale)
> in the transport test suites [1]. Interestingly, this approach would
> allow to cover interactions that are not SOAP based, e.g. one could
> even intercept the database queries executed during the flow.
> 
> Andreas
> 
> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> 
> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
>> Hi Nilupa;
>>
>> When we collect messages from a one location by installing proper
>> handler that will intercept and send messages to to that one location
>> (this one location can be a single server, pub/sub channel etc). There
>> is many ways to make sense of those collected messages. What Andreas
>> mentioned (following complete transaction) is a one possibility.
>>
>> I think you should come up with few scenarios on how you would make
>> sense of the message.
>>
>> Thanks
>> Srinath
>>
>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>>> Hi,
>>>
>>> First let me thank you for commenting.
>>>
>>> As far as I understood, what you would like to see from the proposed
>>> tool is to view set of messages that are exchanged in reponse to a
>>> particular input message. With the understanding that I am having at
>>> the momnet, one way to do it is to filter out the central repository
>>> of messages based on 'To' , 'From' headers and try to contruct the
>>> message chain from it. We can allow the client GUI wich connects to
>>> the central repository to provide the paramenters (For instance the
>>> value of 'To' header) from which an intelligent filtering can be done
>>> for the set of messages avialable at the central repository.
>>>
>>> Perhaps someone has an idea of a better way of doing it and is willing
>>> to share it with us. It would be really nice to hear from them.
>>>
>>> Thanks in advance ..!!
>>>
>>> Best Regards,
>>> Nilupa
>>>
>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> <an...@gmail.com> wrote:
>>>> Personally, I think that the added value of extending the SOAP monitor
>>>> module to collect messages in a central place is too limited to
>>>> attract enough interest from the user community, so that it will be
>>>> difficult to ensure the evolution of such a project in the future.
>>>> However, what many people are looking for is a tool that allows to
>>>> track the flow of a message through a distributed system, or more
>>>> generally to track the sequence of events triggered by a given input
>>>> message (sort of end-to-end transaction monitoring).
>>>>
>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>>> Hello,
>>>>>
>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>> proposing a GSoC project this summer. I would like to take project
>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>> services middleware. I will submit more detailed proposal later. I
>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>> most welcome.
>>>>>
>>>>> Best Regards,
>>>>> Nilupa Bandara
>>>>>
>>>>> [1]
>>>>>
>>>>> Distributed TCP monitor
>>>>>
>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>> pull those messages. Also, we can do something to turn the module on
>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>> on only when they want to debug the system.
>>>>>
>>>>> Then we can take this to next level by adding this module to all
>>>>> services in a system , configuring modules to send all collected
>>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>> TCPMon for a whole system.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>>
>>
>> --
>> ============================
>> Srinath Perera, Ph.D.
>>   WSO2 Inc. http://wso2.com
>>   Blog: http://srinathsview.blogspot.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Sanjiva Weerawarana <sa...@opensource.lk>.

That runtime monitoring / correlation is what the WSO2 Business Activity
Monitor does / is for ...

See: http://wso2.com/products/business-activity-monitor/

Sanjiva.

On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
<an...@gmail.com>wrote:

> I was actually referring to scenarios where a service may call other
> services, which in turn may call again other services. In these
> scenarios, it is not sufficient to simply collect received/sent
> messages on different hosts, unless the system is isolated
> (development environment) or the request rate is sufficiently low so
> that messages can be correlated based on time. Here is a concrete
> scenario from a project in my company:
>
> - Request comes in on an ESB that does security and validation.
> - Request is processed by an application server which persists the
> received information and publishes a JMS message (pub/sub events).
> - The event is consumed by one or more components that may in turn
> interact with other services.
>
> If you want track this flow, it is not sufficient to intercept the
> messages on the different hosts: in addition, you need to instrument
> the services so that the outgoing requests are correlated with the
> incoming requests (by adding SOAP and/or transport headers). What I
> would like to be able to do in the above scenario is to enable
> end-to-end monitoring on a per-message basis: I add a special SOAP or
> HTTP header to the initial request to enable logging and this
> information is propagated along the chain. Every service/component
> then sends a copy of the incoming/outgoing requests/responses to a
> central place where the sequence of events is reconstructed.
>
> One way this could be achieved is with a tool that postprocesses the
> artifacts from the build process. For each artifact, the tool would
> disassemble the artifact, instrument the code using a set of AspectJ
> aspects and reassemble the artifact for deployment. The responsibility
> of the aspects would be to intercept messages, log them and modify
> them to ensure proper correlation. Of course this only works if the
> artifacts are J2EE deployables (WARs or EARs). Note that we already
> use AspectJ for a similar use case (although at a much smaller scale)
> in the transport test suites [1]. Interestingly, this approach would
> allow to cover interactions that are not SOAP based, e.g. one could
> even intercept the database queries executed during the flow.
>
> Andreas
>
> [1]
> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>
> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
> > Hi Nilupa;
> >
> > When we collect messages from a one location by installing proper
> > handler that will intercept and send messages to to that one location
> > (this one location can be a single server, pub/sub channel etc). There
> > is many ways to make sense of those collected messages. What Andreas
> > mentioned (following complete transaction) is a one possibility.
> >
> > I think you should come up with few scenarios on how you would make
> > sense of the message.
> >
> > Thanks
> > Srinath
> >
> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com>
> wrote:
> >> Hi,
> >>
> >> First let me thank you for commenting.
> >>
> >> As far as I understood, what you would like to see from the proposed
> >> tool is to view set of messages that are exchanged in reponse to a
> >> particular input message. With the understanding that I am having at
> >> the momnet, one way to do it is to filter out the central repository
> >> of messages based on 'To' , 'From' headers and try to contruct the
> >> message chain from it. We can allow the client GUI wich connects to
> >> the central repository to provide the paramenters (For instance the
> >> value of 'To' header) from which an intelligent filtering can be done
> >> for the set of messages avialable at the central repository.
> >>
> >> Perhaps someone has an idea of a better way of doing it and is willing
> >> to share it with us. It would be really nice to hear from them.
> >>
> >> Thanks in advance ..!!
> >>
> >> Best Regards,
> >> Nilupa
> >>
> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> >> <an...@gmail.com> wrote:
> >>> Personally, I think that the added value of extending the SOAP monitor
> >>> module to collect messages in a central place is too limited to
> >>> attract enough interest from the user community, so that it will be
> >>> difficult to ensure the evolution of such a project in the future.
> >>> However, what many people are looking for is a tool that allows to
> >>> track the flow of a message through a distributed system, or more
> >>> generally to track the sequence of events triggered by a given input
> >>> message (sort of end-to-end transaction monitoring).
> >>>
> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com>
> wrote:
> >>>> Hello,
> >>>>
> >>>> I am graduate student at Politecnica de Madrid and I am thinking of
> >>>> proposing a GSoC project this summer. I would like to take project
> >>>> "Distribute TCP Monitor" descried[1] which is very interesting and
> >>>> also should be helpful to any Java developer using Apache Axis2 Web
> >>>> services middleware. I will submit more detailed proposal later. I
> >>>> would really like to hear any feedback from you. Any suggestions or
> >>>> features that you would like to see in a "Distributed TCP Monitor" are
> >>>> most welcome.
> >>>>
> >>>> Best Regards,
> >>>> Nilupa Bandara
> >>>>
> >>>> [1]
> >>>>
> >>>> Distributed TCP monitor
> >>>>
> >>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
> >>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
> >>>> problem by writing a Axis2 module (Handler) that intercept messages
> >>>> comes in and goes out of a Axis2 server and send to a UI (via a
> >>>> pub/sub channel or a message Box e.g. ) or record them so user can
> >>>> pull those messages. Also, we can do something to turn the module on
> >>>> and off remotely, so users can have his module deployed, but turn it
> >>>> on only when they want to debug the system.
> >>>>
> >>>> Then we can take this to next level by adding this module to all
> >>>> services in a system , configuring modules to send all collected
> >>>> messages to a pub/sub channel, and subscribing to those messages via a
> >>>> UI, and depicting those messages through a UI. Then users have a
> >>>> TCPMon for a whole system.
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>
> >>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>
> >>
> >
> >
> >
> > --
> > ============================
> > Srinath Perera, Ph.D.
> >   WSO2 Inc. http://wso2.com
> >   Blog: http://srinathsview.blogspot.com/
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>


-- 
Sanjiva Weerawarana, Ph.D.
Founder, Director & Chief Scientist; Lanka Software Foundation;
http://www.opensource.lk/
Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
Member; Apache Software Foundation; http://www.apache.org/
Director; Sahana Software Foundation; http://www.sahanafoundation.org/
Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/

Blog: http://sanjiva.weerawarana.org/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

I was actually referring to scenarios where a service may call other
services, which in turn may call again other services. In these
scenarios, it is not sufficient to simply collect received/sent
messages on different hosts, unless the system is isolated
(development environment) or the request rate is sufficiently low so
that messages can be correlated based on time. Here is a concrete
scenario from a project in my company:

- Request comes in on an ESB that does security and validation.
- Request is processed by an application server which persists the
received information and publishes a JMS message (pub/sub events).
- The event is consumed by one or more components that may in turn
interact with other services.

If you want track this flow, it is not sufficient to intercept the
messages on the different hosts: in addition, you need to instrument
the services so that the outgoing requests are correlated with the
incoming requests (by adding SOAP and/or transport headers). What I
would like to be able to do in the above scenario is to enable
end-to-end monitoring on a per-message basis: I add a special SOAP or
HTTP header to the initial request to enable logging and this
information is propagated along the chain. Every service/component
then sends a copy of the incoming/outgoing requests/responses to a
central place where the sequence of events is reconstructed.

One way this could be achieved is with a tool that postprocesses the
artifacts from the build process. For each artifact, the tool would
disassemble the artifact, instrument the code using a set of AspectJ
aspects and reassemble the artifact for deployment. The responsibility
of the aspects would be to intercept messages, log them and modify
them to ensure proper correlation. Of course this only works if the
artifacts are J2EE deployables (WARs or EARs). Note that we already
use AspectJ for a similar use case (although at a much smaller scale)
in the transport test suites [1]. Interestingly, this approach would
allow to cover interactions that are not SOAP based, e.g. one could
even intercept the database queries executed during the flow.

Andreas

[1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java

On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
> Hi Nilupa;
>
> When we collect messages from a one location by installing proper
> handler that will intercept and send messages to to that one location
> (this one location can be a single server, pub/sub channel etc). There
> is many ways to make sense of those collected messages. What Andreas
> mentioned (following complete transaction) is a one possibility.
>
> I think you should come up with few scenarios on how you would make
> sense of the message.
>
> Thanks
> Srinath
>
> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>> Hi,
>>
>> First let me thank you for commenting.
>>
>> As far as I understood, what you would like to see from the proposed
>> tool is to view set of messages that are exchanged in reponse to a
>> particular input message. With the understanding that I am having at
>> the momnet, one way to do it is to filter out the central repository
>> of messages based on 'To' , 'From' headers and try to contruct the
>> message chain from it. We can allow the client GUI wich connects to
>> the central repository to provide the paramenters (For instance the
>> value of 'To' header) from which an intelligent filtering can be done
>> for the set of messages avialable at the central repository.
>>
>> Perhaps someone has an idea of a better way of doing it and is willing
>> to share it with us. It would be really nice to hear from them.
>>
>> Thanks in advance ..!!
>>
>> Best Regards,
>> Nilupa
>>
>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>> <an...@gmail.com> wrote:
>>> Personally, I think that the added value of extending the SOAP monitor
>>> module to collect messages in a central place is too limited to
>>> attract enough interest from the user community, so that it will be
>>> difficult to ensure the evolution of such a project in the future.
>>> However, what many people are looking for is a tool that allows to
>>> track the flow of a message through a distributed system, or more
>>> generally to track the sequence of events triggered by a given input
>>> message (sort of end-to-end transaction monitoring).
>>>
>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>> Hello,
>>>>
>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>> proposing a GSoC project this summer. I would like to take project
>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>> services middleware. I will submit more detailed proposal later. I
>>>> would really like to hear any feedback from you. Any suggestions or
>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>> most welcome.
>>>>
>>>> Best Regards,
>>>> Nilupa Bandara
>>>>
>>>> [1]
>>>>
>>>> Distributed TCP monitor
>>>>
>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>> pull those messages. Also, we can do something to turn the module on
>>>> and off remotely, so users can have his module deployed, but turn it
>>>> on only when they want to debug the system.
>>>>
>>>> Then we can take this to next level by adding this module to all
>>>> services in a system , configuring modules to send all collected
>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>> UI, and depicting those messages through a UI. Then users have a
>>>> TCPMon for a whole system.
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
>
>
> --
> ============================
> Srinath Perera, Ph.D.
>   WSO2 Inc. http://wso2.com
>   Blog: http://srinathsview.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

I was actually referring to scenarios where a service may call other
services, which in turn may call again other services. In these
scenarios, it is not sufficient to simply collect received/sent
messages on different hosts, unless the system is isolated
(development environment) or the request rate is sufficiently low so
that messages can be correlated based on time. Here is a concrete
scenario from a project in my company:

- Request comes in on an ESB that does security and validation.
- Request is processed by an application server which persists the
received information and publishes a JMS message (pub/sub events).
- The event is consumed by one or more components that may in turn
interact with other services.

If you want track this flow, it is not sufficient to intercept the
messages on the different hosts: in addition, you need to instrument
the services so that the outgoing requests are correlated with the
incoming requests (by adding SOAP and/or transport headers). What I
would like to be able to do in the above scenario is to enable
end-to-end monitoring on a per-message basis: I add a special SOAP or
HTTP header to the initial request to enable logging and this
information is propagated along the chain. Every service/component
then sends a copy of the incoming/outgoing requests/responses to a
central place where the sequence of events is reconstructed.

One way this could be achieved is with a tool that postprocesses the
artifacts from the build process. For each artifact, the tool would
disassemble the artifact, instrument the code using a set of AspectJ
aspects and reassemble the artifact for deployment. The responsibility
of the aspects would be to intercept messages, log them and modify
them to ensure proper correlation. Of course this only works if the
artifacts are J2EE deployables (WARs or EARs). Note that we already
use AspectJ for a similar use case (although at a much smaller scale)
in the transport test suites [1]. Interestingly, this approach would
allow to cover interactions that are not SOAP based, e.g. one could
even intercept the database queries executed during the flow.

Andreas

[1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java

On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
> Hi Nilupa;
>
> When we collect messages from a one location by installing proper
> handler that will intercept and send messages to to that one location
> (this one location can be a single server, pub/sub channel etc). There
> is many ways to make sense of those collected messages. What Andreas
> mentioned (following complete transaction) is a one possibility.
>
> I think you should come up with few scenarios on how you would make
> sense of the message.
>
> Thanks
> Srinath
>
> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>> Hi,
>>
>> First let me thank you for commenting.
>>
>> As far as I understood, what you would like to see from the proposed
>> tool is to view set of messages that are exchanged in reponse to a
>> particular input message. With the understanding that I am having at
>> the momnet, one way to do it is to filter out the central repository
>> of messages based on 'To' , 'From' headers and try to contruct the
>> message chain from it. We can allow the client GUI wich connects to
>> the central repository to provide the paramenters (For instance the
>> value of 'To' header) from which an intelligent filtering can be done
>> for the set of messages avialable at the central repository.
>>
>> Perhaps someone has an idea of a better way of doing it and is willing
>> to share it with us. It would be really nice to hear from them.
>>
>> Thanks in advance ..!!
>>
>> Best Regards,
>> Nilupa
>>
>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>> <an...@gmail.com> wrote:
>>> Personally, I think that the added value of extending the SOAP monitor
>>> module to collect messages in a central place is too limited to
>>> attract enough interest from the user community, so that it will be
>>> difficult to ensure the evolution of such a project in the future.
>>> However, what many people are looking for is a tool that allows to
>>> track the flow of a message through a distributed system, or more
>>> generally to track the sequence of events triggered by a given input
>>> message (sort of end-to-end transaction monitoring).
>>>
>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>> Hello,
>>>>
>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>> proposing a GSoC project this summer. I would like to take project
>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>> services middleware. I will submit more detailed proposal later. I
>>>> would really like to hear any feedback from you. Any suggestions or
>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>> most welcome.
>>>>
>>>> Best Regards,
>>>> Nilupa Bandara
>>>>
>>>> [1]
>>>>
>>>> Distributed TCP monitor
>>>>
>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>> pull those messages. Also, we can do something to turn the module on
>>>> and off remotely, so users can have his module deployed, but turn it
>>>> on only when they want to debug the system.
>>>>
>>>> Then we can take this to next level by adding this module to all
>>>> services in a system , configuring modules to send all collected
>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>> UI, and depicting those messages through a UI. Then users have a
>>>> TCPMon for a whole system.
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
>
>
> --
> ============================
> Srinath Perera, Ph.D.
>   WSO2 Inc. http://wso2.com
>   Blog: http://srinathsview.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

I was actually referring to scenarios where a service may call other
services, which in turn may call again other services. In these
scenarios, it is not sufficient to simply collect received/sent
messages on different hosts, unless the system is isolated
(development environment) or the request rate is sufficiently low so
that messages can be correlated based on time. Here is a concrete
scenario from a project in my company:

- Request comes in on an ESB that does security and validation.
- Request is processed by an application server which persists the
received information and publishes a JMS message (pub/sub events).
- The event is consumed by one or more components that may in turn
interact with other services.

If you want track this flow, it is not sufficient to intercept the
messages on the different hosts: in addition, you need to instrument
the services so that the outgoing requests are correlated with the
incoming requests (by adding SOAP and/or transport headers). What I
would like to be able to do in the above scenario is to enable
end-to-end monitoring on a per-message basis: I add a special SOAP or
HTTP header to the initial request to enable logging and this
information is propagated along the chain. Every service/component
then sends a copy of the incoming/outgoing requests/responses to a
central place where the sequence of events is reconstructed.

One way this could be achieved is with a tool that postprocesses the
artifacts from the build process. For each artifact, the tool would
disassemble the artifact, instrument the code using a set of AspectJ
aspects and reassemble the artifact for deployment. The responsibility
of the aspects would be to intercept messages, log them and modify
them to ensure proper correlation. Of course this only works if the
artifacts are J2EE deployables (WARs or EARs). Note that we already
use AspectJ for a similar use case (although at a much smaller scale)
in the transport test suites [1]. Interestingly, this approach would
allow to cover interactions that are not SOAP based, e.g. one could
even intercept the database queries executed during the flow.

Andreas

[1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java

On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
> Hi Nilupa;
>
> When we collect messages from a one location by installing proper
> handler that will intercept and send messages to to that one location
> (this one location can be a single server, pub/sub channel etc). There
> is many ways to make sense of those collected messages. What Andreas
> mentioned (following complete transaction) is a one possibility.
>
> I think you should come up with few scenarios on how you would make
> sense of the message.
>
> Thanks
> Srinath
>
> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>> Hi,
>>
>> First let me thank you for commenting.
>>
>> As far as I understood, what you would like to see from the proposed
>> tool is to view set of messages that are exchanged in reponse to a
>> particular input message. With the understanding that I am having at
>> the momnet, one way to do it is to filter out the central repository
>> of messages based on 'To' , 'From' headers and try to contruct the
>> message chain from it. We can allow the client GUI wich connects to
>> the central repository to provide the paramenters (For instance the
>> value of 'To' header) from which an intelligent filtering can be done
>> for the set of messages avialable at the central repository.
>>
>> Perhaps someone has an idea of a better way of doing it and is willing
>> to share it with us. It would be really nice to hear from them.
>>
>> Thanks in advance ..!!
>>
>> Best Regards,
>> Nilupa
>>
>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>> <an...@gmail.com> wrote:
>>> Personally, I think that the added value of extending the SOAP monitor
>>> module to collect messages in a central place is too limited to
>>> attract enough interest from the user community, so that it will be
>>> difficult to ensure the evolution of such a project in the future.
>>> However, what many people are looking for is a tool that allows to
>>> track the flow of a message through a distributed system, or more
>>> generally to track the sequence of events triggered by a given input
>>> message (sort of end-to-end transaction monitoring).
>>>
>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>> Hello,
>>>>
>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>> proposing a GSoC project this summer. I would like to take project
>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>> services middleware. I will submit more detailed proposal later. I
>>>> would really like to hear any feedback from you. Any suggestions or
>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>> most welcome.
>>>>
>>>> Best Regards,
>>>> Nilupa Bandara
>>>>
>>>> [1]
>>>>
>>>> Distributed TCP monitor
>>>>
>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>> pull those messages. Also, we can do something to turn the module on
>>>> and off remotely, so users can have his module deployed, but turn it
>>>> on only when they want to debug the system.
>>>>
>>>> Then we can take this to next level by adding this module to all
>>>> services in a system , configuring modules to send all collected
>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>> UI, and depicting those messages through a UI. Then users have a
>>>> TCPMon for a whole system.
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
>
>
> --
> ============================
> Srinath Perera, Ph.D.
>   WSO2 Inc. http://wso2.com
>   Blog: http://srinathsview.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

I was actually referring to scenarios where a service may call other
services, which in turn may call again other services. In these
scenarios, it is not sufficient to simply collect received/sent
messages on different hosts, unless the system is isolated
(development environment) or the request rate is sufficiently low so
that messages can be correlated based on time. Here is a concrete
scenario from a project in my company:

- Request comes in on an ESB that does security and validation.
- Request is processed by an application server which persists the
received information and publishes a JMS message (pub/sub events).
- The event is consumed by one or more components that may in turn
interact with other services.

If you want track this flow, it is not sufficient to intercept the
messages on the different hosts: in addition, you need to instrument
the services so that the outgoing requests are correlated with the
incoming requests (by adding SOAP and/or transport headers). What I
would like to be able to do in the above scenario is to enable
end-to-end monitoring on a per-message basis: I add a special SOAP or
HTTP header to the initial request to enable logging and this
information is propagated along the chain. Every service/component
then sends a copy of the incoming/outgoing requests/responses to a
central place where the sequence of events is reconstructed.

One way this could be achieved is with a tool that postprocesses the
artifacts from the build process. For each artifact, the tool would
disassemble the artifact, instrument the code using a set of AspectJ
aspects and reassemble the artifact for deployment. The responsibility
of the aspects would be to intercept messages, log them and modify
them to ensure proper correlation. Of course this only works if the
artifacts are J2EE deployables (WARs or EARs). Note that we already
use AspectJ for a similar use case (although at a much smaller scale)
in the transport test suites [1]. Interestingly, this approach would
allow to cover interactions that are not SOAP based, e.g. one could
even intercept the database queries executed during the flow.

Andreas

[1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java

On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
> Hi Nilupa;
>
> When we collect messages from a one location by installing proper
> handler that will intercept and send messages to to that one location
> (this one location can be a single server, pub/sub channel etc). There
> is many ways to make sense of those collected messages. What Andreas
> mentioned (following complete transaction) is a one possibility.
>
> I think you should come up with few scenarios on how you would make
> sense of the message.
>
> Thanks
> Srinath
>
> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>> Hi,
>>
>> First let me thank you for commenting.
>>
>> As far as I understood, what you would like to see from the proposed
>> tool is to view set of messages that are exchanged in reponse to a
>> particular input message. With the understanding that I am having at
>> the momnet, one way to do it is to filter out the central repository
>> of messages based on 'To' , 'From' headers and try to contruct the
>> message chain from it. We can allow the client GUI wich connects to
>> the central repository to provide the paramenters (For instance the
>> value of 'To' header) from which an intelligent filtering can be done
>> for the set of messages avialable at the central repository.
>>
>> Perhaps someone has an idea of a better way of doing it and is willing
>> to share it with us. It would be really nice to hear from them.
>>
>> Thanks in advance ..!!
>>
>> Best Regards,
>> Nilupa
>>
>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>> <an...@gmail.com> wrote:
>>> Personally, I think that the added value of extending the SOAP monitor
>>> module to collect messages in a central place is too limited to
>>> attract enough interest from the user community, so that it will be
>>> difficult to ensure the evolution of such a project in the future.
>>> However, what many people are looking for is a tool that allows to
>>> track the flow of a message through a distributed system, or more
>>> generally to track the sequence of events triggered by a given input
>>> message (sort of end-to-end transaction monitoring).
>>>
>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>> Hello,
>>>>
>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>> proposing a GSoC project this summer. I would like to take project
>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>> services middleware. I will submit more detailed proposal later. I
>>>> would really like to hear any feedback from you. Any suggestions or
>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>> most welcome.
>>>>
>>>> Best Regards,
>>>> Nilupa Bandara
>>>>
>>>> [1]
>>>>
>>>> Distributed TCP monitor
>>>>
>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>> pull those messages. Also, we can do something to turn the module on
>>>> and off remotely, so users can have his module deployed, but turn it
>>>> on only when they want to debug the system.
>>>>
>>>> Then we can take this to next level by adding this module to all
>>>> services in a system , configuring modules to send all collected
>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>> UI, and depicting those messages through a UI. Then users have a
>>>> TCPMon for a whole system.
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
>
>
> --
> ============================
> Srinath Perera, Ph.D.
>   WSO2 Inc. http://wso2.com
>   Blog: http://srinathsview.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

I was actually referring to scenarios where a service may call other
services, which in turn may call again other services. In these
scenarios, it is not sufficient to simply collect received/sent
messages on different hosts, unless the system is isolated
(development environment) or the request rate is sufficiently low so
that messages can be correlated based on time. Here is a concrete
scenario from a project in my company:

- Request comes in on an ESB that does security and validation.
- Request is processed by an application server which persists the
received information and publishes a JMS message (pub/sub events).
- The event is consumed by one or more components that may in turn
interact with other services.

If you want track this flow, it is not sufficient to intercept the
messages on the different hosts: in addition, you need to instrument
the services so that the outgoing requests are correlated with the
incoming requests (by adding SOAP and/or transport headers). What I
would like to be able to do in the above scenario is to enable
end-to-end monitoring on a per-message basis: I add a special SOAP or
HTTP header to the initial request to enable logging and this
information is propagated along the chain. Every service/component
then sends a copy of the incoming/outgoing requests/responses to a
central place where the sequence of events is reconstructed.

One way this could be achieved is with a tool that postprocesses the
artifacts from the build process. For each artifact, the tool would
disassemble the artifact, instrument the code using a set of AspectJ
aspects and reassemble the artifact for deployment. The responsibility
of the aspects would be to intercept messages, log them and modify
them to ensure proper correlation. Of course this only works if the
artifacts are J2EE deployables (WARs or EARs). Note that we already
use AspectJ for a similar use case (although at a much smaller scale)
in the transport test suites [1]. Interestingly, this approach would
allow to cover interactions that are not SOAP based, e.g. one could
even intercept the database queries executed during the flow.

Andreas

[1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java

On Tue, Mar 30, 2010 at 09:11, Srinath Perera <he...@gmail.com> wrote:
> Hi Nilupa;
>
> When we collect messages from a one location by installing proper
> handler that will intercept and send messages to to that one location
> (this one location can be a single server, pub/sub channel etc). There
> is many ways to make sense of those collected messages. What Andreas
> mentioned (following complete transaction) is a one possibility.
>
> I think you should come up with few scenarios on how you would make
> sense of the message.
>
> Thanks
> Srinath
>
> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
>> Hi,
>>
>> First let me thank you for commenting.
>>
>> As far as I understood, what you would like to see from the proposed
>> tool is to view set of messages that are exchanged in reponse to a
>> particular input message. With the understanding that I am having at
>> the momnet, one way to do it is to filter out the central repository
>> of messages based on 'To' , 'From' headers and try to contruct the
>> message chain from it. We can allow the client GUI wich connects to
>> the central repository to provide the paramenters (For instance the
>> value of 'To' header) from which an intelligent filtering can be done
>> for the set of messages avialable at the central repository.
>>
>> Perhaps someone has an idea of a better way of doing it and is willing
>> to share it with us. It would be really nice to hear from them.
>>
>> Thanks in advance ..!!
>>
>> Best Regards,
>> Nilupa
>>
>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>> <an...@gmail.com> wrote:
>>> Personally, I think that the added value of extending the SOAP monitor
>>> module to collect messages in a central place is too limited to
>>> attract enough interest from the user community, so that it will be
>>> difficult to ensure the evolution of such a project in the future.
>>> However, what many people are looking for is a tool that allows to
>>> track the flow of a message through a distributed system, or more
>>> generally to track the sequence of events triggered by a given input
>>> message (sort of end-to-end transaction monitoring).
>>>
>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>>> Hello,
>>>>
>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>> proposing a GSoC project this summer. I would like to take project
>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>> services middleware. I will submit more detailed proposal later. I
>>>> would really like to hear any feedback from you. Any suggestions or
>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>> most welcome.
>>>>
>>>> Best Regards,
>>>> Nilupa Bandara
>>>>
>>>> [1]
>>>>
>>>> Distributed TCP monitor
>>>>
>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>> pull those messages. Also, we can do something to turn the module on
>>>> and off remotely, so users can have his module deployed, but turn it
>>>> on only when they want to debug the system.
>>>>
>>>> Then we can take this to next level by adding this module to all
>>>> services in a system , configuring modules to send all collected
>>>> messages to a pub/sub channel, and subscribing to those messages via a
>>>> UI, and depicting those messages through a UI. Then users have a
>>>> TCPMon for a whole system.
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
>
>
> --
> ============================
> Srinath Perera, Ph.D.
>   WSO2 Inc. http://wso2.com
>   Blog: http://srinathsview.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Srinath Perera <he...@gmail.com>.

Hi Nilupa;

When we collect messages from a one location by installing proper
handler that will intercept and send messages to to that one location
(this one location can be a single server, pub/sub channel etc). There
is many ways to make sense of those collected messages. What Andreas
mentioned (following complete transaction) is a one possibility.

I think you should come up with few scenarios on how you would make
sense of the message.

Thanks
Srinath

On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
> Hi,
>
> First let me thank you for commenting.
>
> As far as I understood, what you would like to see from the proposed
> tool is to view set of messages that are exchanged in reponse to a
> particular input message. With the understanding that I am having at
> the momnet, one way to do it is to filter out the central repository
> of messages based on 'To' , 'From' headers and try to contruct the
> message chain from it. We can allow the client GUI wich connects to
> the central repository to provide the paramenters (For instance the
> value of 'To' header) from which an intelligent filtering can be done
> for the set of messages avialable at the central repository.
>
> Perhaps someone has an idea of a better way of doing it and is willing
> to share it with us. It would be really nice to hear from them.
>
> Thanks in advance ..!!
>
> Best Regards,
> Nilupa
>
> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> <an...@gmail.com> wrote:
>> Personally, I think that the added value of extending the SOAP monitor
>> module to collect messages in a central place is too limited to
>> attract enough interest from the user community, so that it will be
>> difficult to ensure the evolution of such a project in the future.
>> However, what many people are looking for is a tool that allows to
>> track the flow of a message through a distributed system, or more
>> generally to track the sequence of events triggered by a given input
>> message (sort of end-to-end transaction monitoring).
>>
>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>> Hello,
>>>
>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>> proposing a GSoC project this summer. I would like to take project
>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>> also should be helpful to any Java developer using Apache Axis2 Web
>>> services middleware. I will submit more detailed proposal later. I
>>> would really like to hear any feedback from you. Any suggestions or
>>> features that you would like to see in a "Distributed TCP Monitor" are
>>> most welcome.
>>>
>>> Best Regards,
>>> Nilupa Bandara
>>>
>>> [1]
>>>
>>> Distributed TCP monitor
>>>
>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>> problem by writing a Axis2 module (Handler) that intercept messages
>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>> pull those messages. Also, we can do something to turn the module on
>>> and off remotely, so users can have his module deployed, but turn it
>>> on only when they want to debug the system.
>>>
>>> Then we can take this to next level by adding this module to all
>>> services in a system , configuring modules to send all collected
>>> messages to a pub/sub channel, and subscribing to those messages via a
>>> UI, and depicting those messages through a UI. Then users have a
>>> TCPMon for a whole system.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>



-- 
============================
Srinath Perera, Ph.D.
   WSO2 Inc. http://wso2.com
   Blog: http://srinathsview.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Srinath Perera <he...@gmail.com>.

Hi Nilupa;

When we collect messages from a one location by installing proper
handler that will intercept and send messages to to that one location
(this one location can be a single server, pub/sub channel etc). There
is many ways to make sense of those collected messages. What Andreas
mentioned (following complete transaction) is a one possibility.

I think you should come up with few scenarios on how you would make
sense of the message.

Thanks
Srinath

On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
> Hi,
>
> First let me thank you for commenting.
>
> As far as I understood, what you would like to see from the proposed
> tool is to view set of messages that are exchanged in reponse to a
> particular input message. With the understanding that I am having at
> the momnet, one way to do it is to filter out the central repository
> of messages based on 'To' , 'From' headers and try to contruct the
> message chain from it. We can allow the client GUI wich connects to
> the central repository to provide the paramenters (For instance the
> value of 'To' header) from which an intelligent filtering can be done
> for the set of messages avialable at the central repository.
>
> Perhaps someone has an idea of a better way of doing it and is willing
> to share it with us. It would be really nice to hear from them.
>
> Thanks in advance ..!!
>
> Best Regards,
> Nilupa
>
> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> <an...@gmail.com> wrote:
>> Personally, I think that the added value of extending the SOAP monitor
>> module to collect messages in a central place is too limited to
>> attract enough interest from the user community, so that it will be
>> difficult to ensure the evolution of such a project in the future.
>> However, what many people are looking for is a tool that allows to
>> track the flow of a message through a distributed system, or more
>> generally to track the sequence of events triggered by a given input
>> message (sort of end-to-end transaction monitoring).
>>
>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>> Hello,
>>>
>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>> proposing a GSoC project this summer. I would like to take project
>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>> also should be helpful to any Java developer using Apache Axis2 Web
>>> services middleware. I will submit more detailed proposal later. I
>>> would really like to hear any feedback from you. Any suggestions or
>>> features that you would like to see in a "Distributed TCP Monitor" are
>>> most welcome.
>>>
>>> Best Regards,
>>> Nilupa Bandara
>>>
>>> [1]
>>>
>>> Distributed TCP monitor
>>>
>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>> problem by writing a Axis2 module (Handler) that intercept messages
>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>> pull those messages. Also, we can do something to turn the module on
>>> and off remotely, so users can have his module deployed, but turn it
>>> on only when they want to debug the system.
>>>
>>> Then we can take this to next level by adding this module to all
>>> services in a system , configuring modules to send all collected
>>> messages to a pub/sub channel, and subscribing to those messages via a
>>> UI, and depicting those messages through a UI. Then users have a
>>> TCPMon for a whole system.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>



-- 
============================
Srinath Perera, Ph.D.
   WSO2 Inc. http://wso2.com
   Blog: http://srinathsview.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Srinath Perera <he...@gmail.com>.

Hi Nilupa;

When we collect messages from a one location by installing proper
handler that will intercept and send messages to to that one location
(this one location can be a single server, pub/sub channel etc). There
is many ways to make sense of those collected messages. What Andreas
mentioned (following complete transaction) is a one possibility.

I think you should come up with few scenarios on how you would make
sense of the message.

Thanks
Srinath

On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
> Hi,
>
> First let me thank you for commenting.
>
> As far as I understood, what you would like to see from the proposed
> tool is to view set of messages that are exchanged in reponse to a
> particular input message. With the understanding that I am having at
> the momnet, one way to do it is to filter out the central repository
> of messages based on 'To' , 'From' headers and try to contruct the
> message chain from it. We can allow the client GUI wich connects to
> the central repository to provide the paramenters (For instance the
> value of 'To' header) from which an intelligent filtering can be done
> for the set of messages avialable at the central repository.
>
> Perhaps someone has an idea of a better way of doing it and is willing
> to share it with us. It would be really nice to hear from them.
>
> Thanks in advance ..!!
>
> Best Regards,
> Nilupa
>
> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> <an...@gmail.com> wrote:
>> Personally, I think that the added value of extending the SOAP monitor
>> module to collect messages in a central place is too limited to
>> attract enough interest from the user community, so that it will be
>> difficult to ensure the evolution of such a project in the future.
>> However, what many people are looking for is a tool that allows to
>> track the flow of a message through a distributed system, or more
>> generally to track the sequence of events triggered by a given input
>> message (sort of end-to-end transaction monitoring).
>>
>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>> Hello,
>>>
>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>> proposing a GSoC project this summer. I would like to take project
>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>> also should be helpful to any Java developer using Apache Axis2 Web
>>> services middleware. I will submit more detailed proposal later. I
>>> would really like to hear any feedback from you. Any suggestions or
>>> features that you would like to see in a "Distributed TCP Monitor" are
>>> most welcome.
>>>
>>> Best Regards,
>>> Nilupa Bandara
>>>
>>> [1]
>>>
>>> Distributed TCP monitor
>>>
>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>> problem by writing a Axis2 module (Handler) that intercept messages
>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>> pull those messages. Also, we can do something to turn the module on
>>> and off remotely, so users can have his module deployed, but turn it
>>> on only when they want to debug the system.
>>>
>>> Then we can take this to next level by adding this module to all
>>> services in a system , configuring modules to send all collected
>>> messages to a pub/sub channel, and subscribing to those messages via a
>>> UI, and depicting those messages through a UI. Then users have a
>>> TCPMon for a whole system.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>



-- 
============================
Srinath Perera, Ph.D.
   WSO2 Inc. http://wso2.com
   Blog: http://srinathsview.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Srinath Perera <he...@gmail.com>.

Hi Nilupa;

When we collect messages from a one location by installing proper
handler that will intercept and send messages to to that one location
(this one location can be a single server, pub/sub channel etc). There
is many ways to make sense of those collected messages. What Andreas
mentioned (following complete transaction) is a one possibility.

I think you should come up with few scenarios on how you would make
sense of the message.

Thanks
Srinath

On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
> Hi,
>
> First let me thank you for commenting.
>
> As far as I understood, what you would like to see from the proposed
> tool is to view set of messages that are exchanged in reponse to a
> particular input message. With the understanding that I am having at
> the momnet, one way to do it is to filter out the central repository
> of messages based on 'To' , 'From' headers and try to contruct the
> message chain from it. We can allow the client GUI wich connects to
> the central repository to provide the paramenters (For instance the
> value of 'To' header) from which an intelligent filtering can be done
> for the set of messages avialable at the central repository.
>
> Perhaps someone has an idea of a better way of doing it and is willing
> to share it with us. It would be really nice to hear from them.
>
> Thanks in advance ..!!
>
> Best Regards,
> Nilupa
>
> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> <an...@gmail.com> wrote:
>> Personally, I think that the added value of extending the SOAP monitor
>> module to collect messages in a central place is too limited to
>> attract enough interest from the user community, so that it will be
>> difficult to ensure the evolution of such a project in the future.
>> However, what many people are looking for is a tool that allows to
>> track the flow of a message through a distributed system, or more
>> generally to track the sequence of events triggered by a given input
>> message (sort of end-to-end transaction monitoring).
>>
>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>> Hello,
>>>
>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>> proposing a GSoC project this summer. I would like to take project
>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>> also should be helpful to any Java developer using Apache Axis2 Web
>>> services middleware. I will submit more detailed proposal later. I
>>> would really like to hear any feedback from you. Any suggestions or
>>> features that you would like to see in a "Distributed TCP Monitor" are
>>> most welcome.
>>>
>>> Best Regards,
>>> Nilupa Bandara
>>>
>>> [1]
>>>
>>> Distributed TCP monitor
>>>
>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>> problem by writing a Axis2 module (Handler) that intercept messages
>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>> pull those messages. Also, we can do something to turn the module on
>>> and off remotely, so users can have his module deployed, but turn it
>>> on only when they want to debug the system.
>>>
>>> Then we can take this to next level by adding this module to all
>>> services in a system , configuring modules to send all collected
>>> messages to a pub/sub channel, and subscribing to those messages via a
>>> UI, and depicting those messages through a UI. Then users have a
>>> TCPMon for a whole system.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>



-- 
============================
Srinath Perera, Ph.D.
   WSO2 Inc. http://wso2.com
   Blog: http://srinathsview.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Srinath Perera <he...@gmail.com>.

Hi Nilupa;

When we collect messages from a one location by installing proper
handler that will intercept and send messages to to that one location
(this one location can be a single server, pub/sub channel etc). There
is many ways to make sense of those collected messages. What Andreas
mentioned (following complete transaction) is a one possibility.

I think you should come up with few scenarios on how you would make
sense of the message.

Thanks
Srinath

On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <ni...@gmail.com> wrote:
> Hi,
>
> First let me thank you for commenting.
>
> As far as I understood, what you would like to see from the proposed
> tool is to view set of messages that are exchanged in reponse to a
> particular input message. With the understanding that I am having at
> the momnet, one way to do it is to filter out the central repository
> of messages based on 'To' , 'From' headers and try to contruct the
> message chain from it. We can allow the client GUI wich connects to
> the central repository to provide the paramenters (For instance the
> value of 'To' header) from which an intelligent filtering can be done
> for the set of messages avialable at the central repository.
>
> Perhaps someone has an idea of a better way of doing it and is willing
> to share it with us. It would be really nice to hear from them.
>
> Thanks in advance ..!!
>
> Best Regards,
> Nilupa
>
> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> <an...@gmail.com> wrote:
>> Personally, I think that the added value of extending the SOAP monitor
>> module to collect messages in a central place is too limited to
>> attract enough interest from the user community, so that it will be
>> difficult to ensure the evolution of such a project in the future.
>> However, what many people are looking for is a tool that allows to
>> track the flow of a message through a distributed system, or more
>> generally to track the sequence of events triggered by a given input
>> message (sort of end-to-end transaction monitoring).
>>
>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>>> Hello,
>>>
>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>> proposing a GSoC project this summer. I would like to take project
>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>> also should be helpful to any Java developer using Apache Axis2 Web
>>> services middleware. I will submit more detailed proposal later. I
>>> would really like to hear any feedback from you. Any suggestions or
>>> features that you would like to see in a "Distributed TCP Monitor" are
>>> most welcome.
>>>
>>> Best Regards,
>>> Nilupa Bandara
>>>
>>> [1]
>>>
>>> Distributed TCP monitor
>>>
>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>> problem by writing a Axis2 module (Handler) that intercept messages
>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>> pull those messages. Also, we can do something to turn the module on
>>> and off remotely, so users can have his module deployed, but turn it
>>> on only when they want to debug the system.
>>>
>>> Then we can take this to next level by adding this module to all
>>> services in a system , configuring modules to send all collected
>>> messages to a pub/sub channel, and subscribing to those messages via a
>>> UI, and depicting those messages through a UI. Then users have a
>>> TCPMon for a whole system.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>



-- 
============================
Srinath Perera, Ph.D.
   WSO2 Inc. http://wso2.com
   Blog: http://srinathsview.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

Hi,

First let me thank you for commenting.

As far as I understood, what you would like to see from the proposed
tool is to view set of messages that are exchanged in reponse to a
particular input message. With the understanding that I am having at
the momnet, one way to do it is to filter out the central repository
of messages based on 'To' , 'From' headers and try to contruct the
message chain from it. We can allow the client GUI wich connects to
the central repository to provide the paramenters (For instance the
value of 'To' header) from which an intelligent filtering can be done
for the set of messages avialable at the central repository.

Perhaps someone has an idea of a better way of doing it and is willing
to share it with us. It would be really nice to hear from them.

Thanks in advance ..!!

Best Regards,
Nilupa

On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
<an...@gmail.com> wrote:
> Personally, I think that the added value of extending the SOAP monitor
> module to collect messages in a central place is too limited to
> attract enough interest from the user community, so that it will be
> difficult to ensure the evolution of such a project in the future.
> However, what many people are looking for is a tool that allows to
> track the flow of a message through a distributed system, or more
> generally to track the sequence of events triggered by a given input
> message (sort of end-to-end transaction monitoring).
>
> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>> Hello,
>>
>> I am graduate student at Politecnica de Madrid and I am thinking of
>> proposing a GSoC project this summer. I would like to take project
>> "Distribute TCP Monitor" descried[1] which is very interesting and
>> also should be helpful to any Java developer using Apache Axis2 Web
>> services middleware. I will submit more detailed proposal later. I
>> would really like to hear any feedback from you. Any suggestions or
>> features that you would like to see in a "Distributed TCP Monitor" are
>> most welcome.
>>
>> Best Regards,
>> Nilupa Bandara
>>
>> [1]
>>
>> Distributed TCP monitor
>>
>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>> problem by writing a Axis2 module (Handler) that intercept messages
>> comes in and goes out of a Axis2 server and send to a UI (via a
>> pub/sub channel or a message Box e.g. ) or record them so user can
>> pull those messages. Also, we can do something to turn the module on
>> and off remotely, so users can have his module deployed, but turn it
>> on only when they want to debug the system.
>>
>> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI. Then users have a
>> TCPMon for a whole system.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

Hi,

First let me thank you for commenting.

As far as I understood, what you would like to see from the proposed
tool is to view set of messages that are exchanged in reponse to a
particular input message. With the understanding that I am having at
the momnet, one way to do it is to filter out the central repository
of messages based on 'To' , 'From' headers and try to contruct the
message chain from it. We can allow the client GUI wich connects to
the central repository to provide the paramenters (For instance the
value of 'To' header) from which an intelligent filtering can be done
for the set of messages avialable at the central repository.

Perhaps someone has an idea of a better way of doing it and is willing
to share it with us. It would be really nice to hear from them.

Thanks in advance ..!!

Best Regards,
Nilupa

On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
<an...@gmail.com> wrote:
> Personally, I think that the added value of extending the SOAP monitor
> module to collect messages in a central place is too limited to
> attract enough interest from the user community, so that it will be
> difficult to ensure the evolution of such a project in the future.
> However, what many people are looking for is a tool that allows to
> track the flow of a message through a distributed system, or more
> generally to track the sequence of events triggered by a given input
> message (sort of end-to-end transaction monitoring).
>
> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>> Hello,
>>
>> I am graduate student at Politecnica de Madrid and I am thinking of
>> proposing a GSoC project this summer. I would like to take project
>> "Distribute TCP Monitor" descried[1] which is very interesting and
>> also should be helpful to any Java developer using Apache Axis2 Web
>> services middleware. I will submit more detailed proposal later. I
>> would really like to hear any feedback from you. Any suggestions or
>> features that you would like to see in a "Distributed TCP Monitor" are
>> most welcome.
>>
>> Best Regards,
>> Nilupa Bandara
>>
>> [1]
>>
>> Distributed TCP monitor
>>
>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>> problem by writing a Axis2 module (Handler) that intercept messages
>> comes in and goes out of a Axis2 server and send to a UI (via a
>> pub/sub channel or a message Box e.g. ) or record them so user can
>> pull those messages. Also, we can do something to turn the module on
>> and off remotely, so users can have his module deployed, but turn it
>> on only when they want to debug the system.
>>
>> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI. Then users have a
>> TCPMon for a whole system.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

Hi,

First let me thank you for commenting.

As far as I understood, what you would like to see from the proposed
tool is to view set of messages that are exchanged in reponse to a
particular input message. With the understanding that I am having at
the momnet, one way to do it is to filter out the central repository
of messages based on 'To' , 'From' headers and try to contruct the
message chain from it. We can allow the client GUI wich connects to
the central repository to provide the paramenters (For instance the
value of 'To' header) from which an intelligent filtering can be done
for the set of messages avialable at the central repository.

Perhaps someone has an idea of a better way of doing it and is willing
to share it with us. It would be really nice to hear from them.

Thanks in advance ..!!

Best Regards,
Nilupa

On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
<an...@gmail.com> wrote:
> Personally, I think that the added value of extending the SOAP monitor
> module to collect messages in a central place is too limited to
> attract enough interest from the user community, so that it will be
> difficult to ensure the evolution of such a project in the future.
> However, what many people are looking for is a tool that allows to
> track the flow of a message through a distributed system, or more
> generally to track the sequence of events triggered by a given input
> message (sort of end-to-end transaction monitoring).
>
> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>> Hello,
>>
>> I am graduate student at Politecnica de Madrid and I am thinking of
>> proposing a GSoC project this summer. I would like to take project
>> "Distribute TCP Monitor" descried[1] which is very interesting and
>> also should be helpful to any Java developer using Apache Axis2 Web
>> services middleware. I will submit more detailed proposal later. I
>> would really like to hear any feedback from you. Any suggestions or
>> features that you would like to see in a "Distributed TCP Monitor" are
>> most welcome.
>>
>> Best Regards,
>> Nilupa Bandara
>>
>> [1]
>>
>> Distributed TCP monitor
>>
>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>> problem by writing a Axis2 module (Handler) that intercept messages
>> comes in and goes out of a Axis2 server and send to a UI (via a
>> pub/sub channel or a message Box e.g. ) or record them so user can
>> pull those messages. Also, we can do something to turn the module on
>> and off remotely, so users can have his module deployed, but turn it
>> on only when they want to debug the system.
>>
>> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI. Then users have a
>> TCPMon for a whole system.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

Hi,

First let me thank you for commenting.

As far as I understood, what you would like to see from the proposed
tool is to view set of messages that are exchanged in reponse to a
particular input message. With the understanding that I am having at
the momnet, one way to do it is to filter out the central repository
of messages based on 'To' , 'From' headers and try to contruct the
message chain from it. We can allow the client GUI wich connects to
the central repository to provide the paramenters (For instance the
value of 'To' header) from which an intelligent filtering can be done
for the set of messages avialable at the central repository.

Perhaps someone has an idea of a better way of doing it and is willing
to share it with us. It would be really nice to hear from them.

Thanks in advance ..!!

Best Regards,
Nilupa

On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
<an...@gmail.com> wrote:
> Personally, I think that the added value of extending the SOAP monitor
> module to collect messages in a central place is too limited to
> attract enough interest from the user community, so that it will be
> difficult to ensure the evolution of such a project in the future.
> However, what many people are looking for is a tool that allows to
> track the flow of a message through a distributed system, or more
> generally to track the sequence of events triggered by a given input
> message (sort of end-to-end transaction monitoring).
>
> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>> Hello,
>>
>> I am graduate student at Politecnica de Madrid and I am thinking of
>> proposing a GSoC project this summer. I would like to take project
>> "Distribute TCP Monitor" descried[1] which is very interesting and
>> also should be helpful to any Java developer using Apache Axis2 Web
>> services middleware. I will submit more detailed proposal later. I
>> would really like to hear any feedback from you. Any suggestions or
>> features that you would like to see in a "Distributed TCP Monitor" are
>> most welcome.
>>
>> Best Regards,
>> Nilupa Bandara
>>
>> [1]
>>
>> Distributed TCP monitor
>>
>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>> problem by writing a Axis2 module (Handler) that intercept messages
>> comes in and goes out of a Axis2 server and send to a UI (via a
>> pub/sub channel or a message Box e.g. ) or record them so user can
>> pull those messages. Also, we can do something to turn the module on
>> and off remotely, so users can have his module deployed, but turn it
>> on only when they want to debug the system.
>>
>> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI. Then users have a
>> TCPMon for a whole system.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by nilupa bandara <ni...@gmail.com>.

Hi,

First let me thank you for commenting.

As far as I understood, what you would like to see from the proposed
tool is to view set of messages that are exchanged in reponse to a
particular input message. With the understanding that I am having at
the momnet, one way to do it is to filter out the central repository
of messages based on 'To' , 'From' headers and try to contruct the
message chain from it. We can allow the client GUI wich connects to
the central repository to provide the paramenters (For instance the
value of 'To' header) from which an intelligent filtering can be done
for the set of messages avialable at the central repository.

Perhaps someone has an idea of a better way of doing it and is willing
to share it with us. It would be really nice to hear from them.

Thanks in advance ..!!

Best Regards,
Nilupa

On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
<an...@gmail.com> wrote:
> Personally, I think that the added value of extending the SOAP monitor
> module to collect messages in a central place is too limited to
> attract enough interest from the user community, so that it will be
> difficult to ensure the evolution of such a project in the future.
> However, what many people are looking for is a tool that allows to
> track the flow of a message through a distributed system, or more
> generally to track the sequence of events triggered by a given input
> message (sort of end-to-end transaction monitoring).
>
> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
>> Hello,
>>
>> I am graduate student at Politecnica de Madrid and I am thinking of
>> proposing a GSoC project this summer. I would like to take project
>> "Distribute TCP Monitor" descried[1] which is very interesting and
>> also should be helpful to any Java developer using Apache Axis2 Web
>> services middleware. I will submit more detailed proposal later. I
>> would really like to hear any feedback from you. Any suggestions or
>> features that you would like to see in a "Distributed TCP Monitor" are
>> most welcome.
>>
>> Best Regards,
>> Nilupa Bandara
>>
>> [1]
>>
>> Distributed TCP monitor
>>
>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>> problem by writing a Axis2 module (Handler) that intercept messages
>> comes in and goes out of a Axis2 server and send to a UI (via a
>> pub/sub channel or a message Box e.g. ) or record them so user can
>> pull those messages. Also, we can do something to turn the module on
>> and off remotely, so users can have his module deployed, but turn it
>> on only when they want to debug the system.
>>
>> Then we can take this to next level by adding this module to all
>> services in a system , configuring modules to send all collected
>> messages to a pub/sub channel, and subscribing to those messages via a
>> UI, and depicting those messages through a UI. Then users have a
>> TCPMon for a whole system.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

Personally, I think that the added value of extending the SOAP monitor
module to collect messages in a central place is too limited to
attract enough interest from the user community, so that it will be
difficult to ensure the evolution of such a project in the future.
However, what many people are looking for is a tool that allows to
track the flow of a message through a distributed system, or more
generally to track the sequence of events triggered by a given input
message (sort of end-to-end transaction monitoring).

On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
> Hello,
>
> I am graduate student at Politecnica de Madrid and I am thinking of
> proposing a GSoC project this summer. I would like to take project
> "Distribute TCP Monitor" descried[1] which is very interesting and
> also should be helpful to any Java developer using Apache Axis2 Web
> services middleware. I will submit more detailed proposal later. I
> would really like to hear any feedback from you. Any suggestions or
> features that you would like to see in a "Distributed TCP Monitor" are
> most welcome.
>
> Best Regards,
> Nilupa Bandara
>
> [1]
>
> Distributed TCP monitor
>
> To use TCP monitor we have to tweak the endpoints, which is bit hard
> sometimes (e.g. two channels Async case for Axis2). We can solve the
> problem by writing a Axis2 module (Handler) that intercept messages
> comes in and goes out of a Axis2 server and send to a UI (via a
> pub/sub channel or a message Box e.g. ) or record them so user can
> pull those messages. Also, we can do something to turn the module on
> and off remotely, so users can have his module deployed, but turn it
> on only when they want to debug the system.
>
> Then we can take this to next level by adding this module to all
> services in a system , configuring modules to send all collected
> messages to a pub/sub channel, and subscribing to those messages via a
> UI, and depicting those messages through a UI. Then users have a
> TCPMon for a whole system.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

>
> Then we can take this to next level by adding this module to all
> services in a system , configuring modules to send all collected
> messages to a pub/sub channel, and subscribing to those messages via a
> UI, and depicting those messages through a UI.


What pub/sub model do you plan to use? WS-Eventing?

Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

>
> Then we can take this to next level by adding this module to all
> services in a system , configuring modules to send all collected
> messages to a pub/sub channel, and subscribing to those messages via a
> UI, and depicting those messages through a UI.


What pub/sub model do you plan to use? WS-Eventing?

Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

Personally, I think that the added value of extending the SOAP monitor
module to collect messages in a central place is too limited to
attract enough interest from the user community, so that it will be
difficult to ensure the evolution of such a project in the future.
However, what many people are looking for is a tool that allows to
track the flow of a message through a distributed system, or more
generally to track the sequence of events triggered by a given input
message (sort of end-to-end transaction monitoring).

On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
> Hello,
>
> I am graduate student at Politecnica de Madrid and I am thinking of
> proposing a GSoC project this summer. I would like to take project
> "Distribute TCP Monitor" descried[1] which is very interesting and
> also should be helpful to any Java developer using Apache Axis2 Web
> services middleware. I will submit more detailed proposal later. I
> would really like to hear any feedback from you. Any suggestions or
> features that you would like to see in a "Distributed TCP Monitor" are
> most welcome.
>
> Best Regards,
> Nilupa Bandara
>
> [1]
>
> Distributed TCP monitor
>
> To use TCP monitor we have to tweak the endpoints, which is bit hard
> sometimes (e.g. two channels Async case for Axis2). We can solve the
> problem by writing a Axis2 module (Handler) that intercept messages
> comes in and goes out of a Axis2 server and send to a UI (via a
> pub/sub channel or a message Box e.g. ) or record them so user can
> pull those messages. Also, we can do something to turn the module on
> and off remotely, so users can have his module deployed, but turn it
> on only when they want to debug the system.
>
> Then we can take this to next level by adding this module to all
> services in a system , configuring modules to send all collected
> messages to a pub/sub channel, and subscribing to those messages via a
> UI, and depicting those messages through a UI. Then users have a
> TCPMon for a whole system.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

>
> Then we can take this to next level by adding this module to all
> services in a system , configuring modules to send all collected
> messages to a pub/sub channel, and subscribing to those messages via a
> UI, and depicting those messages through a UI.


What pub/sub model do you plan to use? WS-Eventing?

Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

Personally, I think that the added value of extending the SOAP monitor
module to collect messages in a central place is too limited to
attract enough interest from the user community, so that it will be
difficult to ensure the evolution of such a project in the future.
However, what many people are looking for is a tool that allows to
track the flow of a message through a distributed system, or more
generally to track the sequence of events triggered by a given input
message (sort of end-to-end transaction monitoring).

On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
> Hello,
>
> I am graduate student at Politecnica de Madrid and I am thinking of
> proposing a GSoC project this summer. I would like to take project
> "Distribute TCP Monitor" descried[1] which is very interesting and
> also should be helpful to any Java developer using Apache Axis2 Web
> services middleware. I will submit more detailed proposal later. I
> would really like to hear any feedback from you. Any suggestions or
> features that you would like to see in a "Distributed TCP Monitor" are
> most welcome.
>
> Best Regards,
> Nilupa Bandara
>
> [1]
>
> Distributed TCP monitor
>
> To use TCP monitor we have to tweak the endpoints, which is bit hard
> sometimes (e.g. two channels Async case for Axis2). We can solve the
> problem by writing a Axis2 module (Handler) that intercept messages
> comes in and goes out of a Axis2 server and send to a UI (via a
> pub/sub channel or a message Box e.g. ) or record them so user can
> pull those messages. Also, we can do something to turn the module on
> and off remotely, so users can have his module deployed, but turn it
> on only when they want to debug the system.
>
> Then we can take this to next level by adding this module to all
> services in a system , configuring modules to send all collected
> messages to a pub/sub channel, and subscribing to those messages via a
> UI, and depicting those messages through a UI. Then users have a
> TCPMon for a whole system.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Samisa Abeysinghe <sa...@gmail.com>.

>
> Then we can take this to next level by adding this module to all
> services in a system , configuring modules to send all collected
> messages to a pub/sub channel, and subscribing to those messages via a
> UI, and depicting those messages through a UI.


What pub/sub model do you plan to use? WS-Eventing?

Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

Personally, I think that the added value of extending the SOAP monitor
module to collect messages in a central place is too limited to
attract enough interest from the user community, so that it will be
difficult to ensure the evolution of such a project in the future.
However, what many people are looking for is a tool that allows to
track the flow of a message through a distributed system, or more
generally to track the sequence of events triggered by a given input
message (sort of end-to-end transaction monitoring).

On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
> Hello,
>
> I am graduate student at Politecnica de Madrid and I am thinking of
> proposing a GSoC project this summer. I would like to take project
> "Distribute TCP Monitor" descried[1] which is very interesting and
> also should be helpful to any Java developer using Apache Axis2 Web
> services middleware. I will submit more detailed proposal later. I
> would really like to hear any feedback from you. Any suggestions or
> features that you would like to see in a "Distributed TCP Monitor" are
> most welcome.
>
> Best Regards,
> Nilupa Bandara
>
> [1]
>
> Distributed TCP monitor
>
> To use TCP monitor we have to tweak the endpoints, which is bit hard
> sometimes (e.g. two channels Async case for Axis2). We can solve the
> problem by writing a Axis2 module (Handler) that intercept messages
> comes in and goes out of a Axis2 server and send to a UI (via a
> pub/sub channel or a message Box e.g. ) or record them so user can
> pull those messages. Also, we can do something to turn the module on
> and off remotely, so users can have his module deployed, but turn it
> on only when they want to debug the system.
>
> Then we can take this to next level by adding this module to all
> services in a system , configuring modules to send all collected
> messages to a pub/sub channel, and subscribing to those messages via a
> UI, and depicting those messages through a UI. Then users have a
> TCPMon for a whole system.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org

Re: [GSoC Project Proposal] Distributed TCP Monitor

Posted by Andreas Veithen <an...@gmail.com>.

Personally, I think that the added value of extending the SOAP monitor
module to collect messages in a central place is too limited to
attract enough interest from the user community, so that it will be
difficult to ensure the evolution of such a project in the future.
However, what many people are looking for is a tool that allows to
track the flow of a message through a distributed system, or more
generally to track the sequence of events triggered by a given input
message (sort of end-to-end transaction monitoring).

On Wed, Mar 24, 2010 at 23:03, nilupa bandara <ni...@gmail.com> wrote:
> Hello,
>
> I am graduate student at Politecnica de Madrid and I am thinking of
> proposing a GSoC project this summer. I would like to take project
> "Distribute TCP Monitor" descried[1] which is very interesting and
> also should be helpful to any Java developer using Apache Axis2 Web
> services middleware. I will submit more detailed proposal later. I
> would really like to hear any feedback from you. Any suggestions or
> features that you would like to see in a "Distributed TCP Monitor" are
> most welcome.
>
> Best Regards,
> Nilupa Bandara
>
> [1]
>
> Distributed TCP monitor
>
> To use TCP monitor we have to tweak the endpoints, which is bit hard
> sometimes (e.g. two channels Async case for Axis2). We can solve the
> problem by writing a Axis2 module (Handler) that intercept messages
> comes in and goes out of a Axis2 server and send to a UI (via a
> pub/sub channel or a message Box e.g. ) or record them so user can
> pull those messages. Also, we can do something to turn the module on
> and off remotely, so users can have his module deployed, but turn it
> on only when they want to debug the system.
>
> Then we can take this to next level by adding this module to all
> services in a system , configuring modules to send all collected
> messages to a pub/sub channel, and subscribing to those messages via a
> UI, and depicting those messages through a UI. Then users have a
> TCPMon for a whole system.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org