You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@synapse.apache.org by "Penkar, Cylus" <cp...@ebay.com> on 2010/08/06 01:13:13 UTC
Parsing XML Response
Folks,
We have a need to further process the response returned by a SOAP service to do some transformation on it. XSLT mediator will not work IMHO because the response is huge and correct me if I am wrong but it will definitely slow down processing.
1) Is there a way to get access to XML stream processing?
2) Can we insert logic in the endpoint to transform the response as the response stream is being read?
Thanks,
Cylus
Re: Parsing XML Response
Posted by indika kumara <in...@gmail.com>.
Hi Cylus
I am afraid I do not have enough knowledge to give an exact answer. However
, what I believe is following
Read a Part (segment) of Input -> Process the part -> write the result to
Output (The bottom line is that there are only a few segments at a given
time in the memory)
Repeat the same process until the end of the Input.
Segment can be a representation of an XML Node, or a byte array, or a line,
etc. Even it is an XML Node, if it is small, it will not incur performance.
It is what Smooks does.
if it is possible to incorporate a pure EDA approach, you may obtain both
performance and scalability.
For example .
Reader -> Transformer -> Writer
Transformer asks Reader to read a segment -> When Reader have read the
segment (when he can), it give to the Transformer -> Transformer processes
the segment and tells the Writer that he want to write -> Writer notifies
the Transformer when he can write > Transformer give the processed segment
to the Writer.
If I am wrong , please forgive me as I do not have an exact idea
Thanks
Indika
On Thu, Aug 19, 2010 at 2:35 AM, Penkar, Cylus <cp...@ebay.com> wrote:
> Thanks Indika, I will try out the Smooks mediator. In the meanwhile I have
> doing some experimentation and I was wondering is there some merit and
> having the input/output streams also available besides message.envelope.body
> access? I am aware that if the users muddle with the payload we could have
> some problems but if they know what they are doing then the transformations
> on the stream can be cheaper in terms of hardware resources (CPU & Memory
> required (1% or 2% cpu overhead & about 5K read-ahead memory - I have tried
> that out by extending Synapse's Axis2 HTTP Sender). Would like to know your
> thoughts?
>
> Thanks,
> Cylus
>
> ________________________________________
> From: indika kumara [indika.kuma@gmail.com]
> Sent: Wednesday, August 18, 2010 11:40 AM
> To: user@synapse.apache.org
> Subject: Re: Parsing XML Response
>
> Hi Cylus
>
> Sorry for late reply, I have not been touching with the Synapse's mailing
> list for last few weeks ...
>
> It is in [1]. It was implemented for a special user for supporting
> specific requirements. It uses the concept of fragment-based processing,
> which is employed by Smooks too.
>
> I believe there should be a Smooks mediator WSO2 ESB 3.0.0 release. You
> may try that.
>
> Thanks
>
> Indika
>
> [1] http://wso2.org/svn/browse/wso2/people/indika/mediators/
>
> code is though. Indika?
> >
> > Also, these sort of requirements can be handled by Smooks a
> transformation
> > library which can be converted to a mediator. To answer question on
> adding
> > logic to endpoint to process the response, it is not possible, becuase of
> > the asynchronous behavior of the synapse engine. When you call
> > endpoint.send
> > the method returns before getting the response, actually even before the
> > message sending being completed.
> >
> > Thanks,
> > Ruwan
> >
> > On Fri, Aug 6, 2010 at 4:43 AM, Penkar, Cylus <cp...@ebay.com> wrote:
> >
> > > Folks,
> > >
> > > We have a need to further process the response returned by a SOAP
> service
> > > to do some transformation on it. XSLT mediator will not work IMHO
> because
> > > the response is huge and correct me if I am wrong but it will
> definitely
> > > slow down processing.
> > >
> > > 1) Is there a way to get access to XML stream processing?
> > > 2) Can we insert logic in the endpoint to transform the response
> as
> > > the response stream is being read?
> > >
> > > Thanks,
> > > Cylus
> > >
> > >
> >
> >
> > --
> > Ruwan Linton
> > Software Architect & Product Manager, WSO2 ESB; http://wso2.org/esb
> > WSO2 <http://wso2.org/esb%0AWSO2> Inc.; http://wso2.org
> >
> > Lean . Enterprise . Middleware
> >
> > phone: +1 408 754 7388 ext 51789
> > email: ruwan@wso2.com; cell: +94 77 341 3097
> > blog: http://blog.ruwan.org
> > linkedin: http://www.linkedin.com/in/ruwanlinton
> > google: http://www.google.com/profiles/ruwan.linton
> > tweet: http://twitter.com/ruwanlinton
> >
>
RE: Parsing XML Response
Posted by "Penkar, Cylus" <cp...@ebay.com>.
Thanks Indika, I will try out the Smooks mediator. In the meanwhile I have doing some experimentation and I was wondering is there some merit and having the input/output streams also available besides message.envelope.body access? I am aware that if the users muddle with the payload we could have some problems but if they know what they are doing then the transformations on the stream can be cheaper in terms of hardware resources (CPU & Memory required (1% or 2% cpu overhead & about 5K read-ahead memory - I have tried that out by extending Synapse's Axis2 HTTP Sender). Would like to know your thoughts?
Thanks,
Cylus
________________________________________
From: indika kumara [indika.kuma@gmail.com]
Sent: Wednesday, August 18, 2010 11:40 AM
To: user@synapse.apache.org
Subject: Re: Parsing XML Response
Hi Cylus
Sorry for late reply, I have not been touching with the Synapse's mailing
list for last few weeks ...
It is in [1]. It was implemented for a special user for supporting
specific requirements. It uses the concept of fragment-based processing,
which is employed by Smooks too.
I believe there should be a Smooks mediator WSO2 ESB 3.0.0 release. You
may try that.
Thanks
Indika
[1] http://wso2.org/svn/browse/wso2/people/indika/mediators/
code is though. Indika?
>
> Also, these sort of requirements can be handled by Smooks a transformation
> library which can be converted to a mediator. To answer question on adding
> logic to endpoint to process the response, it is not possible, becuase of
> the asynchronous behavior of the synapse engine. When you call
> endpoint.send
> the method returns before getting the response, actually even before the
> message sending being completed.
>
> Thanks,
> Ruwan
>
> On Fri, Aug 6, 2010 at 4:43 AM, Penkar, Cylus <cp...@ebay.com> wrote:
>
> > Folks,
> >
> > We have a need to further process the response returned by a SOAP service
> > to do some transformation on it. XSLT mediator will not work IMHO because
> > the response is huge and correct me if I am wrong but it will definitely
> > slow down processing.
> >
> > 1) Is there a way to get access to XML stream processing?
> > 2) Can we insert logic in the endpoint to transform the response as
> > the response stream is being read?
> >
> > Thanks,
> > Cylus
> >
> >
>
>
> --
> Ruwan Linton
> Software Architect & Product Manager, WSO2 ESB; http://wso2.org/esb
> WSO2 <http://wso2.org/esb%0AWSO2> Inc.; http://wso2.org
>
> Lean . Enterprise . Middleware
>
> phone: +1 408 754 7388 ext 51789
> email: ruwan@wso2.com; cell: +94 77 341 3097
> blog: http://blog.ruwan.org
> linkedin: http://www.linkedin.com/in/ruwanlinton
> google: http://www.google.com/profiles/ruwan.linton
> tweet: http://twitter.com/ruwanlinton
>
[ANN]VTD-XML 2.9
Posted by Jimmy Zhang <cr...@comcast.net>.
VTD-XML 2.9, the next generation XML Processing API for SOA and Cloud computing, has been released. Please visit https://sourceforge.net/projects/vtd-xml/files/ to download the latest version.
a.. Strict Conformance
a.. VTD-XML now fully conforms to XML namespace 1.0 spec
b.. Performance Improvement
a.. Significantly improved parsing performance for small XML files
c.. Expand Core VTD-XML API
a.. Adds getPrefixString(), and toNormalizedString2()
d.. Cutting/Splitting
a.. Adds getSiblingElementFragment()
e.. A number of bug fixes and code enhancement including:
a.. Fixes a bug for reading very large XML documents on some platforms
b.. Fixes a bug in parsing processing instruction
c.. Fixes a bug in outputAndReparse()
__._,_.___
Re: Parsing XML Response
Posted by indika kumara <in...@gmail.com>.
Hi Cylus
Sorry for late reply, I have not been touching with the Synapse's mailing
list for last few weeks ...
It is in [1]. It was implemented for a special user for supporting
specific requirements. It uses the concept of fragment-based processing,
which is employed by Smooks too.
I believe there should be a Smooks mediator WSO2 ESB 3.0.0 release. You
may try that.
Thanks
Indika
[1] http://wso2.org/svn/browse/wso2/people/indika/mediators/
code is though. Indika?
>
> Also, these sort of requirements can be handled by Smooks a transformation
> library which can be converted to a mediator. To answer question on adding
> logic to endpoint to process the response, it is not possible, becuase of
> the asynchronous behavior of the synapse engine. When you call
> endpoint.send
> the method returns before getting the response, actually even before the
> message sending being completed.
>
> Thanks,
> Ruwan
>
> On Fri, Aug 6, 2010 at 4:43 AM, Penkar, Cylus <cp...@ebay.com> wrote:
>
> > Folks,
> >
> > We have a need to further process the response returned by a SOAP service
> > to do some transformation on it. XSLT mediator will not work IMHO because
> > the response is huge and correct me if I am wrong but it will definitely
> > slow down processing.
> >
> > 1) Is there a way to get access to XML stream processing?
> > 2) Can we insert logic in the endpoint to transform the response as
> > the response stream is being read?
> >
> > Thanks,
> > Cylus
> >
> >
>
>
> --
> Ruwan Linton
> Software Architect & Product Manager, WSO2 ESB; http://wso2.org/esb
> WSO2 <http://wso2.org/esb%0AWSO2> Inc.; http://wso2.org
>
> Lean . Enterprise . Middleware
>
> phone: +1 408 754 7388 ext 51789
> email: ruwan@wso2.com; cell: +94 77 341 3097
> blog: http://blog.ruwan.org
> linkedin: http://www.linkedin.com/in/ruwanlinton
> google: http://www.google.com/profiles/ruwan.linton
> tweet: http://twitter.com/ruwanlinton
>
Re: Parsing XML Response
Posted by Ruwan Linton <ru...@gmail.com>.
Hi Cylus,
How big is your response XML?
Indika (one of the synapse developers) has done some work related to this on
doing transformations while streaming the message, not quite sure where that
code is though. Indika?
Also, these sort of requirements can be handled by Smooks a transformation
library which can be converted to a mediator. To answer question on adding
logic to endpoint to process the response, it is not possible, becuase of
the asynchronous behavior of the synapse engine. When you call endpoint.send
the method returns before getting the response, actually even before the
message sending being completed.
Thanks,
Ruwan
On Fri, Aug 6, 2010 at 4:43 AM, Penkar, Cylus <cp...@ebay.com> wrote:
> Folks,
>
> We have a need to further process the response returned by a SOAP service
> to do some transformation on it. XSLT mediator will not work IMHO because
> the response is huge and correct me if I am wrong but it will definitely
> slow down processing.
>
> 1) Is there a way to get access to XML stream processing?
> 2) Can we insert logic in the endpoint to transform the response as
> the response stream is being read?
>
> Thanks,
> Cylus
>
>
--
Ruwan Linton
Software Architect & Product Manager, WSO2 ESB; http://wso2.org/esb
WSO2 Inc.; http://wso2.org
Lean . Enterprise . Middleware
phone: +1 408 754 7388 ext 51789
email: ruwan@wso2.com; cell: +94 77 341 3097
blog: http://blog.ruwan.org
linkedin: http://www.linkedin.com/in/ruwanlinton
google: http://www.google.com/profiles/ruwan.linton
tweet: http://twitter.com/ruwanlinton