You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-dev@axis.apache.org by Thomas Diesler <th...@jboss.org> on 2004/06/05 18:21:54 UTC
Using SAAJ-1.2 API to modify the soap header/body content
Hi Axis Dev,
I am working on a J2EE-1.4 compliant implementation for web services (WS4EE) in JBoss based on Axis. As you might know, JBoss has been using Axis in its jboss.net module for quite some time. We are starting a new project called jboss-ws4ee that we would like to use in our J2EE-1.4 certified release.
I'm wondering if you could help me with these issues
#1 SAX recorded events
Axis uses a Serializer/Deserialiser based on SAX events. The recorded events are stored with the SOAPElement.
On serialization the SOAPElement will simply playback the events if the element is not marked dirty.
A JAXRPC handler is allowed to view and modify soap headers and body. For that it can use the saaj api or even the dom api
on javax.xml.soap.Node objects.
When a client makes these modifications, they are not reflected in the axis internal representation of the soap message.
This is especially true for RPCElement, which keeps an internal list of RPCParam objects, which are not linked to SOAPElement.
#2 General soap tree representation
Axis is missing the notion of javax.xml.soap.Node, instead it models javax.xml.soap.Text as javax.xml.soap.SOAPElement.
I'm having a hard time trying to keep all the various SOAPElement representations in sync. For jboss-ws4ee I started a
refactoring effort that would base SOAPElementImpl on NodeImpl which implements the org.w3c.dom.Node interface.
All information about namespaces, attributes, parent/child relationships should be taken from the dom tree.
Serialization/Deserialization would be based on the dom tree as well.
Any suggestions?
cheers
-thomas
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
Senior Developer & Consultant
JBoss Group (Europe) SàRL
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Re: Using SAAJ-1.2 API to modify the soap header/body content
Posted by Davanum Srinivas <da...@gmail.com>.
Thomas,
Yes, there are problems in the current SAAJ implementation. Am all for
making changes to support both #1 and #2. Can we please do this
incrementally? Just start submitting patches into JIRA with some test
cases and we can get both working. Here are some guidelines
(http://nagoya.apache.org/wiki/apachewiki.cgi?AxisProjectPages/SubmitPatches).
Am looking forward to working with your team.
Thanks,
dims
----- Original Message -----
From: Thomas Diesler <th...@jboss.org>
Date: Sat, 5 Jun 2004 11:21:54 -0500
Subject: Using SAAJ-1.2 API to modify the soap header/body content
To: axis-dev@ws.apache.org
Hi Axis Dev,
I
am working on a J2EE-1.4 compliant implementation for web services (WS4EE)
in JBoss based on Axis. As you might know, JBoss has been using Axis in its
jboss.net module for quite some time. We are starting a new project
called jboss-ws4ee that we would like to use in our J2EE-1.4 certified
release.
I'm
wondering if you could help me with these issues
#1 SAX
recorded events
Axis uses a Serializer/Deserialiser
based on SAX events. The recorded events are stored with the
SOAPElement.
On serialization the SOAPElement
will simply playback the events if the element is not marked
dirty.
A JAXRPC handler is allowed to view
and modify soap headers and body. For that it can use the saaj api or even the
dom api
on javax.xml.soap.Node
objects.
When a client makes these
modifications, they are not reflected in the axis internal
representation of the
soap message.
This is especially true for
RPCElement, which keeps an internal list of RPCParam objects, which are not
linked to SOAPElement.
#2 General
soap tree representation
Axis is missing the notion of
javax.xml.soap.Node, instead it models javax.xml.soap.Text as
javax.xml.soap.SOAPElement.
I'm having a hard time trying to
keep all the various SOAPElement representations in sync. For jboss-ws4ee I
started a
refactoring effort that would base SOAPElementImpl on NodeImpl which implements
the org.w3c.dom.Node interface.
All information about namespaces,
attributes, parent/child relationships should be taken from the dom
tree.
Serialization/Deserialization would
be based on the dom tree as well.
Any
suggestions?
cheers
-thomas
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
Senior Developer &
Consultant
JBoss Group (Europe)
SàRL
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Re: Using SAAJ-1.2 API to modify the soap header/body content
Posted by Davanum Srinivas <da...@gmail.com>.
Thomas,
We've reworked a lot of code in the past few days...do you want to
take a quick look and suggest any other changes needed?
Thanks,
dims
----- Original Message -----
From: Thomas Diesler <th...@jboss.org>
Date: Sat, 5 Jun 2004 11:21:54 -0500
Subject: Using SAAJ-1.2 API to modify the soap header/body content
To: axis-dev@ws.apache.org
Hi Axis Dev,
I
am working on a J2EE-1.4 compliant implementation for web services (WS4EE)
in JBoss based on Axis. As you might know, JBoss has been using Axis in its
jboss.net module for quite some time. We are starting a new project
called jboss-ws4ee that we would like to use in our J2EE-1.4 certified
release.
I'm
wondering if you could help me with these issues
#1 SAX
recorded events
Axis uses a Serializer/Deserialiser
based on SAX events. The recorded events are stored with the
SOAPElement.
On serialization the SOAPElement
will simply playback the events if the element is not marked
dirty.
A JAXRPC handler is allowed to view
and modify soap headers and body. For that it can use the saaj api or even the
dom api
on javax.xml.soap.Node
objects.
When a client makes these
modifications, they are not reflected in the axis internal
representation of the
soap message.
This is especially true for
RPCElement, which keeps an internal list of RPCParam objects, which are not
linked to SOAPElement.
#2 General
soap tree representation
Axis is missing the notion of
javax.xml.soap.Node, instead it models javax.xml.soap.Text as
javax.xml.soap.SOAPElement.
I'm having a hard time trying to
keep all the various SOAPElement representations in sync. For jboss-ws4ee I
started a
refactoring effort that would base SOAPElementImpl on NodeImpl which implements
the org.w3c.dom.Node interface.
All information about namespaces,
attributes, parent/child relationships should be taken from the dom
tree.
Serialization/Deserialization would
be based on the dom tree as well.
Any
suggestions?
cheers
-thomas
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
Senior Developer &
Consultant
JBoss Group (Europe)
SàRL
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
--
Davanum Srinivas - http://webservices.apache.org/~dims/
Re: Using SAAJ-1.2 API to modify the soap header/body content
Posted by Davanum Srinivas <da...@gmail.com>.
Ias,
As soon as you confirm that current HEAD of Axis pass the JAX-RPC TCK,
we can finalize 1.2 and move on. So please do let us know your
progress on the axis-tck mailing list.
thanks,
dims
On Sun, 6 Jun 2004 14:46:51 +0900, Ias <ia...@tmax.co.kr> wrote:
>
> Hi Thomas,
>
> Thanks for your pointing the issues out. As a contributor of the SAAJ 1.2
> implementation in Axis (with Dr. Ahn), I have to admit that he and I already
> made our own (refactored) Axis satisfy #1 and #2. He created DOM recorder
> for #1 and I refactored the whole org.apache.axis.message package for #2. As
> for #2, org.apache.axis.message.Node was created and
> org.apache.axis.message.MessageElement and org.apache.axis.message.Text
> extended it.
>
> However, those changes are quite big and wide, so we need to modify Axis a
> lot. Now we are in the middle of finalizing Axis 1.2, I don't think such
> large changes will happen to this upcoming version of Axis. Personally, I
> often have felt that contributing #1 and #2 to the current Axis base seems
> pretty difficult and consider re-structuring the SAAJ part with Axis NG
> (Next Generation).
>
> What I'm designing regarding Axis NG is a complete implementation of JAX-RPC
> 2.0 and Web Services Metadata for high performance with StAX (streaming
> parser). I'm planning to open my thoughts on that as soon as JAX-RPC 2.0 and
> JAXB 2.0 early drafts are unveiled around JavaOne 2004. Briefly, I'd like
> Axis NG to use StAX when deserializing/serializing between SOAP messages and
> SAAJ instances. In addition, Axis NG will use JAXB 2.0 to unmarshal/marshal
> between XMLs and Java instances with StAX. JDK 1.5's metadata enables users
> to develope web services just like JSPs with WSM and generate portable
> artifacts for web services (both client and server) runtime from an
> annotated seed class.
>
> I'd like to hear your opinions about Axis' future, in particular, 1.3 and NG
> even though it's a little early. :-)
>
> Regards,
>
> Ias
>
> ________________________________
>
> From: Thomas Diesler [mailto:thomas.diesler@jboss.org]
> Sent: Sunday, June 06, 2004 1:22 AM
> To: axis-dev@ws.apache.org
> Subject: Using SAAJ-1.2 API to modify the soap header/body content
>
>
>
>
> Hi Axis Dev,
>
> I am working on a J2EE-1.4 compliant implementation for web services
> (WS4EE) in JBoss based on Axis. As you might know, JBoss has been using Axis
> in its jboss.net module for quite some time. We are starting a new project
> called jboss-ws4ee that we would like to use in our J2EE-1.4 certified
> release.
>
> I'm wondering if you could help me with these issues
>
> #1 SAX recorded events
>
> Axis uses a Serializer/Deserialiser based on SAX events. The
> recorded events are stored with the SOAPElement.
> On serialization the SOAPElement will simply playback the events
> if the element is not marked dirty.
>
> A JAXRPC handler is allowed to view and modify soap headers and
> body. For that it can use the saaj api or even the dom api
> on javax.xml.soap.Node objects.
>
> When a client makes these modifications, they are not reflected
> in the axis internal representation of the soap message.
> This is especially true for RPCElement, which keeps an internal
> list of RPCParam objects, which are not linked to SOAPElement.
>
> #2 General soap tree representation
>
> Axis is missing the notion of javax.xml.soap.Node, instead it
> models javax.xml.soap.Text as javax.xml.soap.SOAPElement.
> I'm having a hard time trying to keep all the various SOAPElement
> representations in sync. For jboss-ws4ee I started a
> refactoring effort that would base SOAPElementImpl on NodeImpl
> which implements the org.w3c.dom.Node interface.
>
> All information about namespaces, attributes, parent/child
> relationships should be taken from the dom tree.
> Serialization/Deserialization would be based on the dom tree as
> well.
>
> Any suggestions?
>
> cheers
> -thomas
>
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Thomas Diesler
> Senior Developer & Consultant
> JBoss Group (Europe) SàRL
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
>
>
Re: Using SAAJ-1.2 API to modify the soap header/body content
Posted by Davanum Srinivas <da...@gmail.com>.
Ias,
I was thinking about this today....Can you please send me the diff's
(and the new files)? Let me take a look at them.
thanks,
dims
On Sun, 6 Jun 2004 14:46:51 +0900, Ias <ia...@tmax.co.kr> wrote:
>
> Hi Thomas,
>
> Thanks for your pointing the issues out. As a contributor of the SAAJ 1.2
> implementation in Axis (with Dr. Ahn), I have to admit that he and I already
> made our own (refactored) Axis satisfy #1 and #2. He created DOM recorder
> for #1 and I refactored the whole org.apache.axis.message package for #2. As
> for #2, org.apache.axis.message.Node was created and
> org.apache.axis.message.MessageElement and org.apache.axis.message.Text
> extended it.
>
> However, those changes are quite big and wide, so we need to modify Axis a
> lot. Now we are in the middle of finalizing Axis 1.2, I don't think such
> large changes will happen to this upcoming version of Axis. Personally, I
> often have felt that contributing #1 and #2 to the current Axis base seems
> pretty difficult and consider re-structuring the SAAJ part with Axis NG
> (Next Generation).
>
> What I'm designing regarding Axis NG is a complete implementation of JAX-RPC
> 2.0 and Web Services Metadata for high performance with StAX (streaming
> parser). I'm planning to open my thoughts on that as soon as JAX-RPC 2.0 and
> JAXB 2.0 early drafts are unveiled around JavaOne 2004. Briefly, I'd like
> Axis NG to use StAX when deserializing/serializing between SOAP messages and
> SAAJ instances. In addition, Axis NG will use JAXB 2.0 to unmarshal/marshal
> between XMLs and Java instances with StAX. JDK 1.5's metadata enables users
> to develope web services just like JSPs with WSM and generate portable
> artifacts for web services (both client and server) runtime from an
> annotated seed class.
>
> I'd like to hear your opinions about Axis' future, in particular, 1.3 and NG
> even though it's a little early. :-)
>
> Regards,
>
> Ias
>
> ________________________________
>
> From: Thomas Diesler [mailto:thomas.diesler@jboss.org]
> Sent: Sunday, June 06, 2004 1:22 AM
> To: axis-dev@ws.apache.org
> Subject: Using SAAJ-1.2 API to modify the soap header/body content
>
>
>
>
> Hi Axis Dev,
>
> I am working on a J2EE-1.4 compliant implementation for web services
> (WS4EE) in JBoss based on Axis. As you might know, JBoss has been using Axis
> in its jboss.net module for quite some time. We are starting a new project
> called jboss-ws4ee that we would like to use in our J2EE-1.4 certified
> release.
>
> I'm wondering if you could help me with these issues
>
> #1 SAX recorded events
>
> Axis uses a Serializer/Deserialiser based on SAX events. The
> recorded events are stored with the SOAPElement.
> On serialization the SOAPElement will simply playback the events
> if the element is not marked dirty.
>
> A JAXRPC handler is allowed to view and modify soap headers and
> body. For that it can use the saaj api or even the dom api
> on javax.xml.soap.Node objects.
>
> When a client makes these modifications, they are not reflected
> in the axis internal representation of the soap message.
> This is especially true for RPCElement, which keeps an internal
> list of RPCParam objects, which are not linked to SOAPElement.
>
> #2 General soap tree representation
>
> Axis is missing the notion of javax.xml.soap.Node, instead it
> models javax.xml.soap.Text as javax.xml.soap.SOAPElement.
> I'm having a hard time trying to keep all the various SOAPElement
> representations in sync. For jboss-ws4ee I started a
> refactoring effort that would base SOAPElementImpl on NodeImpl
> which implements the org.w3c.dom.Node interface.
>
> All information about namespaces, attributes, parent/child
> relationships should be taken from the dom tree.
> Serialization/Deserialization would be based on the dom tree as
> well.
>
> Any suggestions?
>
> cheers
> -thomas
>
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Thomas Diesler
> Senior Developer & Consultant
> JBoss Group (Europe) SàRL
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
>
>
RE: Using SAAJ-1.2 API to modify the soap header/body content
Posted by Ias <ia...@tmax.co.kr>.
Hi Thomas,
Thanks for your pointing the issues out. As a contributor of the SAAJ 1.2
implementation in Axis (with Dr. Ahn), I have to admit that he and I already
made our own (refactored) Axis satisfy #1 and #2. He created DOM recorder
for #1 and I refactored the whole org.apache.axis.message package for #2. As
for #2, org.apache.axis.message.Node was created and
org.apache.axis.message.MessageElement and org.apache.axis.message.Text
extended it.
However, those changes are quite big and wide, so we need to modify Axis a
lot. Now we are in the middle of finalizing Axis 1.2, I don't think such
large changes will happen to this upcoming version of Axis. Personally, I
often have felt that contributing #1 and #2 to the current Axis base seems
pretty difficult and consider re-structuring the SAAJ part with Axis NG
(Next Generation).
What I'm designing regarding Axis NG is a complete implementation of JAX-RPC
2.0 and Web Services Metadata for high performance with StAX (streaming
parser). I'm planning to open my thoughts on that as soon as JAX-RPC 2.0 and
JAXB 2.0 early drafts are unveiled around JavaOne 2004. Briefly, I'd like
Axis NG to use StAX when deserializing/serializing between SOAP messages and
SAAJ instances. In addition, Axis NG will use JAXB 2.0 to unmarshal/marshal
between XMLs and Java instances with StAX. JDK 1.5's metadata enables users
to develope web services just like JSPs with WSM and generate portable
artifacts for web services (both client and server) runtime from an
annotated seed class.
I'd like to hear your opinions about Axis' future, in particular, 1.3 and NG
even though it's a little early. :-)
Regards,
Ias
________________________________
From: Thomas Diesler [mailto:thomas.diesler@jboss.org]
Sent: Sunday, June 06, 2004 1:22 AM
To: axis-dev@ws.apache.org
Subject: Using SAAJ-1.2 API to modify the soap header/body content
Hi Axis Dev,
I am working on a J2EE-1.4 compliant implementation for web services
(WS4EE) in JBoss based on Axis. As you might know, JBoss has been using Axis
in its jboss.net module for quite some time. We are starting a new project
called jboss-ws4ee that we would like to use in our J2EE-1.4 certified
release.
I'm wondering if you could help me with these issues
#1 SAX recorded events
Axis uses a Serializer/Deserialiser based on SAX events. The
recorded events are stored with the SOAPElement.
On serialization the SOAPElement will simply playback the events
if the element is not marked dirty.
A JAXRPC handler is allowed to view and modify soap headers and
body. For that it can use the saaj api or even the dom api
on javax.xml.soap.Node objects.
When a client makes these modifications, they are not reflected
in the axis internal representation of the soap message.
This is especially true for RPCElement, which keeps an internal
list of RPCParam objects, which are not linked to SOAPElement.
#2 General soap tree representation
Axis is missing the notion of javax.xml.soap.Node, instead it
models javax.xml.soap.Text as javax.xml.soap.SOAPElement.
I'm having a hard time trying to keep all the various SOAPElement
representations in sync. For jboss-ws4ee I started a
refactoring effort that would base SOAPElementImpl on NodeImpl
which implements the org.w3c.dom.Node interface.
All information about namespaces, attributes, parent/child
relationships should be taken from the dom tree.
Serialization/Deserialization would be based on the dom tree as
well.
Any suggestions?
cheers
-thomas
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
Senior Developer & Consultant
JBoss Group (Europe) SàRL
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Re: opinion Axis' future ... [Re: Using SAAJ-1.2 API to modify the soap header/body content
Posted by Sanjiva Weerawarana <sa...@opensource.lk>.
WSIF with an alternate general protocol?
----- Original Message -----
From: "Davanum Srinivas" <da...@gmail.com>
To: <ax...@ws.apache.org>
Sent: Monday, June 07, 2004 7:50 PM
Subject: Re: opinion Axis' future ... [Re: Using SAAJ-1.2 API to modify the
soap header/body content
> Alek,
>
> have u reviewed this? -
> http://java.sun.com/developer/technicalArticles/WebServices/fastWS/
>
> -- dims
>
> On Sun, 06 Jun 2004 22:24:05 -0500, Aleksander Slominski
> <as...@cs.indiana.edu> wrote:
> >
> > Ias wrote:
> >
> > >Hi Thomas,
> > >
> > >Thanks for your pointing the issues out. As a contributor of the SAAJ
1.2
> > >implementation in Axis (with Dr. Ahn), I have to admit that he and I
already
> > >made our own (refactored) Axis satisfy #1 and #2. He created DOM
recorder
> > >for #1 and I refactored the whole org.apache.axis.message package for
#2. As
> > >for #2, org.apache.axis.message.Node was created and
> > >org.apache.axis.message.MessageElement and org.apache.axis.message.Text
> > >extended it.
> > >
> > >However, those changes are quite big and wide, so we need to modify
Axis a
> > >lot. Now we are in the middle of finalizing Axis 1.2, I don't think
such
> > >large changes will happen to this upcoming version of Axis. Personally,
I
> > >often have felt that contributing #1 and #2 to the current Axis base
seems
> > >pretty difficult and consider re-structuring the SAAJ part with Axis NG
> > >(Next Generation).
> > >
> > >What I'm designing regarding Axis NG is a complete implementation of
JAX-RPC
> > >2.0 and Web Services Metadata for high performance with StAX (streaming
> > >parser). I'm planning to open my thoughts on that as soon as JAX-RPC
2.0 and
> > >JAXB 2.0 early drafts are unveiled around JavaOne 2004. Briefly, I'd
like
> > >Axis NG to use StAX when deserializing/serializing between SOAP
messages and
> > >SAAJ instances.
> > >
> > how will this help avoid the trap of SAX event recording i.e. how it
> > makes any difference if you record SAX or StAX events?!
> >
> > i have pondered this problem and decided that trying to make two things
> > (streaming and DOM-like Infoset) fit into one hole is *the problem*.
> >
> > so my current thinking is that the optimal solution is a super
> > lightweight XML infoset in-memory representation that can be partially
> > materialized or de-materialized (stored in file, stream on-demand
> > parsing backed).
> >
> > to make this happen it is handy to allow to store non XML content in XML
> > infoset tree that is only XML-ized when message is about to be sent
> > (such as pointer to file so really huge files can be sent as BASE64, as
> > attachments, or MOTM or whatever it is called). when receiving message
> > goes through multiple processors in pipeline and is gradually converted
> > from XML to whatever Java representation one needs (such as pointer to
> > file backed representation).
> >
> > this approach tries to avoid converting between multiple XML
> > representation, caching them and trying to keep them consistent (huge
> > memory inflation to keep all message representations or CPU overhead to
> > recompute them on demand which is even worse as XML is CPU bound) and
> > that i think is the root of all current problem ...
> >
> > best,
> >
> > alek
> >
> > --
> > The best way to predict the future is to invent it - Alan Kay
> >
> >
>
Re: Fast Web Services / Fast Infoset / Fast Schema [Re: opinion Axis' future ... [Re: Using SAAJ-1.2 API to modify the soap header/body content
Posted by Sanjiva Weerawarana <sa...@opensource.lk>.
What surprised me was that FWS was only producing like 10X max over
a pure JAXRPC impl. IMO that's not a good enough benefit .. AxisMora
comes close to that against Axis/Java with a typed pull parser (yours
+ extensions ;-)) and a different internal architecture.
> interesting problem would be to build XML infoset API that would work
> both for XML 1.0/1.1 serialization, supports binary data (MTOM), allows
> partial in-memory materialization *and* can be (de)serialized to FWS ....
+1 for this. The JROM project we (my group in IBM) did a while back
(and release via alphaWorks) was an attempt at a pure Infoset model
with PSVI info too. IBM has a new thing called SDO (which is an open
source project at eclipse.org I believe) which has now replaced that
but its a bit more complicated unfortunately as the scope is much
broader. I haven't looked at the beehive stuff yet but it seems that
BEA must have something along these lines there too (something beyond
XMLBeans I mean).
Sanjiva.
----- Original Message -----
From: "Aleksander Slominski" <as...@cs.indiana.edu>
To: <ax...@ws.apache.org>
Sent: Monday, June 07, 2004 9:45 PM
Subject: Fast Web Services / Fast Infoset / Fast Schema [Re: opinion Axis'
future ... [Re: Using SAAJ-1.2 API to modify the soap header/body content
> Davanum Srinivas wrote:
>
> >have u reviewed this? -
> >http://java.sun.com/developer/technicalArticles/WebServices/fastWS/
> >
> >
> i think it is interesting approach to doing _binary_ XML but requires
> both sides to talk Fast Web Services but generally will not work with
> "slow" Web Services unless client knows WSDL endpoint (that would be the
> case for WSIF) or there is some dynamic metadata discovery protocol to
> identify that endpoint supports FWS.
>
> there is clearly a tradeoff between performance of binary encoding and
> flexibility/readability of text protocols and that may be become even
> more of issue when doc/literal message exchange dominates use of XML - i
> think that there is really no point of converting complex XML schema
> that has lot of extensibility points (and xsd:any) to ASN.1 (even if it
> was possible) ...
>
> still the most important factor to consider is network latency - when
> you deploy service on Internet the bottleneck is network and not how
> fast you can process messages (you can always do load balancing and
> throw more and always cheaper hardware) so using FWS will just mean less
> hardware. moreover self-describing XML is clearly an advantage when you
> try to create complex Internet scale distributed application (so that
> may pay all extra hardware as debugging/maintenance time is lower).
>
> > however FWS (and any other binary protocol) has a definite advantage as
> an optimized alternative to XML when used in *controlled* environment
> such as inside enterprise, mobile computing for very constrained devices
> (but here one should look on WS vs XHTML lesson) and even on Internet
> when there is a need to transfer large amounts of data but even in this
> case MTOM may be the winner as it keeps binary data integrated in XML
> infoset and will not require any special FWS-enabled wsdl2java
> stub/skeleton generator ....
> interesting problem would be to build XML infoset API that would work
> both for XML 1.0/1.1 serialization, supports binary data (MTOM), allows
> partial in-memory materialization *and* can be (de)serialized to FWS ....
>
> just dumped my thoughts - they may change when i actually see working
> FWS prototype (can it be downloaded from somewhere?
> https://jwsdp.dev.java.net/fast/ asks to "Stayed tuned ...") :)
>
> alek
>
> >On Sun, 06 Jun 2004 22:24:05 -0500, Aleksander Slominski
> ><as...@cs.indiana.edu> wrote:
> >
> >
> >>Ias wrote:
> >>
> >>
> >>
> >>>Hi Thomas,
> >>>
> >>>Thanks for your pointing the issues out. As a contributor of the SAAJ
1.2
> >>>implementation in Axis (with Dr. Ahn), I have to admit that he and I
already
> >>>made our own (refactored) Axis satisfy #1 and #2. He created DOM
recorder
> >>>for #1 and I refactored the whole org.apache.axis.message package for
#2. As
> >>>for #2, org.apache.axis.message.Node was created and
> >>>org.apache.axis.message.MessageElement and org.apache.axis.message.Text
> >>>extended it.
> >>>
> >>>However, those changes are quite big and wide, so we need to modify
Axis a
> >>>lot. Now we are in the middle of finalizing Axis 1.2, I don't think
such
> >>>large changes will happen to this upcoming version of Axis. Personally,
I
> >>>often have felt that contributing #1 and #2 to the current Axis base
seems
> >>>pretty difficult and consider re-structuring the SAAJ part with Axis NG
> >>>(Next Generation).
> >>>
> >>>What I'm designing regarding Axis NG is a complete implementation of
JAX-RPC
> >>>2.0 and Web Services Metadata for high performance with StAX (streaming
> >>>parser). I'm planning to open my thoughts on that as soon as JAX-RPC
2.0 and
> >>>JAXB 2.0 early drafts are unveiled around JavaOne 2004. Briefly, I'd
like
> >>>Axis NG to use StAX when deserializing/serializing between SOAP
messages and
> >>>SAAJ instances.
> >>>
> >>>
> >>>
> >>how will this help avoid the trap of SAX event recording i.e. how it
> >>makes any difference if you record SAX or StAX events?!
> >>
> >>i have pondered this problem and decided that trying to make two things
> >>(streaming and DOM-like Infoset) fit into one hole is *the problem*.
> >>
> >>so my current thinking is that the optimal solution is a super
> >>lightweight XML infoset in-memory representation that can be partially
> >>materialized or de-materialized (stored in file, stream on-demand
> >>parsing backed).
> >>
> >>to make this happen it is handy to allow to store non XML content in XML
> >>infoset tree that is only XML-ized when message is about to be sent
> >>(such as pointer to file so really huge files can be sent as BASE64, as
> >>attachments, or MOTM or whatever it is called). when receiving message
> >>goes through multiple processors in pipeline and is gradually converted
> >>from XML to whatever Java representation one needs (such as pointer to
> >>file backed representation).
> >>
> >>this approach tries to avoid converting between multiple XML
> >>representation, caching them and trying to keep them consistent (huge
> >>memory inflation to keep all message representations or CPU overhead to
> >>recompute them on demand which is even worse as XML is CPU bound) and
> >>that i think is the root of all current problem ...
> >>
> >>best,
> >>
> >>alek
> >>
> >>--
> >>The best way to predict the future is to invent it - Alan Kay
> >>
> >>
> >>
> >>
>
>
> --
> The best way to predict the future is to invent it - Alan Kay
>
>
Fast Web Services / Fast Infoset / Fast Schema [Re: opinion Axis'
future ... [Re: Using SAAJ-1.2 API to modify the soap header/body content
Posted by Aleksander Slominski <as...@cs.indiana.edu>.
Davanum Srinivas wrote:
>have u reviewed this? -
>http://java.sun.com/developer/technicalArticles/WebServices/fastWS/
>
>
i think it is interesting approach to doing _binary_ XML but requires
both sides to talk Fast Web Services but generally will not work with
"slow" Web Services unless client knows WSDL endpoint (that would be the
case for WSIF) or there is some dynamic metadata discovery protocol to
identify that endpoint supports FWS.
there is clearly a tradeoff between performance of binary encoding and
flexibility/readability of text protocols and that may be become even
more of issue when doc/literal message exchange dominates use of XML - i
think that there is really no point of converting complex XML schema
that has lot of extensibility points (and xsd:any) to ASN.1 (even if it
was possible) ...
still the most important factor to consider is network latency - when
you deploy service on Internet the bottleneck is network and not how
fast you can process messages (you can always do load balancing and
throw more and always cheaper hardware) so using FWS will just mean less
hardware. moreover self-describing XML is clearly an advantage when you
try to create complex Internet scale distributed application (so that
may pay all extra hardware as debugging/maintenance time is lower).
however FWS (and any other binary protocol) has a definite advantage as
an optimized alternative to XML when used in *controlled* environment
such as inside enterprise, mobile computing for very constrained devices
(but here one should look on WS vs XHTML lesson) and even on Internet
when there is a need to transfer large amounts of data but even in this
case MTOM may be the winner as it keeps binary data integrated in XML
infoset and will not require any special FWS-enabled wsdl2java
stub/skeleton generator ....
interesting problem would be to build XML infoset API that would work
both for XML 1.0/1.1 serialization, supports binary data (MTOM), allows
partial in-memory materialization *and* can be (de)serialized to FWS ....
just dumped my thoughts - they may change when i actually see working
FWS prototype (can it be downloaded from somewhere?
https://jwsdp.dev.java.net/fast/ asks to "Stayed tuned ...") :)
alek
>On Sun, 06 Jun 2004 22:24:05 -0500, Aleksander Slominski
><as...@cs.indiana.edu> wrote:
>
>
>>Ias wrote:
>>
>>
>>
>>>Hi Thomas,
>>>
>>>Thanks for your pointing the issues out. As a contributor of the SAAJ 1.2
>>>implementation in Axis (with Dr. Ahn), I have to admit that he and I already
>>>made our own (refactored) Axis satisfy #1 and #2. He created DOM recorder
>>>for #1 and I refactored the whole org.apache.axis.message package for #2. As
>>>for #2, org.apache.axis.message.Node was created and
>>>org.apache.axis.message.MessageElement and org.apache.axis.message.Text
>>>extended it.
>>>
>>>However, those changes are quite big and wide, so we need to modify Axis a
>>>lot. Now we are in the middle of finalizing Axis 1.2, I don't think such
>>>large changes will happen to this upcoming version of Axis. Personally, I
>>>often have felt that contributing #1 and #2 to the current Axis base seems
>>>pretty difficult and consider re-structuring the SAAJ part with Axis NG
>>>(Next Generation).
>>>
>>>What I'm designing regarding Axis NG is a complete implementation of JAX-RPC
>>>2.0 and Web Services Metadata for high performance with StAX (streaming
>>>parser). I'm planning to open my thoughts on that as soon as JAX-RPC 2.0 and
>>>JAXB 2.0 early drafts are unveiled around JavaOne 2004. Briefly, I'd like
>>>Axis NG to use StAX when deserializing/serializing between SOAP messages and
>>>SAAJ instances.
>>>
>>>
>>>
>>how will this help avoid the trap of SAX event recording i.e. how it
>>makes any difference if you record SAX or StAX events?!
>>
>>i have pondered this problem and decided that trying to make two things
>>(streaming and DOM-like Infoset) fit into one hole is *the problem*.
>>
>>so my current thinking is that the optimal solution is a super
>>lightweight XML infoset in-memory representation that can be partially
>>materialized or de-materialized (stored in file, stream on-demand
>>parsing backed).
>>
>>to make this happen it is handy to allow to store non XML content in XML
>>infoset tree that is only XML-ized when message is about to be sent
>>(such as pointer to file so really huge files can be sent as BASE64, as
>>attachments, or MOTM or whatever it is called). when receiving message
>>goes through multiple processors in pipeline and is gradually converted
>>from XML to whatever Java representation one needs (such as pointer to
>>file backed representation).
>>
>>this approach tries to avoid converting between multiple XML
>>representation, caching them and trying to keep them consistent (huge
>>memory inflation to keep all message representations or CPU overhead to
>>recompute them on demand which is even worse as XML is CPU bound) and
>>that i think is the root of all current problem ...
>>
>>best,
>>
>>alek
>>
>>--
>>The best way to predict the future is to invent it - Alan Kay
>>
>>
>>
>>
--
The best way to predict the future is to invent it - Alan Kay
Re: opinion Axis' future ... [Re: Using SAAJ-1.2 API to modify the soap header/body content
Posted by Davanum Srinivas <da...@gmail.com>.
Alek,
have u reviewed this? -
http://java.sun.com/developer/technicalArticles/WebServices/fastWS/
-- dims
On Sun, 06 Jun 2004 22:24:05 -0500, Aleksander Slominski
<as...@cs.indiana.edu> wrote:
>
> Ias wrote:
>
> >Hi Thomas,
> >
> >Thanks for your pointing the issues out. As a contributor of the SAAJ 1.2
> >implementation in Axis (with Dr. Ahn), I have to admit that he and I already
> >made our own (refactored) Axis satisfy #1 and #2. He created DOM recorder
> >for #1 and I refactored the whole org.apache.axis.message package for #2. As
> >for #2, org.apache.axis.message.Node was created and
> >org.apache.axis.message.MessageElement and org.apache.axis.message.Text
> >extended it.
> >
> >However, those changes are quite big and wide, so we need to modify Axis a
> >lot. Now we are in the middle of finalizing Axis 1.2, I don't think such
> >large changes will happen to this upcoming version of Axis. Personally, I
> >often have felt that contributing #1 and #2 to the current Axis base seems
> >pretty difficult and consider re-structuring the SAAJ part with Axis NG
> >(Next Generation).
> >
> >What I'm designing regarding Axis NG is a complete implementation of JAX-RPC
> >2.0 and Web Services Metadata for high performance with StAX (streaming
> >parser). I'm planning to open my thoughts on that as soon as JAX-RPC 2.0 and
> >JAXB 2.0 early drafts are unveiled around JavaOne 2004. Briefly, I'd like
> >Axis NG to use StAX when deserializing/serializing between SOAP messages and
> >SAAJ instances.
> >
> how will this help avoid the trap of SAX event recording i.e. how it
> makes any difference if you record SAX or StAX events?!
>
> i have pondered this problem and decided that trying to make two things
> (streaming and DOM-like Infoset) fit into one hole is *the problem*.
>
> so my current thinking is that the optimal solution is a super
> lightweight XML infoset in-memory representation that can be partially
> materialized or de-materialized (stored in file, stream on-demand
> parsing backed).
>
> to make this happen it is handy to allow to store non XML content in XML
> infoset tree that is only XML-ized when message is about to be sent
> (such as pointer to file so really huge files can be sent as BASE64, as
> attachments, or MOTM or whatever it is called). when receiving message
> goes through multiple processors in pipeline and is gradually converted
> from XML to whatever Java representation one needs (such as pointer to
> file backed representation).
>
> this approach tries to avoid converting between multiple XML
> representation, caching them and trying to keep them consistent (huge
> memory inflation to keep all message representations or CPU overhead to
> recompute them on demand which is even worse as XML is CPU bound) and
> that i think is the root of all current problem ...
>
> best,
>
> alek
>
> --
> The best way to predict the future is to invent it - Alan Kay
>
>
Re: opinion Axis' future ... [Re: Using SAAJ-1.2 API to modify the soap header/body content
Posted by Sanjiva Weerawarana <sa...@opensource.lk>.
> Ias wrote:
>
> >What I'm designing regarding Axis NG is a complete implementation of
JAX-RPC
> >2.0 and Web Services Metadata for high performance with StAX (streaming
> >parser). I'm planning to open my thoughts on that as soon as JAX-RPC 2.0
and
> >JAXB 2.0 early drafts are unveiled around JavaOne 2004. Briefly, I'd like
> >Axis NG to use StAX when deserializing/serializing between SOAP messages
and
> >SAAJ instances.
Have you looked at AxisMora? That's basically what AxisMora is ..
and it gets a major perf improvement by the pull model. That needs to
be completed more but the guts are solid IMO.
See ws-axis/contrib/axismora.
Sanjiva.
Re: opinion Axis' future ... [Re: Using SAAJ-1.2 API to modify the
soap header/body content
Posted by Richard Martin <rm...@essex.ac.uk>.
Aleksander Slominski wrote:
> Steve Loughran wrote:
>
>> Aleksander Slominski wrote:
>>
>>> so my current thinking is that the optimal solution is a super
>>> lightweight XML infoset in-memory representation that can be
>>> partially materialized or de-materialized (stored in file, stream
>>> on-demand parsing backed).
>>>
>>> to make this happen it is handy to allow to store non XML content in
>>> XML infoset tree that is only XML-ized when message is about to be
>>> sent (such as pointer to file so really huge files can be sent as
>>> BASE64, as attachments, or MOTM or whatever it is called). when
>>> receiving message goes through multiple processors in pipeline and
>>> is gradually converted from XML to whatever Java representation one
>>> needs (such as pointer to file backed representation).
>>
>>
>>
>> +1 to being able to offload v large portions of the infoset to HDD;
>> the reason you cannot stick big binary stuff in as base64 stuff today
>> is not really network efficiency (we base-64 all email), but
>> implementation -XML parsers werent written for 15MB strings, or 30MB
>> infosets. That needs to change.
>>
> true (i know i wrote one and struggled wit this problem) but i think
> that MTOM/XOP will be more "optimal" solution to it as potentially
> huge binary parts will be optimized as attachments and can be easily
> identified and offloaded to disk without need to guess which XML
> element text/base64 content needs must be handled as a large binary
> blob ...
>
> alek
This sounds very interesting; I'm currently working on a streaming DOM
API and would love to help out (if you think there's anything I can
do). MTOM seems to be the way forward - if you could combine this with
a lightweight infoset that's transparently built by SAX _during_ web
service invocation then this would reduce the memory overhead and
latency for large documents.
Regards,
Richard Martin
Re: SOAP streaming and faults, security everywhere and performance
implications
Posted by Steve Loughran <st...@iseran.com>.
Aleksander Slominski wrote:
>
> maybe we need SSOAP?
Look at what UPnP1.0 did for their work; they needed the ability to
stream a 400 page doc with inline images to a printer.
>>
> IMHO as business and scientific worlds converges on Web services and
> Grids those distinctions are blurred.
oh, are we converging :)
>
> for example one of the most important requirements for grid services (if
> not the most important ...) is to have security.
>
> in current WS world security has serious impact on performance (like one
> order of magnitude slower for WSS and even more when you start doing SSO
> with capabilities ....).
>
> so the time of simple non secure and non reliable SOAP messages may be
> going away but i hope it will not require more and more complex WS-*
> specs that are harder and harder to understand and work with (WSS and
> WS-Policy comes to mind ....).
>
> alek
>
I am sitting in a WS-RF meeting as I write this. Complexity is rushing
towards us. Soon the "simplicity" of SOAP will be a distant memory :(
-steve
Re: Re: message/transport level security [Re: SOAP streaming and
faults, security everywhere and performanceimplications [Re: opinion Axis'
future ... [Re: Using SAAJ-1.2 API tomodify the soap header/body content
Posted by Samuel Meder <me...@mcs.anl.gov>.
On Wed, 2004-06-09 at 15:13, Davanum Srinivas wrote:
> I just browsed their code....No, they don't have WS-SecConv support as
> far as i can tell.
We ended up implementing our own equivalent (for timing and other
reasons). So no, we don't have WS-SecConv. From a protocol performance
perspective the issues should be exactly the same though.
/Sam
>
> -- dims
>
> On Wed, 09 Jun 2004 15:10:37 -0500, Aleksander Slominski
> <as...@cs.indiana.edu> wrote:
> >
> > Davanum Srinivas wrote:
> >
> > >Do you have a WS-SecConv impl? (open source?)
> > >
> > >
> > we looked on Globus Toolkit 3.2 which i think is an early/prototype
> > implementation and i am not sure about its long term status.
> >
> > i am sure that Globus guys will have more definitve answer to that :)
> >
> > thanks,
> >
> > alek
> >
> >
> >
> > >On Wed, 09 Jun 2004 11:15:58 -0500, Aleksander Slominski
> > ><as...@cs.indiana.edu> wrote:
> > >
> > >
> > >>Sanjiva Weerawarana wrote:
> > >>
> > >>
> > >>
> > >>>"Samuel Meder" <me...@mcs.anl.gov> writes:
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>>I completely agree on Alex's point that security (and other QoS aspects)
> > >>>>are very important for us working in the Grid world. It is hard to
> > >>>>justify to users that turning on security will cause a 10x slowdown (it
> > >>>>is actually even worse with larger payloads) and since grid scenarios
> > >>>>are generally multi organization security is a ubiquitous requirement (I
> > >>>>would think that the same is true for B2B scenarios).
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>Of course. However, that's a fundamental issue with XML security
> > >>>and not with the WS-* specs in particular. The XML sec stuff is
> > >>>defined in terms of DOMs and DOM hash's .. which can be computed
> > >>>in a streaming model for many XPaths ..
> > >>>
> > >>>
> > >>>
> > >>>
> > >>hi Sanjiva
> > >>
> > >>i think that it is true for verification but not for signing as you need
> > >>to compute signature: you *can* compute it in a streaming fashion but
> > >>then you need to put signature in header so you have to buffer whole
> > >>stream output (why, oh why, there is no footer element in SOAP ...)
> > >>
> > >>
> > >>
> > >>>Furthermore using WS-Security without WS-Secure Conversation is
> > >>>retarded (IMO).
> > >>>
> > >>>
> > >>>
> > >>>
> > >>there are many cases to consider and each situation is unique.
> > >>
> > >>WS-SecConv does not buy you much if you do very coarse grained messaging
> > >>(only one message sent to service) still the overhead is small enough
> > >>that may be doable to do WS-SecConv for every message.
> > >>
> > >>however there is a potential scalability problem as server needs to
> > >>remember all those opened sessions so implementation must be very
> > >>careful here (we did some testing and in one test setup after a thousand
> > >>or so connections server refused to accept any new connections ...)
> > >>
> > >>
> > >>
> > >>>In summary, IMO there's no reason why WS-Security/WS-Secure Conversation
> > >>>have to be any less performant than any other public key security
> > >>>technology IMO for many cases.
> > >>>
> > >>>
> > >>>
> > >>yes - if we do have better libs to do efficient
> > >>streaming/canonicalization/verification/signing and generally avoiding
> > >>excessive memory usage.
> > >>
> > >>
> > >>
> > >>>There may be degenerate cases where
> > >>>it is, but if we can make those rare then we're in business.
> > >>>
> > >>>
> > >>>
> > >>>
> > >>that is still to be seen :)
> > >>
> > >>currently SSL without keep alive (so it does key exchange each time) is
> > >>2-5x faster than WS-SecConv after first message (this is just estimate
> > >>...) and SSL with keep alive (no key exchange so it is comparable to
> > >>session in WS-SecConv) it is faster by *order of magnitude* than
> > >>WS-SecConv in what we tested (caveat emperor!)
> > >>
> > >>plus with SSL you can easily by hardware accelerator and load balancers
> > >>that will understand SSL it will take some time before WS-SecConv is
> > >>real standard and has this kind of support ...
> > >>
> > >>we are going to submit a paper (Satoshi did a great work on this) with
> > >>details on tests to Workshop on Grid Computing (Grid 2004) and hope that
> > >>it will lead to some discussion and ultimately faster (streaming?) code
> > >>in future :)
> > >>
> > >>thanks,
> > >>
> > >>alek
> > >>
> > >>--
> > >>The best way to predict the future is to invent it - Alan Kay
> > >>
> > >>
> > >>
> > >>
> > >
> > >
> > >
> > >
> >
> > --
> > The best way to predict the future is to invent it - Alan Kay
> >
> >
--
Sam Meder <me...@mcs.anl.gov>
The Globus Alliance - University of Chicago
630-252-1752
Re: Re: message/transport level security [Re: SOAP streaming and faults, security everywhere and performanceimplications [Re: opinion Axis' future ... [Re: Using SAAJ-1.2 API tomodify the soap header/body content
Posted by Davanum Srinivas <da...@gmail.com>.
I just browsed their code....No, they don't have WS-SecConv support as
far as i can tell.
-- dims
On Wed, 09 Jun 2004 15:10:37 -0500, Aleksander Slominski
<as...@cs.indiana.edu> wrote:
>
> Davanum Srinivas wrote:
>
> >Do you have a WS-SecConv impl? (open source?)
> >
> >
> we looked on Globus Toolkit 3.2 which i think is an early/prototype
> implementation and i am not sure about its long term status.
>
> i am sure that Globus guys will have more definitve answer to that :)
>
> thanks,
>
> alek
>
>
>
> >On Wed, 09 Jun 2004 11:15:58 -0500, Aleksander Slominski
> ><as...@cs.indiana.edu> wrote:
> >
> >
> >>Sanjiva Weerawarana wrote:
> >>
> >>
> >>
> >>>"Samuel Meder" <me...@mcs.anl.gov> writes:
> >>>
> >>>
> >>>
> >>>
> >>>>I completely agree on Alex's point that security (and other QoS aspects)
> >>>>are very important for us working in the Grid world. It is hard to
> >>>>justify to users that turning on security will cause a 10x slowdown (it
> >>>>is actually even worse with larger payloads) and since grid scenarios
> >>>>are generally multi organization security is a ubiquitous requirement (I
> >>>>would think that the same is true for B2B scenarios).
> >>>>
> >>>>
> >>>>
> >>>>
> >>>Of course. However, that's a fundamental issue with XML security
> >>>and not with the WS-* specs in particular. The XML sec stuff is
> >>>defined in terms of DOMs and DOM hash's .. which can be computed
> >>>in a streaming model for many XPaths ..
> >>>
> >>>
> >>>
> >>>
> >>hi Sanjiva
> >>
> >>i think that it is true for verification but not for signing as you need
> >>to compute signature: you *can* compute it in a streaming fashion but
> >>then you need to put signature in header so you have to buffer whole
> >>stream output (why, oh why, there is no footer element in SOAP ...)
> >>
> >>
> >>
> >>>Furthermore using WS-Security without WS-Secure Conversation is
> >>>retarded (IMO).
> >>>
> >>>
> >>>
> >>>
> >>there are many cases to consider and each situation is unique.
> >>
> >>WS-SecConv does not buy you much if you do very coarse grained messaging
> >>(only one message sent to service) still the overhead is small enough
> >>that may be doable to do WS-SecConv for every message.
> >>
> >>however there is a potential scalability problem as server needs to
> >>remember all those opened sessions so implementation must be very
> >>careful here (we did some testing and in one test setup after a thousand
> >>or so connections server refused to accept any new connections ...)
> >>
> >>
> >>
> >>>In summary, IMO there's no reason why WS-Security/WS-Secure Conversation
> >>>have to be any less performant than any other public key security
> >>>technology IMO for many cases.
> >>>
> >>>
> >>>
> >>yes - if we do have better libs to do efficient
> >>streaming/canonicalization/verification/signing and generally avoiding
> >>excessive memory usage.
> >>
> >>
> >>
> >>>There may be degenerate cases where
> >>>it is, but if we can make those rare then we're in business.
> >>>
> >>>
> >>>
> >>>
> >>that is still to be seen :)
> >>
> >>currently SSL without keep alive (so it does key exchange each time) is
> >>2-5x faster than WS-SecConv after first message (this is just estimate
> >>...) and SSL with keep alive (no key exchange so it is comparable to
> >>session in WS-SecConv) it is faster by *order of magnitude* than
> >>WS-SecConv in what we tested (caveat emperor!)
> >>
> >>plus with SSL you can easily by hardware accelerator and load balancers
> >>that will understand SSL it will take some time before WS-SecConv is
> >>real standard and has this kind of support ...
> >>
> >>we are going to submit a paper (Satoshi did a great work on this) with
> >>details on tests to Workshop on Grid Computing (Grid 2004) and hope that
> >>it will lead to some discussion and ultimately faster (streaming?) code
> >>in future :)
> >>
> >>thanks,
> >>
> >>alek
> >>
> >>--
> >>The best way to predict the future is to invent it - Alan Kay
> >>
> >>
> >>
> >>
> >
> >
> >
> >
>
> --
> The best way to predict the future is to invent it - Alan Kay
>
>
--
Davanum Srinivas - http://webservices.apache.org/~dims/
Re: message/transport level security [Re: SOAP streaming and faults,
security everywhere and performanceimplications [Re: opinion Axis' future
... [Re: Using SAAJ-1.2 API tomodify the soap header/body content
Posted by Aleksander Slominski <as...@cs.indiana.edu>.
Davanum Srinivas wrote:
>Do you have a WS-SecConv impl? (open source?)
>
>
we looked on Globus Toolkit 3.2 which i think is an early/prototype
implementation and i am not sure about its long term status.
i am sure that Globus guys will have more definitve answer to that :)
thanks,
alek
>On Wed, 09 Jun 2004 11:15:58 -0500, Aleksander Slominski
><as...@cs.indiana.edu> wrote:
>
>
>>Sanjiva Weerawarana wrote:
>>
>>
>>
>>>"Samuel Meder" <me...@mcs.anl.gov> writes:
>>>
>>>
>>>
>>>
>>>>I completely agree on Alex's point that security (and other QoS aspects)
>>>>are very important for us working in the Grid world. It is hard to
>>>>justify to users that turning on security will cause a 10x slowdown (it
>>>>is actually even worse with larger payloads) and since grid scenarios
>>>>are generally multi organization security is a ubiquitous requirement (I
>>>>would think that the same is true for B2B scenarios).
>>>>
>>>>
>>>>
>>>>
>>>Of course. However, that's a fundamental issue with XML security
>>>and not with the WS-* specs in particular. The XML sec stuff is
>>>defined in terms of DOMs and DOM hash's .. which can be computed
>>>in a streaming model for many XPaths ..
>>>
>>>
>>>
>>>
>>hi Sanjiva
>>
>>i think that it is true for verification but not for signing as you need
>>to compute signature: you *can* compute it in a streaming fashion but
>>then you need to put signature in header so you have to buffer whole
>>stream output (why, oh why, there is no footer element in SOAP ...)
>>
>>
>>
>>>Furthermore using WS-Security without WS-Secure Conversation is
>>>retarded (IMO).
>>>
>>>
>>>
>>>
>>there are many cases to consider and each situation is unique.
>>
>>WS-SecConv does not buy you much if you do very coarse grained messaging
>>(only one message sent to service) still the overhead is small enough
>>that may be doable to do WS-SecConv for every message.
>>
>>however there is a potential scalability problem as server needs to
>>remember all those opened sessions so implementation must be very
>>careful here (we did some testing and in one test setup after a thousand
>>or so connections server refused to accept any new connections ...)
>>
>>
>>
>>>In summary, IMO there's no reason why WS-Security/WS-Secure Conversation
>>>have to be any less performant than any other public key security
>>>technology IMO for many cases.
>>>
>>>
>>>
>>yes - if we do have better libs to do efficient
>>streaming/canonicalization/verification/signing and generally avoiding
>>excessive memory usage.
>>
>>
>>
>>>There may be degenerate cases where
>>>it is, but if we can make those rare then we're in business.
>>>
>>>
>>>
>>>
>>that is still to be seen :)
>>
>>currently SSL without keep alive (so it does key exchange each time) is
>>2-5x faster than WS-SecConv after first message (this is just estimate
>>...) and SSL with keep alive (no key exchange so it is comparable to
>>session in WS-SecConv) it is faster by *order of magnitude* than
>>WS-SecConv in what we tested (caveat emperor!)
>>
>>plus with SSL you can easily by hardware accelerator and load balancers
>>that will understand SSL it will take some time before WS-SecConv is
>>real standard and has this kind of support ...
>>
>>we are going to submit a paper (Satoshi did a great work on this) with
>>details on tests to Workshop on Grid Computing (Grid 2004) and hope that
>>it will lead to some discussion and ultimately faster (streaming?) code
>>in future :)
>>
>>thanks,
>>
>>alek
>>
>>--
>>The best way to predict the future is to invent it - Alan Kay
>>
>>
>>
>>
>
>
>
>
--
The best way to predict the future is to invent it - Alan Kay
Re: message/transport level security [Re: SOAP streaming and faults, security everywhere and performanceimplications [Re: opinion Axis' future ... [Re: Using SAAJ-1.2 API tomodify the soap header/body content
Posted by Davanum Srinivas <da...@gmail.com>.
Alek,
Do you have a WS-SecConv impl? (open source?)
-- dims
On Wed, 09 Jun 2004 11:15:58 -0500, Aleksander Slominski
<as...@cs.indiana.edu> wrote:
>
> Sanjiva Weerawarana wrote:
>
> >"Samuel Meder" <me...@mcs.anl.gov> writes:
> >
> >
> >>I completely agree on Alex's point that security (and other QoS aspects)
> >>are very important for us working in the Grid world. It is hard to
> >>justify to users that turning on security will cause a 10x slowdown (it
> >>is actually even worse with larger payloads) and since grid scenarios
> >>are generally multi organization security is a ubiquitous requirement (I
> >>would think that the same is true for B2B scenarios).
> >>
> >>
> >
> >Of course. However, that's a fundamental issue with XML security
> >and not with the WS-* specs in particular. The XML sec stuff is
> >defined in terms of DOMs and DOM hash's .. which can be computed
> >in a streaming model for many XPaths ..
> >
> >
> hi Sanjiva
>
> i think that it is true for verification but not for signing as you need
> to compute signature: you *can* compute it in a streaming fashion but
> then you need to put signature in header so you have to buffer whole
> stream output (why, oh why, there is no footer element in SOAP ...)
>
> >Furthermore using WS-Security without WS-Secure Conversation is
> >retarded (IMO).
> >
> >
> there are many cases to consider and each situation is unique.
>
> WS-SecConv does not buy you much if you do very coarse grained messaging
> (only one message sent to service) still the overhead is small enough
> that may be doable to do WS-SecConv for every message.
>
> however there is a potential scalability problem as server needs to
> remember all those opened sessions so implementation must be very
> careful here (we did some testing and in one test setup after a thousand
> or so connections server refused to accept any new connections ...)
>
> >In summary, IMO there's no reason why WS-Security/WS-Secure Conversation
> >have to be any less performant than any other public key security
> >technology IMO for many cases.
> >
> yes - if we do have better libs to do efficient
> streaming/canonicalization/verification/signing and generally avoiding
> excessive memory usage.
>
> >There may be degenerate cases where
> >it is, but if we can make those rare then we're in business.
> >
> >
> that is still to be seen :)
>
> currently SSL without keep alive (so it does key exchange each time) is
> 2-5x faster than WS-SecConv after first message (this is just estimate
> ...) and SSL with keep alive (no key exchange so it is comparable to
> session in WS-SecConv) it is faster by *order of magnitude* than
> WS-SecConv in what we tested (caveat emperor!)
>
> plus with SSL you can easily by hardware accelerator and load balancers
> that will understand SSL it will take some time before WS-SecConv is
> real standard and has this kind of support ...
>
> we are going to submit a paper (Satoshi did a great work on this) with
> details on tests to Workshop on Grid Computing (Grid 2004) and hope that
> it will lead to some discussion and ultimately faster (streaming?) code
> in future :)
>
> thanks,
>
> alek
>
> --
> The best way to predict the future is to invent it - Alan Kay
>
>
--
Davanum Srinivas - http://webservices.apache.org/~dims/
message/transport level security [Re: SOAP streaming and faults,
security everywhere and performanceimplications [Re: opinion Axis' future
... [Re: Using SAAJ-1.2 API tomodify the soap header/body content
Posted by Aleksander Slominski <as...@cs.indiana.edu>.
Sanjiva Weerawarana wrote:
>"Samuel Meder" <me...@mcs.anl.gov> writes:
>
>
>>I completely agree on Alex's point that security (and other QoS aspects)
>>are very important for us working in the Grid world. It is hard to
>>justify to users that turning on security will cause a 10x slowdown (it
>>is actually even worse with larger payloads) and since grid scenarios
>>are generally multi organization security is a ubiquitous requirement (I
>>would think that the same is true for B2B scenarios).
>>
>>
>
>Of course. However, that's a fundamental issue with XML security
>and not with the WS-* specs in particular. The XML sec stuff is
>defined in terms of DOMs and DOM hash's .. which can be computed
>in a streaming model for many XPaths ..
>
>
hi Sanjiva
i think that it is true for verification but not for signing as you need
to compute signature: you *can* compute it in a streaming fashion but
then you need to put signature in header so you have to buffer whole
stream output (why, oh why, there is no footer element in SOAP ...)
>Furthermore using WS-Security without WS-Secure Conversation is
>retarded (IMO).
>
>
there are many cases to consider and each situation is unique.
WS-SecConv does not buy you much if you do very coarse grained messaging
(only one message sent to service) still the overhead is small enough
that may be doable to do WS-SecConv for every message.
however there is a potential scalability problem as server needs to
remember all those opened sessions so implementation must be very
careful here (we did some testing and in one test setup after a thousand
or so connections server refused to accept any new connections ...)
>In summary, IMO there's no reason why WS-Security/WS-Secure Conversation
>have to be any less performant than any other public key security
>technology IMO for many cases.
>
yes - if we do have better libs to do efficient
streaming/canonicalization/verification/signing and generally avoiding
excessive memory usage.
>There may be degenerate cases where
>it is, but if we can make those rare then we're in business.
>
>
that is still to be seen :)
currently SSL without keep alive (so it does key exchange each time) is
2-5x faster than WS-SecConv after first message (this is just estimate
...) and SSL with keep alive (no key exchange so it is comparable to
session in WS-SecConv) it is faster by *order of magnitude* than
WS-SecConv in what we tested (caveat emperor!)
plus with SSL you can easily by hardware accelerator and load balancers
that will understand SSL it will take some time before WS-SecConv is
real standard and has this kind of support ...
we are going to submit a paper (Satoshi did a great work on this) with
details on tests to Workshop on Grid Computing (Grid 2004) and hope that
it will lead to some discussion and ultimately faster (streaming?) code
in future :)
thanks,
alek
--
The best way to predict the future is to invent it - Alan Kay
Re: SOAP streaming and faults, security everywhere and performanceimplications [Re: opinion Axis' future ... [Re: Using SAAJ-1.2 API tomodify the soap header/body content
Posted by Sanjiva Weerawarana <sa...@opensource.lk>.
"Samuel Meder" <me...@mcs.anl.gov> writes:
> I completely agree on Alex's point that security (and other QoS aspects)
> are very important for us working in the Grid world. It is hard to
> justify to users that turning on security will cause a 10x slowdown (it
> is actually even worse with larger payloads) and since grid scenarios
> are generally multi organization security is a ubiquitous requirement (I
> would think that the same is true for B2B scenarios).
Of course. However, that's a fundamental issue with XML security
and not with the WS-* specs in particular. The XML sec stuff is
defined in terms of DOMs and DOM hash's .. which can be computed
in a streaming model for many XPaths ..
Furthermore using WS-Security without WS-Secure Conversation is
retarded (IMO).
In summary, IMO there's no reason why WS-Security/WS-Secure Conversation
have to be any less performant than any other public key security
technology IMO for many cases. There may be degenerate cases where
it is, but if we can make those rare then we're in business.
Sanjiva.
Re: SOAP streaming and faults, security everywhere and performance
implications [Re: opinion Axis' future ... [Re: Using SAAJ-1.2 API to
modify the soap header/body content
Posted by Samuel Meder <me...@mcs.anl.gov>.
I completely agree on Alex's point that security (and other QoS aspects)
are very important for us working in the Grid world. It is hard to
justify to users that turning on security will cause a 10x slowdown (it
is actually even worse with larger payloads) and since grid scenarios
are generally multi organization security is a ubiquitous requirement (I
would think that the same is true for B2B scenarios).
/Sam
On Tue, 2004-06-08 at 13:29, Aleksander Slominski wrote:
> Sanjiva Weerawarana wrote:
>
> >Hi Alek,
> >
> >
> Hi Sanjiva,
>
> much more comments inline.
>
> >>but it seems that reality of current SOAP is not allowing any of
> >>those desirable features to be easily expressed ...
> >>
> >>
> >
> >I agree with your comment about streaming .. but I'm willing to
> >"break" SOAP semantics for it.
> >
> >
> :)
>
> >>now that we are stuffing more and into headers (such as binary
> >>security tokens) there seems to be quite common case that size
> >>of headers will well outweigh size of payload.
> >>
> >>
> >
> >I agree 100%. However, in the presence of such headers the
> >performance requirements change. For example, if RM is being
> >used with a QoS of exactly once, ordered delivery, then the raw
> >performance is not so critical in a business process that's
> >probably driving it. Most likely the process is a long running
> >business transaction (like a mortgage application) and then
> >there's not much point in focusing on whether a message takes
> >10ms or 20ms when the whole process runs over a 3 month period.
> >
> >
> >
> i think security will soon be required for any message exchange and the
> same may be about WS-RM when we have more multi-hop services and
> messages that must pass multiple gateways (next gen firewalls).
>
> >>also i think that MTOM (and attachment) can be referenced
> >>both from headers and payloads so binary large data can be
> >>
> >>
> >
> >WSDL 2.0 does not support that; MTOM is for the payload only ..
> >headers are not part of the fundamental description of a Web service;
> >they come into existence due to the application of certain QoS
> >(like RM or security).
> >
> >
> i will need to check this i thought MTOM can be applied to any base64
> content ...
>
> >>easily added anywhere in SOAP envelope and we should make
> >>sure that all this can be efficiently processed ...
> >>
> >>
> >
> >Sure, but you can't have the cake and eat it. My believe today is
> >that header processing is expensive and is going to be so. SOAP
> >requires that all mustUnderstand headers be processed correctly ..
> >that means you can't "stream" headers thru - the framework has to
> >process everything. Furthermore, as more and more QoS characteristics
> >(aka policies) are applied, header processing becomes even more
> >complex. In particular, it is quite non-trivial to determine which
> >order the various policy handlers can run .. in the case of transactions
> >and reliable messaging and security you can convince yourself that the
> >right order is security, then reliable messaging and then Tx. In the
> >general case there is no algorithm to compute the order AFAIK.
> >
> >
> yes we need policies about how to apply policies (and handlers) ...
>
> >>still clearly the one case to keep in mind is how to handle very
> >>large payloads as it may be very important case for some class of
> >>applications such as scientific computing.
> >>
> >>
> >Sure.
> >
> >
> >>i started with just considering generic XML messaging (and that
> >>is how XSOAP4/XSUL is designed) so i did not feel like there is
> >>a need to have a special separate treatment of header and body
> >>(payload) which seems to be now validated by current "core"
> >>WS-* specs (see below).
> >>
> >>
> >
> >I'm confused by the last comment; which WS-* specs blur the ditinction
> >between header and body?? (I couldn't see a clarification further
> >below; sorry.)
> >
> >
> for example: WS-RM has some special (infrastructure level) messages that
> has empty body and action is really part of header (but no payload ...)
>
> >>if you describe everything as XML infoset that it is very
> >>important to allow simple access to whole message XML infoset
> >>(and make possible to use such tools like XPath or XQuery) -
> >>runtime XML infoset is just optimized to take hints (such as
> >>provided by MTOM) to make decisions about in-memory
> >>materialization of XML content.
> >>
> >>
> >
> >Yep; I'm definitely +1 for an MTOM-aware Infoset model being the
> >default, dynamically typed representation of arbitray XML content.
> >
> >
> then why to make the distinction between XML elements <S:Header> and
> <S:Body> - isnt body like a special header targeted to ultimate receiver?
>
> >>so my thinking is that there needs to be set of flexible layers
> >>and developer can choose on which layer to plugin their code:
> >>transport, XML message infoset, SOAP headers/payload (header
> >>processors). XML Schema data binding, RelaxNG validation, whatnot ...
> >>
> >>
> >
> >IMO we need to classify different types of developers and provide
> >different programming models for them. I want to keep app developers
> >a level above all this .. basically at the level WSDL 2.0 is putting
> >them at; anything else are policies they express or are expressed
> >to them by someone else.
> >
> >
> i think it would be very valuable to have enumeration of roles (like
> EJB) but applied to building WS and SO(A).
>
> >The developers who write such policy handlers of course have a
> >different view of the world and get access to a different/additional
> >set of information .. for example, the security handler is someone
> >who will likely re-write the entire payload (if it were encrypted).
> >
> >
> yes - we do it all the time including dreaded conversion to DOM so
> existing XML security libs can work on the message.
>
> >>unfortunately XML is not designed for streaming and moreover
> >>streaming of payload is not supported in SOAP 1.1/1.2.
> >>
> >>
> >Yes, unfortunately.
> >
> >
> >
> unfortunately :(
>
> maybe we need SSOAP?
>
> >>i think i can make this bold (?!) statement based on my past
> >>experience: only by breaking SOAP semantics it is i possible
> >> to do limited streaming of vanilla SOAP envelopes with just
> >>
> >>
> >
> >I'm ok with "breaking" SOAP semantics a bit here ..
> >
> i wish it could be expressed without any breakage - the heart of it is
> that if you want to stream output you can not know if there some
> processor that produces output (such as serializer or actual service
> when it computes stream content) that may fail *during* output - in this
> case you can either buffer whole thing (preferably to DB or file for
> really large outputs so at least memory footprint is not too bad ...) or
> start sending it immediately but then you can not signal fault in
> mid-stream.
>
> the only robust but sub-optimal way to deal with it (i found and
> implemented in XSOAP1) is to close abruptly output stream.
>
> it would be much better if i could write <S:Fault> as last children of
> <S:Body> and have receiving processor to discard input and act on Fault.
>
> >in fact the
> >breakage would only be visible to an external observer in the form
> >of the error they will get.
> >
> breakage will be only if error happens but if you do what i did in
> XSOAP1 the receiver will not know what caused problem as no Fault is
> sent only that connection was lost ...
>
> >Two well-behaved SOAP processors will
> >not have any problem .. where well-behaved means that they basically
> >general well-formed XML; a very low bar to pass.
> >
> >
> >
> it is more complicated than that as i described above: Fault is special
> and is supposed to be first child of Body ...
>
> do you have some other solution in mind?
>
> >>payload (why no way to signal in-stream faults?) and it gets
> >>very difficult with WSS (why there is no footers so one could
> >>compute signature on the fly and then write it in footer!) and
> >>does not make sense for WS-ReliableMessaging (message is stored
> >> before sending so it can be easily re-sent ...) except of course
> >>for re-streaming out of (database) persistent storage later but
> >>that is just replay and will require coding to make sure it
> >>scales well ....
> >>
> >>
> >
> >Again, I believe the realistic business scenarios for WS-Security
> >and WS-Reliable Messaging have different performance characteristics
> >than a raw SOAP message exchange. Thus, I'm not convinced there's
> >much value in doing raw comparison of how long it takes an RM
> >exchange to occur .. that's not what RM is intended to be used for
> >IMO.
> >
> >
> IMHO as business and scientific worlds converges on Web services and
> Grids those distinctions are blurred.
>
> for example one of the most important requirements for grid services (if
> not the most important ...) is to have security.
>
> in current WS world security has serious impact on performance (like one
> order of magnitude slower for WSS and even more when you start doing SSO
> with capabilities ....).
>
> so the time of simple non secure and non reliable SOAP messages may be
> going away but i hope it will not require more and more complex WS-*
> specs that are harder and harder to understand and work with (WSS and
> WS-Policy comes to mind ....).
>
> alek
--
Sam Meder <me...@mcs.anl.gov>
The Globus Alliance - University of Chicago
630-252-1752
SOAP streaming and faults, security everywhere and performance implications
[Re: opinion Axis' future ... [Re: Using SAAJ-1.2 API to modify the soap
header/body content
Posted by Aleksander Slominski <as...@cs.indiana.edu>.
Sanjiva Weerawarana wrote:
>Hi Alek,
>
>
Hi Sanjiva,
much more comments inline.
>>but it seems that reality of current SOAP is not allowing any of
>>those desirable features to be easily expressed ...
>>
>>
>
>I agree with your comment about streaming .. but I'm willing to
>"break" SOAP semantics for it.
>
>
:)
>>now that we are stuffing more and into headers (such as binary
>>security tokens) there seems to be quite common case that size
>>of headers will well outweigh size of payload.
>>
>>
>
>I agree 100%. However, in the presence of such headers the
>performance requirements change. For example, if RM is being
>used with a QoS of exactly once, ordered delivery, then the raw
>performance is not so critical in a business process that's
>probably driving it. Most likely the process is a long running
>business transaction (like a mortgage application) and then
>there's not much point in focusing on whether a message takes
>10ms or 20ms when the whole process runs over a 3 month period.
>
>
>
i think security will soon be required for any message exchange and the
same may be about WS-RM when we have more multi-hop services and
messages that must pass multiple gateways (next gen firewalls).
>>also i think that MTOM (and attachment) can be referenced
>>both from headers and payloads so binary large data can be
>>
>>
>
>WSDL 2.0 does not support that; MTOM is for the payload only ..
>headers are not part of the fundamental description of a Web service;
>they come into existence due to the application of certain QoS
>(like RM or security).
>
>
i will need to check this i thought MTOM can be applied to any base64
content ...
>>easily added anywhere in SOAP envelope and we should make
>>sure that all this can be efficiently processed ...
>>
>>
>
>Sure, but you can't have the cake and eat it. My believe today is
>that header processing is expensive and is going to be so. SOAP
>requires that all mustUnderstand headers be processed correctly ..
>that means you can't "stream" headers thru - the framework has to
>process everything. Furthermore, as more and more QoS characteristics
>(aka policies) are applied, header processing becomes even more
>complex. In particular, it is quite non-trivial to determine which
>order the various policy handlers can run .. in the case of transactions
>and reliable messaging and security you can convince yourself that the
>right order is security, then reliable messaging and then Tx. In the
>general case there is no algorithm to compute the order AFAIK.
>
>
yes we need policies about how to apply policies (and handlers) ...
>>still clearly the one case to keep in mind is how to handle very
>>large payloads as it may be very important case for some class of
>>applications such as scientific computing.
>>
>>
>Sure.
>
>
>>i started with just considering generic XML messaging (and that
>>is how XSOAP4/XSUL is designed) so i did not feel like there is
>>a need to have a special separate treatment of header and body
>>(payload) which seems to be now validated by current "core"
>>WS-* specs (see below).
>>
>>
>
>I'm confused by the last comment; which WS-* specs blur the ditinction
>between header and body?? (I couldn't see a clarification further
>below; sorry.)
>
>
for example: WS-RM has some special (infrastructure level) messages that
has empty body and action is really part of header (but no payload ...)
>>if you describe everything as XML infoset that it is very
>>important to allow simple access to whole message XML infoset
>>(and make possible to use such tools like XPath or XQuery) -
>>runtime XML infoset is just optimized to take hints (such as
>>provided by MTOM) to make decisions about in-memory
>>materialization of XML content.
>>
>>
>
>Yep; I'm definitely +1 for an MTOM-aware Infoset model being the
>default, dynamically typed representation of arbitray XML content.
>
>
then why to make the distinction between XML elements <S:Header> and
<S:Body> - isnt body like a special header targeted to ultimate receiver?
>>so my thinking is that there needs to be set of flexible layers
>>and developer can choose on which layer to plugin their code:
>>transport, XML message infoset, SOAP headers/payload (header
>>processors). XML Schema data binding, RelaxNG validation, whatnot ...
>>
>>
>
>IMO we need to classify different types of developers and provide
>different programming models for them. I want to keep app developers
>a level above all this .. basically at the level WSDL 2.0 is putting
>them at; anything else are policies they express or are expressed
>to them by someone else.
>
>
i think it would be very valuable to have enumeration of roles (like
EJB) but applied to building WS and SO(A).
>The developers who write such policy handlers of course have a
>different view of the world and get access to a different/additional
>set of information .. for example, the security handler is someone
>who will likely re-write the entire payload (if it were encrypted).
>
>
yes - we do it all the time including dreaded conversion to DOM so
existing XML security libs can work on the message.
>>unfortunately XML is not designed for streaming and moreover
>>streaming of payload is not supported in SOAP 1.1/1.2.
>>
>>
>Yes, unfortunately.
>
>
>
unfortunately :(
maybe we need SSOAP?
>>i think i can make this bold (?!) statement based on my past
>>experience: only by breaking SOAP semantics it is i possible
>> to do limited streaming of vanilla SOAP envelopes with just
>>
>>
>
>I'm ok with "breaking" SOAP semantics a bit here ..
>
i wish it could be expressed without any breakage - the heart of it is
that if you want to stream output you can not know if there some
processor that produces output (such as serializer or actual service
when it computes stream content) that may fail *during* output - in this
case you can either buffer whole thing (preferably to DB or file for
really large outputs so at least memory footprint is not too bad ...) or
start sending it immediately but then you can not signal fault in
mid-stream.
the only robust but sub-optimal way to deal with it (i found and
implemented in XSOAP1) is to close abruptly output stream.
it would be much better if i could write <S:Fault> as last children of
<S:Body> and have receiving processor to discard input and act on Fault.
>in fact the
>breakage would only be visible to an external observer in the form
>of the error they will get.
>
breakage will be only if error happens but if you do what i did in
XSOAP1 the receiver will not know what caused problem as no Fault is
sent only that connection was lost ...
>Two well-behaved SOAP processors will
>not have any problem .. where well-behaved means that they basically
>general well-formed XML; a very low bar to pass.
>
>
>
it is more complicated than that as i described above: Fault is special
and is supposed to be first child of Body ...
do you have some other solution in mind?
>>payload (why no way to signal in-stream faults?) and it gets
>>very difficult with WSS (why there is no footers so one could
>>compute signature on the fly and then write it in footer!) and
>>does not make sense for WS-ReliableMessaging (message is stored
>> before sending so it can be easily re-sent ...) except of course
>>for re-streaming out of (database) persistent storage later but
>>that is just replay and will require coding to make sure it
>>scales well ....
>>
>>
>
>Again, I believe the realistic business scenarios for WS-Security
>and WS-Reliable Messaging have different performance characteristics
>than a raw SOAP message exchange. Thus, I'm not convinced there's
>much value in doing raw comparison of how long it takes an RM
>exchange to occur .. that's not what RM is intended to be used for
>IMO.
>
>
IMHO as business and scientific worlds converges on Web services and
Grids those distinctions are blurred.
for example one of the most important requirements for grid services (if
not the most important ...) is to have security.
in current WS world security has serious impact on performance (like one
order of magnitude slower for WSS and even more when you start doing SSO
with capabilities ....).
so the time of simple non secure and non reliable SOAP messages may be
going away but i hope it will not require more and more complex WS-*
specs that are harder and harder to understand and work with (WSS and
WS-Policy comes to mind ....).
alek
--
The best way to predict the future is to invent it - Alan Kay
Re: opinion Axis' future ... [Re: Using SAAJ-1.2 API to modify the soap header/body content
Posted by Sanjiva Weerawarana <sa...@opensource.lk>.
Hi Alek,
> i can see how you are thinking in performance/streaming/RPC terms
I've actually eliminated RPCness from my mind now .. that's just a
pattern which may be inferred if so desired. See how WSDL 2.0 is
designed.
> but it seems that reality of current SOAP is not allowing any of
> those desirable features to be easily expressed ...
I agree with your comment about streaming .. but I'm willing to
"break" SOAP semantics for it.
> now that we are stuffing more and into headers (such as binary
> security tokens) there seems to be quite common case that size
> of headers will well outweigh size of payload.
I agree 100%. However, in the presence of such headers the
performance requirements change. For example, if RM is being
used with a QoS of exactly once, ordered delivery, then the raw
performance is not so critical in a business process that's
probably driving it. Most likely the process is a long running
business transaction (like a mortgage application) and then
there's not much point in focusing on whether a message takes
10ms or 20ms when the whole process runs over a 3 month period.
> also i think that MTOM (and attachment) can be referenced
> both from headers and payloads so binary large data can be
WSDL 2.0 does not support that; MTOM is for the payload only ..
headers are not part of the fundamental description of a Web service;
they come into existence due to the application of certain QoS
(like RM or security).
> easily added anywhere in SOAP envelope and we should make
> sure that all this can be efficiently processed ...
Sure, but you can't have the cake and eat it. My believe today is
that header processing is expensive and is going to be so. SOAP
requires that all mustUnderstand headers be processed correctly ..
that means you can't "stream" headers thru - the framework has to
process everything. Furthermore, as more and more QoS characteristics
(aka policies) are applied, header processing becomes even more
complex. In particular, it is quite non-trivial to determine which
order the various policy handlers can run .. in the case of transactions
and reliable messaging and security you can convince yourself that the
right order is security, then reliable messaging and then Tx. In the
general case there is no algorithm to compute the order AFAIK.
> still clearly the one case to keep in mind is how to handle very
> large payloads as it may be very important case for some class of
> applications such as scientific computing.
Sure.
> i started with just considering generic XML messaging (and that
> is how XSOAP4/XSUL is designed) so i did not feel like there is
> a need to have a special separate treatment of header and body
> (payload) which seems to be now validated by current "core"
> WS-* specs (see below).
I'm confused by the last comment; which WS-* specs blur the ditinction
between header and body?? (I couldn't see a clarification further
below; sorry.)
> if you describe everything as XML infoset that it is very
> important to allow simple access to whole message XML infoset
> (and make possible to use such tools like XPath or XQuery) -
> runtime XML infoset is just optimized to take hints (such as
> provided by MTOM) to make decisions about in-memory
> materialization of XML content.
Yep; I'm definitely +1 for an MTOM-aware Infoset model being the
default, dynamically typed representation of arbitray XML content.
> so my thinking is that there needs to be set of flexible layers
> and developer can choose on which layer to plugin their code:
> transport, XML message infoset, SOAP headers/payload (header
> processors). XML Schema data binding, RelaxNG validation, whatnot ...
IMO we need to classify different types of developers and provide
different programming models for them. I want to keep app developers
a level above all this .. basically at the level WSDL 2.0 is putting
them at; anything else are policies they express or are expressed
to them by someone else.
The developers who write such policy handlers of course have a
different view of the world and get access to a different/additional
set of information .. for example, the security handler is someone
who will likely re-write the entire payload (if it were encrypted).
> unfortunately XML is not designed for streaming and moreover
> streaming of payload is not supported in SOAP 1.1/1.2.
Yes, unfortunately.
> i think i can make this bold (?!) statement based on my past
> experience: only by breaking SOAP semantics it is i possible
> to do limited streaming of vanilla SOAP envelopes with just
I'm ok with "breaking" SOAP semantics a bit here .. in fact the
breakage would only be visible to an external observer in the form
of the error they will get. Two well-behaved SOAP processors will
not have any problem .. where well-behaved means that they basically
general well-formed XML; a very low bar to pass.
> payload (why no way to signal in-stream faults?) and it gets
> very difficult with WSS (why there is no footers so one could
> compute signature on the fly and then write it in footer!) and
> does not make sense for WS-ReliableMessaging (message is stored
> before sending so it can be easily re-sent ...) except of course
> for re-streaming out of (database) persistent storage later but
> that is just replay and will require coding to make sure it
> scales well ....
Again, I believe the realistic business scenarios for WS-Security
and WS-Reliable Messaging have different performance characteristics
than a raw SOAP message exchange. Thus, I'm not convinced there's
much value in doing raw comparison of how long it takes an RM
exchange to occur .. that's not what RM is intended to be used for
IMO.
Sanjiva.
Re: opinion Axis' future ... [Re: Using SAAJ-1.2 API to modify the
soap header/body content
Posted by Aleksander Slominski <as...@cs.indiana.edu>.
Sanjiva Weerawarana wrote:
>"Aleksander Slominski" <as...@cs.indiana.edu> writes:
>
>
>>>>so my current thinking is that the optimal solution is a super
>>>>lightweight XML infoset in-memory representation that can be
>>>>partially materialized or de-materialized (stored in file, stream
>>>>on-demand parsing backed).
>>>>
>>>>
>
>Does this cover the entire envelope or just the payload?
>
Sanjiva,
i can see how you are thinking in performance/streaming/RPC terms but it
seems that reality of current SOAP is not allowing any of those
desirable features to be easily expressed ...
now that we are stuffing more and into headers (such as binary security
tokens) there seems to be quite common case that size of headers will
well outweigh size of payload. also i think that MTOM (and attachment)
can be referenced both from headers and payloads so binary large data
can be easily added anywhere in SOAP envelope and we should make sure
that all this can be efficiently processed ...
still clearly the one case to keep in mind is how to handle very large
payloads as it may be very important case for some class of applications
such as scientific computing.
>IMO it
>has to be the latter ..
>
i started with just considering generic XML messaging (and that is how
XSOAP4/XSUL is designed) so i did not feel like there is a need to have
a special separate treatment of header and body (payload) which seems to
be now validated by current "core" WS-* specs (see below).
if you describe everything as XML infoset that it is very important to
allow simple access to whole message XML infoset (and make possible to
use such tools like XPath or XQuery) - runtime XML infoset is /just/
optimized to take hints (such as provided by MTOM) to make decisions
about in-memory materialization of XML content.
>I'm now convinced JAXRPC et al. are flat
>out wrong in providing XML level access to the entire SOAP envelope.
>Headers should be provided to any handler (or application code
>if so desired even though that's busted too in my mind) but not
>at the Infoset level.
>
developers will always find ways to make things happen and even if it is
difficult when it needs to be done it is done even if it ain't pretty :)
>That will permit the Axis framework to do
>things like mustunderstand handling and not allow a handler
>to go screw with that stuff.
>
>
so my thinking is that there needs to be set of flexible layers and
developer can choose on which layer to plugin their code: transport, XML
message infoset, SOAP headers/payload (header processors). XML Schema
data binding, RelaxNG validation, whatnot ...
>Payload is a different beast ..
>
>
unfortunately XML is not designed for streaming and moreover streaming
of payload is not supported in SOAP 1.1/1.2 - i think i can make this
bold (?!) statement based on my past experience: only by breaking SOAP
semantics it is i possible to do *limited* streaming of vanilla SOAP
envelopes with just payload (why no way to signal in-stream faults?) and
it gets very difficult with WSS (why there is no footers so one could
compute signature on the fly and then write it in footer!) and does not
make sense for WS-ReliableMessaging (message is *stored* before sending
so it can be easily re-sent ...) except of course for re-streaming out
of (database) persistent storage later but that is just replay and will
require coding to make sure it scales well ....
alek
--
The best way to predict the future is to invent it - Alan Kay
Re: opinion Axis' future ... [Re: Using SAAJ-1.2 API to modify the soap header/body content
Posted by Sanjiva Weerawarana <sa...@opensource.lk>.
"Aleksander Slominski" <as...@cs.indiana.edu> writes:
> >> so my current thinking is that the optimal solution is a super
> >> lightweight XML infoset in-memory representation that can be
> >> partially materialized or de-materialized (stored in file, stream
> >> on-demand parsing backed).
Does this cover the entire envelope or just the payload? IMO it
has to be the latter .. I'm now convinced JAXRPC et al. are flat
out wrong in providing XML level access to the entire SOAP envelope.
Headers should be provided to any handler (or application code
if so desired even though that's busted too in my mind) but not
at the Infoset level. That will permit the Axis framework to do
things like mustunderstand handling and not allow a handler
to go screw with that stuff.
Payload is a different beast ..
Sanjiva.
Re: opinion Axis' future ... [Re: Using SAAJ-1.2 API to modify the
soap header/body content
Posted by Aleksander Slominski <as...@cs.indiana.edu>.
Steve Loughran wrote:
> Aleksander Slominski wrote:
>
>> so my current thinking is that the optimal solution is a super
>> lightweight XML infoset in-memory representation that can be
>> partially materialized or de-materialized (stored in file, stream
>> on-demand parsing backed).
>>
>> to make this happen it is handy to allow to store non XML content in
>> XML infoset tree that is only XML-ized when message is about to be
>> sent (such as pointer to file so really huge files can be sent as
>> BASE64, as attachments, or MOTM or whatever it is called). when
>> receiving message goes through multiple processors in pipeline and is
>> gradually converted from XML to whatever Java representation one
>> needs (such as pointer to file backed representation).
>
>
> +1 to being able to offload v large portions of the infoset to HDD;
> the reason you cannot stick big binary stuff in as base64 stuff today
> is not really network efficiency (we base-64 all email), but
> implementation -XML parsers werent written for 15MB strings, or 30MB
> infosets. That needs to change.
>
true (i know i wrote one and struggled wit this problem) but i think
that MTOM/XOP will be more "optimal" solution to it as potentially huge
binary parts will be optimized as attachments and can be easily
identified and offloaded to disk without need to guess which XML element
text/base64 content needs must be handled as a large binary blob ...
alek
--
The best way to predict the future is to invent it - Alan Kay
Re: opinion Axis' future ... [Re: Using SAAJ-1.2 API to modify the
soap header/body content
Posted by Steve Loughran <st...@iseran.com>.
Aleksander Slominski wrote:
> so my current thinking is that the optimal solution is a super
> lightweight XML infoset in-memory representation that can be partially
> materialized or de-materialized (stored in file, stream on-demand
> parsing backed).
>
> to make this happen it is handy to allow to store non XML content in XML
> infoset tree that is only XML-ized when message is about to be sent
> (such as pointer to file so really huge files can be sent as BASE64, as
> attachments, or MOTM or whatever it is called). when receiving message
> goes through multiple processors in pipeline and is gradually converted
> from XML to whatever Java representation one needs (such as pointer to
> file backed representation).
+1 to being able to offload v large portions of the infoset to HDD; the
reason you cannot stick big binary stuff in as base64 stuff today is not
really network efficiency (we base-64 all email), but implementation
-XML parsers werent written for 15MB strings, or 30MB infosets. That
needs to change.
Aleksander, are you at GGF11 right now? I am there, though in a state of
jetlag induced detachment.
-steve
opinion Axis' future ... [Re: Using SAAJ-1.2 API to modify the soap
header/body content
Posted by Aleksander Slominski <as...@cs.indiana.edu>.
Ias wrote:
>Hi Thomas,
>
>Thanks for your pointing the issues out. As a contributor of the SAAJ 1.2
>implementation in Axis (with Dr. Ahn), I have to admit that he and I already
>made our own (refactored) Axis satisfy #1 and #2. He created DOM recorder
>for #1 and I refactored the whole org.apache.axis.message package for #2. As
>for #2, org.apache.axis.message.Node was created and
>org.apache.axis.message.MessageElement and org.apache.axis.message.Text
>extended it.
>
>However, those changes are quite big and wide, so we need to modify Axis a
>lot. Now we are in the middle of finalizing Axis 1.2, I don't think such
>large changes will happen to this upcoming version of Axis. Personally, I
>often have felt that contributing #1 and #2 to the current Axis base seems
>pretty difficult and consider re-structuring the SAAJ part with Axis NG
>(Next Generation).
>
>What I'm designing regarding Axis NG is a complete implementation of JAX-RPC
>2.0 and Web Services Metadata for high performance with StAX (streaming
>parser). I'm planning to open my thoughts on that as soon as JAX-RPC 2.0 and
>JAXB 2.0 early drafts are unveiled around JavaOne 2004. Briefly, I'd like
>Axis NG to use StAX when deserializing/serializing between SOAP messages and
>SAAJ instances.
>
how will this help avoid the trap of SAX event recording i.e. how it
makes any difference if you record SAX or StAX events?!
i have pondered this problem and decided that trying to make two things
(streaming and DOM-like Infoset) fit into one hole is *the problem*.
so my current thinking is that the optimal solution is a super
lightweight XML infoset in-memory representation that can be partially
materialized or de-materialized (stored in file, stream on-demand
parsing backed).
to make this happen it is handy to allow to store non XML content in XML
infoset tree that is only XML-ized when message is about to be sent
(such as pointer to file so really huge files can be sent as BASE64, as
attachments, or MOTM or whatever it is called). when receiving message
goes through multiple processors in pipeline and is gradually converted
from XML to whatever Java representation one needs (such as pointer to
file backed representation).
this approach tries to avoid converting between multiple XML
representation, caching them and trying to keep them consistent (huge
memory inflation to keep all message representations or CPU overhead to
recompute them on demand which is even worse as XML is CPU bound) and
that i think is the root of all current problem ...
best,
alek
--
The best way to predict the future is to invent it - Alan Kay
RE: Using SAAJ-1.2 API to modify the soap header/body content
Posted by Ias <ia...@tmax.co.kr>.
Hi Thomas,
Thanks for your pointing the issues out. As a contributor of the SAAJ 1.2
implementation in Axis (with Dr. Ahn), I have to admit that he and I already
made our own (refactored) Axis satisfy #1 and #2. He created DOM recorder
for #1 and I refactored the whole org.apache.axis.message package for #2. As
for #2, org.apache.axis.message.Node was created and
org.apache.axis.message.MessageElement and org.apache.axis.message.Text
extended it.
However, those changes are quite big and wide, so we need to modify Axis a
lot. Now we are in the middle of finalizing Axis 1.2, I don't think such
large changes will happen to this upcoming version of Axis. Personally, I
often have felt that contributing #1 and #2 to the current Axis base seems
pretty difficult and consider re-structuring the SAAJ part with Axis NG
(Next Generation).
What I'm designing regarding Axis NG is a complete implementation of JAX-RPC
2.0 and Web Services Metadata for high performance with StAX (streaming
parser). I'm planning to open my thoughts on that as soon as JAX-RPC 2.0 and
JAXB 2.0 early drafts are unveiled around JavaOne 2004. Briefly, I'd like
Axis NG to use StAX when deserializing/serializing between SOAP messages and
SAAJ instances. In addition, Axis NG will use JAXB 2.0 to unmarshal/marshal
between XMLs and Java instances with StAX. JDK 1.5's metadata enables users
to develope web services just like JSPs with WSM and generate portable
artifacts for web services (both client and server) runtime from an
annotated seed class.
I'd like to hear your opinions about Axis' future, in particular, 1.3 and NG
even though it's a little early. :-)
Regards,
Ias
________________________________
From: Thomas Diesler [mailto:thomas.diesler@jboss.org]
Sent: Sunday, June 06, 2004 1:22 AM
To: axis-dev@ws.apache.org
Subject: Using SAAJ-1.2 API to modify the soap header/body content
Hi Axis Dev,
I am working on a J2EE-1.4 compliant implementation for web services
(WS4EE) in JBoss based on Axis. As you might know, JBoss has been using Axis
in its jboss.net module for quite some time. We are starting a new project
called jboss-ws4ee that we would like to use in our J2EE-1.4 certified
release.
I'm wondering if you could help me with these issues
#1 SAX recorded events
Axis uses a Serializer/Deserialiser based on SAX events. The
recorded events are stored with the SOAPElement.
On serialization the SOAPElement will simply playback the events
if the element is not marked dirty.
A JAXRPC handler is allowed to view and modify soap headers and
body. For that it can use the saaj api or even the dom api
on javax.xml.soap.Node objects.
When a client makes these modifications, they are not reflected
in the axis internal representation of the soap message.
This is especially true for RPCElement, which keeps an internal
list of RPCParam objects, which are not linked to SOAPElement.
#2 General soap tree representation
Axis is missing the notion of javax.xml.soap.Node, instead it
models javax.xml.soap.Text as javax.xml.soap.SOAPElement.
I'm having a hard time trying to keep all the various SOAPElement
representations in sync. For jboss-ws4ee I started a
refactoring effort that would base SOAPElementImpl on NodeImpl
which implements the org.w3c.dom.Node interface.
All information about namespaces, attributes, parent/child
relationships should be taken from the dom tree.
Serialization/Deserialization would be based on the dom tree as
well.
Any suggestions?
cheers
-thomas
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
Senior Developer & Consultant
JBoss Group (Europe) SàRL
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx