Re: [Axis2] Data binding attachments support

[Glen Daniels <gl...@thoughtcraft.com>]

Hi dims:

> May be i am missing something...the difference in my mind is a person
> implementing a databinding layer should be able to access the
> attachements without having to build the om tree. straight from stax
> to java objects with no om and use whatever they need to store the
> attachments byte arrays or data handlers or some databinding specific
> construct.

+1.  OM was built to allow you to optimize out the 
tree-building/buffering for the normal XML case - you call 
getXMLStreamReaderWithoutCaching() and go.  MTOM/attachments are sort of 
the fly in the ointment there, in that you need another layer below StAX 
in order to get at the attachments.  We've got that layer now, but it's 
hidden and tightly coupled to the OM tree framework.  The suggestion is 
simply to open it up so you can do exactly what dims describes here.

--Glen

Re: [Axis2] Data binding attachments support

[Dennis Sosnoski <dm...@sosnoski.com>]

Yes, I think that's the crux of the matter - we need to be able to build 
a tree *if required* by ws-sec or such, but the common case is no ws-sec 
and hence no tree needed. And if you *do* use ws-sec, the OM 
tree-building is the least of your performance concerns.

For the JiBX interface to Axis2 I've got this part dummied out for now, 
but the intent is to just marshal output to a memory buffer and then 
build an OM tree from that output. The OMElement implementation that 
represents a JiBX data item will then just delegate to the OMElement 
constructed from the output. I'll split this apart and move the 
delegating OMElement implementation over to Axiom, both because this 
makes it easier to handle the continuing changes in the Axiom API and 
because that way the same technique can be used to support JAXB 2.0 and 
other data binding frameworks.

  - Dennis

Davanum Srinivas wrote:

>if ws-sec is turned on, then we force a build of the om tree...just
>like we do now.
>
>-- dims
>
>On 3/31/06, Sanjiva Weerawarana <sa...@opensource.lk> wrote:
>  
>
>>On Fri, 2006-03-31 at 10:55 -0500, Davanum Srinivas wrote:
>>    
>>
>>>Let me try again...The DB framework will build the java objects
>>>directly from the MIME root part (this is the first step always!) and
>>>*then* accesses the other mime parts and sticks them where it is
>>>needed (or adds a reference) on the java objects that it already
>>>created. Except that OM tree is *never* built.
>>>      
>>>
>>Ah but that's inconsistent with XOP .. if you do XOP, then when you look
>>at the XML at the Infoset level (which is what you do when you look at
>>the root part thru Axiom) then you have to un-XOPify it and just see the
>>XML Infoset. There's no halfway point.
>>
>>What you're looking at is SwA .. MTOM is not that IMO.
>>
>>I guess we could put a flag saying "don't unXOPify" but that seems like
>>a hack.
>>
>>    
>>
>>> And on the sending
>>>side, it generates stax events directly from the the java objects into
>>>the MIME root part and adds the attachments into a bag while it is
>>>doing so...again no OM tree in the picture at all.
>>>      
>>>
>>Again, you're thinking like SwA and not like a single unified Infoset
>>that has the binary parts logically in it. Think of WS-Security- how
>>will your model work with WS-Sec turned on to sign the whole shebang?
>>
>>Sanjiva.
>>
>>
>>    
>>
>
>
>--
>Davanum Srinivas : http://wso2.com/blogs/
>
>  
>

Re: [Axis2] Data binding attachments support

[Davanum Srinivas <da...@gmail.com>]

if ws-sec is turned on, then we force a build of the om tree...just
like we do now.

-- dims

On 3/31/06, Sanjiva Weerawarana <sa...@opensource.lk> wrote:
> On Fri, 2006-03-31 at 10:55 -0500, Davanum Srinivas wrote:
> > Let me try again...The DB framework will build the java objects
> > directly from the MIME root part (this is the first step always!) and
> > *then* accesses the other mime parts and sticks them where it is
> > needed (or adds a reference) on the java objects that it already
> > created. Except that OM tree is *never* built.
>
> Ah but that's inconsistent with XOP .. if you do XOP, then when you look
> at the XML at the Infoset level (which is what you do when you look at
> the root part thru Axiom) then you have to un-XOPify it and just see the
> XML Infoset. There's no halfway point.
>
> What you're looking at is SwA .. MTOM is not that IMO.
>
> I guess we could put a flag saying "don't unXOPify" but that seems like
> a hack.
>
> >  And on the sending
> > side, it generates stax events directly from the the java objects into
> > the MIME root part and adds the attachments into a bag while it is
> > doing so...again no OM tree in the picture at all.
>
> Again, you're thinking like SwA and not like a single unified Infoset
> that has the binary parts logically in it. Think of WS-Security- how
> will your model work with WS-Sec turned on to sign the whole shebang?
>
> Sanjiva.
>
>


--
Davanum Srinivas : http://wso2.com/blogs/

Re: [Axis2] Data binding attachments support

[Thilina Gunarathne <cs...@gmail.com>]

A quick comment before I run for my lecture :)
> > What you're looking at is SwA .. MTOM is not that IMO.
>
> Same deal for either really, just in one case you have hrefs and in
> another you have xop:Includes.
>
There's a huge difference.. SwA results in two data models. One for
the XML and other for the attachments.
XOP avoids this.. It's just this one XOP:include element..But it makes
a huge difference..
AFAIK when we see a href in a some XML there's no way we can tell for
sure that there's a referenced attachment which should logically map
here.It's just a ordinary attribute..  But whenever we see a
XOP:Include we can definitely tell that "okay.. There's a attachment
which should be logically map to here.". This automatic mapping
results in just one data model.. XML infoset with XOP..

This point comes in to play mostly when these two kinds of messages
were encountered by programs that look only at the payload without
looking at schema or other things.. One example is WS-Security..


~Thilina

Re: [Axis2] Data binding attachments support

[Glen Daniels <gl...@thoughtcraft.com>]

Hey Sanjiva:

> On Fri, 2006-03-31 at 10:55 -0500, Davanum Srinivas wrote:
>> Let me try again...The DB framework will build the java objects
>> directly from the MIME root part (this is the first step always!) and
>> *then* accesses the other mime parts and sticks them where it is
>> needed (or adds a reference) on the java objects that it already
>> created. Except that OM tree is *never* built.
> 
> Ah but that's inconsistent with XOP .. if you do XOP, then when you look
> at the XML at the Infoset level (which is what you do when you look at
> the root part thru Axiom) then you have to un-XOPify it and just see the
> XML Infoset. There's no halfway point. 

Rrright, exactly.  The point is that while what you just said is 
perfectly true, you see the INFOSET (i.e. what you get out of 
XmlStreamReader) but not necessarily the OM tree.  So if a DB framework 
like JIBX wants to handle the XOP support itself, it can do that by 
doing something like this in the deserializer:

   ...deserializing from StAX events...
   if (currentElement.getQName().equals(XOP_INCLUDE)) {
     String contentID = getIDFromXopInclude();
     XOPThing binaryThing =
         new XOPThing(attachmentContext, contentID);
     // insert binaryThing into the object we're deserializing
   }
   ...continue...

XOPThing is this "future DataHandler" that we were talking about, maybe 
just a DataHandler.  But the point is it knows how to ask the 
AttachmentContext object (with an API like what Dennis proposes) for the 
actual InputStream and process it at the right time (obviously after all 
the XML has been pulled/deserialized from the root part - when the 
object tree is compete and the application asks for the image/data/etc).

As I'm thinking about it I wonder if we want to add a configuration 
option which would automatically "pre-cache" all the attachments into 
files immediately upon parsing the end of the root part, or if that 
should indeed be the default behaviour.  In other words, do we want to 
support pausing the read on the actual HTTP InputStream until the 
application asks for an attachment, much as we do with the SOAP envelope?

> What you're looking at is SwA .. MTOM is not that IMO.

Same deal for either really, just in one case you have hrefs and in 
another you have xop:Includes.

> I guess we could put a flag saying "don't unXOPify" but that seems like
> a hack.

You don't need a flag other than "don't cache" - which is really "don't 
build the Object Model".  If OM isn't building the Object Model, it 
can't very well do unXOPification (where would it put anything?).

>>  And on the sending
>> side, it generates stax events directly from the the java objects into
>> the MIME root part and adds the attachments into a bag while it is
>> doing so...again no OM tree in the picture at all.

+1 dims.

> Again, you're thinking like SwA and not like a single unified Infoset
> that has the binary parts logically in it. Think of WS-Security- how
> will your model work with WS-Sec turned on to sign the whole shebang?

Great point.  WS-Security (or anything else which requires the full 
infoset to be preserved in the OM as well as the databound objects) will 
need to switch on a flag which indicates "build the object model 
always".  So really, on the receiving side that flag should be an option 
on the OM builder which overrides a call to 
getXMLStreamReaderWithoutCaching() and turns it into a caching call. 
This is transparent to both the security code (which wants an OM) and 
the DB code (which wants StAX events) - both sides can get at the 
attachments using the correct APIs.  The optimization only works when no 
one sets the "always build the OM" flag, but other than that it should 
work transparently.

The same is true on the outbound side - if the flag is set, the StAX 
events won't go directly out to the OutputStream representing the root 
part of the MIME envelope (as they could in the optimized case), but 
instead build up an OM so that a security (or checksum, or whatever) 
handler can deal with it later.

Make sense?

--Glen

Re: [Axis2] Data binding attachments support

[Sanjiva Weerawarana <sa...@opensource.lk>]

On Fri, 2006-03-31 at 10:55 -0500, Davanum Srinivas wrote:
> Let me try again...The DB framework will build the java objects
> directly from the MIME root part (this is the first step always!) and
> *then* accesses the other mime parts and sticks them where it is
> needed (or adds a reference) on the java objects that it already
> created. Except that OM tree is *never* built.

Ah but that's inconsistent with XOP .. if you do XOP, then when you look
at the XML at the Infoset level (which is what you do when you look at
the root part thru Axiom) then you have to un-XOPify it and just see the
XML Infoset. There's no halfway point. 

What you're looking at is SwA .. MTOM is not that IMO.

I guess we could put a flag saying "don't unXOPify" but that seems like
a hack.

>  And on the sending
> side, it generates stax events directly from the the java objects into
> the MIME root part and adds the attachments into a bag while it is
> doing so...again no OM tree in the picture at all.

Again, you're thinking like SwA and not like a single unified Infoset
that has the binary parts logically in it. Think of WS-Security- how
will your model work with WS-Sec turned on to sign the whole shebang?

Sanjiva.

Re: [Axis2] Data binding attachments support

[Davanum Srinivas <da...@gmail.com>]

Sanjiva,

/me scratches his head and wonders How come glen understands what he
says - "exactly what dims describes here" and sanjiva doesn't :)

Let me try again...The DB framework will build the java objects
directly from the MIME root part (this is the first step always!) and
*then* accesses the other mime parts and sticks them where it is
needed (or adds a reference) on the java objects that it already
created. Except that OM tree is *never* built. And on the sending
side, it generates stax events directly from the the java objects into
the MIME root part and adds the attachments into a bag while it is
doing so...again no OM tree in the picture at all.

-- dims

On 3/31/06, Sanjiva Weerawarana <sa...@opensource.lk> wrote:
> On Fri, 2006-03-31 at 09:52 -0500, Glen Daniels wrote:
> > Hi dims:
> >
> > > May be i am missing something...the difference in my mind is a person
> > > implementing a databinding layer should be able to access the
> > > attachements without having to build the om tree. straight from stax
> > > to java objects with no om and use whatever they need to store the
> > > attachments byte arrays or data handlers or some databinding specific
> > > construct.
>
> What's buffered are the bytes of the MIME root part that contains the
> SOAP envelope. I agree we shouldn't build the tree for that part (and
> I'm pretty certain we don't) until its needed but the bytes have to be
> read and buffered. Unless you guys know of some magic technology I don't
> see how that can be done any other way ;-) .. a stream is ordered you
> know and those bytes come after these bytes. Simple as that.
>
> > +1.  OM was built to allow you to optimize out the
> > tree-building/buffering for the normal XML case - you call
> > getXMLStreamReaderWithoutCaching() and go.  MTOM/attachments are sort of
> > the fly in the ointment there, in that you need another layer below StAX
> > in order to get at the attachments.  We've got that layer now, but it's
> > hidden and tightly coupled to the OM tree framework.  The suggestion is
> > simply to open it up so you can do exactly what dims describes here.
>
> Ah ok that part I agree with .. that getting to the MIME data via the
> MIMEHelper is a good thing. Is that what you're looking for?
>
> Sanjiva.
>
>


--
Davanum Srinivas : http://wso2.com/blogs/

Re: [Axis2] Data binding attachments support

[Sanjiva Weerawarana <sa...@opensource.lk>]

On Fri, 2006-03-31 at 09:52 -0500, Glen Daniels wrote:
> Hi dims:
> 
> > May be i am missing something...the difference in my mind is a person
> > implementing a databinding layer should be able to access the
> > attachements without having to build the om tree. straight from stax
> > to java objects with no om and use whatever they need to store the
> > attachments byte arrays or data handlers or some databinding specific
> > construct.

What's buffered are the bytes of the MIME root part that contains the
SOAP envelope. I agree we shouldn't build the tree for that part (and
I'm pretty certain we don't) until its needed but the bytes have to be
read and buffered. Unless you guys know of some magic technology I don't
see how that can be done any other way ;-) .. a stream is ordered you
know and those bytes come after these bytes. Simple as that.

> +1.  OM was built to allow you to optimize out the 
> tree-building/buffering for the normal XML case - you call 
> getXMLStreamReaderWithoutCaching() and go.  MTOM/attachments are sort of 
> the fly in the ointment there, in that you need another layer below StAX 
> in order to get at the attachments.  We've got that layer now, but it's 
> hidden and tightly coupled to the OM tree framework.  The suggestion is 
> simply to open it up so you can do exactly what dims describes here.

Ah ok that part I agree with .. that getting to the MIME data via the
MIMEHelper is a good thing. Is that what you're looking for?

Sanjiva.