You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@axis.apache.org by Chad La Joie <la...@georgetown.edu> on 2007/08/03 23:57:54 UTC
End-to-End Streaming Support
I'm working on a project that may (read: almost certainly will) require
very large binary blobs (many GBs) to be transported. The specification
I'm working with requires at least SOAP over HTTP support and allows for
the use of MTOM.
My question is, does Axis 2 (client and service side) support true
end-to-end streaming. By this I mean can a service stream data from
disk, over HTTP to the client and provide a stream on the client side
which can be dumped direct to disk?
I had read the MTOM guide [1] and while I see data handlers and sources
being used it doesn't say whether Axis 2 is architected in such a way
that the data is never buffered in memory.
Thanks for the help.
[1] http://ws.apache.org/axis2/1_0/mtom-guide.html
--
Chad La Joie 2052-C Harris Bldg
OIS-Middleware 202.687.0124
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: End-to-End Streaming Support
Posted by Chad La Joie <la...@georgetown.edu>.
Yep, thanks Dims.
The part of
"write it directly to the http stream"
is what I need to verify.
Thanks to both you and Thilina. Sorry I wasn't able to be more clear in
the very beginning.
Davanum Srinivas wrote:
> Chad,
>
> Take a deep breath and start from scratch :)
>
> You a WSDL with a Schema with an element whose schema type="xsd:base64Binary"
> [See http://ws.apache.org/axis2/1_2/mtom-guide.html, search for MyBinaryData]
>
> When you run WSDL2Java and review Base64Binary.java, it will have a
> setBase64Binary/getBase64Binary with DataHandler as the parameter
>
> In either your client or service, just use say a file backed
> DataHandler to send the data. Your file will have binary data (NOT
> base64 encoded).
>
> At runtime, *IF* you have enabled MTOM in axis2.xml or as a property
> in options on the client side, the engine will create a mime separated
> stream with the soap xml in the root mime part and will ask *your*
> data handler for the binary stream and write it directly to the http
> stream.
> However if MTOM is *disabled*, the engine will try to base64 encode
> the *WHOLE* byte stream from your data handler and create a giant text
> node inside the SOAP request/response.
>
> Is the process clear now?
>
> thanks,
> dims
>
>
> BTW, I don't recommend HTTP/SOAP for multi-gig content
>
> Step #2: When you run WSDL2Java *USING ADB data binding*,
>
> On 8/5/07, Chad La Joie <la...@georgetown.edu> wrote:
>> Thilina Gunarathne wrote:
>>>> If, as we discussed in the first part of this thread, Axis2/Axiom
>>>> supports true end-to-end streaming, and Axiom can correctly stream
>>>> content from/into a DataHandler, what causes problems for arbitrarily
>>>> large files?
>>> Problems like what.. AFAIK it does not cause any problems.
>> I don't know what the problem would be. You said
>>
>> "Do you want to send/receive binary in base64 form.. If so it won't
>> tolerate large files.."
>>
>> So, I was trying to find out why it wouldn't tolerate large files.
>>
>> May be we
>>> can hep you a bit if you state your use case clearly.
>> I thought I had, or at least I tried. Let me try a third time.
>>
>> I have a SOAP body with an element defined as:
>>
>> <element name="foo" type="base64Binary">
>>
>> I need support for putting multi-gig (pre-base64 encoding) content in
>> that element and transporting over HTTP w/ MTOM. I'm trying to be
>> obtuse, but I don't know what additional information I need to give either.
>>
>> At this point I'm really just trying to verify the large-content case
>> over MTOM.
>>
>> --
>> Chad La Joie 2052-C Harris Bldg
>> OIS-Middleware 202.687.0124
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
>> For additional commands, e-mail: axis-user-help@ws.apache.org
>>
>>
>
>
--
Chad La Joie 2052-C Harris Bldg
OIS-Middleware 202.687.0124
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: End-to-End Streaming Support
Posted by Davanum Srinivas <da...@gmail.com>.
Chad,
Take a deep breath and start from scratch :)
You a WSDL with a Schema with an element whose schema type="xsd:base64Binary"
[See http://ws.apache.org/axis2/1_2/mtom-guide.html, search for MyBinaryData]
When you run WSDL2Java and review Base64Binary.java, it will have a
setBase64Binary/getBase64Binary with DataHandler as the parameter
In either your client or service, just use say a file backed
DataHandler to send the data. Your file will have binary data (NOT
base64 encoded).
At runtime, *IF* you have enabled MTOM in axis2.xml or as a property
in options on the client side, the engine will create a mime separated
stream with the soap xml in the root mime part and will ask *your*
data handler for the binary stream and write it directly to the http
stream.
However if MTOM is *disabled*, the engine will try to base64 encode
the *WHOLE* byte stream from your data handler and create a giant text
node inside the SOAP request/response.
Is the process clear now?
thanks,
dims
BTW, I don't recommend HTTP/SOAP for multi-gig content
Step #2: When you run WSDL2Java *USING ADB data binding*,
On 8/5/07, Chad La Joie <la...@georgetown.edu> wrote:
> Thilina Gunarathne wrote:
> >> If, as we discussed in the first part of this thread, Axis2/Axiom
> >> supports true end-to-end streaming, and Axiom can correctly stream
> >> content from/into a DataHandler, what causes problems for arbitrarily
> >> large files?
> > Problems like what.. AFAIK it does not cause any problems.
>
> I don't know what the problem would be. You said
>
> "Do you want to send/receive binary in base64 form.. If so it won't
> tolerate large files.."
>
> So, I was trying to find out why it wouldn't tolerate large files.
>
> May be we
> > can hep you a bit if you state your use case clearly.
>
> I thought I had, or at least I tried. Let me try a third time.
>
> I have a SOAP body with an element defined as:
>
> <element name="foo" type="base64Binary">
>
> I need support for putting multi-gig (pre-base64 encoding) content in
> that element and transporting over HTTP w/ MTOM. I'm trying to be
> obtuse, but I don't know what additional information I need to give either.
>
> At this point I'm really just trying to verify the large-content case
> over MTOM.
>
> --
> Chad La Joie 2052-C Harris Bldg
> OIS-Middleware 202.687.0124
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>
--
Davanum Srinivas :: http://davanum.wordpress.com
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: End-to-End Streaming Support
Posted by Chad La Joie <la...@georgetown.edu>.
Thilina Gunarathne wrote:
>> If, as we discussed in the first part of this thread, Axis2/Axiom
>> supports true end-to-end streaming, and Axiom can correctly stream
>> content from/into a DataHandler, what causes problems for arbitrarily
>> large files?
> Problems like what.. AFAIK it does not cause any problems.
I don't know what the problem would be. You said
"Do you want to send/receive binary in base64 form.. If so it won't
tolerate large files.."
So, I was trying to find out why it wouldn't tolerate large files.
May be we
> can hep you a bit if you state your use case clearly.
I thought I had, or at least I tried. Let me try a third time.
I have a SOAP body with an element defined as:
<element name="foo" type="base64Binary">
I need support for putting multi-gig (pre-base64 encoding) content in
that element and transporting over HTTP w/ MTOM. I'm trying to be
obtuse, but I don't know what additional information I need to give either.
At this point I'm really just trying to verify the large-content case
over MTOM.
--
Chad La Joie 2052-C Harris Bldg
OIS-Middleware 202.687.0124
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: End-to-End Streaming Support
Posted by Thilina Gunarathne <cs...@gmail.com>.
> If, as we discussed in the first part of this thread, Axis2/Axiom
> supports true end-to-end streaming, and Axiom can correctly stream
> content from/into a DataHandler, what causes problems for arbitrarily
> large files?
Problems like what.. AFAIK it does not cause any problems. May be we
can hep you a bit if you state your use case clearly.
thanks,
Thilina
>
> Maybe I misunderstood something in the first part of the thread or
> didn't properly state what I was trying to accomplish.
> --
> Chad La Joie 2052-C Harris Bldg
> OIS-Middleware 202.687.0124
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>
--
Thilina Gunarathne - http://www.wso2.com - http://thilinag.blogspot.com
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: End-to-End Streaming Support
Posted by Chad La Joie <la...@georgetown.edu>.
Okay, so it does seems that I haven't explained what I need clear enough.
I'm implementing a SOAP service based on a particular specification.
One operation of this service allows for a client to upload arbitrarily
large binary objects, as base64 encoded content of an element, within
the SOAP body. The specification then requires the use MTOM as a means
of optimizing the transfer of this information. The SOAP service has
another operation that allows for a client to retrieve this uploaded
information in the same manner (base64 encoded content of SOAP body
element using MTOM during transmission).
The binary content being uploaded could be something small (1-2MB) or
could be quite large (many GBs). In the later case, if, at any point
during transmission, on either the client or server side, some bit of
code tries to buffer, in memory, the whole message or even just the
base64 encoded content the process will exhaust the server's memory.
So, what I need is true end-to-end IO streaming support.
By this I mean I need the client to be able to begin building and
sending the XML over to the service. When the client gets to the point
where it needs to send the contents of the element carrying the base64
encoded binary content, it would pipe the DataHandler's input stream
into the established output stream to the service.
Likewise, on the service side, it would receive the first parts of the
SOAP request (the headers and the start of the body) and when it got to
the point where it was receiving the base64 encoded binary content,
would pipe its input stream into the output stream of the DataHandler.
Reverse this for the second operation where the client is pulling the
information down from the service.
I then asked, if Axis2 did have such support, was it possible to, on the
service side, for the data handler to receive the information in base64
encode form during the first operation and provide it in base64 encoded
form during the second operation. The service itself never operates on
this encoded binary content (except to store it). So, if it could
receive, store, and provide it all in the base64 form that would be
significantly more efficient.
So, in summary, the key point is that neither the client or service ever
make the assumption that it can fit any MIME body of the XOP package in
memory, other than the initial aplication/xop+xml body, in memory.
Martin Gainty wrote:
> Chad-
> By streaming your definition is not necessarily read or write (XML |
> Text) streams but something closer to RSS Feed I would look at tweaking
> axis2/databinding/utils/reader/ADBXMLStreamReaderTest.java specifically
> take a look at newDocument method
> notice the wrap of ByteArrayInputStream("SomePreviouslyReadByteArray");
> and use that for your InputStream
> you'll have to think thru how much to read (is this entity RLE?)
> (Please confirm whether your stream is an RSS or using some other
> Streaming mechanism)
> HTH/
> Martin
> This email message and any files transmitted with it contain confidential
> information intended only for the person(s) to whom this email message is
> addressed. If you have received this email message in error, please notify
> the sender immediately by telephone or email and destroy the original
> message without making a copy. Thank you.
>
> ----- Original Message ----- From: "Chad La Joie" <la...@georgetown.edu>
> To: <ax...@ws.apache.org>
> Sent: Sunday, August 05, 2007 1:08 PM
> Subject: Re: End-to-End Streaming Support
>
>
>> Thilina Gunarathne wrote:
>>>> Does Axis, or Axiom, handle the Base64 encoding/decoding of the binary
>>>> data to/from a DataHandler?
>>> Yes... Only if needed...
>>>
>>>> If so, is there a way to disable such handling?
>>> Nope... But if you use the API properly you can avoid any unncessary
>>> encoding/decodings.
>>>
>>>> Characteristics for my service have a client submitting large files and
>>>> then some number of clients retrieving them. The service itself treats
>>>> these files as opaque blobs. It could receive, and return, them to
>>>> Axis
>>>> in a Base64 encoded form saving a lot of time when clients retrieved a
>>>> document be avoiding a re-encode.
>>> Above is not clear to me. Is your service written in Axis2. Do you
>>> want to send/receive binary in base64 form.. If so it won't tolerate
>>> large files..
>>
>> I'm not using anything yet. My hope was to use Axis2 so right now I'm
>> just trying to make sure it supports what I'll need. I know that the
>> need to send large files, encoded as base64 element content, is not
>> something that most SOAP stacks handle. This is why I started by asking
>> about true end-to-end streaming support when using MTOM.
>>
>> If, as we discussed in the first part of this thread, Axis2/Axiom
>> supports true end-to-end streaming, and Axiom can correctly stream
>> content from/into a DataHandler, what causes problems for arbitrarily
>> large files?
>>
>> Maybe I misunderstood something in the first part of the thread or
>> didn't properly state what I was trying to accomplish.
>> --
>> Chad La Joie 2052-C Harris Bldg
>> OIS-Middleware 202.687.0124
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
>> For additional commands, e-mail: axis-user-help@ws.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
--
Chad La Joie 2052-C Harris Bldg
OIS-Middleware 202.687.0124
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: End-to-End Streaming Support
Posted by Martin Gainty <mg...@hotmail.com>.
Chad-
By streaming your definition is not necessarily read or write (XML | Text)
streams but something closer to RSS Feed I would look at tweaking
axis2/databinding/utils/reader/ADBXMLStreamReaderTest.java specifically take
a look at newDocument method
notice the wrap of ByteArrayInputStream("SomePreviouslyReadByteArray");
and use that for your InputStream
you'll have to think thru how much to read (is this entity RLE?)
(Please confirm whether your stream is an RSS or using some other Streaming
mechanism)
HTH/
Martin
This email message and any files transmitted with it contain confidential
information intended only for the person(s) to whom this email message is
addressed. If you have received this email message in error, please notify
the sender immediately by telephone or email and destroy the original
message without making a copy. Thank you.
----- Original Message -----
From: "Chad La Joie" <la...@georgetown.edu>
To: <ax...@ws.apache.org>
Sent: Sunday, August 05, 2007 1:08 PM
Subject: Re: End-to-End Streaming Support
> Thilina Gunarathne wrote:
>>> Does Axis, or Axiom, handle the Base64 encoding/decoding of the binary
>>> data to/from a DataHandler?
>> Yes... Only if needed...
>>
>>> If so, is there a way to disable such handling?
>> Nope... But if you use the API properly you can avoid any unncessary
>> encoding/decodings.
>>
>>> Characteristics for my service have a client submitting large files and
>>> then some number of clients retrieving them. The service itself treats
>>> these files as opaque blobs. It could receive, and return, them to Axis
>>> in a Base64 encoded form saving a lot of time when clients retrieved a
>>> document be avoiding a re-encode.
>> Above is not clear to me. Is your service written in Axis2. Do you
>> want to send/receive binary in base64 form.. If so it won't tolerate
>> large files..
>
> I'm not using anything yet. My hope was to use Axis2 so right now I'm
> just trying to make sure it supports what I'll need. I know that the
> need to send large files, encoded as base64 element content, is not
> something that most SOAP stacks handle. This is why I started by asking
> about true end-to-end streaming support when using MTOM.
>
> If, as we discussed in the first part of this thread, Axis2/Axiom
> supports true end-to-end streaming, and Axiom can correctly stream
> content from/into a DataHandler, what causes problems for arbitrarily
> large files?
>
> Maybe I misunderstood something in the first part of the thread or
> didn't properly state what I was trying to accomplish.
> --
> Chad La Joie 2052-C Harris Bldg
> OIS-Middleware 202.687.0124
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: End-to-End Streaming Support
Posted by Chad La Joie <la...@georgetown.edu>.
Thilina Gunarathne wrote:
>> Does Axis, or Axiom, handle the Base64 encoding/decoding of the binary
>> data to/from a DataHandler?
> Yes... Only if needed...
>
>> If so, is there a way to disable such handling?
> Nope... But if you use the API properly you can avoid any unncessary
> encoding/decodings.
>
>> Characteristics for my service have a client submitting large files and
>> then some number of clients retrieving them. The service itself treats
>> these files as opaque blobs. It could receive, and return, them to Axis
>> in a Base64 encoded form saving a lot of time when clients retrieved a
>> document be avoiding a re-encode.
> Above is not clear to me. Is your service written in Axis2. Do you
> want to send/receive binary in base64 form.. If so it won't tolerate
> large files..
I'm not using anything yet. My hope was to use Axis2 so right now I'm
just trying to make sure it supports what I'll need. I know that the
need to send large files, encoded as base64 element content, is not
something that most SOAP stacks handle. This is why I started by asking
about true end-to-end streaming support when using MTOM.
If, as we discussed in the first part of this thread, Axis2/Axiom
supports true end-to-end streaming, and Axiom can correctly stream
content from/into a DataHandler, what causes problems for arbitrarily
large files?
Maybe I misunderstood something in the first part of the thread or
didn't properly state what I was trying to accomplish.
--
Chad La Joie 2052-C Harris Bldg
OIS-Middleware 202.687.0124
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: End-to-End Streaming Support
Posted by Thilina Gunarathne <cs...@gmail.com>.
> Does Axis, or Axiom, handle the Base64 encoding/decoding of the binary
> data to/from a DataHandler?
Yes... Only if needed...
> If so, is there a way to disable such handling?
Nope... But if you use the API properly you can avoid any unncessary
encoding/decodings.
> Characteristics for my service have a client submitting large files and
> then some number of clients retrieving them. The service itself treats
> these files as opaque blobs. It could receive, and return, them to Axis
> in a Base64 encoded form saving a lot of time when clients retrieved a
> document be avoiding a re-encode.
Above is not clear to me. Is your service written in Axis2. Do you
want to send/receive binary in base64 form.. If so it won't tolerate
large files..
thanks,
Thilina
>
> Chad La Joie wrote:
> > I'm working on a project that may (read: almost certainly will) require
> > very large binary blobs (many GBs) to be transported. The specification
> > I'm working with requires at least SOAP over HTTP support and allows for
> > the use of MTOM.
> >
> > My question is, does Axis 2 (client and service side) support true
> > end-to-end streaming. By this I mean can a service stream data from
> > disk, over HTTP to the client and provide a stream on the client side
> > which can be dumped direct to disk?
> >
> > I had read the MTOM guide [1] and while I see data handlers and sources
> > being used it doesn't say whether Axis 2 is architected in such a way
> > that the data is never buffered in memory.
> >
> > Thanks for the help.
> >
> >
> > [1] http://ws.apache.org/axis2/1_0/mtom-guide.html
>
> --
> Chad La Joie 2052-C Harris Bldg
> OIS-Middleware 202.687.0124
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>
--
Thilina Gunarathne - http://www.wso2.com - http://thilinag.blogspot.com
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: End-to-End Streaming Support
Posted by Chad La Joie <la...@georgetown.edu>.
Sorry, I have one more question related to this topic.
Does Axis, or Axiom, handle the Base64 encoding/decoding of the binary
data to/from a DataHandler? If so, is there a way to disable such handling?
Characteristics for my service have a client submitting large files and
then some number of clients retrieving them. The service itself treats
these files as opaque blobs. It could receive, and return, them to Axis
in a Base64 encoded form saving a lot of time when clients retrieved a
document be avoiding a re-encode.
Chad La Joie wrote:
> I'm working on a project that may (read: almost certainly will) require
> very large binary blobs (many GBs) to be transported. The specification
> I'm working with requires at least SOAP over HTTP support and allows for
> the use of MTOM.
>
> My question is, does Axis 2 (client and service side) support true
> end-to-end streaming. By this I mean can a service stream data from
> disk, over HTTP to the client and provide a stream on the client side
> which can be dumped direct to disk?
>
> I had read the MTOM guide [1] and while I see data handlers and sources
> being used it doesn't say whether Axis 2 is architected in such a way
> that the data is never buffered in memory.
>
> Thanks for the help.
>
>
> [1] http://ws.apache.org/axis2/1_0/mtom-guide.html
--
Chad La Joie 2052-C Harris Bldg
OIS-Middleware 202.687.0124
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: End-to-End Streaming Support
Posted by Thilina Gunarathne <cs...@gmail.com>.
Not sure whether it's mentioned in that thread.. But you can use the
getIncomingAttachmentStreams() method of the Axiom Attachments
class[1] to access the underlying attachment streams too...
thanks,
Thilina
[1] http://ws.apache.org/commons/axiom/apidocs/org/apache/axiom/attachments/Attachments.html
> Great, that's what I was hoping to hear. Thanks to both of you.
>
> Thilina Gunarathne wrote:
> > Yes..
> >
> > On 8/4/07, Chad La Joie <la...@georgetown.edu> wrote:
> >> Thanks Thilina & Dims.
> >>
> >> Just one other question then. The thread that Dims pointed me at is
> >> about SwA. Does the same discussion apply to Axis's MTOM support? The
> >> spec I'm following allows MTOM but not SwA.
> >>
> >>
> >> Thilina Gunarathne wrote:
> >>>>>> My question is, does Axis 2 (client and service side) support true
> >>>>>> end-to-end streaming. By this I mean can a service stream data from
> >>>>>> disk, over HTTP to the client and provide a stream on the client side
> >>>>>> which can be dumped direct to disk?
> >>> Yes... You can use the attachment streaming capability as mentioned in
> >>> the above thread.
> >>>
> >>> You can also use the Axis2 file caching in a smart way if your
> >>> requirement is only to dump the file in to the disk in the client
> >>> side. Axis2 file caching dumps the received bytes directly to the disk
> >>> if the receiving attachment is larger than a certain limit.
> >>>
> >>> You can manipulate the cached files in the following way by adding some
> >>> code to the client side business logic (Make sure to make the file size
> >>> threshold 0).
> >>>
> >>> DataHandler dataHandler = (DataHandler)omText.getDataHandler();
> >>> // You can check the data source type for added safety
> >>> FileDataSource fileDataSource = (FileDataSource)dataHandler.getDataSource();
> >>> File file = fileDataSource.getFile();
> >>> //Rename to your desired file name
> >>> file.renameTo(new File("MyFile.bin"));
> >>>
> >>> Make sure to increase the socket time out and also enable HTTP
> >>> Chunking when sending larger data..
> >>>
> >>> thanks,
> >>> Thilina
> >>>
> >>>>>> I had read the MTOM guide [1] and while I see data handlers and sources
> >>>>>> being used it doesn't say whether Axis 2 is architected in such a way
> >>>>>> that the data is never buffered in memory.
> >>>>>>
> >>>>>> Thanks for the help.
> >>>>>>
> >>>>>>
> >>>>>> [1] http://ws.apache.org/axis2/1_0/mtom-guide.html
> >>>>> --
> >>>>> Chad La Joie 2052-C Harris Bldg
> >>>>> OIS-Middleware 202.687.0124
> >>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> >>>>> For additional commands, e-mail: axis-user-help@ws.apache.org
> >>>>>
> >>>>>
> >>>> --
> >>>> Davanum Srinivas :: http://davanum.wordpress.com
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> >>>> For additional commands, e-mail: axis-user-help@ws.apache.org
> >>>>
> >>>>
> >>>
> >> --
> >> Chad La Joie 2052-C Harris Bldg
> >> OIS-Middleware 202.687.0124
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> >> For additional commands, e-mail: axis-user-help@ws.apache.org
> >>
> >>
> >
> >
>
> --
> Chad La Joie 2052-C Harris Bldg
> OIS-Middleware 202.687.0124
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>
--
Thilina Gunarathne - http://www.wso2.com - http://thilinag.blogspot.com
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: End-to-End Streaming Support
Posted by Chad La Joie <la...@georgetown.edu>.
Great, that's what I was hoping to hear. Thanks to both of you.
Thilina Gunarathne wrote:
> Yes..
>
> On 8/4/07, Chad La Joie <la...@georgetown.edu> wrote:
>> Thanks Thilina & Dims.
>>
>> Just one other question then. The thread that Dims pointed me at is
>> about SwA. Does the same discussion apply to Axis's MTOM support? The
>> spec I'm following allows MTOM but not SwA.
>>
>>
>> Thilina Gunarathne wrote:
>>>>>> My question is, does Axis 2 (client and service side) support true
>>>>>> end-to-end streaming. By this I mean can a service stream data from
>>>>>> disk, over HTTP to the client and provide a stream on the client side
>>>>>> which can be dumped direct to disk?
>>> Yes... You can use the attachment streaming capability as mentioned in
>>> the above thread.
>>>
>>> You can also use the Axis2 file caching in a smart way if your
>>> requirement is only to dump the file in to the disk in the client
>>> side. Axis2 file caching dumps the received bytes directly to the disk
>>> if the receiving attachment is larger than a certain limit.
>>>
>>> You can manipulate the cached files in the following way by adding some
>>> code to the client side business logic (Make sure to make the file size
>>> threshold 0).
>>>
>>> DataHandler dataHandler = (DataHandler)omText.getDataHandler();
>>> // You can check the data source type for added safety
>>> FileDataSource fileDataSource = (FileDataSource)dataHandler.getDataSource();
>>> File file = fileDataSource.getFile();
>>> //Rename to your desired file name
>>> file.renameTo(new File("MyFile.bin"));
>>>
>>> Make sure to increase the socket time out and also enable HTTP
>>> Chunking when sending larger data..
>>>
>>> thanks,
>>> Thilina
>>>
>>>>>> I had read the MTOM guide [1] and while I see data handlers and sources
>>>>>> being used it doesn't say whether Axis 2 is architected in such a way
>>>>>> that the data is never buffered in memory.
>>>>>>
>>>>>> Thanks for the help.
>>>>>>
>>>>>>
>>>>>> [1] http://ws.apache.org/axis2/1_0/mtom-guide.html
>>>>> --
>>>>> Chad La Joie 2052-C Harris Bldg
>>>>> OIS-Middleware 202.687.0124
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
>>>>> For additional commands, e-mail: axis-user-help@ws.apache.org
>>>>>
>>>>>
>>>> --
>>>> Davanum Srinivas :: http://davanum.wordpress.com
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
>>>> For additional commands, e-mail: axis-user-help@ws.apache.org
>>>>
>>>>
>>>
>> --
>> Chad La Joie 2052-C Harris Bldg
>> OIS-Middleware 202.687.0124
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
>> For additional commands, e-mail: axis-user-help@ws.apache.org
>>
>>
>
>
--
Chad La Joie 2052-C Harris Bldg
OIS-Middleware 202.687.0124
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: End-to-End Streaming Support
Posted by Thilina Gunarathne <cs...@gmail.com>.
Yes..
On 8/4/07, Chad La Joie <la...@georgetown.edu> wrote:
> Thanks Thilina & Dims.
>
> Just one other question then. The thread that Dims pointed me at is
> about SwA. Does the same discussion apply to Axis's MTOM support? The
> spec I'm following allows MTOM but not SwA.
>
>
> Thilina Gunarathne wrote:
> >>>> My question is, does Axis 2 (client and service side) support true
> >>>> end-to-end streaming. By this I mean can a service stream data from
> >>>> disk, over HTTP to the client and provide a stream on the client side
> >>>> which can be dumped direct to disk?
> > Yes... You can use the attachment streaming capability as mentioned in
> > the above thread.
> >
> > You can also use the Axis2 file caching in a smart way if your
> > requirement is only to dump the file in to the disk in the client
> > side. Axis2 file caching dumps the received bytes directly to the disk
> > if the receiving attachment is larger than a certain limit.
> >
> > You can manipulate the cached files in the following way by adding some
> > code to the client side business logic (Make sure to make the file size
> > threshold 0).
> >
> > DataHandler dataHandler = (DataHandler)omText.getDataHandler();
> > // You can check the data source type for added safety
> > FileDataSource fileDataSource = (FileDataSource)dataHandler.getDataSource();
> > File file = fileDataSource.getFile();
> > //Rename to your desired file name
> > file.renameTo(new File("MyFile.bin"));
> >
> > Make sure to increase the socket time out and also enable HTTP
> > Chunking when sending larger data..
> >
> > thanks,
> > Thilina
> >
> >>>> I had read the MTOM guide [1] and while I see data handlers and sources
> >>>> being used it doesn't say whether Axis 2 is architected in such a way
> >>>> that the data is never buffered in memory.
> >>>>
> >>>> Thanks for the help.
> >>>>
> >>>>
> >>>> [1] http://ws.apache.org/axis2/1_0/mtom-guide.html
> >>> --
> >>> Chad La Joie 2052-C Harris Bldg
> >>> OIS-Middleware 202.687.0124
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> >>> For additional commands, e-mail: axis-user-help@ws.apache.org
> >>>
> >>>
> >>
> >> --
> >> Davanum Srinivas :: http://davanum.wordpress.com
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> >> For additional commands, e-mail: axis-user-help@ws.apache.org
> >>
> >>
> >
> >
>
> --
> Chad La Joie 2052-C Harris Bldg
> OIS-Middleware 202.687.0124
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>
--
Thilina Gunarathne - http://www.wso2.com - http://thilinag.blogspot.com
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: End-to-End Streaming Support
Posted by Chad La Joie <la...@georgetown.edu>.
Thanks Thilina & Dims.
Just one other question then. The thread that Dims pointed me at is
about SwA. Does the same discussion apply to Axis's MTOM support? The
spec I'm following allows MTOM but not SwA.
Thilina Gunarathne wrote:
>>>> My question is, does Axis 2 (client and service side) support true
>>>> end-to-end streaming. By this I mean can a service stream data from
>>>> disk, over HTTP to the client and provide a stream on the client side
>>>> which can be dumped direct to disk?
> Yes... You can use the attachment streaming capability as mentioned in
> the above thread.
>
> You can also use the Axis2 file caching in a smart way if your
> requirement is only to dump the file in to the disk in the client
> side. Axis2 file caching dumps the received bytes directly to the disk
> if the receiving attachment is larger than a certain limit.
>
> You can manipulate the cached files in the following way by adding some
> code to the client side business logic (Make sure to make the file size
> threshold 0).
>
> DataHandler dataHandler = (DataHandler)omText.getDataHandler();
> // You can check the data source type for added safety
> FileDataSource fileDataSource = (FileDataSource)dataHandler.getDataSource();
> File file = fileDataSource.getFile();
> //Rename to your desired file name
> file.renameTo(new File("MyFile.bin"));
>
> Make sure to increase the socket time out and also enable HTTP
> Chunking when sending larger data..
>
> thanks,
> Thilina
>
>>>> I had read the MTOM guide [1] and while I see data handlers and sources
>>>> being used it doesn't say whether Axis 2 is architected in such a way
>>>> that the data is never buffered in memory.
>>>>
>>>> Thanks for the help.
>>>>
>>>>
>>>> [1] http://ws.apache.org/axis2/1_0/mtom-guide.html
>>> --
>>> Chad La Joie 2052-C Harris Bldg
>>> OIS-Middleware 202.687.0124
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
>>> For additional commands, e-mail: axis-user-help@ws.apache.org
>>>
>>>
>>
>> --
>> Davanum Srinivas :: http://davanum.wordpress.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
>> For additional commands, e-mail: axis-user-help@ws.apache.org
>>
>>
>
>
--
Chad La Joie 2052-C Harris Bldg
OIS-Middleware 202.687.0124
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: End-to-End Streaming Support
Posted by Thilina Gunarathne <cs...@gmail.com>.
> > > My question is, does Axis 2 (client and service side) support true
> > > end-to-end streaming. By this I mean can a service stream data from
> > > disk, over HTTP to the client and provide a stream on the client side
> > > which can be dumped direct to disk?
Yes... You can use the attachment streaming capability as mentioned in
the above thread.
You can also use the Axis2 file caching in a smart way if your
requirement is only to dump the file in to the disk in the client
side. Axis2 file caching dumps the received bytes directly to the disk
if the receiving attachment is larger than a certain limit.
You can manipulate the cached files in the following way by adding some
code to the client side business logic (Make sure to make the file size
threshold 0).
DataHandler dataHandler = (DataHandler)omText.getDataHandler();
// You can check the data source type for added safety
FileDataSource fileDataSource = (FileDataSource)dataHandler.getDataSource();
File file = fileDataSource.getFile();
//Rename to your desired file name
file.renameTo(new File("MyFile.bin"));
Make sure to increase the socket time out and also enable HTTP
Chunking when sending larger data..
thanks,
Thilina
> > >
> > > I had read the MTOM guide [1] and while I see data handlers and sources
> > > being used it doesn't say whether Axis 2 is architected in such a way
> > > that the data is never buffered in memory.
> > >
> > > Thanks for the help.
> > >
> > >
> > > [1] http://ws.apache.org/axis2/1_0/mtom-guide.html
> >
> > --
> > Chad La Joie 2052-C Harris Bldg
> > OIS-Middleware 202.687.0124
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > For additional commands, e-mail: axis-user-help@ws.apache.org
> >
> >
>
>
> --
> Davanum Srinivas :: http://davanum.wordpress.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>
--
Thilina Gunarathne - http://www.wso2.com - http://thilinag.blogspot.com
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: End-to-End Streaming Support
Posted by Davanum Srinivas <da...@gmail.com>.
Please check the archives, example:
http://marc.info/?t=118353301600006&r=1&w=2
thanks,
dims
On 8/3/07, Chad La Joie <la...@georgetown.edu> wrote:
> After thinking about this a bit more I realized that I didn't provide
> enough information to be able to answer this as external things, outside
> of Axis, will impact this, specifically I'm thinking the DataSource and
> the Servlet container Axis is deployed in.
>
> In way of DataSources, it will be a file based data source and hence
> able to report the number of bytes available to the stream and I can
> deploy the application in any container, the environment is not set.
>
> So, I guess my question is expanded a bit. Does Axis support streaming
> and if so, are there particular containers that support Axis's support
> for streaming?
>
> If there are other factors/constraints that I'm missing let me know.
> Thanks.
>
> Chad La Joie wrote:
> > I'm working on a project that may (read: almost certainly will) require
> > very large binary blobs (many GBs) to be transported. The specification
> > I'm working with requires at least SOAP over HTTP support and allows for
> > the use of MTOM.
> >
> > My question is, does Axis 2 (client and service side) support true
> > end-to-end streaming. By this I mean can a service stream data from
> > disk, over HTTP to the client and provide a stream on the client side
> > which can be dumped direct to disk?
> >
> > I had read the MTOM guide [1] and while I see data handlers and sources
> > being used it doesn't say whether Axis 2 is architected in such a way
> > that the data is never buffered in memory.
> >
> > Thanks for the help.
> >
> >
> > [1] http://ws.apache.org/axis2/1_0/mtom-guide.html
>
> --
> Chad La Joie 2052-C Harris Bldg
> OIS-Middleware 202.687.0124
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>
--
Davanum Srinivas :: http://davanum.wordpress.com
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: End-to-End Streaming Support
Posted by Chad La Joie <la...@georgetown.edu>.
After thinking about this a bit more I realized that I didn't provide
enough information to be able to answer this as external things, outside
of Axis, will impact this, specifically I'm thinking the DataSource and
the Servlet container Axis is deployed in.
In way of DataSources, it will be a file based data source and hence
able to report the number of bytes available to the stream and I can
deploy the application in any container, the environment is not set.
So, I guess my question is expanded a bit. Does Axis support streaming
and if so, are there particular containers that support Axis's support
for streaming?
If there are other factors/constraints that I'm missing let me know.
Thanks.
Chad La Joie wrote:
> I'm working on a project that may (read: almost certainly will) require
> very large binary blobs (many GBs) to be transported. The specification
> I'm working with requires at least SOAP over HTTP support and allows for
> the use of MTOM.
>
> My question is, does Axis 2 (client and service side) support true
> end-to-end streaming. By this I mean can a service stream data from
> disk, over HTTP to the client and provide a stream on the client side
> which can be dumped direct to disk?
>
> I had read the MTOM guide [1] and while I see data handlers and sources
> being used it doesn't say whether Axis 2 is architected in such a way
> that the data is never buffered in memory.
>
> Thanks for the help.
>
>
> [1] http://ws.apache.org/axis2/1_0/mtom-guide.html
--
Chad La Joie 2052-C Harris Bldg
OIS-Middleware 202.687.0124
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org