You are viewing a plain text version of this content. The canonical link for it is here.
Posted to soap-user@ws.apache.org by Simon Wail <si...@comview.com> on 2001/03/08 00:54:54 UTC
RPC calls with LARGE attachments
If I wish to send large attachments (many MBs) in my RPC calls, is the data
read into memory before being sent over the wire. I'm currently using the
ByteArrayDataSource with a File parameter. Looking at the code it seems the
ByteArrayDataSource reads the complete file into memory which I want to avoid.
If I was to create a new DataSource object that implemented the getInputStream
method correctly, would the rest of the SOAP toolkit use this input stream to
send the data over the wire, or would it some how read all the data into memory
before streaming - highly undesirable. Obviously on the server side I'd like
the same mechanism to obtain an input stream for the attachments and avoid
reading the data into memory before writing to a file. This might not be
possible as I assume all the MIME headers are read and parsed before control is
passed to the RPC called method. Alternatively, on the server, the SOAP root
part could be read, and then it is up to the called method to access each
attachment in the correct order - goes back to my previous problem of the
called method having no access to the SOAP context, let alone the MIME
attachments.
Thanks,
Simon.
-------------------------------------------------------------------------------
| Dr. Simon Wail | ComView Corporation - Research and Development |
| 220 White Plains Rd | Internet: simon.wail@comview.COM |
| Tarrytown NY USA 10591 | (TEL) +1 914 332-4800 (FAX) +1 914 206-3566 |
|=============================================================================|
| "The Truth is out there... Time is an illusion, lunch-time doubly so" |
| Chris Carter Douglas Adams |
-------------------------------------------------------------------------------
Re: RPC calls with LARGE attachments
Posted by Wouter Cloetens <wo...@mind.be>.
On Wed, Mar 07, 2001 at 06:54:54PM -0500, Simon Wail wrote:
> If I wish to send large attachments (many MBs) in my RPC calls, is the data
> read into memory before being sent over the wire.
Yes.
> I'm currently using the
> ByteArrayDataSource with a File parameter. Looking at the code it seems the
> ByteArrayDataSource reads the complete file into memory which I want to avoid.
> If I was to create a new DataSource object that implemented the getInputStream
> method correctly, would the rest of the SOAP toolkit use this input stream to
> send the data over the wire, or would it some how read all the data into memory
> before streaming - highly undesirable.
The latter, I'm afraid. JavaMail will do that.
> Obviously on the server side I'd like
> the same mechanism to obtain an input stream for the attachments and avoid
> reading the data into memory before writing to a file. This might not be
> possible as I assume all the MIME headers are read and parsed before control is
> passed to the RPC called method.
Well, no, it is possible. In theory, nothing forces us to read the attachments'
content before the transport layer is sending off the message.
> Alternatively, on the server, the SOAP root
> part could be read, and then it is up to the called method to access each
> attachment in the correct order - goes back to my previous problem of the
> called method having no access to the SOAP context, let alone the MIME
> attachments.
Correct. I discussed this as a performance issue several months ago on this
mailing list. Someone else recently offered a mechanism to store attachments
on the filesystem while the message is being built, rather than in memory.
Thought that's not always desirable, it's certainly an interesting option
too.
The big hurdle is that we should ditch JavaMail and implement generic Java
MIME support, light-weight, not focused on e-mail, and with low memory
consumption (by delaying reading from the input stream during serialisation,
and providing an InputStream implementation that delays reading MIME part
content from the transport input stream (e.g. ServletInputStream for HTTP)
until its read method is invoked).
Alas, I have time nor energy (nor need at the moment anymore)... I'm booked
for the next 5 months, but after that, if you'll pay my employer for 2
weeks consultancy fee, I'll gladly do it. ;-)
Re: RPC calls with LARGE attachments
Posted by Wouter Cloetens <wo...@mind.be>.
On Wed, Mar 07, 2001 at 06:54:54PM -0500, Simon Wail wrote:
> If I wish to send large attachments (many MBs) in my RPC calls, is the data
> read into memory before being sent over the wire.
Yes.
> I'm currently using the
> ByteArrayDataSource with a File parameter. Looking at the code it seems the
> ByteArrayDataSource reads the complete file into memory which I want to avoid.
> If I was to create a new DataSource object that implemented the getInputStream
> method correctly, would the rest of the SOAP toolkit use this input stream to
> send the data over the wire, or would it some how read all the data into memory
> before streaming - highly undesirable.
The latter, I'm afraid. JavaMail will do that.
> Obviously on the server side I'd like
> the same mechanism to obtain an input stream for the attachments and avoid
> reading the data into memory before writing to a file. This might not be
> possible as I assume all the MIME headers are read and parsed before control is
> passed to the RPC called method.
Well, no, it is possible. In theory, nothing forces us to read the attachments'
content before the transport layer is sending off the message.
> Alternatively, on the server, the SOAP root
> part could be read, and then it is up to the called method to access each
> attachment in the correct order - goes back to my previous problem of the
> called method having no access to the SOAP context, let alone the MIME
> attachments.
Correct. I discussed this as a performance issue several months ago on this
mailing list. Someone else recently offered a mechanism to store attachments
on the filesystem while the message is being built, rather than in memory.
Thought that's not always desirable, it's certainly an interesting option
too.
The big hurdle is that we should ditch JavaMail and implement generic Java
MIME support, light-weight, not focused on e-mail, and with low memory
consumption (by delaying reading from the input stream during serialisation,
and providing an InputStream implementation that delays reading MIME part
content from the transport input stream (e.g. ServletInputStream for HTTP)
until its read method is invoked).
Alas, I have time nor energy (nor need at the moment anymore)... I'm booked
for the next 5 months, but after that, if you'll pay my employer for 2
weeks consultancy fee, I'll gladly do it. ;-)