You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@axis.apache.org by Tammy Dugan <td...@regenstrief.org> on 2007/06/06 20:45:15 UTC
Re: MTOM error "java.lang.OutOfMemoryError: Java heap space"
Just following up on this bug. I really can't switch over to Axis2 until
this is working so I would appreciate any help you could give me.
Thanks,
Tammy
Tammy Dugan wrote:
> I submitted bug Axis2-2743. Here is an excerpt from the JIRA:
>
> When I try to return 270 MB of xml from an axis2 service, I get an
> OutOfMemory error. I wrote my own CustomDataSource and the error
> occurs in the following serialize method:
>
> public void serialize(XMLStreamWriter xmlWriter) throws
> XMLStreamException
> {
> XMLStreamReader reader = null;
>
> StreamingOMSerializer serializer = new StreamingOMSerializer();
> reader = getReader();
> System.out.println("before out of memory error");
> serializer.serialize(reader, xmlWriter);// OutOfMemory error here
> System.out.println("after out of memory error");
> xmlWriter.flush();
> }
>
> If I run the serialize locally and create my own xmlWriter, there is
> no error. However, if the serialize gets called from an axis2 service,
> an OutOfMemory error occurs. Because of this, I really think that
> axis2 is using a writer as input to XMLStreamWriter that is buffering
> all the data. It shouldn't be doing that.
>
>
> Thanks,
>
> Tammy
>
> Davanum Srinivas wrote:
>> Tammy,
>>
>> Can you please create a JIRA and upload your sample test case? We need
>> something concrete to recreate the bug on our side. No, you cannot use
>> OutputStream directly as the data has to be in xml.
>>
>> thanks,
>> dims
>>
>> On 5/30/07, Tammy Dugan <td...@regenstrief.org> wrote:
>>> 2007-05-29 16:59:18,828 INFO - (Thread[http-8080-Processor3,5,main])
>>> End
>>> soap service //this is my logging at the end of the service
>>> 2007-05-29 17:00:19,906 ERROR - java.lang.OutOfMemoryError
>>>
>>> I'm not sure which http version I am using. Where do I find it? I am
>>> using Tomcat 4.1.
>>>
>>> Instead of messing with the XMLStreamWriter, is there a way to get
>>> axis2
>>> to use this message signature from the OMDataSource interface:
>>>
>>> public void serialize(OutputStream output, OMOutputFormat format)
>>> throws
>>> XMLStreamException
>>>
>>> so I can directly stream my input stream to an output stream without
>>> the
>>> apparent problems with XMLStreamWriter serialization?
>>>
>>> Thanks,
>>>
>>> Tammy
>>>
>>> Davanum Srinivas wrote:
>>> > can u please paste the stack trace(s)? Are u using http 1.1 or 1.0?
>>> >
>>> > thx,
>>> > dims
>>> >
>>> > On 5/29/07, Tammy Dugan <td...@regenstrief.org> wrote:
>>> >> I created a OMSourcedElementImpl from an input stream:
>>> >>
>>> >>
>>> >> OMFactory factory = OMAbstractFactory.getOMFactory();
>>> >> OMNamespace ns = new OMNamespaceImpl("", "");
>>> >> OMDataSource dataSource = new CustomDataSource(inputStream);
>>> >> return new OMSourcedElementImpl(localName, ns, factory, dataSource);
>>> >>
>>> >> I also created a custom data source. Here are some snippets of code
>>> >> from it:
>>> >>
>>> >> public class CustomDataSource implements OMDataSource
>>> >> {
>>> >> private final InputStream data;
>>> >>
>>> >> public CustomDataSource(InputStream data)
>>> >> {
>>> >> this.data = data;
>>> >> }
>>> >>
>>> >> public void serialize(XMLStreamWriter xmlWriter) throws
>>> >> XMLStreamException
>>> >> {
>>> >> XMLStreamReader reader = null;
>>> >>
>>> >> StreamingOMSerializer serializer = new
>>> StreamingOMSerializer();
>>> >>
>>> >> reader = getReader();
>>> >>
>>> >> serializer.serialize(reader, xmlWriter);// OutOfMemory error
>>> >> here
>>> >>
>>> >> xmlWriter.flush();
>>> >> }
>>> >>
>>> >> public XMLStreamReader getReader() throws XMLStreamException
>>> >> {
>>> >> return StAXUtils.createXMLStreamReader(this.data);
>>> >> }
>>> >> }
>>> >>
>>> >>
>>> >> As you can see from the comment, I get an OutOfMemory error at
>>> >> serializer.serialize when I use a StreamingOMSerializer. Is there
>>> >> another way I can transfer data from an input stream to an
>>> >> XMLStreamWriter? What in StreamingOMSerializer is buffering the
>>> data?
>>> >>
>>> >>
>>> >>
>>> >> Tammy
>>> >>
>>> >>
>>> >>
>>> >> Dennis Sosnoski wrote:
>>> >> > Hi Paul,
>>> >> >
>>> >> > I'm not sure exactly what you mean with this code. This thread has
>>> >> > been repurposed a time or two, but the question from Tammy was
>>> how to
>>> >> > stream out data from a database as the body of a response. So
>>> the data
>>> >> > is not in the form of an OMElement, and the point of using
>>> >> > OMDataSource is to avoid building it as an OMElement. The data
>>> has to
>>> >> > be supplied as the AXIOM representation is being written to the
>>> output
>>> >> > stream, which is what OMDataSource allows - it's basically a
>>> >> > placeholder that says XML output will be generated on demand.
>>> >> >
>>> >> > If you *do* build the data as a full AXIOM model you'll run out of
>>> >> > memory pretty quickly. AXIOM is relatively fat by comparison
>>> with most
>>> >> > other document models, and from what I've seen takes at least
>>> 10 times
>>> >> > the size of the document in memory usage. So the only practical
>>> way to
>>> >> > handle large documents (say >10 MB) is to stream the data using
>>> >> > OMDataSource or an equivalent.
>>> >> >
>>> >> > Incidentally, note that this also means that anyone using
>>> WS-Security
>>> >> > is going to effectively be limited to document sizes of about
>>> 10 MB or
>>> >> > so. Again based on what I've seen, Rampart always ends up
>>> building the
>>> >> > full AXIOM model even if the only thing you're doing is using a
>>> >> > TimeStamp. I haven't verified this, but I suspect roughly the same
>>> >> > size limit will apply for attachment handling - expanding the tree
>>> >> > will inline the attachment data as base64 text, so the fact
>>> that it's
>>> >> > actually transmitted as a binary blob doesn't help when Rampart is
>>> >> > engaged.
>>> >> >
>>> >> > - Dennis
>>> >> >
>>> >>
>>> >> --
>>> >> Tammy Dugan
>>> >> Computer Programmer
>>> >>
>>> >> Regenstrief Institute, Inc.
>>> >> Medical Informatics
>>> >> Health Information and Translational Sciences (HITS) Building
>>> >> 410 West 10th Street, Suite 2000
>>> >> Indianapolis, IN 46202
>>> >> Main: 317.423.5500
>>> >> Fax: 317.423.5695
>>> >> IU campus mail address: HS, 2000
>>> >>
>>> >> (317) 423 - 5541
>>> >>
>>> >> Confidentiality Notice: The contents of this message and any files
>>> >> transmitted with it may contain confidential and/or privileged
>>> >> information and are intended solely for the use of the named
>>> >> addressee(s). Additionally, the information contained herein may
>>> have
>>> >> been disclosed to you from medical records with confidentiality
>>> >> protected by federal and state laws. Federal regulations and State
>>> >> laws prohibit you from making further disclosure of such information
>>> >> without the specific written consent of the person to whom the
>>> >> information pertains or as otherwise permitted by such
>>> regulations. A
>>> >> general authorization for the release of medical or other
>>> information
>>> >> is not sufficient for this purpose.
>>> >>
>>> >> If you have received this message in error, please notify the sender
>>> >> by return e-mail and delete the original message. Any retention,
>>> >> disclosure, copying, distribution or use of this information by
>>> >> anyone other than the intended recipient is strictly prohibited.
>>> >>
>>> >>
>>> >>
>>> >>
>>> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
>>> >> For additional commands, e-mail: axis-user-help@ws.apache.org
>>> >>
>>> >>
>>> >
>>> >
>>>
>>> --
>>> Tammy Dugan
>>> Computer Programmer
>>>
>>> Regenstrief Institute, Inc.
>>> Medical Informatics
>>> Health Information and Translational Sciences (HITS) Building
>>> 410 West 10th Street, Suite 2000
>>> Indianapolis, IN 46202
>>> Main: 317.423.5500
>>> Fax: 317.423.5695
>>> IU campus mail address: HS, 2000
>>>
>>> (317) 423 - 5541
>>>
>>> Confidentiality Notice: The contents of this message and any files
>>> transmitted with it may contain confidential and/or privileged
>>> information and are intended solely for the use of the named
>>> addressee(s). Additionally, the information contained herein may
>>> have been disclosed to you from medical records with confidentiality
>>> protected by federal and state laws. Federal regulations and State
>>> laws prohibit you from making further disclosure of such information
>>> without the specific written consent of the person to whom the
>>> information pertains or as otherwise permitted by such regulations.
>>> A general authorization for the release of medical or other
>>> information is not sufficient for this purpose.
>>>
>>> If you have received this message in error, please notify the sender
>>> by return e-mail and delete the original message. Any retention,
>>> disclosure, copying, distribution or use of this information by
>>> anyone other than the intended recipient is strictly prohibited.
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
>>> For additional commands, e-mail: axis-user-help@ws.apache.org
>>>
>>>
>>
>>
>
--
Tammy Dugan
Computer Programmer
Regenstrief Institute, Inc.
Medical Informatics
Health Information and Translational Sciences (HITS) Building
410 West 10th Street, Suite 2000
Indianapolis, IN 46202
Main: 317.423.5500
Fax: 317.423.5695
IU campus mail address: HS, 2000
(317) 423 - 5541
Confidentiality Notice: The contents of this message and any files transmitted with it may contain confidential and/or privileged information and are intended solely for the use of the named addressee(s). Additionally, the information contained herein may have been disclosed to you from medical records with confidentiality protected by federal and state laws. Federal regulations and State laws prohibit you from making further disclosure of such information without the specific written consent of the person to whom the information pertains or as otherwise permitted by such regulations. A general authorization for the release of medical or other information is not sufficient for this purpose.
If you have received this message in error, please notify the sender by return e-mail and delete the original message. Any retention, disclosure, copying, distribution or use of this information by anyone other than the intended recipient is strictly prohibited.
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org