You are viewing a plain text version of this content. The canonical link for it is here.
Posted to j-users@xerces.apache.org by "Xue, Sean (MTO)" <Se...@mto.gov.on.ca> on 2002/05/28 15:25:31 UTC
Problem in large xml reading with SAX parser.
Hi all,
Is there anybody has some experience with reading large xml file. I met an
out of memory problem when reading a xml file with more than 1,000,000
records. It always ran into out of memory problem at 180,904 record. I
delete all code except counting code in events. Any suggestion is welcome.
Thanking all in anticipation,
regards,
Sean
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org
Re: Problem in large xml reading with SAX parser.
Posted by Leonardo Holanda <le...@atlantico.com.br>.
Hello Sean.
We have sucessfully read a 6 GB XML file using SAX instance of Xerces. This
XML conforms to IPDR.org master XML Schema
(http://www.ipdr.org/public/IPDRDoc3.0.xsd). In this process, we read a XML
input (IPDRDoc) and save as a flat text file. We did it with a 10,000,000
record XML file (each record is a XML complex type element with about 15 XML
simple type elements) producing a 10,000,000 line txt file (2 GB file size).
Are you sure you're using a SAX parser? We had this problem when using a DOM
parser. The JVM runs out of memory with files grater than
80MB.
Maybe you're loading some elements in memory while traversing the XML
document. So, you have to free those elements every you reach an endElement
in your record scope.
regards,
Leo Holanda
Instituto Atlântico
Fortaleza - CE - Brazil
----- Original Message -----
From: "Xue, Sean (MTO)" <Se...@mto.gov.on.ca>
To: <xe...@xml.apache.org>
Sent: Tuesday, May 28, 2002 10:25 AM
Subject: Problem in large xml reading with SAX parser.
> Hi all,
>
> Is there anybody has some experience with reading large xml file. I met an
> out of memory problem when reading a xml file with more than 1,000,000
> records. It always ran into out of memory problem at 180,904 record. I
> delete all code except counting code in events. Any suggestion is welcome.
>
> Thanking all in anticipation,
> regards,
> Sean
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-j-user-help@xml.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org