core dump when parser->parse'ing a 98Mb file

[Richard Docksey <Ri...@liffe.com>]

Hi,

I have a program which builds a DOM tree from an input
file, it works fine for medium size files for core dumps
when doing a 'new' for an input file of 98Mb in size.

the process gets to 780Mb in size just before it exits,
I wondered if this is a known bug/feature and if there's
anything I can do (other then convert to sax) to avoid it.

(version from $XERCES_ROOT/version.incl is 1_4 and I'm 
running on SunOS 5.6)

thanks in advance,

Richard.


  ---- hidden frames, use 'where -h' to see them all ----
  [41] default_new_handler(0xc, 0x0, 0x4f398, 0x0, 0x0, 0xb0c00), at 0x4aa84
  [42] operator new(0xc, 0xb25c0, 0xbd128, 0x4aa80, 0x0, 0x0), at 0x4a9e8
  [43] NamedNodeMapImpl::setNamedItem(0x3310b908, 0xef1725aa, 0xffffffff,
0x0, 0x3310b908, 0x0), at 0xef0f5e88
  [44] ElementImpl::setAttributeNS(0x3310e188, 0x3310b908, 0xefffe3cc,
0xefffe3c8, 0x3310d400, 0xef15a640), at 0xef0f082c
  [45] DOMParser::startElement(0xc5250, 0xef1725b4, 0xef15fae8, 0x1,
0x3310d400, 0xcb4e8), at 0xef0d767c
  [46] XMLScanner::scanStartTagNS(0xc6700, 0xefffe4e4, 0x3, 0x1, 0x0,
0xc6718), at 0xef11d5b0
  [47] XMLScanner::scanContent(0xc6700, 0x0, 0xef15a640, 0xc67cc, 0x1802c,
0xc6718), at 0xef11b3b0
  [48] XMLScanner::scanDocument(0xc6700, 0xc4e00, 0x0, 0xc6770, 0xc4e00,
0x81), at 0xef119774
  [49] XMLScanner::scanDocument(0xc6700, 0xcb380, 0x0, 0xc4e00, 0xc4e00,
0x20), at 0xef119590
  [50] XMLScanner::scanDocument(0xc6700, 0xcb380, 0x0, 0xef06bf11, 0xc4de8,
0xb0c00), at 0xef11961c
  [51] DOMParser::parse(0xc5250, 0xefffeb80, 0x0, 0xef15a640, 0xc5264,
0xc4dd0), at 0xef0d6844
=>[52] main(argC = 3, argV = 0xefffe92c), line 362 in "HDE.cpp"

(line 362 is 'parser->parse(szFileName);')


----------------------------------------------------------------------
The information contained in this e-mail is confidential and solely for 
the intended addressee(s). Unauthorised reproduction, disclosure, modification, 
and/or distribution of this email may be unlawful. If you have received 
this email in error, please notify the sender immediately and delete it 
from your system. The views expressed in this message do not necessarily 
reflect those of LIFFE (Holdings) Plc or any of its subsidiary companies.
----------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-c-dev-help@xml.apache.org

Re: core dump when parser->parse'ing a 98Mb file

["Jason E. Stewart" <ja...@openinformatics.com>]

"Richard Docksey" <Ri...@liffe.com> writes:

> I have a program which builds a DOM tree from an input file, it
> works fine for medium size files for core dumps when doing a 'new'
> for an input file of 98Mb in size.

DOM is a memory resident image of your entire XML file. So it's going
to be at least as big, right? Then the issue is how many bytes of
storage are required for each node in the DOM tree, that gives you the
multiplicative effect. 

The current DOM is known to be inefficient, hence the new IDOM
implementation. Give it a try instead, but you're not likely to get
away from using a huge amount of memory. Use SAX 1/2 instead.

jas.

---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-c-dev-help@xml.apache.org