Xerces Exception with very large XML file

[Michael Schmidt <m....@web.de>]

Hi everybody,

I've installed the latest (stable) version of Xerces, i.e. libxerces-x.so.27. My OS is Ubuntu Linux 6.06LTS. I've built from sources and installation seems to be fine. I have two questions:

1.) Xerces causes a runtime exception when executed on very large XML files (5GB):

Fatal Error at file , line 0, char 0
  Message: An exception occurred! Type:RuntimeException, Message:The primary document entity could not be opened. Id=/home/michael/research/cvs/cwprojection/experiments/xquery_xmark/xml/auction5000.xml

I had similar problems when using a self-written XML parser, but easily fixed the problem by using fopen64() instead of fopen() to open the input document. Is there any compile time option I can use to support these large files? Or does anybody know how to fix this problem? I did not yet try to change xerces source code...

2.) Xerces by default uses the "-O" compile time option to generate the librarises. Is there any benefit when using "-O3" instead?

Help would be appreciated
Michael
_______________________________________________________________________
Viren-Scan für Ihren PC! Jetzt für jeden. Sofort, online und kostenlos.
Gleich testen! http://www.pc-sicherheit.web.de/freescan/?mc=022222

Re: Xerces Exception with very large XML file

[Boris Kolpackov <bo...@codesynthesis.com>]

Hi Michael,

Michael Schmidt <m....@web.de> writes:

> I had similar problems when using a self-written XML parser, but
> easily fixed the problem by using fopen64() instead of fopen()
> to open the input document. Is there any compile time option I
> can use to support these large files? Or does anybody know how
> to fix this problem? I did not yet try to change xerces source
> code...

You can probably fix it without changing Xerces-C++ source by
implementing your own InputSource and InputStream that use
fopen64 to open the file.


hth,
-boris


-- 
Boris Kolpackov
Code Synthesis Tools CC
http://www.codesynthesis.com
Open-Source, Cross-Platform C++ XML Data Binding

Re: Xerces Exception with very large XML file

[Alberto Massari <am...@datadirect.com>]

Hi Michael,

At 14.20 08/03/2007 +0100, Michael Schmidt wrote:
>Hi everybody,
>
>I've installed the latest (stable) version of Xerces, i.e. 
>libxerces-x.so.27. My OS is Ubuntu Linux 6.06LTS. I've built from 
>sources and installation seems to be fine. I have two questions:
>
>1.) Xerces causes a runtime exception when executed on very large 
>XML files (5GB):
>
>Fatal Error at file , line 0, char 0
>   Message: An exception occurred! Type:RuntimeException, 
> Message:The primary document entity could not be opened. 
> Id=/home/michael/research/cvs/cwprojection/experiments/xquery_xmark/xml/auction5000.xml
>
>I had similar problems when using a self-written XML parser, but 
>easily fixed the problem by using fopen64() instead of fopen() to 
>open the input document. Is there any compile time option I can use 
>to support these large files? Or does anybody know how to fix this 
>problem? I did not yet try to change xerces source code...

If you want to try the fopen64 call, you should modify the 
XMLPlatformUtils::openFile and XMLPlatformUtils::openFileToWrite 
methods in <xerces-dir>/src/xercesc/util/Platforms/Linux/LinuxPlatformUtils.cpp

>2.) Xerces by default uses the "-O" compile time option to generate 
>the librarises. Is there any benefit when using "-O3" instead?

I am not an expert of gcc and Linx, but I think somebody was worried 
of aggressive optimizations; nothing stops you from editing 
Makefile.incl to try a different set of flags.

Hope this helps,
Alberto