You are viewing a plain text version of this content. The canonical link for it is here.
Posted to p-dev@xerces.apache.org by Chris Cheung <ch...@clc.cuhk.edu.hk> on 2003/11/14 06:04:56 UTC
Memory Leak?
Dear all,
I need to use Xerces-Perl to repeatedly
- parse XML to DOM tree
- manipulate the DOM tree
- pretty-print the DOM tree
It seems that serious memory leak occurs. I do not hold any reference to
the Perl Xerces object used in the programme and hence the Perl objects
should be DESTROYed after use, but memory usage keeps on increasing as
reported by the OS.
Is there any solution to this problem?
Thank you very much in advance.
--
Best Regards,
Chris Cheung
Center for Large-Scale Computation
Have a nice day!
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
Re: DOMInputSource vs InputSource (was Re: Memory Leak?)
Posted by "Jason E. Stewart" <ja...@openinformatics.com>.
Brian Faull <bf...@mitre.org> writes:
> > So since I don't know how to do automatic wrapping of the docs, I've
> > been trying to generate as many example files as possible that show
> > how to use the API.
>
> ... often more helpful anwyay! The steal-and-modify programming paradigm
> is much easier than the figure-it-out-yourself technique. :)
;-)
Yup.
> > Ok, will do. What error do you get when you run:
> >
> > svn co http://svn.apache.org/repos/asf/xml/xerces-p/trunk/
>
> svn: RA layer request failed
> svn: REPORT request failed on '/repos/asf/!svn/vcc/default'
> svn: REPORT of '/repos/asf/!svn/vcc/default': 400 Bad Request
> (http://svn.apache.org)
>
> > Are you behind a firewall? If so you need to set up your proxy
> > attributes in ~/.subversion.
>
> Already did... am behind firewall -- modified proxy settings accordingly
> in ~/.subversion/servers which gives the above error. Just tried it
> without proxy settings, gives "could not connect to server" as
> expected...
This sounds like your HTTP proxy is blocking some of the protocol
commands that SVN needs (but web browsers don't). See the SNV FAQ for
details.
> On subversion.tigris.org, it suggests using the form
> $ svn co <URL> <project_name>
Only if you want to rename the directory created by the checkout.
> Did you (or where did you?) post the tarball? (sorry... not trying to be
> impatient, no hurry, just couldn't find it...)
No, I'm still running memory tests to figure out where the problem
lies. Once I post the tarball, I'll message the list.
Cheers,
jas.
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
Re: DOMInputSource vs InputSource (was Re: Memory Leak?)
Posted by Brian Faull <bf...@mitre.org>.
"Jason E. Stewart" wrote:
<snip>
> The issue I have always had is maintenance of static documentation -
> or docs would be based on the Xerces-C API, and if it changes or docs
> need to change. It's one thing if you're documenting your *own* API -
> if you change the code, you change the docs. But in this case it's
> monitoring and changing someone elses docs, and that rubs me the wrong
> way.
Agree completely -- no blame from here!
> Unfortunately, while there are excellent automatic documentation tools
> (like the doxygen system used by Xerces-C), there are no systems (that
> I'm aware of) that will auto-convert from one API (in C++) to another
> API (in Perl), unfortunately SWIG only handles wrapping of the
> code. So, for example, I would want any method in Xerces that requires
> an XMLCh* (their unicode string), to be documented in Perl as simply
> taking a scalar (I do a bit under the hood to ensure that you can pass
> any scalar value, string, int, or float and have it handled properly).
Sounds like a darn good idea. Also sounds like a difficult task, to get
auto-documentation to get the semantics of arguments right in such a
loosely-typed language as Perl...
> So since I don't know how to do automatic wrapping of the docs, I've
> been trying to generate as many example files as possible that show
> how to use the API.
... often more helpful anwyay! The steal-and-modify programming paradigm
is much easier than the figure-it-out-yourself technique. :)
> > > I'll add it and commit it to the SVN repository. You can either grab
> > > it from there or I can make a dev snapshot tarball.
> >
> > Installed and like svn, but don't seem to have the right configuration yet
> > -- can't snarf the repository. If you would post a tarball at your
> > convenience, that would be super.
>
> Ok, will do. What error do you get when you run:
>
> svn co http://svn.apache.org/repos/asf/xml/xerces-p/trunk/
svn: RA layer request failed
svn: REPORT request failed on '/repos/asf/!svn/vcc/default'
svn: REPORT of '/repos/asf/!svn/vcc/default': 400 Bad Request
(http://svn.apache.org)
> Are you behind a firewall? If so you need to set up your proxy
> attributes in ~/.subversion.
Already did... am behind firewall -- modified proxy settings accordingly
in ~/.subversion/servers which gives the above error. Just tried it
without proxy settings, gives "could not connect to server" as expected...
On subversion.tigris.org, it suggests using the form
$ svn co <URL> <project_name>
Or using port 81 rather than 80, or using SSL (https)... also tried
those... no luck...
Did you (or where did you?) post the tarball? (sorry... not trying to be
impatient, no hurry, just couldn't find it...)
Thanks again,
-brian
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
Re: DOMInputSource vs InputSource (was Re: Memory Leak?)
Posted by "Jason E. Stewart" <ja...@openinformatics.com>.
Brian Faull <bf...@mitre.org> writes:
> > Aha! I wondered what that class was for (blush) ...
>
> So did I. It was mind-numbing looking at the API docs for DOMInputSource,
> InputSource, and Wrapper4* classes for both... :-\ I'm no OO expert,
> but... hmmm...
Yes... At least the Xerces-C team is better at documentation than I
have been. Making POD docs for XML-Xerces has been on the TODO list
for, well, ever since the project started!
The issue I have always had is maintenance of static documentation -
or docs would be based on the Xerces-C API, and if it changes or docs
need to change. It's one thing if you're documenting your *own* API -
if you change the code, you change the docs. But in this case it's
monitoring and changing someone elses docs, and that rubs me the wrong
way.
Unfortunately, while there are excellent automatic documentation tools
(like the doxygen system used by Xerces-C), there are no systems (that
I'm aware of) that will auto-convert from one API (in C++) to another
API (in Perl), unfortunately SWIG only handles wrapping of the
code. So, for example, I would want any method in Xerces that requires
an XMLCh* (their unicode string), to be documented in Perl as simply
taking a scalar (I do a bit under the hood to ensure that you can pass
any scalar value, string, int, or float and have it handled properly).
So since I don't know how to do automatic wrapping of the docs, I've
been trying to generate as many example files as possible that show
how to use the API.
> > I'll add it and commit it to the SVN repository. You can either grab
> > it from there or I can make a dev snapshot tarball.
>
> Installed and like svn, but don't seem to have the right configuration yet
> -- can't snarf the repository. If you would post a tarball at your
> convenience, that would be super.
Ok, will do. What error do you get when you run:
svn co http://svn.apache.org/repos/asf/xml/xerces-p/trunk/
Are you behind a firewall? If so you need to set up your proxy
attributes in ~/.subversion.
Cheers,
jas.
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
Re: DOMInputSource vs InputSource (was Re: Memory Leak?)
Posted by Brian Faull <bf...@mitre.org>.
"Jason E. Stewart" wrote:
>
> Brian Faull <bf...@mitre.org> writes:
>
> > Trying to implement the solution you list -- but I need to take
> > input from a local variable. My previous attempt used a
> > XercesDOMParser rather than DOMBuilder... and XercesDOMParser::parse
> > was AbstractDOMParser::parse, which took an InputSource (can be
> > StdIn*, MemBuf*, LocalFile*, ...). DOMBuilder::parse rather takes a
> > DOMInputSource, which seems that it may only be a file or URL
> > natively.
> >
> > There is a handy class called Wrapper4InputSource which isa
> > DOMInputSource which can allow an InputSource to be used as a
> > DOMInputSource. However... Wrapper4InputSource doesn't appear to be
> > available in XML::Xerces. I didn't see any tests with a
> > DOMInputSource used, either... Have I missed something, or...?
>
> Aha! I wondered what that class was for (blush) ...
So did I. It was mind-numbing looking at the API docs for DOMInputSource,
InputSource, and Wrapper4* classes for both... :-\ I'm no OO expert,
but... hmmm...
> The file Xerces.i controls which Xerces-C API classes that I
> wrap. Currently, this is done manually - I just update the file an a
> per-need basis. So if a class' header file is not there, XML-Xerces
> won't know anything about it.
>
> I was just thinking that it would be convenient to know what public
> header files are available and which ones I'm not currently wrapping.
Good to know, and great idea -- I'll view that file in the future.
> I'll add it and commit it to the SVN repository. You can either grab
> it from there or I can make a dev snapshot tarball.
Installed and like svn, but don't seem to have the right configuration yet
-- can't snarf the repository. If you would post a tarball at your
convenience, that would be super.
Thanks a bunch!
-brian
> Cheers,
> jas.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
Re: DOMInputSource vs InputSource (was Re: Memory Leak?)
Posted by "Jason E. Stewart" <ja...@openinformatics.com>.
Brian Faull <bf...@mitre.org> writes:
> Trying to implement the solution you list -- but I need to take
> input from a local variable. My previous attempt used a
> XercesDOMParser rather than DOMBuilder... and XercesDOMParser::parse
> was AbstractDOMParser::parse, which took an InputSource (can be
> StdIn*, MemBuf*, LocalFile*, ...). DOMBuilder::parse rather takes a
> DOMInputSource, which seems that it may only be a file or URL
> natively.
>
> There is a handy class called Wrapper4InputSource which isa
> DOMInputSource which can allow an InputSource to be used as a
> DOMInputSource. However... Wrapper4InputSource doesn't appear to be
> available in XML::Xerces. I didn't see any tests with a
> DOMInputSource used, either... Have I missed something, or...?
Aha! I wondered what that class was for (blush) ...
The file Xerces.i controls which Xerces-C API classes that I
wrap. Currently, this is done manually - I just update the file an a
per-need basis. So if a class' header file is not there, XML-Xerces
won't know anything about it.
I was just thinking that it would be convenient to know what public
header files are available and which ones I'm not currently wrapping.
I'll add it and commit it to the SVN repository. You can either grab
it from there or I can make a dev snapshot tarball.
Cheers,
jas.
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
DOMInputSource vs InputSource (was Re: Memory Leak?)
Posted by Brian Faull <bf...@mitre.org>.
Jason, et al,
Trying to implement the solution you list -- but I need to take input from
a local variable. My previous attempt used a XercesDOMParser rather than
DOMBuilder... and XercesDOMParser::parse was AbstractDOMParser::parse,
which took an InputSource (can be StdIn*, MemBuf*, LocalFile*, ...).
DOMBuilder::parse rather takes a DOMInputSource, which seems that it may
only be a file or URL natively.
There is a handy class called Wrapper4InputSource which isa DOMInputSource
which can allow an InputSource to be used as a DOMInputSource. However...
Wrapper4InputSource doesn't appear to be available in XML::Xerces. I
didn't see any tests with a DOMInputSource used, either... Have I missed
something, or...?
-brian
"Jason E. Stewart" wrote:
>
> Chris Cheung <ch...@clc.cuhk.edu.hk> writes:
>
> > I need to use Xerces-Perl to repeatedly
> >
> > - parse XML to DOM tree
> > - manipulate the DOM tree
> > - pretty-print the DOM tree
> >
> > It seems that serious memory leak occurs. I do not hold any reference to
> > the Perl Xerces object used in the programme and hence the Perl objects
> > should be DESTROYed after use, but memory usage keeps on increasing as
> > reported by the OS.
>
> Hi Chris,
>
> So I think I've solved this. First use DOMBuilder, and second ensure
> that resetDocumentPool() gets called after you are finished with the
> DOM tree. Here's some code stolen from my memtest.pl:
>
> my $impl = XML::Xerces::DOMImplementationRegistry::getDOMImplementation('LS');
> my $parser = $impl->createDOMBuilder($XML::Xerces::DOMImplementationLS::MODE_SYNCHRONOUS,'');
>
> sub validate_builder {
> my $xml = shift ;
> my $parser = shift;
>
> eval {
> $parser->setFeature("$XML::Xerces::XMLUni::fgDOMNamespaces", $namespace) ;
> $parser->setFeature("$XML::Xerces::XMLUni::fgXercesSchema", $schema) ;
> $parser->setFeature("$XML::Xerces::XMLUni::fgXercesSchemaFullChecking", $schema) ;
> # $parser->setFeature("http://apache.org/xml/features/validation-error-as-fatal", $validate) ;
> };
> XML::Xerces::error($@) if $@;
>
> eval {
> $parser->setFeature("$XML::Xerces::XMLUni::fgDOMValidation", $validate) ;
> # $parser->setFeature("$XML::Xerces::XMLUni::fgDOMValidateIfSchema", 0) ;
> };
> XML::Xerces::error($@) if $@;
>
> eval {
> # my $is = XML::Xerces::MemBufInputSource->new($xml);
> # my $is = XML::Xerces::LocalFileInputSource->new($OPTIONS{file});
> $parser->parseURI($OPTIONS{file}) ;
> } ;
> XML::Xerces::error($@) if $@;
> $parser->resetDocumentPool();
> }
>
> Using this code I'm not getting any perceptable leaks with 2.3.0-4.
>
> Let me know if this helps,
> jas.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
--
Brian Faull
Senior Integrated Electronics Engineer
D620 - Communications and Networking
The MITRE Corporation
202 Burlington Road, MS E015
Bedford, MA 01730-1420
V:781.271.5736 F:781.271.8875
mailto:bfaull@mitre.org
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
Re: Memory Leak?
Posted by Chris Cheung <ch...@clc.cuhk.edu.hk>.
Hi Jas
Thanks a lot for your help. I installed XML-Xerces-2.3.0-4
and run the attached test script modified according to your attached
code. I used the Linux tool "top" to monitor the memory usage of the
progress while it is running. However, the memory usage still grow from
9M to 29M for 500 iteration of parsing. Did I do anything wrong?
On 15 Nov 2003, Jason E. Stewart wrote:
> Hi Chris,
>
> So I think I've solved this. First use DOMBuilder, and second ensure
> that resetDocumentPool() gets called after you are finished with the
> DOM tree. Here's some code stolen from my memtest.pl:
>
> my $impl = XML::Xerces::DOMImplementationRegistry::getDOMImplementation('LS');
> my $parser = $impl->createDOMBuilder($XML::Xerces::DOMImplementationLS::MODE_SYNCHRONOUS,'');
>
>
> sub validate_builder {
> my $xml = shift ;
> my $parser = shift;
>
> eval {
> $parser->setFeature("$XML::Xerces::XMLUni::fgDOMNamespaces", $namespace) ;
> $parser->setFeature("$XML::Xerces::XMLUni::fgXercesSchema", $schema) ;
> $parser->setFeature("$XML::Xerces::XMLUni::fgXercesSchemaFullChecking", $schema) ;
> # $parser->setFeature("http://apache.org/xml/features/validation-error-as-fatal", $validate) ;
> };
> XML::Xerces::error($@) if $@;
>
>
> eval {
> $parser->setFeature("$XML::Xerces::XMLUni::fgDOMValidation", $validate) ;
> # $parser->setFeature("$XML::Xerces::XMLUni::fgDOMValidateIfSchema", 0) ;
> };
> XML::Xerces::error($@) if $@;
>
> eval {
> # my $is = XML::Xerces::MemBufInputSource->new($xml);
> # my $is = XML::Xerces::LocalFileInputSource->new($OPTIONS{file});
> $parser->parseURI($OPTIONS{file}) ;
> } ;
> XML::Xerces::error($@) if $@;
> $parser->resetDocumentPool();
> }
>
> Using this code I'm not getting any perceptable leaks with 2.3.0-4.
>
> Let me know if this helps,
> jas.
>
--
Best Regards,
Chris Cheung
Center for Large-Scale Computation
Have a nice day!
Re: Memory Leak?
Posted by "Jason E. Stewart" <ja...@openinformatics.com>.
Chris Cheung <ch...@clc.cuhk.edu.hk> writes:
> I need to use Xerces-Perl to repeatedly
>
> - parse XML to DOM tree
> - manipulate the DOM tree
> - pretty-print the DOM tree
>
> It seems that serious memory leak occurs. I do not hold any reference to
> the Perl Xerces object used in the programme and hence the Perl objects
> should be DESTROYed after use, but memory usage keeps on increasing as
> reported by the OS.
Hi Chris,
So I think I've solved this. First use DOMBuilder, and second ensure
that resetDocumentPool() gets called after you are finished with the
DOM tree. Here's some code stolen from my memtest.pl:
my $impl = XML::Xerces::DOMImplementationRegistry::getDOMImplementation('LS');
my $parser = $impl->createDOMBuilder($XML::Xerces::DOMImplementationLS::MODE_SYNCHRONOUS,'');
sub validate_builder {
my $xml = shift ;
my $parser = shift;
eval {
$parser->setFeature("$XML::Xerces::XMLUni::fgDOMNamespaces", $namespace) ;
$parser->setFeature("$XML::Xerces::XMLUni::fgXercesSchema", $schema) ;
$parser->setFeature("$XML::Xerces::XMLUni::fgXercesSchemaFullChecking", $schema) ;
# $parser->setFeature("http://apache.org/xml/features/validation-error-as-fatal", $validate) ;
};
XML::Xerces::error($@) if $@;
eval {
$parser->setFeature("$XML::Xerces::XMLUni::fgDOMValidation", $validate) ;
# $parser->setFeature("$XML::Xerces::XMLUni::fgDOMValidateIfSchema", 0) ;
};
XML::Xerces::error($@) if $@;
eval {
# my $is = XML::Xerces::MemBufInputSource->new($xml);
# my $is = XML::Xerces::LocalFileInputSource->new($OPTIONS{file});
$parser->parseURI($OPTIONS{file}) ;
} ;
XML::Xerces::error($@) if $@;
$parser->resetDocumentPool();
}
Using this code I'm not getting any perceptable leaks with 2.3.0-4.
Let me know if this helps,
jas.
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
Re: Memory Leak?
Posted by "Jason E. Stewart" <ja...@openinformatics.com>.
Chris Cheung <ch...@clc.cuhk.edu.hk> writes:
> Dear all,
>
> I need to use Xerces-Perl to repeatedly
>
> - parse XML to DOM tree
> - manipulate the DOM tree
> - pretty-print the DOM tree
>
> It seems that serious memory leak occurs. I do not hold any reference to
> the Perl Xerces object used in the programme and hence the Perl objects
> should be DESTROYed after use, but memory usage keeps on increasing as
> reported by the OS.
>
> Is there any solution to this problem?
> Thank you very much in advance.
Hi Chris,
You're the second person this week to notice the problem, see the
archive for more details:
http://nagoya.apache.org/eyebrowse/ReadMsg?listName=xerces-p-dev@xml.apache.org&msgNo=411
We've still not pinpointed the problem yet, so if you can give us more
details - a short piece of code that exhibits the problem - that would
be most helpful.
Cheers,
jas.
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
Re: Memory Leak?
Posted by "Jason E. Stewart" <ja...@openinformatics.com>.
Chris Cheung <ch...@clc.cuhk.edu.hk> writes:
> my $parser = XML::Xerces::XercesDOMParser->new();
> $parser->setDoNamespaces(1);
>
> $parser->setValidationScheme($XML::Xerces::AbstractDOMParser::Val_Never);
>
> $parser->parse("example.xml");
> my $doc = $parser->adoptDocument();
If you just want access to the underlying document I would call
getDocument(). adoptDocument() is if you want to inform the parser
that you are accepting responsibility for the memory of the $doc. So
when you call $parser->resetDocumentPool() it will not free up the
memory for $doc, you must do so using $doc->release().
But this is not the cause of the leak here. Just FYI.
Cheers,
jas.
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
Re: {Spam?} Re: Memory Leak?
Posted by "Jason E. Stewart" <ja...@openinformatics.com>.
Chris Cheung <ch...@clc.cuhk.edu.hk> writes:
> Dear Jason,
>
> On Tue, 22 Jun 2004, Jason E. Stewart wrote:
>
>> Chris Cheung <ch...@clc.cuhk.edu.hk> writes:
>>
>> > My project uses both DOM parsing and SAX parsing. DOM parsing is for
>> > some small-scale XML configuration file and the application will
>> > traverse the DOM Tree directly using the API of XML-Xerces. We use
>> > SAX for parsing some VERY LARGE XML data file which transform the
>> > XML data into some Perl data structure with filtering.
>> >
>> > I will try to list all API we are currently as soon as possible.
>>
>> There are 5 major API's that I need to consider:
>>
>> * XercesDOMParser
>> * DOMBuilder
>> * SAXParser
>> * SAX2XMLReader
>> * DOMWriter
>>
>
> Thank you very much for your help.
>
> The APIs which our project uses are listed as follows.
> (Sorry that I just grep them from source code, but the class and method
> name should be obvious)
> If you find that we used Xerces-Perl inappropreately somewhere or have
> some better suggestion, please kindly let us know.
Ok, it looks like you're using XercesDOMParser and not the new DOM
Level 3 system, DOMBuilder. You are using the DOM Level 3
serialization, DOMWriter. And you are using SAX Level 2, XMLReader.
Cheers,
jas.
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
Re: {Spam?} Re: Memory Leak?
Posted by Chris Cheung <ch...@clc.cuhk.edu.hk>.
Dear Jason,
On Tue, 22 Jun 2004, Jason E. Stewart wrote:
> Chris Cheung <ch...@clc.cuhk.edu.hk> writes:
>
> > My project uses both DOM parsing and SAX parsing. DOM parsing is for
> > some small-scale XML configuration file and the application will
> > traverse the DOM Tree directly using the API of XML-Xerces. We use
> > SAX for parsing some VERY LARGE XML data file which transform the
> > XML data into some Perl data structure with filtering.
> >
> > I will try to list all API we are currently as soon as possible.
>
> There are 5 major API's that I need to consider:
>
> * XercesDOMParser
> * DOMBuilder
> * SAXParser
> * SAX2XMLReader
> * DOMWriter
>
Thank you very much for your help.
The APIs which our project uses are listed as follows.
(Sorry that I just grep them from source code, but the class and method
name should be obvious)
If you find that we used Xerces-Perl inappropreately somewhere or have
some better suggestion, please kindly let us know.
DOM parsing:
------------
1. my $parser = XML::Xerces::XercesDOMParser->new();
2. $parser->setDoNamespaces(1);
3. $parser->setCreateEntityReferenceNodes(1);
4. $parser->setDoSchema(1);
5. $parser->setErrorHandler(...);
6. $parser->setValidationScheme($XML::Xerces::AbstractDOMParser::Val_Auto);
7. $parser->parse(XML::Xerces::MemBufInputSource->new($xmlString));
8. $parser->parse($xmlFile);
9. my $doc = $parser->getDocument();
SAX parsing:
------------
1. my $parser = XML::Xerces::XMLReaderFactory::createXMLReader();
2. $parser->setFeature("$XML::Xerces::XMLUni::fgSAX2CoreNameSpaces", 1);
3. $parser->setFeature("$XML::Xerces::XMLUni::fgSAX2CoreValidation",1);
4. $parser->setFeature("$XML::Xerces::XMLUni::fgXercesDynamic", 1);
5. $parser->setFeature("$XML::Xerces::XMLUni::fgXercesSchema", 1);
6. $parser->setErrorHandler(...);
7. $parser->setContentHandler(...);
8. $parser->parse(XML::Xerces::MemBufInputSource->new($xmlString));
9. $parser->parse(XML::Xerces::LocalFileInputSource->new($xmlFile));
XML Serialization:
------------------
1. my $domImpl =
XML::Xerces::DOMImplementationRegistry::getDOMImplementation('LS');
2. my $writer = $domImpl->createDOMWriter();
3. $writer->canSetFeature('format-pretty-print',1);
4. $writer->setFeature('format-pretty-print',1);
5. my $target = XML::Xerces::MemBufFormatTarget->new();
6. $writer->writeNode($target, $doc);
7. $target->getRawBuffer();
--
Best Regards,
Chris Cheung
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
Re: {Spam?} Re: Memory Leak?
Posted by "Jason E. Stewart" <ja...@openinformatics.com>.
Chris Cheung <ch...@clc.cuhk.edu.hk> writes:
> My project uses both DOM parsing and SAX parsing. DOM parsing is for
> some small-scale XML configuration file and the application will
> traverse the DOM Tree directly using the API of XML-Xerces. We use
> SAX for parsing some VERY LARGE XML data file which transform the
> XML data into some Perl data structure with filtering.
>
> I will try to list all API we are currently as soon as possible.
There are 5 major API's that I need to consider:
* XercesDOMParser
* DOMBuilder
* SAXParser
* SAX2XMLReader
* DOMWriter
Let me know which of these you are most concerned with.
Cheers,
jas.
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
Re: {Spam?} Re: Memory Leak?
Posted by Chris Cheung <ch...@clc.cuhk.edu.hk>.
Hi Jason,
Sorry for my late reply.
I am very pleased to hear that -- NO MEMORY LEAK. And we want to express
our appreciation to you again -- Thanks a lot!
My project uses both DOM parsing and SAX parsing. DOM parsing is for
some small-scale XML configuration file and the application will
traverse the DOM Tree directly using the API of XML-Xerces. We use SAX
for parsing some VERY LARGE XML data file which transform the XML data
into some Perl data structure with filtering.
I will try to list all API we are currently as soon as possible.
best regards,
Chris
On Tue, 22 Jun 2004, Jason E. Stewart wrote:
> Hi Chris,
>
> There are two MAJOR core issues with XML-Xerces at the moment:
> 1) Memory leaks
> 2) Multiple Inheritance strangeness with the DOMNode hierarchy
>
> Because I've been unable to debug these in the past, I've taken a new
> approach - completely re-build XML-Xerces from the ground up. This is
> not a code rewrite because all I am doing is constructing a SWIG
> interface file, Xerces.i, which enables SWIG to autowrap Xerces-C.
>
> Xerces is a rather large API - at last count I am wrapping 108
> different header files.
>
> So to make debugging simple I am starting over from scratch - only
> wrapping the barest minimum of files to get working tests - and then
> testing all aspects of the system - including memory leaks.
>
> With only 10 headers wrapped, I get the basic XercesDOMParser
> implementation, and the good news is:
>
> NO MEMORY LEAKS!!
>
> So I am going to slowly add the other DOM components and check my
> progress until I have recreated the complete API and located the
> problems.
>
> I am building this against the Xerces-C-2.6 from CVS, and I am happy
> to make my intermediate builds available for your testing. What are
> the API's which you desperately need for your application - I will be
> happy to include them in my testing.
>
> Cheers,
> jas.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
Re: Memory Leak?
Posted by "Jason E. Stewart" <ja...@openinformatics.com>.
Hi Chris,
There are two MAJOR core issues with XML-Xerces at the moment:
1) Memory leaks
2) Multiple Inheritance strangeness with the DOMNode hierarchy
Because I've been unable to debug these in the past, I've taken a new
approach - completely re-build XML-Xerces from the ground up. This is
not a code rewrite because all I am doing is constructing a SWIG
interface file, Xerces.i, which enables SWIG to autowrap Xerces-C.
Xerces is a rather large API - at last count I am wrapping 108
different header files.
So to make debugging simple I am starting over from scratch - only
wrapping the barest minimum of files to get working tests - and then
testing all aspects of the system - including memory leaks.
With only 10 headers wrapped, I get the basic XercesDOMParser
implementation, and the good news is:
NO MEMORY LEAKS!!
So I am going to slowly add the other DOM components and check my
progress until I have recreated the complete API and located the
problems.
I am building this against the Xerces-C-2.6 from CVS, and I am happy
to make my intermediate builds available for your testing. What are
the API's which you desperately need for your application - I will be
happy to include them in my testing.
Cheers,
jas.
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
XML-Xerces-2.5 and Memory Leak
Posted by "Jason E. Stewart" <ja...@openinformatics.com>.
Hi Chris,
XML-Xerces is still leaking memory - the sax2 parser leak is really
huge, but the good news is that the DOM Builder leak seems to be
almost nonexistent. Perhaps you can see if using the DOM Builder for
your code helps.
Cheers,
jas.
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-p-dev-help@xml.apache.org
Re: Memory Leak?
Posted by Chris Cheung <ch...@clc.cuhk.edu.hk>.
Dear Jason,
On 14 Nov 2003, Jason E. Stewart wrote:
> Apparently, one user indicated that if he turned validation off the
> memory leak went away. He was using a SAX2 parser, however. So please
> try this for your application and let us know if it helps.
Thanks a lot for your help. I tried turning validation off but the
memory leak still occurs. Attached please find two Perl scripts producing
the memory leak - one for XML parsing and one for pretty-printing
independently. The example XML I used is about 60k bytes. The variables
are local in the for-loop so I suppose it will be destroyed at the end of
the for-loop in each iteration.
The memory leak by mleak2.pl grow to >400M (500 iterations).
If I comment out the statement with $parser->adoptDocument, the memory
leak grow only to 35M.
Do I need to call some API to clean up after calling
XML::Xerces::AbstractDOMParser::adoptDocument() ?
The memory leak by mleak3.pl grow to about 150M (2500 iterations)
Hope this help. Thanks again.
--
Best Regards,
Chris Cheung
Center for Large-Scale Computation
Have a nice day!
Re: Memory Leak?
Posted by "Jason E. Stewart" <ja...@openinformatics.com>.
Chris Cheung <ch...@clc.cuhk.edu.hk> writes:
> Is there any solution to this problem?
Apparently, one user indicated that if he turned validation off the
memory leak went away. He was using a SAX2 parser, however. So please
try this for your application and let us know if it helps.
See:
http://nagoya.apache.org/eyebrowse/ReadMsg?listName=xerces-p-dev@xml.apache.org&msgNo=413
for details.
Cheers,
jas.
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-p-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-p-dev-help@xml.apache.org