You are viewing a plain text version of this content. The canonical link for it is here.
Posted to c-dev@xerces.apache.org by Simon Fell <so...@zaks.demon.co.uk> on 2000/07/05 01:58:37 UTC
Memory leak in SAXParser::parse() ?
Hi,
I'm using the SAXParser in Xerces-C 1.2.0a on Windows 2000 / VC++ v6.
Boundschecker is reporting a consistent 128 byte leak for every call
to SAXParser::parse (using a MemBufInputSource as the source). Has
anyone else seen this problem, or is it possible that i am using
something wrong ? (everything seems to work).
Cheers
Simon
Re: Memory leak in SAXParser::parse() ?
Posted by Simon Fell <so...@zaks.demon.co.uk>.
Thanks !, that did the trick
Cheers
Simon
On Tue, 04 Jul 2000 20:45:37 -0700, in soap you wrote:
>Hmmm... It looks like the fMap field of the element stack entries is not
>getting deleted upon destruction time. If you can rebuild the parser, try
>this:
>
>ElemStack::~ElemStack()
>{
> //
> // Start working from the bottom of the stack and clear it out as we
> // go up. Once we hit an uninitialized one, we can break out.
> //
> for (unsigned int stackInd = 0; stackInd < fStackCapacity; stackInd++)
> {
> // If this entry has been set, then lets clean it up
> if (!fStack[stackInd])
> break;
>
> // Delete the row for this entry, then delete the row structure
> delete [] fStack[stackInd]->fChildIds;
>+ delete [] fStack[stackInd]->fMap;
> delete fStack[stackInd];
> }
>
> // Delete the stack array itself now
> delete [] fStack;
>}
>
>
>So I added a line to the destructor to delete the map array for each entry
>in the element stack. I think that's it. I bet if you made your test file
>more deeply nested and mapped some prefixes in each level, it would start
>leaking even more (since each nested element that maps prefixes must have a
>map entry for that nesting level.)
>
>If that turns out to be it, then I'll put this fix in. Too bad it didn't get
>caught before 1.2.0 went out!
>
>--------------------------
>Dean Roddey
>The CIDLib C++ Frameworks
>Charmed Quark Software
>droddey@charmedquark.com
>http://www.charmedquark.com
>
>"You young, and you gotcha health. Whatchoo wanna job fer?"
>
>
>----- Original Message -----
>From: "Simon Fell" <so...@zaks.demon.co.uk>
>To: <xe...@xml.apache.org>
>Sent: Tuesday, July 04, 2000 8:13 PM
>Subject: Re: Memory leak in SAXParser::parse() ?
>
>
>> but from what i can see, its not a static buffer, it should get
>> cleared up when the SAXParser goes out of scope.
>>
>> I have a test program which use SAXParser, to parse the same XML over
>> and over, each iteration of the loop leaks 128 bytes. If i don't
>> enable namespaces then i don't see the leak.
>> I'm using the parser code like this,
>>
>> SAXParser parser;
>> parser.setDoValidation(false);
>> parser.setDoNamespaces(true) ;
>>
>> SOAPHandler handler;
>> parser.setDocumentHandler(&handler);
>> parser.setErrorHandler(&handler);
>>
>> try
>> {
>> MemBufInputSource input(requestXML, cbrequestXML, L"", false) ;
>> parser.parse(input);
>> _methodname = handler.getMethodName() ;
>> _params = handler.getParams();
>> }
>>
>> the above code is in a function, which gets called in a loop, with
>> 10000 iterations, boundschecker reports 10000 leaks of 128 bytes (I've
>> confirmed this by checking the private bytes with perfmon).
>>
>> I'll see if i can duplicate the problem with one of the sample
>> programs.
>>
>> Cheers
>> Simon
>>
>> On Tue, 04 Jul 2000 19:43:28 -0700, in soap you wrote:
>>
>> >The element stack grows as needed. Therefore, it has to reallocate the
>stack
>> >buffer. This will happen the first time you parse any file with
>namespaces
>> >on. As long as you just reparse the same file, it shouldn't happen again
>> >during any one run of the process. Once you stop the process and start
>> >again, its going to do it again. If you parse an XML source which maps
>> >enough more prefixes, then it will expand again to handle that larger
>number
>> >of prefixes.
>> >
>> >So, unless you mean that you put the parser in a loop and call parse()
>over
>> >and over and it grows every time, its not a leak, its just an expansion
>of a
>> >data structure.
>> >
>> >--------------------------
>> >Dean Roddey
>> >The CIDLib C++ Frameworks
>> >Charmed Quark Software
>> >droddey@charmedquark.com
>> >http://www.charmedquark.com
>> >
>> >"You young, and you gotcha health. Whatchoo wanna job fer?"
>> >
>> >
>> >----- Original Message -----
>> >From: "Simon Fell" <so...@zaks.demon.co.uk>
>> >To: <xe...@xml.apache.org>
>> >Sent: Tuesday, July 04, 2000 6:40 PM
>> >Subject: Re: Memory leak in SAXParser::parse() ?
>> >
>> >
>> >> I built a debug version of Xerces,and boundsChecker shows the
>> >> following call stack for the leak
>> >>
>> >> Function File Line
>> >> operator new() crtdbg.h 552
>> >> ElemStack::expandMap() ElemStack.cpp 459
>> >> ElemStack::addPrefix() ElemStack.cpp 330
>> >> XMLScanner::updateNSMap() XMLScanner2.cpp 1034
>> >> XMLScanner::scanStartTagNS()XMLScanner.cpp 2662
>> >> XMLScanner::scanContent() XMLScanner.cpp 1475
>> >> XMLScanner::scanDocument() XMLScanner.cpp 408
>> >> SAXParser::parse() SAXParser.cpp 334
>> >>
>> >> I have validation turned off, and namespaces turned on.
>> >>
>> >> Cheers
>> >> Simon
>> >>
>> >> On Tue, 04 Jul 2000 16:58:37 -0700, in xerces-c-dev you wrote:
>> >>
>> >> >Hi,
>> >> >
>> >> >I'm using the SAXParser in Xerces-C 1.2.0a on Windows 2000 / VC++ v6.
>> >> >Boundschecker is reporting a consistent 128 byte leak for every call
>> >> >to SAXParser::parse (using a MemBufInputSource as the source). Has
>> >> >anyone else seen this problem, or is it possible that i am using
>> >> >something wrong ? (everything seems to work).
>> >> >
>> >> >Cheers
>> >> >Simon
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
>> >> For additional commands, e-mail: xerces-c-dev-help@xml.apache.org
>> >>
>> >
>> >
>> >---------------------------------------------------------------------
>> >To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
>> >For additional commands, e-mail: xerces-c-dev-help@xml.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
>> For additional commands, e-mail: xerces-c-dev-help@xml.apache.org
>>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
>For additional commands, e-mail: xerces-c-dev-help@xml.apache.org
Re: Memory leak in SAXParser::parse() ?
Posted by Dean Roddey <dr...@charmedquark.com>.
Hmmm... It looks like the fMap field of the element stack entries is not
getting deleted upon destruction time. If you can rebuild the parser, try
this:
ElemStack::~ElemStack()
{
//
// Start working from the bottom of the stack and clear it out as we
// go up. Once we hit an uninitialized one, we can break out.
//
for (unsigned int stackInd = 0; stackInd < fStackCapacity; stackInd++)
{
// If this entry has been set, then lets clean it up
if (!fStack[stackInd])
break;
// Delete the row for this entry, then delete the row structure
delete [] fStack[stackInd]->fChildIds;
+ delete [] fStack[stackInd]->fMap;
delete fStack[stackInd];
}
// Delete the stack array itself now
delete [] fStack;
}
So I added a line to the destructor to delete the map array for each entry
in the element stack. I think that's it. I bet if you made your test file
more deeply nested and mapped some prefixes in each level, it would start
leaking even more (since each nested element that maps prefixes must have a
map entry for that nesting level.)
If that turns out to be it, then I'll put this fix in. Too bad it didn't get
caught before 1.2.0 went out!
--------------------------
Dean Roddey
The CIDLib C++ Frameworks
Charmed Quark Software
droddey@charmedquark.com
http://www.charmedquark.com
"You young, and you gotcha health. Whatchoo wanna job fer?"
----- Original Message -----
From: "Simon Fell" <so...@zaks.demon.co.uk>
To: <xe...@xml.apache.org>
Sent: Tuesday, July 04, 2000 8:13 PM
Subject: Re: Memory leak in SAXParser::parse() ?
> but from what i can see, its not a static buffer, it should get
> cleared up when the SAXParser goes out of scope.
>
> I have a test program which use SAXParser, to parse the same XML over
> and over, each iteration of the loop leaks 128 bytes. If i don't
> enable namespaces then i don't see the leak.
> I'm using the parser code like this,
>
> SAXParser parser;
> parser.setDoValidation(false);
> parser.setDoNamespaces(true) ;
>
> SOAPHandler handler;
> parser.setDocumentHandler(&handler);
> parser.setErrorHandler(&handler);
>
> try
> {
> MemBufInputSource input(requestXML, cbrequestXML, L"", false) ;
> parser.parse(input);
> _methodname = handler.getMethodName() ;
> _params = handler.getParams();
> }
>
> the above code is in a function, which gets called in a loop, with
> 10000 iterations, boundschecker reports 10000 leaks of 128 bytes (I've
> confirmed this by checking the private bytes with perfmon).
>
> I'll see if i can duplicate the problem with one of the sample
> programs.
>
> Cheers
> Simon
>
> On Tue, 04 Jul 2000 19:43:28 -0700, in soap you wrote:
>
> >The element stack grows as needed. Therefore, it has to reallocate the
stack
> >buffer. This will happen the first time you parse any file with
namespaces
> >on. As long as you just reparse the same file, it shouldn't happen again
> >during any one run of the process. Once you stop the process and start
> >again, its going to do it again. If you parse an XML source which maps
> >enough more prefixes, then it will expand again to handle that larger
number
> >of prefixes.
> >
> >So, unless you mean that you put the parser in a loop and call parse()
over
> >and over and it grows every time, its not a leak, its just an expansion
of a
> >data structure.
> >
> >--------------------------
> >Dean Roddey
> >The CIDLib C++ Frameworks
> >Charmed Quark Software
> >droddey@charmedquark.com
> >http://www.charmedquark.com
> >
> >"You young, and you gotcha health. Whatchoo wanna job fer?"
> >
> >
> >----- Original Message -----
> >From: "Simon Fell" <so...@zaks.demon.co.uk>
> >To: <xe...@xml.apache.org>
> >Sent: Tuesday, July 04, 2000 6:40 PM
> >Subject: Re: Memory leak in SAXParser::parse() ?
> >
> >
> >> I built a debug version of Xerces,and boundsChecker shows the
> >> following call stack for the leak
> >>
> >> Function File Line
> >> operator new() crtdbg.h 552
> >> ElemStack::expandMap() ElemStack.cpp 459
> >> ElemStack::addPrefix() ElemStack.cpp 330
> >> XMLScanner::updateNSMap() XMLScanner2.cpp 1034
> >> XMLScanner::scanStartTagNS()XMLScanner.cpp 2662
> >> XMLScanner::scanContent() XMLScanner.cpp 1475
> >> XMLScanner::scanDocument() XMLScanner.cpp 408
> >> SAXParser::parse() SAXParser.cpp 334
> >>
> >> I have validation turned off, and namespaces turned on.
> >>
> >> Cheers
> >> Simon
> >>
> >> On Tue, 04 Jul 2000 16:58:37 -0700, in xerces-c-dev you wrote:
> >>
> >> >Hi,
> >> >
> >> >I'm using the SAXParser in Xerces-C 1.2.0a on Windows 2000 / VC++ v6.
> >> >Boundschecker is reporting a consistent 128 byte leak for every call
> >> >to SAXParser::parse (using a MemBufInputSource as the source). Has
> >> >anyone else seen this problem, or is it possible that i am using
> >> >something wrong ? (everything seems to work).
> >> >
> >> >Cheers
> >> >Simon
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
> >> For additional commands, e-mail: xerces-c-dev-help@xml.apache.org
> >>
> >
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
> >For additional commands, e-mail: xerces-c-dev-help@xml.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-c-dev-help@xml.apache.org
>
Re: Memory leak in SAXParser::parse() ?
Posted by Simon Fell <so...@zaks.demon.co.uk>.
but from what i can see, its not a static buffer, it should get
cleared up when the SAXParser goes out of scope.
I have a test program which use SAXParser, to parse the same XML over
and over, each iteration of the loop leaks 128 bytes. If i don't
enable namespaces then i don't see the leak.
I'm using the parser code like this,
SAXParser parser;
parser.setDoValidation(false);
parser.setDoNamespaces(true) ;
SOAPHandler handler;
parser.setDocumentHandler(&handler);
parser.setErrorHandler(&handler);
try
{
MemBufInputSource input(requestXML, cbrequestXML, L"", false) ;
parser.parse(input);
_methodname = handler.getMethodName() ;
_params = handler.getParams();
}
the above code is in a function, which gets called in a loop, with
10000 iterations, boundschecker reports 10000 leaks of 128 bytes (I've
confirmed this by checking the private bytes with perfmon).
I'll see if i can duplicate the problem with one of the sample
programs.
Cheers
Simon
On Tue, 04 Jul 2000 19:43:28 -0700, in soap you wrote:
>The element stack grows as needed. Therefore, it has to reallocate the stack
>buffer. This will happen the first time you parse any file with namespaces
>on. As long as you just reparse the same file, it shouldn't happen again
>during any one run of the process. Once you stop the process and start
>again, its going to do it again. If you parse an XML source which maps
>enough more prefixes, then it will expand again to handle that larger number
>of prefixes.
>
>So, unless you mean that you put the parser in a loop and call parse() over
>and over and it grows every time, its not a leak, its just an expansion of a
>data structure.
>
>--------------------------
>Dean Roddey
>The CIDLib C++ Frameworks
>Charmed Quark Software
>droddey@charmedquark.com
>http://www.charmedquark.com
>
>"You young, and you gotcha health. Whatchoo wanna job fer?"
>
>
>----- Original Message -----
>From: "Simon Fell" <so...@zaks.demon.co.uk>
>To: <xe...@xml.apache.org>
>Sent: Tuesday, July 04, 2000 6:40 PM
>Subject: Re: Memory leak in SAXParser::parse() ?
>
>
>> I built a debug version of Xerces,and boundsChecker shows the
>> following call stack for the leak
>>
>> Function File Line
>> operator new() crtdbg.h 552
>> ElemStack::expandMap() ElemStack.cpp 459
>> ElemStack::addPrefix() ElemStack.cpp 330
>> XMLScanner::updateNSMap() XMLScanner2.cpp 1034
>> XMLScanner::scanStartTagNS()XMLScanner.cpp 2662
>> XMLScanner::scanContent() XMLScanner.cpp 1475
>> XMLScanner::scanDocument() XMLScanner.cpp 408
>> SAXParser::parse() SAXParser.cpp 334
>>
>> I have validation turned off, and namespaces turned on.
>>
>> Cheers
>> Simon
>>
>> On Tue, 04 Jul 2000 16:58:37 -0700, in xerces-c-dev you wrote:
>>
>> >Hi,
>> >
>> >I'm using the SAXParser in Xerces-C 1.2.0a on Windows 2000 / VC++ v6.
>> >Boundschecker is reporting a consistent 128 byte leak for every call
>> >to SAXParser::parse (using a MemBufInputSource as the source). Has
>> >anyone else seen this problem, or is it possible that i am using
>> >something wrong ? (everything seems to work).
>> >
>> >Cheers
>> >Simon
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
>> For additional commands, e-mail: xerces-c-dev-help@xml.apache.org
>>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
>For additional commands, e-mail: xerces-c-dev-help@xml.apache.org
Re: Memory leak in SAXParser::parse() ?
Posted by Dean Roddey <dr...@charmedquark.com>.
The element stack grows as needed. Therefore, it has to reallocate the stack
buffer. This will happen the first time you parse any file with namespaces
on. As long as you just reparse the same file, it shouldn't happen again
during any one run of the process. Once you stop the process and start
again, its going to do it again. If you parse an XML source which maps
enough more prefixes, then it will expand again to handle that larger number
of prefixes.
So, unless you mean that you put the parser in a loop and call parse() over
and over and it grows every time, its not a leak, its just an expansion of a
data structure.
--------------------------
Dean Roddey
The CIDLib C++ Frameworks
Charmed Quark Software
droddey@charmedquark.com
http://www.charmedquark.com
"You young, and you gotcha health. Whatchoo wanna job fer?"
----- Original Message -----
From: "Simon Fell" <so...@zaks.demon.co.uk>
To: <xe...@xml.apache.org>
Sent: Tuesday, July 04, 2000 6:40 PM
Subject: Re: Memory leak in SAXParser::parse() ?
> I built a debug version of Xerces,and boundsChecker shows the
> following call stack for the leak
>
> Function File Line
> operator new() crtdbg.h 552
> ElemStack::expandMap() ElemStack.cpp 459
> ElemStack::addPrefix() ElemStack.cpp 330
> XMLScanner::updateNSMap() XMLScanner2.cpp 1034
> XMLScanner::scanStartTagNS()XMLScanner.cpp 2662
> XMLScanner::scanContent() XMLScanner.cpp 1475
> XMLScanner::scanDocument() XMLScanner.cpp 408
> SAXParser::parse() SAXParser.cpp 334
>
> I have validation turned off, and namespaces turned on.
>
> Cheers
> Simon
>
> On Tue, 04 Jul 2000 16:58:37 -0700, in xerces-c-dev you wrote:
>
> >Hi,
> >
> >I'm using the SAXParser in Xerces-C 1.2.0a on Windows 2000 / VC++ v6.
> >Boundschecker is reporting a consistent 128 byte leak for every call
> >to SAXParser::parse (using a MemBufInputSource as the source). Has
> >anyone else seen this problem, or is it possible that i am using
> >something wrong ? (everything seems to work).
> >
> >Cheers
> >Simon
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-c-dev-help@xml.apache.org
>
Re: Memory leak in SAXParser::parse() ?
Posted by Simon Fell <so...@zaks.demon.co.uk>.
I built a debug version of Xerces,and boundsChecker shows the
following call stack for the leak
Function File Line
operator new() crtdbg.h 552
ElemStack::expandMap() ElemStack.cpp 459
ElemStack::addPrefix() ElemStack.cpp 330
XMLScanner::updateNSMap() XMLScanner2.cpp 1034
XMLScanner::scanStartTagNS()XMLScanner.cpp 2662
XMLScanner::scanContent() XMLScanner.cpp 1475
XMLScanner::scanDocument() XMLScanner.cpp 408
SAXParser::parse() SAXParser.cpp 334
I have validation turned off, and namespaces turned on.
Cheers
Simon
On Tue, 04 Jul 2000 16:58:37 -0700, in xerces-c-dev you wrote:
>Hi,
>
>I'm using the SAXParser in Xerces-C 1.2.0a on Windows 2000 / VC++ v6.
>Boundschecker is reporting a consistent 128 byte leak for every call
>to SAXParser::parse (using a MemBufInputSource as the source). Has
>anyone else seen this problem, or is it possible that i am using
>something wrong ? (everything seems to work).
>
>Cheers
>Simon