You are viewing a plain text version of this content. The canonical link for it is here.
Posted to c-dev@xerces.apache.org by Sree Harsha <sr...@gmail.com> on 2008/02/26 11:40:07 UTC
Please help me out with this code..Can i reuse DOM objects in a while loop??????
/*Can i reuse the DOM objects like XercesDOMParser object in a do... while
loop as follows.... or will it cause any memory leaks how to reuse them...*/
do
{
/*create a source file */
parser_.parse(source); //parser_ is a XercesDOMParser object
/*parse the file source*/
/*delete the source file*/
/*what function should i call to reuse parser_ object in the next iteration
so that memory consumption reduces */
}while(dwbytes==bufsize);
/*some condition*/
--
Regards
Sree Harsha Vardhana S.N
"When you want something, all the universe conspires in helping you to
achieve it."
--
Regards
Sree Harsha Vardhana S.N
"When you want something, all the universe conspires in helping you to
achieve it."
RE: Please help me out with this code..Can i reuse DOM objects in a while loop??????
Posted by Jesse Pelton <js...@PKC.com>.
Oops. Alberto already corrected the first point. I stand by the second,
though.
________________________________
From: Jesse Pelton [mailto:jsp@PKC.com]
Sent: Wednesday, February 27, 2008 8:59 AM
To: c-dev@xerces.apache.org
Subject: RE: Please help me out with this code..Can i reuse DOM objects
in a while loop??????
I haven't examined the reset code, but I'd expect the document to be
released when you reset the parser (unless you adopt the document via a
call to adoptDocument(), in which case it's your responsibility to
release it).
Given that you're processing huge files, you should consider using a SAX
parser. SAX is a stream-oriented interface. You install callbacks to
handle SAX events as the input is processed. You use the callbacks to
extract whatever information you need and store it however it makes
sense; the parser does not construct any representation of the document
itself. If you store the extracted data efficiently, this should require
much less memory than a DOM representing the entire document. In the
ideal case, you have chunks of data that can be processed and then
discarded; a SAX parser should be able to handle arbitrarily large
documents in this scenario, assuming the chunks aren't so big that they
exhaust memory.
________________________________
From: Sree Harsha [mailto:sree.harsha.sn@gmail.com]
Sent: Tuesday, February 26, 2008 10:20 PM
To: c-dev@xerces.apache.org
Subject: Re: Please help me out with this code..Can i reuse DOM objects
in a while loop??????
Thanks for your reply, but still the memory consumption has not come
down. Is there any way i can reuse DOMDocument variable also like in the
following snippet...
do
{
/*create a source file */
parser_.parse(source); //parser_ is a XercesDOMParser object
xercesc::DOMDocument* xmlDoc = parser_.getDocument() ;
/*How to release the memory allocated to the DOMDocument pointer*/
/*parse the file source*/
/*delete the source file*/
/*I used the function which you suggested*/
parser_.reset();
/*still it takes up a lot of memory when parsing a big file*/
}while(dwbytes==bufsize);
/*some condition*/
Finally I also have one more doubt... When we use DOM, the entire file
to be parsed is brought into the memory isnt it... Now if I have to
parse a large file, say 380 MB or 1GB, we cant Bring the entire file to
the memory. So I thought it would be better to split the file into
smaller sizes and parse them bit by bit (4K long)... that is the reason
i have used create source file in the above code....
Am I in the Right path????
On 2/26/08, Jesse Pelton <js...@pkc.com> wrote:
It looks to me like you could call the various
XercesDOMParser::reset...() and AbstractDOMParser::reset...() methods.
Or, to be paranoid and/or lazy, you could use a new parser for each
iteration.
________________________________
From: Sree Harsha [mailto:sree.harsha.sn@gmail.com]
Sent: Tuesday, February 26, 2008 5:40 AM
To: boris@codesynthesis.com; c-dev@xerces.apache.org
Subject: Please help me out with this code..Can i reuse DOM
objects in a while loop??????
/*Can i reuse the DOM objects like XercesDOMParser object in a
do... while loop as follows.... or will it cause any memory leaks how to
reuse them...*/
do
{
/*create a source file */
parser_.parse(source); //parser_ is a XercesDOMParser object
/*parse the file source*/
/*delete the source file*/
/*what function should i call to reuse parser_ object in the
next iteration so that memory consumption reduces */
}while(dwbytes==bufsize);
/*some condition*/
--
Regards
Sree Harsha Vardhana S.N
"When you want something, all the universe conspires in helping
you to
achieve it."
--
Regards
Sree Harsha Vardhana S.N
"When you want something, all the universe conspires in helping
you to
achieve it."
--
Regards
Sree Harsha Vardhana S.N
"When you want something, all the universe conspires in helping you to
achieve it."
RE: Please help me out with this code..Can i reuse DOM objects in a while loop??????
Posted by Jesse Pelton <js...@PKC.com>.
I haven't examined the reset code, but I'd expect the document to be
released when you reset the parser (unless you adopt the document via a
call to adoptDocument(), in which case it's your responsibility to
release it).
Given that you're processing huge files, you should consider using a SAX
parser. SAX is a stream-oriented interface. You install callbacks to
handle SAX events as the input is processed. You use the callbacks to
extract whatever information you need and store it however it makes
sense; the parser does not construct any representation of the document
itself. If you store the extracted data efficiently, this should require
much less memory than a DOM representing the entire document. In the
ideal case, you have chunks of data that can be processed and then
discarded; a SAX parser should be able to handle arbitrarily large
documents in this scenario, assuming the chunks aren't so big that they
exhaust memory.
________________________________
From: Sree Harsha [mailto:sree.harsha.sn@gmail.com]
Sent: Tuesday, February 26, 2008 10:20 PM
To: c-dev@xerces.apache.org
Subject: Re: Please help me out with this code..Can i reuse DOM objects
in a while loop??????
Thanks for your reply, but still the memory consumption has not come
down. Is there any way i can reuse DOMDocument variable also like in the
following snippet...
do
{
/*create a source file */
parser_.parse(source); //parser_ is a XercesDOMParser object
xercesc::DOMDocument* xmlDoc = parser_.getDocument() ;
/*How to release the memory allocated to the DOMDocument pointer*/
/*parse the file source*/
/*delete the source file*/
/*I used the function which you suggested*/
parser_.reset();
/*still it takes up a lot of memory when parsing a big file*/
}while(dwbytes==bufsize);
/*some condition*/
Finally I also have one more doubt... When we use DOM, the entire file
to be parsed is brought into the memory isnt it... Now if I have to
parse a large file, say 380 MB or 1GB, we cant Bring the entire file to
the memory. So I thought it would be better to split the file into
smaller sizes and parse them bit by bit (4K long)... that is the reason
i have used create source file in the above code....
Am I in the Right path????
On 2/26/08, Jesse Pelton <js...@pkc.com> wrote:
It looks to me like you could call the various
XercesDOMParser::reset...() and AbstractDOMParser::reset...() methods.
Or, to be paranoid and/or lazy, you could use a new parser for each
iteration.
________________________________
From: Sree Harsha [mailto:sree.harsha.sn@gmail.com]
Sent: Tuesday, February 26, 2008 5:40 AM
To: boris@codesynthesis.com; c-dev@xerces.apache.org
Subject: Please help me out with this code..Can i reuse DOM
objects in a while loop??????
/*Can i reuse the DOM objects like XercesDOMParser object in a
do... while loop as follows.... or will it cause any memory leaks how to
reuse them...*/
do
{
/*create a source file */
parser_.parse(source); //parser_ is a XercesDOMParser object
/*parse the file source*/
/*delete the source file*/
/*what function should i call to reuse parser_ object in the
next iteration so that memory consumption reduces */
}while(dwbytes==bufsize);
/*some condition*/
--
Regards
Sree Harsha Vardhana S.N
"When you want something, all the universe conspires in helping
you to
achieve it."
--
Regards
Sree Harsha Vardhana S.N
"When you want something, all the universe conspires in helping
you to
achieve it."
--
Regards
Sree Harsha Vardhana S.N
"When you want something, all the universe conspires in helping you to
achieve it."
Re: Please help me out with this code..Can i reuse DOM objects in a while loop??????
Posted by Sree Harsha <sr...@gmail.com>.
Thanks for the tip on releasing the XMLstring memory ....... there was a
reduction in the use of memory...
please tell me why i am getting an error when i resetDocumentPool();
I get a run time error R6025 pure virtual function call
thank you
On 2/27/08, Sree Harsha <sr...@gmail.com> wrote:
>
> Thanks Alberto,
> I used resetDocumentPool(),
> But I get a run time error R6025 pure virtual function call.
>
> Thanks and regards
> Sree Harsha Vardhana S.N
>
> On 2/27/08, Alberto Massari <am...@datadirect.com> wrote:
> >
> > Hi Sree,
> > you should invoke resetDocumentPool(), not reset(). As for the code you
> > posted previously, I noticed you were not releasing the transcoded
> > strings (you actually commented the call to XMLString::release): that
> > would be a substantial leak.
> > On the other hand, if you want to parse files so big, you should
> > investigate switching to SAX parsing.
> >
> > Alberto
> >
> > Sree Harsha wrote:
> > > Thanks for your reply, but still the memory consumption has not come
> > > down. Is there any way i can reuse DOMDocument variable also like in
> > > the following snippet...
> > >
> > > do
> > > {
> > >
> > > /*create a source file */
> > >
> > > parser_.parse(source); //parser_ is a XercesDOMParser object
> > >
> > > xercesc::DOMDocument* xmlDoc = parser_.getDocument() ;
> > >
> > > /*How to release the memory allocated to the DOMDocument pointer*/
> > >
> > > /*parse the file source*/
> > >
> > > /*delete the source file*/
> > >
> > > /*I used the function which you suggested*/
> > >
> > > parser_.reset();
> > >
> > > /*still it takes up a lot of memory when parsing a big file*/
> > >
> > > }while(dwbytes==bufsize);
> > > /*some condition*/
> > >
> > > Finally I also have one more doubt... When we use DOM, the entire file
> > > to be parsed is brought into the memory isnt it... Now if I have to
> > > parse a large file, say 380 MB or 1GB, we cant Bring the entire file
> > > to the memory. So I thought it would be better to split the file into
> > > smaller sizes and parse them bit by bit (4K long)... that is the
> > > reason i have used create source file in the above code....
> > > Am I in the Right path????
> > >
> > > On 2/26/08, *Jesse Pelton* <jsp@pkc.com <ma...@pkc.com>> wrote:
> > >
> > > It looks to me like you could call the various
> > > XercesDOMParser::reset...() and AbstractDOMParser::reset...()
> > > methods. Or, to be paranoid and/or lazy, you could use a new
> > > parser for each iteration.
> > >
> > >
> > ------------------------------------------------------------------------
> > > *From:* Sree Harsha [mailto:sree.harsha.sn@gmail.com
> > > <ma...@gmail.com>]
> > > *Sent:* Tuesday, February 26, 2008 5:40 AM
> > > *To:* boris@codesynthesis.com <ma...@codesynthesis.com>;
> > > c-dev@xerces.apache.org <ma...@xerces.apache.org>
> > > *Subject:* Please help me out with this code..Can i reuse DOM
> > > objects in a while loop??????
> > >
> > >
> > > /*Can i reuse the DOM objects like XercesDOMParser object in a
> > > do... while loop as follows.... or will it cause any memory leaks
> > > how to reuse them...*/
> > > do
> > > {
> > >
> > > /*create a source file */
> > >
> > > parser_.parse(source); //parser_ is a XercesDOMParser object
> > >
> > > /*parse the file source*/
> > >
> > > /*delete the source file*/
> > >
> > > /*what function should i call to reuse parser_ object in the next
> > > iteration so that memory consumption reduces */
> > >
> > > }while(dwbytes==bufsize);
> > > /*some condition*/
> > >
> > >
> > > --
> > > Regards
> > > Sree Harsha Vardhana S.N
> > > "When you want something, all the universe conspires in helping
> > you to
> > > achieve it."
> > >
> > > --
> > > Regards
> > > Sree Harsha Vardhana S.N
> > > "When you want something, all the universe conspires in helping
> > you to
> > > achieve it."
> > >
> > >
> > >
> > >
> > > --
> > > Regards
> > > Sree Harsha Vardhana S.N
> > > "When you want something, all the universe conspires in helping you to
> > > achieve it."
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
> > For additional commands, e-mail: c-dev-help@xerces.apache.org
> >
> >
>
>
> --
> Regards
> Sree Harsha Vardhana S.N
> "When you want something, all the universe conspires in helping you to
> achieve it."
>
--
Regards
Sree Harsha Vardhana S.N
"When you want something, all the universe conspires in helping you to
achieve it."
Re: Please help me out with this code..Can i reuse DOM objects in a while loop??????
Posted by Sree Harsha <sr...@gmail.com>.
XercesDOMParser parser_;
do
{
/*create a source file */
/*During the next iteration the following code will give runtime error*/
parser_.parse(source);
xercesc::DOMDocument* xmlDoc = parser_.getDocument() ;
/*How to release the memory allocated to the DOMDocument pointer*/
/*parse the file source*/
/*delete the source file*/
/*I used the function which you suggested*/
parser_.resetDocumentPool();
}while(dwbytes==bufsize);
/*some condition*/
On 2/27/08, Alberto Massari <am...@datadirect.com> wrote:
>
> Sree Harsha wrote:
> > Thanks Alberto,
> > I used resetDocumentPool(),
> > But I get a run time error R6025 pure virtual function call.
>
> When/where do you call resetDocumentPool? And where do you get the pure
> virtual function call? Are you attempting to access one of the DOM tree
> that you have just released?
>
> Alberto
>
> >
> > Thanks and regards
> > Sree Harsha Vardhana S.N
> >
> > On 2/27/08, *Alberto Massari* <amassari@datadirect.com
> > <ma...@datadirect.com>> wrote:
> >
> > Hi Sree,
> > you should invoke resetDocumentPool(), not reset(). As for the
> > code you
> > posted previously, I noticed you were not releasing the transcoded
> > strings (you actually commented the call to XMLString::release):
> that
> > would be a substantial leak.
> > On the other hand, if you want to parse files so big, you should
> > investigate switching to SAX parsing.
> >
> > Alberto
> >
> > Sree Harsha wrote:
> > > Thanks for your reply, but still the memory consumption has not
> come
> > > down. Is there any way i can reuse DOMDocument variable also like
> in
> > > the following snippet...
> > >
> > > do
> > > {
> > >
> > > /*create a source file */
> > >
> > > parser_.parse(source); //parser_ is a XercesDOMParser object
> > >
> > > xercesc::DOMDocument* xmlDoc = parser_.getDocument() ;
> > >
> > > /*How to release the memory allocated to the DOMDocument pointer*/
> > >
> > > /*parse the file source*/
> > >
> > > /*delete the source file*/
> > >
> > > /*I used the function which you suggested*/
> > >
> > > parser_.reset();
> > >
> > > /*still it takes up a lot of memory when parsing a big file*/
> > >
> > > }while(dwbytes==bufsize);
> > > /*some condition*/
> > >
> > > Finally I also have one more doubt... When we use DOM, the
> > entire file
> > > to be parsed is brought into the memory isnt it... Now if I have
> to
> > > parse a large file, say 380 MB or 1GB, we cant Bring the entire
> file
> > > to the memory. So I thought it would be better to split the file
> > into
> > > smaller sizes and parse them bit by bit (4K long)... that is the
> > > reason i have used create source file in the above code....
> > > Am I in the Right path????
> > >
> > > On 2/26/08, *Jesse Pelton* <jsp@pkc.com <ma...@pkc.com>
> > <mailto:jsp@pkc.com <ma...@pkc.com>>> wrote:
> > >
> > > It looks to me like you could call the various
> > > XercesDOMParser::reset...() and AbstractDOMParser::reset...()
> > > methods. Or, to be paranoid and/or lazy, you could use a new
> > > parser for each iteration.
> > >
> > >
> >
> ------------------------------------------------------------------------
> > > *From:* Sree Harsha [mailto:sree.harsha.sn@gmail.com
> > <ma...@gmail.com>
> > > <mailto:sree.harsha.sn@gmail.com
> > <ma...@gmail.com>>]
> > > *Sent:* Tuesday, February 26, 2008 5:40 AM
> > > *To:* boris@codesynthesis.com
> > <ma...@codesynthesis.com> <mailto:boris@codesynthesis.com
> > <ma...@codesynthesis.com>>;
> > > c-dev@xerces.apache.org <ma...@xerces.apache.org>
> > <mailto:c-dev@xerces.apache.org <ma...@xerces.apache.org>>
> > > *Subject:* Please help me out with this code..Can i reuse DOM
> > > objects in a while loop??????
> > >
> > >
> > > /*Can i reuse the DOM objects like XercesDOMParser object in a
> > > do... while loop as follows.... or will it cause any memory
> > leaks
> > > how to reuse them...*/
> > > do
> > > {
> > >
> > > /*create a source file */
> > >
> > > parser_.parse(source); //parser_ is a XercesDOMParser object
> > >
> > > /*parse the file source*/
> > >
> > > /*delete the source file*/
> > >
> > > /*what function should i call to reuse parser_ object in the
> > next
> > > iteration so that memory consumption reduces */
> > >
> > > }while(dwbytes==bufsize);
> > > /*some condition*/
> > >
> > >
> > > --
> > > Regards
> > > Sree Harsha Vardhana S.N
> > > "When you want something, all the universe conspires in
> > helping you to
> > > achieve it."
> > >
> > > --
> > > Regards
> > > Sree Harsha Vardhana S.N
> > > "When you want something, all the universe conspires in
> > helping you to
> > > achieve it."
> > >
> > >
> > >
> > >
> > > --
> > > Regards
> > > Sree Harsha Vardhana S.N
> > > "When you want something, all the universe conspires in helping
> > you to
> > > achieve it."
> >
> >
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
> > <ma...@xerces.apache.org>
> > For additional commands, e-mail: c-dev-help@xerces.apache.org
> > <ma...@xerces.apache.org>
> >
> >
> >
> >
> > --
> > Regards
> > Sree Harsha Vardhana S.N
> > "When you want something, all the universe conspires in helping you to
> > achieve it."
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
> For additional commands, e-mail: c-dev-help@xerces.apache.org
>
>
--
Regards
Sree Harsha Vardhana S.N
"When you want something, all the universe conspires in helping you to
achieve it."
Re: Please help me out with this code..Can i reuse DOM objects in a while loop??????
Posted by Sree Harsha <sr...@gmail.com>.
/*Here is how the code looks like*/
XercesDOMParser parser_;
do
{
/*code to create a source file */
/*During the next iteration the following code will give runtime error*/
parser_.parse(source);
xercesc::DOMDocument* xmlDoc = parser_.getDocument() ;
/*code to parse the file source*/
/*code to delete the source file*/
/*I used the function which you suggested*/
parser_.resetDocumentPool();
}while(dwbytes==bufsize);
/*some condition*/
Re: Please help me out with this code..Can i reuse DOM objects in
a while loop??????
Posted by Alberto Massari <am...@datadirect.com>.
Sree Harsha wrote:
> Thanks Alberto,
> I used resetDocumentPool(),
> But I get a run time error R6025 pure virtual function call.
When/where do you call resetDocumentPool? And where do you get the pure
virtual function call? Are you attempting to access one of the DOM tree
that you have just released?
Alberto
>
> Thanks and regards
> Sree Harsha Vardhana S.N
>
> On 2/27/08, *Alberto Massari* <amassari@datadirect.com
> <ma...@datadirect.com>> wrote:
>
> Hi Sree,
> you should invoke resetDocumentPool(), not reset(). As for the
> code you
> posted previously, I noticed you were not releasing the transcoded
> strings (you actually commented the call to XMLString::release): that
> would be a substantial leak.
> On the other hand, if you want to parse files so big, you should
> investigate switching to SAX parsing.
>
> Alberto
>
> Sree Harsha wrote:
> > Thanks for your reply, but still the memory consumption has not come
> > down. Is there any way i can reuse DOMDocument variable also like in
> > the following snippet...
> >
> > do
> > {
> >
> > /*create a source file */
> >
> > parser_.parse(source); //parser_ is a XercesDOMParser object
> >
> > xercesc::DOMDocument* xmlDoc = parser_.getDocument() ;
> >
> > /*How to release the memory allocated to the DOMDocument pointer*/
> >
> > /*parse the file source*/
> >
> > /*delete the source file*/
> >
> > /*I used the function which you suggested*/
> >
> > parser_.reset();
> >
> > /*still it takes up a lot of memory when parsing a big file*/
> >
> > }while(dwbytes==bufsize);
> > /*some condition*/
> >
> > Finally I also have one more doubt... When we use DOM, the
> entire file
> > to be parsed is brought into the memory isnt it... Now if I have to
> > parse a large file, say 380 MB or 1GB, we cant Bring the entire file
> > to the memory. So I thought it would be better to split the file
> into
> > smaller sizes and parse them bit by bit (4K long)... that is the
> > reason i have used create source file in the above code....
> > Am I in the Right path????
> >
> > On 2/26/08, *Jesse Pelton* <jsp@pkc.com <ma...@pkc.com>
> <mailto:jsp@pkc.com <ma...@pkc.com>>> wrote:
> >
> > It looks to me like you could call the various
> > XercesDOMParser::reset...() and AbstractDOMParser::reset...()
> > methods. Or, to be paranoid and/or lazy, you could use a new
> > parser for each iteration.
> >
> >
> ------------------------------------------------------------------------
> > *From:* Sree Harsha [mailto:sree.harsha.sn@gmail.com
> <ma...@gmail.com>
> > <mailto:sree.harsha.sn@gmail.com
> <ma...@gmail.com>>]
> > *Sent:* Tuesday, February 26, 2008 5:40 AM
> > *To:* boris@codesynthesis.com
> <ma...@codesynthesis.com> <mailto:boris@codesynthesis.com
> <ma...@codesynthesis.com>>;
> > c-dev@xerces.apache.org <ma...@xerces.apache.org>
> <mailto:c-dev@xerces.apache.org <ma...@xerces.apache.org>>
> > *Subject:* Please help me out with this code..Can i reuse DOM
> > objects in a while loop??????
> >
> >
> > /*Can i reuse the DOM objects like XercesDOMParser object in a
> > do... while loop as follows.... or will it cause any memory
> leaks
> > how to reuse them...*/
> > do
> > {
> >
> > /*create a source file */
> >
> > parser_.parse(source); //parser_ is a XercesDOMParser object
> >
> > /*parse the file source*/
> >
> > /*delete the source file*/
> >
> > /*what function should i call to reuse parser_ object in the
> next
> > iteration so that memory consumption reduces */
> >
> > }while(dwbytes==bufsize);
> > /*some condition*/
> >
> >
> > --
> > Regards
> > Sree Harsha Vardhana S.N
> > "When you want something, all the universe conspires in
> helping you to
> > achieve it."
> >
> > --
> > Regards
> > Sree Harsha Vardhana S.N
> > "When you want something, all the universe conspires in
> helping you to
> > achieve it."
> >
> >
> >
> >
> > --
> > Regards
> > Sree Harsha Vardhana S.N
> > "When you want something, all the universe conspires in helping
> you to
> > achieve it."
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
> <ma...@xerces.apache.org>
> For additional commands, e-mail: c-dev-help@xerces.apache.org
> <ma...@xerces.apache.org>
>
>
>
>
> --
> Regards
> Sree Harsha Vardhana S.N
> "When you want something, all the universe conspires in helping you to
> achieve it."
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
Re: Please help me out with this code..Can i reuse DOM objects in a while loop??????
Posted by Sree Harsha <sr...@gmail.com>.
Thanks Alberto,
I used resetDocumentPool(),
But I get a run time error R6025 pure virtual function call.
Thanks and regards
Sree Harsha Vardhana S.N
On 2/27/08, Alberto Massari <am...@datadirect.com> wrote:
>
> Hi Sree,
> you should invoke resetDocumentPool(), not reset(). As for the code you
> posted previously, I noticed you were not releasing the transcoded
> strings (you actually commented the call to XMLString::release): that
> would be a substantial leak.
> On the other hand, if you want to parse files so big, you should
> investigate switching to SAX parsing.
>
> Alberto
>
> Sree Harsha wrote:
> > Thanks for your reply, but still the memory consumption has not come
> > down. Is there any way i can reuse DOMDocument variable also like in
> > the following snippet...
> >
> > do
> > {
> >
> > /*create a source file */
> >
> > parser_.parse(source); //parser_ is a XercesDOMParser object
> >
> > xercesc::DOMDocument* xmlDoc = parser_.getDocument() ;
> >
> > /*How to release the memory allocated to the DOMDocument pointer*/
> >
> > /*parse the file source*/
> >
> > /*delete the source file*/
> >
> > /*I used the function which you suggested*/
> >
> > parser_.reset();
> >
> > /*still it takes up a lot of memory when parsing a big file*/
> >
> > }while(dwbytes==bufsize);
> > /*some condition*/
> >
> > Finally I also have one more doubt... When we use DOM, the entire file
> > to be parsed is brought into the memory isnt it... Now if I have to
> > parse a large file, say 380 MB or 1GB, we cant Bring the entire file
> > to the memory. So I thought it would be better to split the file into
> > smaller sizes and parse them bit by bit (4K long)... that is the
> > reason i have used create source file in the above code....
> > Am I in the Right path????
> >
> > On 2/26/08, *Jesse Pelton* <jsp@pkc.com <ma...@pkc.com>> wrote:
> >
> > It looks to me like you could call the various
> > XercesDOMParser::reset...() and AbstractDOMParser::reset...()
> > methods. Or, to be paranoid and/or lazy, you could use a new
> > parser for each iteration.
> >
> >
> ------------------------------------------------------------------------
> > *From:* Sree Harsha [mailto:sree.harsha.sn@gmail.com
> > <ma...@gmail.com>]
> > *Sent:* Tuesday, February 26, 2008 5:40 AM
> > *To:* boris@codesynthesis.com <ma...@codesynthesis.com>;
> > c-dev@xerces.apache.org <ma...@xerces.apache.org>
> > *Subject:* Please help me out with this code..Can i reuse DOM
> > objects in a while loop??????
> >
> >
> > /*Can i reuse the DOM objects like XercesDOMParser object in a
> > do... while loop as follows.... or will it cause any memory leaks
> > how to reuse them...*/
> > do
> > {
> >
> > /*create a source file */
> >
> > parser_.parse(source); //parser_ is a XercesDOMParser object
> >
> > /*parse the file source*/
> >
> > /*delete the source file*/
> >
> > /*what function should i call to reuse parser_ object in the next
> > iteration so that memory consumption reduces */
> >
> > }while(dwbytes==bufsize);
> > /*some condition*/
> >
> >
> > --
> > Regards
> > Sree Harsha Vardhana S.N
> > "When you want something, all the universe conspires in helping you
> to
> > achieve it."
> >
> > --
> > Regards
> > Sree Harsha Vardhana S.N
> > "When you want something, all the universe conspires in helping you
> to
> > achieve it."
> >
> >
> >
> >
> > --
> > Regards
> > Sree Harsha Vardhana S.N
> > "When you want something, all the universe conspires in helping you to
> > achieve it."
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
> For additional commands, e-mail: c-dev-help@xerces.apache.org
>
>
--
Regards
Sree Harsha Vardhana S.N
"When you want something, all the universe conspires in helping you to
achieve it."
Re: Please help me out with this code..Can i reuse DOM objects in
a while loop??????
Posted by Alberto Massari <am...@datadirect.com>.
Hi Sree,
you should invoke resetDocumentPool(), not reset(). As for the code you
posted previously, I noticed you were not releasing the transcoded
strings (you actually commented the call to XMLString::release): that
would be a substantial leak.
On the other hand, if you want to parse files so big, you should
investigate switching to SAX parsing.
Alberto
Sree Harsha wrote:
> Thanks for your reply, but still the memory consumption has not come
> down. Is there any way i can reuse DOMDocument variable also like in
> the following snippet...
>
> do
> {
>
> /*create a source file */
>
> parser_.parse(source); //parser_ is a XercesDOMParser object
>
> xercesc::DOMDocument* xmlDoc = parser_.getDocument() ;
>
> /*How to release the memory allocated to the DOMDocument pointer*/
>
> /*parse the file source*/
>
> /*delete the source file*/
>
> /*I used the function which you suggested*/
>
> parser_.reset();
>
> /*still it takes up a lot of memory when parsing a big file*/
>
> }while(dwbytes==bufsize);
> /*some condition*/
>
> Finally I also have one more doubt... When we use DOM, the entire file
> to be parsed is brought into the memory isnt it... Now if I have to
> parse a large file, say 380 MB or 1GB, we cant Bring the entire file
> to the memory. So I thought it would be better to split the file into
> smaller sizes and parse them bit by bit (4K long)... that is the
> reason i have used create source file in the above code....
> Am I in the Right path????
>
> On 2/26/08, *Jesse Pelton* <jsp@pkc.com <ma...@pkc.com>> wrote:
>
> It looks to me like you could call the various
> XercesDOMParser::reset...() and AbstractDOMParser::reset...()
> methods. Or, to be paranoid and/or lazy, you could use a new
> parser for each iteration.
>
> ------------------------------------------------------------------------
> *From:* Sree Harsha [mailto:sree.harsha.sn@gmail.com
> <ma...@gmail.com>]
> *Sent:* Tuesday, February 26, 2008 5:40 AM
> *To:* boris@codesynthesis.com <ma...@codesynthesis.com>;
> c-dev@xerces.apache.org <ma...@xerces.apache.org>
> *Subject:* Please help me out with this code..Can i reuse DOM
> objects in a while loop??????
>
>
> /*Can i reuse the DOM objects like XercesDOMParser object in a
> do... while loop as follows.... or will it cause any memory leaks
> how to reuse them...*/
> do
> {
>
> /*create a source file */
>
> parser_.parse(source); //parser_ is a XercesDOMParser object
>
> /*parse the file source*/
>
> /*delete the source file*/
>
> /*what function should i call to reuse parser_ object in the next
> iteration so that memory consumption reduces */
>
> }while(dwbytes==bufsize);
> /*some condition*/
>
>
> --
> Regards
> Sree Harsha Vardhana S.N
> "When you want something, all the universe conspires in helping you to
> achieve it."
>
> --
> Regards
> Sree Harsha Vardhana S.N
> "When you want something, all the universe conspires in helping you to
> achieve it."
>
>
>
>
> --
> Regards
> Sree Harsha Vardhana S.N
> "When you want something, all the universe conspires in helping you to
> achieve it."
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
Re: Please help me out with this code..Can i reuse DOM objects in a while loop??????
Posted by Sree Harsha <sr...@gmail.com>.
Thanks for your reply, but still the memory consumption has not come down.
Is there any way i can reuse DOMDocument variable also like in the following
snippet...
do
{
/*create a source file */
parser_.parse(source); //parser_ is a XercesDOMParser object
xercesc::DOMDocument* xmlDoc = parser_.getDocument() ;
/*How to release the memory allocated to the DOMDocument pointer*/
/*parse the file source*/
/*delete the source file*/
/*I used the function which you suggested*/
parser_.reset();
/*still it takes up a lot of memory when parsing a big file*/
}while(dwbytes==bufsize);
/*some condition*/
Finally I also have one more doubt... When we use DOM, the entire file to be
parsed is brought into the memory isnt it... Now if I have to parse a large
file, say 380 MB or 1GB, we cant Bring the entire file to the memory. So I
thought it would be better to split the file into smaller sizes and parse
them bit by bit (4K long)... that is the reason i have used create source
file in the above code....
Am I in the Right path????
On 2/26/08, Jesse Pelton <js...@pkc.com> wrote:
>
> It looks to me like you could call the various
> XercesDOMParser::reset...() and AbstractDOMParser::reset...() methods. Or,
> to be paranoid and/or lazy, you could use a new parser for each iteration.
>
> ------------------------------
> *From:* Sree Harsha [mailto:sree.harsha.sn@gmail.com]
> *Sent:* Tuesday, February 26, 2008 5:40 AM
> *To:* boris@codesynthesis.com; c-dev@xerces.apache.org
> *Subject:* Please help me out with this code..Can i reuse DOM objects in a
> while loop??????
>
>
> /*Can i reuse the DOM objects like XercesDOMParser object in a do...
> while loop as follows.... or will it cause any memory leaks how to reuse
> them...*/
> do
> {
>
> /*create a source file */
>
> parser_.parse(source); //parser_ is a XercesDOMParser object
>
> /*parse the file source*/
>
> /*delete the source file*/
>
> /*what function should i call to reuse parser_ object in the next
> iteration so that memory consumption reduces */
>
> }while(dwbytes==bufsize);
> /*some condition*/
>
>
> --
> Regards
> Sree Harsha Vardhana S.N
> "When you want something, all the universe conspires in helping you to
> achieve it."
>
> --
> Regards
> Sree Harsha Vardhana S.N
> "When you want something, all the universe conspires in helping you to
> achieve it."
>
--
Regards
Sree Harsha Vardhana S.N
"When you want something, all the universe conspires in helping you to
achieve it."
RE: Please help me out with this code..Can i reuse DOM objects in a while loop??????
Posted by Jesse Pelton <js...@PKC.com>.
It looks to me like you could call the various
XercesDOMParser::reset...() and AbstractDOMParser::reset...() methods.
Or, to be paranoid and/or lazy, you could use a new parser for each
iteration.
________________________________
From: Sree Harsha [mailto:sree.harsha.sn@gmail.com]
Sent: Tuesday, February 26, 2008 5:40 AM
To: boris@codesynthesis.com; c-dev@xerces.apache.org
Subject: Please help me out with this code..Can i reuse DOM objects in a
while loop??????
/*Can i reuse the DOM objects like XercesDOMParser object in a do...
while loop as follows.... or will it cause any memory leaks how to reuse
them...*/
do
{
/*create a source file */
parser_.parse(source); //parser_ is a XercesDOMParser object
/*parse the file source*/
/*delete the source file*/
/*what function should i call to reuse parser_ object in the next
iteration so that memory consumption reduces */
}while(dwbytes==bufsize);
/*some condition*/
--
Regards
Sree Harsha Vardhana S.N
"When you want something, all the universe conspires in helping you to
achieve it."
--
Regards
Sree Harsha Vardhana S.N
"When you want something, all the universe conspires in helping you to
achieve it."