You are viewing a plain text version of this content. The canonical link for it is here.
Posted to c-dev@xerces.apache.org by Krishnan RG <rk...@icope.com> on 2002/08/22 10:40:46 UTC
Re: PROBLEM SOLVED - A Question on DOM_Node::cloneNode() Usage...
Hi Everybody!!
Thanks to your all your help I was able to solve my problem.
The solution is as follows...
Tinny was right in saying that cloning a document creates a NEW Document.
The problem is not wth cloneNode().
The problem was that I WAS NOT cloning the DOM_Document. Instead I was
cloning the ROOT ELEMENT of the master document (i.e.
masterDocument.getDocumentelement().cloneNode(deep)). Hence the new copy was
also associated with the master document and released ONLY when the master
document was released.
I could have either followed Erik's suggestion and used importNode(). Rather
I chose to correct the mistake and I am now cloning the master document.
(i.e. masterDocument.cloneNode(deep) )
Thanks a million Erik, Jesse and Tinny for your help. I think this mailing
list is great! :)
Krishnan.
Something to be thought about.
-------------------------------
Let us say that my xml indeed had OVER 997 id attributes. In this case the
moment the number of id attributes crosses 997, the table should be grown.
After the table has grown the previous objects should still be valid.
BUT from my experience I think once the table grows the previous objects are
all invalid.
I say this because my previous program used to crash EVERYTIME the table
grew. Not only when there were no more primes in the gPrimes[] array.
Can somebody please cross verify on this??
Thanks Once Again!!
Krishnan
----- Original Message -----
From: "Tinny Ng" <tn...@ca.ibm.com>
To: <xe...@xml.apache.org>
Sent: Wednesday, August 21, 2002 8:54 PM
Subject: Re: A Question on DOM_Node::cloneNode() Usage...
Krishnan,
One second, when you cloneNode(), you can cloning the doc itself (e.g.
doc.cloneNode(true)), right? According to the code, the clone of the
document will have its own memory:
NodeImpl *DocumentImpl::cloneNode(bool deep) {
// clone the node itself
DocumentImpl *newdoc = new DocumentImpl(); <== the newdoc has its own
heap
// then the children by _importing_ them
if (deep) {
for (ChildNode *n = firstChild; n != null; n = n->nextSibling) {
newdoc->appendChild(newdoc->importNode(n, true));
}
}
newdoc->setErrorChecking(errorChecking);
return newdoc;
};
So I don't think the problem is related to the clone.
In fact the NodeIdMap is used only for ID type attribute, does your big xml
document have a LOT of ID attribute, more than 999983?
Tinny
----- Original Message -----
From: "Erik Rydgren" <er...@mandarinen.se>
To: <xe...@xml.apache.org>
Sent: Wednesday, August 21, 2002 9:50 AM
Subject: RE: A Question on DOM_Node::cloneNode() Usage...
> There it is!!!. CloneNode creates new nodes inside the original documents
> memory allocator and that memory is only released when the document dies.
> Because you never release the document then the memory never is either.
Use
> ImportNode(deep) into another newly created empty document instead. It
will
> then release the memory when that document is released. Problem solved.
>
> / Erik
>
>
> -----Original Message-----
> From: Krishnan RG [mailto:rkrishnan@icope.com]
> Sent: den 21 augusti 2002 15:31
> To: xerces-c-dev@xml.apache.org
> Subject: Re: A Question on DOM_Node::cloneNode() Usage...
>
>
> Yes...
> I would also like to find out why the table is filling up in the first
> place. I think this is the ley to the problem. I would like to add more to
> what I have already written before.
>
> 1) I create a master DOM by parsing a document.
> 2) Each time I need a new copy of the DOM I use cloneNode(deep=true) to
> generate one.
> 3) As soon as I am finished with the copy I release it!!
> 4) BUT it seems that these copies are actually released from memory ONLY
> when the master DOM is also released.
>
> Is this the correct behaviour??
> The cloneNode() documentation says that the parent of the newly generates
> duplicate node will be NULL. Hence I assumed that the duplicate would be
> independent of the Original node. But this does not seem to be the case.
>
> If the duplicate copies are released from memory as soon as I release
them,
> then I think the problem should be solved. Otherwise the accumulation of
> nodes (node leak) will continue.
>
> How can I make this happen? Any clues?
>
> Many many thanks in your efforts to help me,
>
> Krishnan.
>
> ----- Original Message -----
> From: "Jesse Pelton" <js...@PKC.com>
> To: <xe...@xml.apache.org>
> Sent: Wednesday, August 21, 2002 6:21 PM
> Subject: RE: A Question on DOM_Node::cloneNode() Usage...
>
>
> Looks like each prime in the list is the largest prime below a power of
ten.
> The first entry looks like the largest prime less than 1,000, the second
> like the largest prime below 10,000, and so on. The next one in the series
> would be the largest prime below 10,000,000, which according to
> http://www.rsok.com/cgi-bin/printprimes_cgi is 9,999,991. (If we're
> determined to pursue this approach, the next two numbers in the series
would
> be 99,999,989 and 999,999,937.)
>
> Before we enable the creation of huge NodeIDMaps, thouhg, it might be
worth
> exploring whether there's some way to avoid filling it in the first place.
> Given the presented scenario, the table will fill eventually, unless the
> system runs out of memory first.
>
> -----Original Message-----
> From: Erik Rydgren [mailto:erik.rydgren@mandarinen.se]
> Sent: Wednesday, August 21, 2002 8:23 AM
> To: xerces-c-dev@xml.apache.org
> Subject: RE: A Question on DOM_Node::cloneNode() Usage...
>
>
> The error is produced in this code.
>
> //
> // Grow the table to the next larger size.
> // It has gotten too full for efficient operation.
> // (We never fill it all the way)
> //
> void NodeIDMap::growTable()
> {
> AttrImpl **oldTable = fTable;
> unsigned int oldSize = fSize;
>
> //
> // Figure the new table size.
> //
> fprintf(stderr, "growing...\n");
> fSizeIndex++;
> fSize = gPrimes[fSizeIndex];
> if (fSize == 0)
> {
> // We need to grow bigger than the largest available size.
> // Big trouble.
> fSizeIndex--;
> throw "NodeIDMap::growTable - big trouble."; <<-- My guess is that
> this is the exception you get.
> }
>
> ... and higher up in the same file i found:
>
> static const int gPrimes[] = {997, 9973, 99991, 999983, 0 }; // To do -
add
> a few more.
> static const float gMaxFill = 0.8f; // The maximum fraction of the total
> table entries to consume before exanding.
>
> So my guess is that you have exceeded 999983*0.8 number of nodes and the
> system needs a bigger table.
> But what the next prime should be I leave up to others.
> Anyone?
>
> Best regards
> Erik Rydgren
>
> -----Original Message-----
> From: Krishnan RG [mailto:rkrishnan@icope.com]
> Sent: den 21 augusti 2002 12:56
> To: xerces-c-dev@xml.apache.org
> Subject: Fw: A Question on DOM_Node::cloneNode() Usage...
>
>
> Hi Everybody,
>
> I introduce myself as Krishnan, a software developer, working for iCOPE
> Technologies, Bangalore.
>
> I have been trying to use the Xerces C++ XML Parser to parse XML documents
> to build a server application on Windows 2000. But I have been having some
> small problems. I would warmly appreciate and welcome any help that can be
> offered in this regard. Please find below the details of my usage of the
> Xerces Parser and the problem that I face.
>
> Thanks a ton for your time and help,
> Krishnan.
>
> USAGE OF XERCES-1-7-0
> ----------------------
> 1) When my server starts, I initialize the XMLPlatform. My server is
> multithreaded. I ensure that XMLPlatfrUtils::Initialize() is called ONLY
> ONCE in the entire app.
>
> 2) The Main thread of my app instantiates a SINGLE instance of the Parser.
>
> 3) Worker threads in my app need to parse documents and process requests
at
> run time.
>
> 4) The app maintains a cached list of DOM_Document objects containing
> already parsed documents. (Initially, this cache is empty.)
>
> 5) When a worker thread needs to parse a document, it checks the cache to
> see if the required document is already parsed...
> 5a)If already parsed, it uses the cloneNode(deep = true) method to
> generate a duplicate of the document and uses this duplicate document for
> further processing.
> 5b)If not already parsed, it parses the document and first adds it to
> the cache. It then uses cloneNode(deep = true) to generate a duplicate of
> this document and uses this duplicate document for further processing.
>
> NOTE: Only one worker thread can acces the cache and call cloneNode() at a
> particular point in time. Access to the cache is mutally exclusive.
>
> 6) The duplicate is released by the thread after completion of document
> processing.
>
> 7) When the server shuts down, the main thread cleans the cache by
releasing
> all DOM_Document objects. Finally it calls XMLPlatformUtils::Terminate()
> ONLY ONCE and winds up.
>
> MY PROBLEM
> -----------
> My server needs to process some 500 requests a minute. And the processing
of
> each request requires a parsed copy of document. Hence I implemented
> caching. The strategy I use works fine most of the time. But after some
> time, approximately 10 or 15 minutes (sooner if I use a bigger document),
> Xerces prints the message
> "growing..."
> on the console. The very next call to cloneNode(), immediately after this
> "growing..." message is displayed, throws an exception in the XERCES DLL.
>
> ******First-chance exception in Switch.exe (XERCES-C_1_7_0D.DLL):
> 0xC0000005: Access Violation.*******
>
> I think this is because Xerces reallocates some internal memory and hence
my
> DOM_Document object becomes invalid. But how do I ensure that my
> DOM_Document object and other Xerces objects remains valid even after
> REALLOCATION?? Is there any way for this?
>
> I have been trying to solve this from a long time. But I have not found
the
> solution to it. I would highly appreciate if anyone could throw some light
> on this problem!
>
>
> Many Thanks,
> Krishnan RG
> Systems Engineer,
> iCOPE Technologies,
> Bangalore.
> mailto:rkrishnan@icope.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-c-dev-help@xml.apache.org
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-c-dev-help@xml.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-c-dev-help@xml.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-c-dev-help@xml.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-c-dev-help@xml.apache.org