Memory leaks in xerces-c_1_1.dll

[Alex Matevossian <am...@ipnet-solutions.com>]

Hi Andy,

This is a follow-up to the email sent 2 days ago by Steve Chuang re memory
leaks in Xerces.

Summary
Xerces DLL allocates memory on the heap during initialization.  Since the
memory is not cleaned automatically by the DLL and the DLL provides no
mechanism to release the memory explicitly, the memory will be released only
when the process hosting the DLL terminates.

This creates memory leaks if the DLL is dynamically loaded and unloaded by
the hosting process.

Details
Before using any functionality provided by xerces-c_1_1.dll, one needs to
call void XMLPlatformUtils::Initialize() to initialize the library.  Several
objects are allocated on the heap during this initialization (it is possible
that other resources are allocated, but we did not conduct a thorough
examination).

To avoid redundant object allocations, the bool XMLPlatformUtils::fgInitFlag
is used to indicate whether this is the first or subsequent invocation of
XMLPlatformUtils::Initialize().

The approach outlined above works perfectly well if the DLL is loaded no
more than once into the hosting process.  If the DLL is loaded more than
once, the XMLPlatformUtils::fgInitFlag is reset every on every load.  This
will cause another allocation of the objects on the heap.  Since there is no
way to release the resources allocated on the heap by the DLL, every time
the DLL is loaded and initialized, another set of objects will be allocated
on the heap.

On the other hand, I believe it is fair to say that the implementer of the
DLL cannot assume how the DLL is used (i.e., statically linked or
dynamically loaded, loaded once or many times, etc.).  That is why we would
like to ask you to either (1) add automatic cleanup (possibly triggered in
the DllMain() on DLL_PROCESS_DETACH event) or (2) provide an explicit
mechanism for releasing the resources (e.g.,
XMLPlatformUtils::UnInitialize() or something similar in nature).  We would
prefer to have both features; however, if you can only implement one of the
features, the explicit control of resources seems more important to us.

I am attaching a zip file containing the VC 6 projects illustrating the
problem.  In this project I also create mutexes (named and unnamed) and file
(just to show that those types of resources leak as well as heap memory; I
don't know whether xerces-c_1_1.dll creates those during initialization or
not).

The project structure is as follows:

XC\
  Common.h - file included by all projects.  This file controls which
objects are allocated.

  client\ - the project that generates the hosting exe

  comLib\ - the project that generates an intermediate DLL.  This DLL is
dynamically loaded/freed by the client.exe.  This DLL is statically linked
to the XercesLib.dll

  XercesLib\ - the project that generates XercesLib.dll.  This DLL simulates
xerces-c_1_1.dll.
    XercesLib.dsw - the workspace that includes all three projects with the
appropriately set dependencies.

  Debug\ - the directory containing the debug builds of all three projects.
  Release\ - the directory containing the release builds of all three
projects.

To simplify the testing, XercesLib exports only this:

class XERCESLIB_API XMLPlatformUtils
{
public :
    static void Initialize();

private :

    static bool     fgInitFlag;
};

There are #define's that allow you control the number of times XercesLib.dll
is loaded/called/unloaded, the size of heap allocations, and whether to
create mutex and files (files are created in "C:\Temp", if you don't have
this directory, you should either create it or modify the path in *.cpp
files).

Thank you in advance for your help.

Regards,
Alex Matevossian.
IPNet Solutions.

 <<Xc.zip>> 

Re: Memory leaks in xerces-c_1_1.dll

[Andy Heninger <he...@us.ibm.com>]

Hi Alex,

   You are correct.  Xerces will leak if the DLL is dynamically loaded,
unloaded, and loaded again.  As Dean notes in his earlier reply, he has
added a termination method and a hook by which this kind of storage can be
recovered.

An example of use of the new mechanism for registering an object for
deletion at termination is in the file src/internal/XMLScanner, at line 197.

The application code triggers the termination cleanup  by calling
XMLPlatformUtils::Terminate() just before unloading the Xerces DLL.

As Dean noted, not all of the lazily allocated storage is currently set up
to use this mechanism - actually, most of it doesn't yet.  We will be
getting to this over time, but if you would like to speed things along, I'll
be more than happy to put any patches for this back into CVS.

  Best regards,

   -- Andy


From: "Alex Matevossian" <am...@ipnet-solutions.com>
> This is a follow-up to the email sent 2 days ago by Steve Chuang re memory
> leaks in Xerces.
>
> Summary
> Xerces DLL allocates memory on the heap during initialization.  Since the
> memory is not cleaned automatically by the DLL and the DLL provides no
> mechanism to release the memory explicitly, the memory will be released
only
> when the process hosting the DLL terminates.
>
> This creates memory leaks if the DLL is dynamically loaded and unloaded by
> the hosting process.
>
  [Details deleted]