You are viewing a plain text version of this content. The canonical link for it is here.
Posted to c-users@xerces.apache.org by Александр Федоров <a....@gmail.com> on 2019/05/29 07:24:51 UTC
leak
Hello!
I have simple program which reuses created document with different elements
that seems to be leaking. To demonstrate the problem I create new
DOMElement, add it to document, then immediately remove it and release. I
do it for rather big number of times.
Here is the code snippet:
#include <xercesc/util/PlatformUtils.hpp>
#include <xercesc/dom/DOM.hpp>
#include <xercesc/util/XMLString.hpp>
#include <iostream>
#include <fstream>
#include <zconf.h>
void process_mem_usage() {
double vm = 0.;
double rss = 0.;
unsigned long vsize;
long lrss;
{
std::string ignore;
std::ifstream ifs("/proc/self/stat", std::ios_base::in);
ifs >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore
>> ignore >> ignore >> ignore
>> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >>
ignore >> ignore >> ignore
>> ignore >> ignore >> vsize >> lrss;
}
long page_size_kb = sysconf(_SC_PAGE_SIZE) / 1024;
vm = vsize / 1024.;
rss = lrss * page_size_kb;
std::cout << "VM: " << vm << "; RSS: " << rss << " (Kb)" << std::endl;
}
int main() {
using namespace xercesc_3_2;
XMLPlatformUtils::Initialize();
XMLCh * const tempStr = new XMLCh[100];
XMLString::transcode("Range", tempStr, 99);
DOMImplementation *impl =
DOMImplementationRegistry::getDOMImplementation(tempStr);
XMLString::transcode("root", tempStr, 99);
DOMDocument *doc = impl->createDocument(nullptr, tempStr, nullptr);
DOMElement *root = doc->getDocumentElement();
process_mem_usage();
for(auto i = 0; i < 100; ++i) {
for(auto j = 0; j < 1000; ++j) {
XMLString::transcode("FirstElement", tempStr, 99);
DOMElement *el = doc->createElement(tempStr);
root->appendChild(el);
auto removed = (DOMElement *) root->removeChild(el);
removed->release();
}
process_mem_usage();
}
doc->release();
process_mem_usage();
delete[] tempStr;
process_mem_usage();
XMLPlatformUtils::Terminate()
return 0;
}
So I expect RSS to be unchanged or quite stable, but it rises continuously.
--
*Kind regards,*
* Alexander.*
Re: leak
Posted by Ben Griffin <be...@redsnapper.net>.
I don’t really use this stuff anymore as it’s not actively developed.
However, I recommend you separate your transcoding test from everything else. IIRC transcoding had a memory leak in it.
The code I used in the end for transcoding can be found here https://github.com/RedSnapper/Obyx/blob/master/commons/xml/manager.cpp
It’s not great code, but it did resolve the leaks I otherwise found.
Hope it is of some help to you in your journey.
> On 1 Jun 2019, at 07:03, Александр Федоров <a....@gmail.com> wrote:
>
> Hello Joao!
> Thank you!
> I do not expect any interesting results from valgrind. As you can see,
> memory successfully released with the DOMDocument object release.
> What help do you think I can get from valgrind?
>
> пт, 31 мая 2019 г., 1:53 João M. S. Silva <jo...@gmail.com>:
>
>> Hello Александр,
>>
>> Have you tried Valgrind?
>>
>> João M. S. Silva
>>
>>> On 5/29/19 8:24 AM, Александр Федоров wrote:
>>> Hello!
>>> I have simple program which reuses created document with different
>> elements
>>> that seems to be leaking. To demonstrate the problem I create new
>>> DOMElement, add it to document, then immediately remove it and release. I
>>> do it for rather big number of times.
>>>
>>> Here is the code snippet:
>>>
>>> #include <xercesc/util/PlatformUtils.hpp>
>>> #include <xercesc/dom/DOM.hpp>
>>> #include <xercesc/util/XMLString.hpp>
>>> #include <iostream>
>>> #include <fstream>
>>> #include <zconf.h>
>>>
>>> void process_mem_usage() {
>>> double vm = 0.;
>>> double rss = 0.;
>>>
>>> unsigned long vsize;
>>> long lrss;
>>> {
>>> std::string ignore;
>>> std::ifstream ifs("/proc/self/stat", std::ios_base::in);
>>> ifs >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore
>>>>> ignore >> ignore >> ignore
>>>>> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >>
>>> ignore >> ignore >> ignore
>>>>> ignore >> ignore >> vsize >> lrss;
>>> }
>>>
>>> long page_size_kb = sysconf(_SC_PAGE_SIZE) / 1024;
>>> vm = vsize / 1024.;
>>> rss = lrss * page_size_kb;
>>> std::cout << "VM: " << vm << "; RSS: " << rss << " (Kb)" << std::endl;
>>> }
>>>
>>> int main() {
>>> using namespace xercesc_3_2;
>>>
>>> XMLPlatformUtils::Initialize();
>>>
>>> XMLCh * const tempStr = new XMLCh[100];
>>> XMLString::transcode("Range", tempStr, 99);
>>> DOMImplementation *impl =
>>> DOMImplementationRegistry::getDOMImplementation(tempStr);
>>>
>>> XMLString::transcode("root", tempStr, 99);
>>> DOMDocument *doc = impl->createDocument(nullptr, tempStr, nullptr);
>>> DOMElement *root = doc->getDocumentElement();
>>>
>>> process_mem_usage();
>>>
>>> for(auto i = 0; i < 100; ++i) {
>>> for(auto j = 0; j < 1000; ++j) {
>>> XMLString::transcode("FirstElement", tempStr, 99);
>>> DOMElement *el = doc->createElement(tempStr);
>>> root->appendChild(el);
>>>
>>> auto removed = (DOMElement *) root->removeChild(el);
>>> removed->release();
>>> }
>>> process_mem_usage();
>>> }
>>>
>>> doc->release();
>>> process_mem_usage();
>>> delete[] tempStr;
>>> process_mem_usage();
>>>
>>> XMLPlatformUtils::Terminate()
>>> return 0;
>>> }
>>>
>>> So I expect RSS to be unchanged or quite stable, but it rises
>> continuously.
>>>
>>>
>>
Re: leak
Posted by "João M. S. Silva" <jo...@gmail.com>.
I'm not sure, but Valgrind should be able to pinpoint where the leak is?
João M. S. Silva
On 6/1/19 7:03 AM, Александр Федоров wrote:
> Hello Joao!
> Thank you!
> I do not expect any interesting results from valgrind. As you can see,
> memory successfully released with the DOMDocument object release.
> What help do you think I can get from valgrind?
>
> пт, 31 мая 2019 г., 1:53 João M. S. Silva <jo...@gmail.com>:
>
>> Hello Александр,
>>
>> Have you tried Valgrind?
>>
>> João M. S. Silva
>>
>> On 5/29/19 8:24 AM, Александр Федоров wrote:
>>> Hello!
>>> I have simple program which reuses created document with different
>> elements
>>> that seems to be leaking. To demonstrate the problem I create new
>>> DOMElement, add it to document, then immediately remove it and release. I
>>> do it for rather big number of times.
>>>
>>> Here is the code snippet:
>>>
>>> #include <xercesc/util/PlatformUtils.hpp>
>>> #include <xercesc/dom/DOM.hpp>
>>> #include <xercesc/util/XMLString.hpp>
>>> #include <iostream>
>>> #include <fstream>
>>> #include <zconf.h>
>>>
>>> void process_mem_usage() {
>>> double vm = 0.;
>>> double rss = 0.;
>>>
>>> unsigned long vsize;
>>> long lrss;
>>> {
>>> std::string ignore;
>>> std::ifstream ifs("/proc/self/stat", std::ios_base::in);
>>> ifs >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore
>>>>> ignore >> ignore >> ignore
>>>>> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >>
>>> ignore >> ignore >> ignore
>>>>> ignore >> ignore >> vsize >> lrss;
>>> }
>>>
>>> long page_size_kb = sysconf(_SC_PAGE_SIZE) / 1024;
>>> vm = vsize / 1024.;
>>> rss = lrss * page_size_kb;
>>> std::cout << "VM: " << vm << "; RSS: " << rss << " (Kb)" << std::endl;
>>> }
>>>
>>> int main() {
>>> using namespace xercesc_3_2;
>>>
>>> XMLPlatformUtils::Initialize();
>>>
>>> XMLCh * const tempStr = new XMLCh[100];
>>> XMLString::transcode("Range", tempStr, 99);
>>> DOMImplementation *impl =
>>> DOMImplementationRegistry::getDOMImplementation(tempStr);
>>>
>>> XMLString::transcode("root", tempStr, 99);
>>> DOMDocument *doc = impl->createDocument(nullptr, tempStr, nullptr);
>>> DOMElement *root = doc->getDocumentElement();
>>>
>>> process_mem_usage();
>>>
>>> for(auto i = 0; i < 100; ++i) {
>>> for(auto j = 0; j < 1000; ++j) {
>>> XMLString::transcode("FirstElement", tempStr, 99);
>>> DOMElement *el = doc->createElement(tempStr);
>>> root->appendChild(el);
>>>
>>> auto removed = (DOMElement *) root->removeChild(el);
>>> removed->release();
>>> }
>>> process_mem_usage();
>>> }
>>>
>>> doc->release();
>>> process_mem_usage();
>>> delete[] tempStr;
>>> process_mem_usage();
>>>
>>> XMLPlatformUtils::Terminate()
>>> return 0;
>>> }
>>>
>>> So I expect RSS to be unchanged or quite stable, but it rises
>> continuously.
>>>
>>>
>>
>
Re: leak
Posted by Александр Федоров <a....@gmail.com>.
Hello Joao!
Thank you!
I do not expect any interesting results from valgrind. As you can see,
memory successfully released with the DOMDocument object release.
What help do you think I can get from valgrind?
пт, 31 мая 2019 г., 1:53 João M. S. Silva <jo...@gmail.com>:
> Hello Александр,
>
> Have you tried Valgrind?
>
> João M. S. Silva
>
> On 5/29/19 8:24 AM, Александр Федоров wrote:
> > Hello!
> > I have simple program which reuses created document with different
> elements
> > that seems to be leaking. To demonstrate the problem I create new
> > DOMElement, add it to document, then immediately remove it and release. I
> > do it for rather big number of times.
> >
> > Here is the code snippet:
> >
> > #include <xercesc/util/PlatformUtils.hpp>
> > #include <xercesc/dom/DOM.hpp>
> > #include <xercesc/util/XMLString.hpp>
> > #include <iostream>
> > #include <fstream>
> > #include <zconf.h>
> >
> > void process_mem_usage() {
> > double vm = 0.;
> > double rss = 0.;
> >
> > unsigned long vsize;
> > long lrss;
> > {
> > std::string ignore;
> > std::ifstream ifs("/proc/self/stat", std::ios_base::in);
> > ifs >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore
> >>> ignore >> ignore >> ignore
> >>> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >>
> > ignore >> ignore >> ignore
> >>> ignore >> ignore >> vsize >> lrss;
> > }
> >
> > long page_size_kb = sysconf(_SC_PAGE_SIZE) / 1024;
> > vm = vsize / 1024.;
> > rss = lrss * page_size_kb;
> > std::cout << "VM: " << vm << "; RSS: " << rss << " (Kb)" << std::endl;
> > }
> >
> > int main() {
> > using namespace xercesc_3_2;
> >
> > XMLPlatformUtils::Initialize();
> >
> > XMLCh * const tempStr = new XMLCh[100];
> > XMLString::transcode("Range", tempStr, 99);
> > DOMImplementation *impl =
> > DOMImplementationRegistry::getDOMImplementation(tempStr);
> >
> > XMLString::transcode("root", tempStr, 99);
> > DOMDocument *doc = impl->createDocument(nullptr, tempStr, nullptr);
> > DOMElement *root = doc->getDocumentElement();
> >
> > process_mem_usage();
> >
> > for(auto i = 0; i < 100; ++i) {
> > for(auto j = 0; j < 1000; ++j) {
> > XMLString::transcode("FirstElement", tempStr, 99);
> > DOMElement *el = doc->createElement(tempStr);
> > root->appendChild(el);
> >
> > auto removed = (DOMElement *) root->removeChild(el);
> > removed->release();
> > }
> > process_mem_usage();
> > }
> >
> > doc->release();
> > process_mem_usage();
> > delete[] tempStr;
> > process_mem_usage();
> >
> > XMLPlatformUtils::Terminate()
> > return 0;
> > }
> >
> > So I expect RSS to be unchanged or quite stable, but it rises
> continuously.
> >
> >
>
Re: leak
Posted by "João M. S. Silva" <jo...@gmail.com>.
Hello Александр,
Have you tried Valgrind?
João M. S. Silva
On 5/29/19 8:24 AM, Александр Федоров wrote:
> Hello!
> I have simple program which reuses created document with different elements
> that seems to be leaking. To demonstrate the problem I create new
> DOMElement, add it to document, then immediately remove it and release. I
> do it for rather big number of times.
>
> Here is the code snippet:
>
> #include <xercesc/util/PlatformUtils.hpp>
> #include <xercesc/dom/DOM.hpp>
> #include <xercesc/util/XMLString.hpp>
> #include <iostream>
> #include <fstream>
> #include <zconf.h>
>
> void process_mem_usage() {
> double vm = 0.;
> double rss = 0.;
>
> unsigned long vsize;
> long lrss;
> {
> std::string ignore;
> std::ifstream ifs("/proc/self/stat", std::ios_base::in);
> ifs >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore
>>> ignore >> ignore >> ignore
>>> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >>
> ignore >> ignore >> ignore
>>> ignore >> ignore >> vsize >> lrss;
> }
>
> long page_size_kb = sysconf(_SC_PAGE_SIZE) / 1024;
> vm = vsize / 1024.;
> rss = lrss * page_size_kb;
> std::cout << "VM: " << vm << "; RSS: " << rss << " (Kb)" << std::endl;
> }
>
> int main() {
> using namespace xercesc_3_2;
>
> XMLPlatformUtils::Initialize();
>
> XMLCh * const tempStr = new XMLCh[100];
> XMLString::transcode("Range", tempStr, 99);
> DOMImplementation *impl =
> DOMImplementationRegistry::getDOMImplementation(tempStr);
>
> XMLString::transcode("root", tempStr, 99);
> DOMDocument *doc = impl->createDocument(nullptr, tempStr, nullptr);
> DOMElement *root = doc->getDocumentElement();
>
> process_mem_usage();
>
> for(auto i = 0; i < 100; ++i) {
> for(auto j = 0; j < 1000; ++j) {
> XMLString::transcode("FirstElement", tempStr, 99);
> DOMElement *el = doc->createElement(tempStr);
> root->appendChild(el);
>
> auto removed = (DOMElement *) root->removeChild(el);
> removed->release();
> }
> process_mem_usage();
> }
>
> doc->release();
> process_mem_usage();
> delete[] tempStr;
> process_mem_usage();
>
> XMLPlatformUtils::Terminate()
> return 0;
> }
>
> So I expect RSS to be unchanged or quite stable, but it rises continuously.
>
>
Possible leakage
Posted by Александр Федоров <a....@gmail.com>.
Hello!
I have simple program which reuses created document with different elements
that seems to be leaking. To demonstrate the problem I create new
DOMElement, add it to document, then immediately remove it and release. I
do it for rather big number of times.
Here is the code snippet:
#include <xercesc/util/PlatformUtils.hpp>
#include <xercesc/dom/DOM.hpp>
#include <xercesc/util/XMLString.hpp>
#include <iostream>
#include <fstream>
#include <zconf.h>
void process_mem_usage() {
double vm = 0.;
double rss = 0.;
unsigned long vsize;
long lrss;
{
std::string ignore;
std::ifstream ifs("/proc/self/stat", std::ios_base::in);
ifs >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore
>> ignore >> ignore >> ignore
>> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >> ignore >>
ignore >> ignore >> ignore
>> ignore >> ignore >> vsize >> lrss;
}
long page_size_kb = sysconf(_SC_PAGE_SIZE) / 1024;
vm = vsize / 1024.;
rss = lrss * page_size_kb;
std::cout << "VM: " << vm << "; RSS: " << rss << " (Kb)" << std::endl;
}
int main() {
using namespace xercesc_3_2;
XMLPlatformUtils::Initialize();
XMLCh * const tempStr = new XMLCh[100];
XMLString::transcode("Range", tempStr, 99);
DOMImplementation *impl =
DOMImplementationRegistry::getDOMImplementation(tempStr);
XMLString::transcode("root", tempStr, 99);
DOMDocument *doc = impl->createDocument(nullptr, tempStr, nullptr);
DOMElement *root = doc->getDocumentElement();
process_mem_usage();
for(auto i = 0; i < 100; ++i) {
for(auto j = 0; j < 1000; ++j) {
XMLString::transcode("FirstElement", tempStr, 99);
DOMElement *el = doc->createElement(tempStr);
root->appendChild(el);
auto removed = (DOMElement *) root->removeChild(el);
removed->release();
}
process_mem_usage();
}
doc->release();
process_mem_usage();
delete[] tempStr;
process_mem_usage();
XMLPlatformUtils::Terminate()
return 0;
}
So I expect RSS to be unchanged or quite stable, but it rises continuously.
--
*Kind regards,*
* Alexander.*