You are viewing a plain text version of this content. The canonical link for it is here.
Posted to c-dev@xerces.apache.org by "tao xiaochuan (JIRA)" <xe...@xml.apache.org> on 2009/12/14 09:58:18 UTC
[jira] Created: (XERCESC-1901) memory address leak when running
many threads which will parsing xml files
memory address leak when running many threads which will parsing xml files
--------------------------------------------------------------------------
Key: XERCESC-1901
URL: https://issues.apache.org/jira/browse/XERCESC-1901
Project: Xerces-C++
Issue Type: Bug
Components: SAX/SAX2
Affects Versions: 3.0.1
Environment: Red Hat Enterprise Linux Server release 5.1 (Tikanga)
-g++ --version : x86_64-redhat-linux-g++ (GCC) 4.1.1 20070105 (Red Hat 4.1.1-52)
Reporter: tao xiaochuan
when running 5 threads , in each thread a xml file will (for test purpost xerces-c-3.0.1\samples\data\long.xml is used)
be parsed, it is found, by top command , VIRT increase large , by pmap command , 3 (some time 4) memory address block about 64 m
are found , when by 8 threads , there will be 5 (some time 6) such blocks :
Address Kbytes Mode Offset Device Mapping
00002aaab00b9000 64796 ----- 00002aaab00b9000 000:00000 [ anon ]
...
00002aaab40b9000 64796 ----- 00002aaab40b9000 000:00000 [ anon ]
...
00002aaab80bb000 64788 ----- 00002aaab80bb000 000:00000 [ anon ]
these blocks will not be released even after XMLPlatformUtils::Terminate();
if there is only one thead , there is no such a problem
chief code structure:
main(){
XMLPlatformUtils::Initialize();
//start many threads
//wait for alll threads ends
XMLPlatformUtils::Terminate();
}
in each thread :
while(true){
...
auto_ptr<SAXParser> parser( new SAXParser() );
parser->setDocumentHandler(this); //the handler do nothing but just extend from HandlerBase
parser->setErrorHandler(this); //the handler do nothing but just extend from HandlerBase
parser->setValidationScheme(SAXParser::Val_Never);
parser->setDoSchema(false);
parser->setDoNamespaces(true);
parser->parse(inputFile.c_str());
}
refer to attachment for detail about the simple test code
one more related question :
in the doc , it is said "client application needs to Terminate() (or multiple Terminate() in the case where multiple Initialize() have been invoked before" , does it also suit for more than one threads ?
for example :
thread 1 : XMLPlatformUtils::Initialize(); //A
thread 1 : do some parsing //B
thread 2 : XMLPlatformUtils::Initialize(); //C
thread 2 : do some parsing //D
thread 1 : do some parsing //E
thread 1 : XMLPlatformUtils::Terminate(); //F
thread 2 : do some parsing //G
thread 1 : XMLPlatformUtils::Terminate(); //H
}
since these two api are all static , then does C will affect E and does F affect G
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
[jira] Closed: (XERCESC-1901) memory address leak when running many
threads which will parsing xml files
Posted by "tao xiaochuan (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESC-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
tao xiaochuan closed XERCESC-1901.
----------------------------------
Resolution: Not A Problem
it is caused by version of glibc
rpm -qf /lib64/ld-2.5.so
glibc-2.5-24
when it is changed to
rpm -qf /lib64/ld-2.5.so
glibc-2.5-42
problem is gone
> memory address leak when running many threads which will parsing xml files
> --------------------------------------------------------------------------
>
> Key: XERCESC-1901
> URL: https://issues.apache.org/jira/browse/XERCESC-1901
> Project: Xerces-C++
> Issue Type: Bug
> Components: SAX/SAX2
> Affects Versions: 3.0.1
> Environment: Red Hat Enterprise Linux Server release 5.1 (Tikanga)
> -g++ --version : x86_64-redhat-linux-g++ (GCC) 4.1.1 20070105 (Red Hat 4.1.1-52)
> Reporter: tao xiaochuan
> Attachments: mainNew, test.zip
>
>
> when running 5 threads , in each thread a xml file will (for test purpost xerces-c-3.0.1\samples\data\long.xml is used)
> be parsed, it is found, by top command , VIRT increase large , by pmap command , 3 (some time 4) memory address block about 64 m
> are found , when by 8 threads , there will be 5 (some time 6) such blocks :
> Address Kbytes Mode Offset Device Mapping
> 00002aaab00b9000 64796 ----- 00002aaab00b9000 000:00000 [ anon ]
> ...
> 00002aaab40b9000 64796 ----- 00002aaab40b9000 000:00000 [ anon ]
> ...
> 00002aaab80bb000 64788 ----- 00002aaab80bb000 000:00000 [ anon ]
> these blocks will not be released even after XMLPlatformUtils::Terminate();
> if there is only one thead , there is no such a problem
> chief code structure:
> main(){
> XMLPlatformUtils::Initialize();
> //start many threads
> //wait for alll threads ends
> XMLPlatformUtils::Terminate();
> }
> in each thread :
> while(true){
> ...
>
> auto_ptr<SAXParser> parser( new SAXParser() );
> parser->setDocumentHandler(this); //the handler do nothing but just extend from HandlerBase
> parser->setErrorHandler(this); //the handler do nothing but just extend from HandlerBase
> parser->setValidationScheme(SAXParser::Val_Never);
> parser->setDoSchema(false);
> parser->setDoNamespaces(true);
> parser->parse(inputFile.c_str());
>
> }
> refer to attachment for detail about the simple test code
> one more related question :
> in the doc , it is said "client application needs to Terminate() (or multiple Terminate() in the case where multiple Initialize() have been invoked before" , does it also suit for more than one threads ?
> for example :
> thread 1 : XMLPlatformUtils::Initialize(); //A
> thread 1 : do some parsing //B
> thread 2 : XMLPlatformUtils::Initialize(); //C
> thread 2 : do some parsing //D
> thread 1 : do some parsing //E
> thread 1 : XMLPlatformUtils::Terminate(); //F
> thread 2 : do some parsing //G
> thread 1 : XMLPlatformUtils::Terminate(); //H
> }
> since these two api are all static , then does C will affect E and does F affect G
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
[jira] Updated: (XERCESC-1901) memory address leak when running
many threads which will parsing xml files
Posted by "tao xiaochuan (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESC-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
tao xiaochuan updated XERCESC-1901:
-----------------------------------
Attachment: mainNew
> memory address leak when running many threads which will parsing xml files
> --------------------------------------------------------------------------
>
> Key: XERCESC-1901
> URL: https://issues.apache.org/jira/browse/XERCESC-1901
> Project: Xerces-C++
> Issue Type: Bug
> Components: SAX/SAX2
> Affects Versions: 3.0.1
> Environment: Red Hat Enterprise Linux Server release 5.1 (Tikanga)
> -g++ --version : x86_64-redhat-linux-g++ (GCC) 4.1.1 20070105 (Red Hat 4.1.1-52)
> Reporter: tao xiaochuan
> Attachments: mainNew, test.zip
>
>
> when running 5 threads , in each thread a xml file will (for test purpost xerces-c-3.0.1\samples\data\long.xml is used)
> be parsed, it is found, by top command , VIRT increase large , by pmap command , 3 (some time 4) memory address block about 64 m
> are found , when by 8 threads , there will be 5 (some time 6) such blocks :
> Address Kbytes Mode Offset Device Mapping
> 00002aaab00b9000 64796 ----- 00002aaab00b9000 000:00000 [ anon ]
> ...
> 00002aaab40b9000 64796 ----- 00002aaab40b9000 000:00000 [ anon ]
> ...
> 00002aaab80bb000 64788 ----- 00002aaab80bb000 000:00000 [ anon ]
> these blocks will not be released even after XMLPlatformUtils::Terminate();
> if there is only one thead , there is no such a problem
> chief code structure:
> main(){
> XMLPlatformUtils::Initialize();
> //start many threads
> //wait for alll threads ends
> XMLPlatformUtils::Terminate();
> }
> in each thread :
> while(true){
> ...
>
> auto_ptr<SAXParser> parser( new SAXParser() );
> parser->setDocumentHandler(this); //the handler do nothing but just extend from HandlerBase
> parser->setErrorHandler(this); //the handler do nothing but just extend from HandlerBase
> parser->setValidationScheme(SAXParser::Val_Never);
> parser->setDoSchema(false);
> parser->setDoNamespaces(true);
> parser->parse(inputFile.c_str());
>
> }
> refer to attachment for detail about the simple test code
> one more related question :
> in the doc , it is said "client application needs to Terminate() (or multiple Terminate() in the case where multiple Initialize() have been invoked before" , does it also suit for more than one threads ?
> for example :
> thread 1 : XMLPlatformUtils::Initialize(); //A
> thread 1 : do some parsing //B
> thread 2 : XMLPlatformUtils::Initialize(); //C
> thread 2 : do some parsing //D
> thread 1 : do some parsing //E
> thread 1 : XMLPlatformUtils::Terminate(); //F
> thread 2 : do some parsing //G
> thread 1 : XMLPlatformUtils::Terminate(); //H
> }
> since these two api are all static , then does C will affect E and does F affect G
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
[jira] Commented: (XERCESC-1901) memory address leak when running
many threads which will parsing xml files
Posted by "tao xiaochuan (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESC-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12790526#action_12790526 ]
tao xiaochuan commented on XERCESC-1901:
----------------------------------------
hello . the leak problem we focus on can not be detected by valgrind , since it is not caused by not using new and delete properly , the memory leak related to ACE TASK found by valgrind is not the core of the problem , sorry for that the example
code has made some confusing by it is not a good code
a leak case which can not be found by vargrind :
for example , for thread which is not PTHREAD_CREATE_DETACHED, it will casue below leak when its parent thread do not join it after it finish :
Address Kbytes Mode Offset Device Mapping
0000000040001000 10240 rwx-- 0000000040001000 000:00000 [ anon ]
such memory address block is gotten by mmap api , not by new , valgrind can not detect such leak
for my problem, i have used valgrind already at beginning and also use strace command to trace mmap calling , but nothing found , then there is the problem reported
can you repeat the test (for example 5 thread nor more ) and watch it by pmap
( copy xerces-c-3.0.1\samples\data\long.xml to Test1.xml ~ Test10.xml , or change all TestN.xml to long.xml in main.cc)
and you can also fiind that when there is only one thread , there will be no such 64 m block no matter how long it will run.
> memory address leak when running many threads which will parsing xml files
> --------------------------------------------------------------------------
>
> Key: XERCESC-1901
> URL: https://issues.apache.org/jira/browse/XERCESC-1901
> Project: Xerces-C++
> Issue Type: Bug
> Components: SAX/SAX2
> Affects Versions: 3.0.1
> Environment: Red Hat Enterprise Linux Server release 5.1 (Tikanga)
> -g++ --version : x86_64-redhat-linux-g++ (GCC) 4.1.1 20070105 (Red Hat 4.1.1-52)
> Reporter: tao xiaochuan
> Attachments: test.zip
>
>
> when running 5 threads , in each thread a xml file will (for test purpost xerces-c-3.0.1\samples\data\long.xml is used)
> be parsed, it is found, by top command , VIRT increase large , by pmap command , 3 (some time 4) memory address block about 64 m
> are found , when by 8 threads , there will be 5 (some time 6) such blocks :
> Address Kbytes Mode Offset Device Mapping
> 00002aaab00b9000 64796 ----- 00002aaab00b9000 000:00000 [ anon ]
> ...
> 00002aaab40b9000 64796 ----- 00002aaab40b9000 000:00000 [ anon ]
> ...
> 00002aaab80bb000 64788 ----- 00002aaab80bb000 000:00000 [ anon ]
> these blocks will not be released even after XMLPlatformUtils::Terminate();
> if there is only one thead , there is no such a problem
> chief code structure:
> main(){
> XMLPlatformUtils::Initialize();
> //start many threads
> //wait for alll threads ends
> XMLPlatformUtils::Terminate();
> }
> in each thread :
> while(true){
> ...
>
> auto_ptr<SAXParser> parser( new SAXParser() );
> parser->setDocumentHandler(this); //the handler do nothing but just extend from HandlerBase
> parser->setErrorHandler(this); //the handler do nothing but just extend from HandlerBase
> parser->setValidationScheme(SAXParser::Val_Never);
> parser->setDoSchema(false);
> parser->setDoNamespaces(true);
> parser->parse(inputFile.c_str());
>
> }
> refer to attachment for detail about the simple test code
> one more related question :
> in the doc , it is said "client application needs to Terminate() (or multiple Terminate() in the case where multiple Initialize() have been invoked before" , does it also suit for more than one threads ?
> for example :
> thread 1 : XMLPlatformUtils::Initialize(); //A
> thread 1 : do some parsing //B
> thread 2 : XMLPlatformUtils::Initialize(); //C
> thread 2 : do some parsing //D
> thread 1 : do some parsing //E
> thread 1 : XMLPlatformUtils::Terminate(); //F
> thread 2 : do some parsing //G
> thread 1 : XMLPlatformUtils::Terminate(); //H
> }
> since these two api are all static , then does C will affect E and does F affect G
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
[jira] Commented: (XERCESC-1901) memory address leak when running
many threads which will parsing xml files
Posted by "tao xiaochuan (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESC-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12793896#action_12793896 ]
tao xiaochuan commented on XERCESC-1901:
----------------------------------------
hello . this ticket can be closed , the reason is found
rpm -qf /lib64/ld-2.5.so
glibc-2.5-24
after using new version
rpm -qf /lib64/ld-2.5.so
glibc-2.5-42
the problem is gone
> memory address leak when running many threads which will parsing xml files
> --------------------------------------------------------------------------
>
> Key: XERCESC-1901
> URL: https://issues.apache.org/jira/browse/XERCESC-1901
> Project: Xerces-C++
> Issue Type: Bug
> Components: SAX/SAX2
> Affects Versions: 3.0.1
> Environment: Red Hat Enterprise Linux Server release 5.1 (Tikanga)
> -g++ --version : x86_64-redhat-linux-g++ (GCC) 4.1.1 20070105 (Red Hat 4.1.1-52)
> Reporter: tao xiaochuan
> Attachments: mainNew, test.zip
>
>
> when running 5 threads , in each thread a xml file will (for test purpost xerces-c-3.0.1\samples\data\long.xml is used)
> be parsed, it is found, by top command , VIRT increase large , by pmap command , 3 (some time 4) memory address block about 64 m
> are found , when by 8 threads , there will be 5 (some time 6) such blocks :
> Address Kbytes Mode Offset Device Mapping
> 00002aaab00b9000 64796 ----- 00002aaab00b9000 000:00000 [ anon ]
> ...
> 00002aaab40b9000 64796 ----- 00002aaab40b9000 000:00000 [ anon ]
> ...
> 00002aaab80bb000 64788 ----- 00002aaab80bb000 000:00000 [ anon ]
> these blocks will not be released even after XMLPlatformUtils::Terminate();
> if there is only one thead , there is no such a problem
> chief code structure:
> main(){
> XMLPlatformUtils::Initialize();
> //start many threads
> //wait for alll threads ends
> XMLPlatformUtils::Terminate();
> }
> in each thread :
> while(true){
> ...
>
> auto_ptr<SAXParser> parser( new SAXParser() );
> parser->setDocumentHandler(this); //the handler do nothing but just extend from HandlerBase
> parser->setErrorHandler(this); //the handler do nothing but just extend from HandlerBase
> parser->setValidationScheme(SAXParser::Val_Never);
> parser->setDoSchema(false);
> parser->setDoNamespaces(true);
> parser->parse(inputFile.c_str());
>
> }
> refer to attachment for detail about the simple test code
> one more related question :
> in the doc , it is said "client application needs to Terminate() (or multiple Terminate() in the case where multiple Initialize() have been invoked before" , does it also suit for more than one threads ?
> for example :
> thread 1 : XMLPlatformUtils::Initialize(); //A
> thread 1 : do some parsing //B
> thread 2 : XMLPlatformUtils::Initialize(); //C
> thread 2 : do some parsing //D
> thread 1 : do some parsing //E
> thread 1 : XMLPlatformUtils::Terminate(); //F
> thread 2 : do some parsing //G
> thread 1 : XMLPlatformUtils::Terminate(); //H
> }
> since these two api are all static , then does C will affect E and does F affect G
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
[jira] Commented: (XERCESC-1901) memory address leak when running
many threads which will parsing xml files
Posted by "tao xiaochuan (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESC-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791149#action_12791149 ]
tao xiaochuan commented on XERCESC-1901:
----------------------------------------
it is found that the problem happen in only 64 bit system , i have tested two 32 bit system and one 64 bit system (what i can get in company at present)
32 bit system (no such leak problem)
Linux version 2.6.9-42.0.10.ELsmp (root@yort.fnal.gov) (gcc version 3.4.5 20051201 (Red Hat 3.4.5-2)) #1 SMP Tue Feb 27 08:38:56 CST 2007
g++ (GCC) 3.4.6 20060404 (Red Hat 3.4.6-3)
Copyright (C) 2006 Free Software Foundation, Inc.
32 bit system (no such leak problem)
cat /proc/version
Linux version 2.6.18-128.1.1.el5 (mockbuild@hs20-bc1-5.build.redhat.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Mon Jan 26 13:59:00 EST 2009
cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.2 (Tikanga)
64 bit system (has such leak problem)
cat /proc/version
Linux version 2.6.18-53.1.13.el5 (brewbuilder@hs20-bc1-6.build.redhat.com) (gcc version 4.1.2 20070626 (Red Hat 4.1.2-14)) #1 SMP Mon Feb 11 13:27:27 EST 2008
cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.1 (Tikanga)
the test code has also been simpled in one file mainNew.cc (ACE is not used but pthread),see attachment
i do not know whether this is a problem caused by 64 bit system or a problem special to xerces-c of 64 bit version
> memory address leak when running many threads which will parsing xml files
> --------------------------------------------------------------------------
>
> Key: XERCESC-1901
> URL: https://issues.apache.org/jira/browse/XERCESC-1901
> Project: Xerces-C++
> Issue Type: Bug
> Components: SAX/SAX2
> Affects Versions: 3.0.1
> Environment: Red Hat Enterprise Linux Server release 5.1 (Tikanga)
> -g++ --version : x86_64-redhat-linux-g++ (GCC) 4.1.1 20070105 (Red Hat 4.1.1-52)
> Reporter: tao xiaochuan
> Attachments: test.zip
>
>
> when running 5 threads , in each thread a xml file will (for test purpost xerces-c-3.0.1\samples\data\long.xml is used)
> be parsed, it is found, by top command , VIRT increase large , by pmap command , 3 (some time 4) memory address block about 64 m
> are found , when by 8 threads , there will be 5 (some time 6) such blocks :
> Address Kbytes Mode Offset Device Mapping
> 00002aaab00b9000 64796 ----- 00002aaab00b9000 000:00000 [ anon ]
> ...
> 00002aaab40b9000 64796 ----- 00002aaab40b9000 000:00000 [ anon ]
> ...
> 00002aaab80bb000 64788 ----- 00002aaab80bb000 000:00000 [ anon ]
> these blocks will not be released even after XMLPlatformUtils::Terminate();
> if there is only one thead , there is no such a problem
> chief code structure:
> main(){
> XMLPlatformUtils::Initialize();
> //start many threads
> //wait for alll threads ends
> XMLPlatformUtils::Terminate();
> }
> in each thread :
> while(true){
> ...
>
> auto_ptr<SAXParser> parser( new SAXParser() );
> parser->setDocumentHandler(this); //the handler do nothing but just extend from HandlerBase
> parser->setErrorHandler(this); //the handler do nothing but just extend from HandlerBase
> parser->setValidationScheme(SAXParser::Val_Never);
> parser->setDoSchema(false);
> parser->setDoNamespaces(true);
> parser->parse(inputFile.c_str());
>
> }
> refer to attachment for detail about the simple test code
> one more related question :
> in the doc , it is said "client application needs to Terminate() (or multiple Terminate() in the case where multiple Initialize() have been invoked before" , does it also suit for more than one threads ?
> for example :
> thread 1 : XMLPlatformUtils::Initialize(); //A
> thread 1 : do some parsing //B
> thread 2 : XMLPlatformUtils::Initialize(); //C
> thread 2 : do some parsing //D
> thread 1 : do some parsing //E
> thread 1 : XMLPlatformUtils::Terminate(); //F
> thread 2 : do some parsing //G
> thread 1 : XMLPlatformUtils::Terminate(); //H
> }
> since these two api are all static , then does C will affect E and does F affect G
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
[jira] Commented: (XERCESC-1901) memory address leak when running
many threads which will parsing xml files
Posted by "Boris Kolpackov (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESC-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12790307#action_12790307 ]
Boris Kolpackov commented on XERCESC-1901:
------------------------------------------
I ran your test under valgrind. While there are a number of leaks in it (e.g., you don't free ACE task), none of them are Xerces-C++ related (output below). Can you run valgrind and see what you get?
All thread exit
==19096==
==19096== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 9 from 1)
==19096== malloc/free: in use at exit: 19,381 bytes in 24 blocks.
==19096== malloc/free: 30,730 allocs, 30,706 frees, 50,514,225 bytes allocated.
==19096== For counts of detected errors, rerun with: -v
==19096== searching for pointers to 24 not-freed blocks.
==19096== checked 677,216 bytes.
==19096==
==19096== 8 bytes in 1 blocks are still reachable in loss record 1 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x54DB6B6: ACE_TSS<ACE_Service_Config::TSS_Resources>::make_TSS_TYPE() const (TSS_T.cpp:60)
==19096== by 0x54DB4D6: ACE_TSS<ACE_Service_Config::TSS_Resources>::ts_get() const (TSS_T.cpp:219)
==19096== by 0x54DB1DA: ACE_TSS<ACE_Service_Config::TSS_Resources>::operator ACE_Service_Config::TSS_Resources*() const (TSS_T.cpp:53)
==19096== by 0x54DA333: ACE_Service_Config::current_i(ACE_Service_Gestalt*) (Service_Config.cpp:378)
==19096== by 0x54DA196: ACE_Service_Config::current() (Service_Config.cpp:354)
==19096== by 0x54DA35C: ACE_Service_Config::static_svcs() (Service_Config.cpp:394)
==19096== by 0x5542060: ACE_Object_Manager_Preallocations::ACE_Object_Manager_Preallocations() (Object_Manager.cpp:134)
==19096== by 0x55428B2: ACE_Object_Manager::init() (Object_Manager.cpp:257)
==19096== by 0x5542C33: ACE_Object_Manager::ACE_Object_Manager() (Object_Manager.cpp:313)
==19096== by 0x5542E90: ACE_Object_Manager::instance() (Object_Manager.cpp:333)
==19096== by 0x5543D97: ACE_Object_Manager_Manager::ACE_Object_Manager_Manager() (Object_Manager.cpp:758)
==19096==
==19096==
==19096== 64 bytes in 1 blocks are still reachable in loss record 2 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x54DA286: ACE_Service_Config::current_i(ACE_Service_Gestalt*) (Service_Config.cpp:375)
==19096== by 0x54DA196: ACE_Service_Config::current() (Service_Config.cpp:354)
==19096== by 0x54DA35C: ACE_Service_Config::static_svcs() (Service_Config.cpp:394)
==19096== by 0x5542060: ACE_Object_Manager_Preallocations::ACE_Object_Manager_Preallocations() (Object_Manager.cpp:134)
==19096== by 0x55428B2: ACE_Object_Manager::init() (Object_Manager.cpp:257)
==19096== by 0x5542C33: ACE_Object_Manager::ACE_Object_Manager() (Object_Manager.cpp:313)
==19096== by 0x5542E90: ACE_Object_Manager::instance() (Object_Manager.cpp:333)
==19096== by 0x5543D97: ACE_Object_Manager_Manager::ACE_Object_Manager_Manager() (Object_Manager.cpp:758)
==19096== by 0x5543FC4: __static_initialization_and_destruction_0(int, int) (Object_Manager.cpp:774)
==19096== by 0x554400C: global constructors keyed to _ZN18ACE_Object_Manager9instance_E (Object_Manager.cpp:855)
==19096== by 0x5596A45: (within /home/boris/work/ace/ACE-5.5.2/ace/libACE.so.5.5.2)
==19096==
==19096==
==19096== 72 bytes in 1 blocks are still reachable in loss record 3 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x54E2803: ACE_Service_Repository::instance(unsigned long) (Service_Repository.cpp:63)
==19096== by 0x54DCC99: ACE_Service_Gestalt::ACE_Service_Gestalt(unsigned long, bool, bool) (Service_Gestalt.cpp:243)
==19096== by 0x54DA4AE: ACE_Service_Config::ACE_Service_Config(int, unsigned long, int) (Service_Config.cpp:447)
==19096== by 0x54DB40F: ACE_Singleton<ACE_Service_Config, ACE_Thread_Mutex>::ACE_Singleton() (Singleton.inl:14)
==19096== by 0x54DAF9D: ACE_Singleton<ACE_Service_Config, ACE_Thread_Mutex>::instance() (Singleton.cpp:77)
==19096== by 0x54DA084: ACE_Service_Config::global() (Service_Config.cpp:284)
==19096== by 0x54DA18E: ACE_Service_Config::current() (Service_Config.cpp:354)
==19096== by 0x54DA35C: ACE_Service_Config::static_svcs() (Service_Config.cpp:394)
==19096== by 0x5542060: ACE_Object_Manager_Preallocations::ACE_Object_Manager_Preallocations() (Object_Manager.cpp:134)
==19096== by 0x55428B2: ACE_Object_Manager::init() (Object_Manager.cpp:257)
==19096== by 0x5542C33: ACE_Object_Manager::ACE_Object_Manager() (Object_Manager.cpp:313)
==19096==
==19096==
==19096== 80 bytes in 1 blocks are still reachable in loss record 4 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x54DAF90: ACE_Singleton<ACE_Service_Config, ACE_Thread_Mutex>::instance() (Singleton.cpp:77)
==19096== by 0x54DA084: ACE_Service_Config::global() (Service_Config.cpp:284)
==19096== by 0x54DA18E: ACE_Service_Config::current() (Service_Config.cpp:354)
==19096== by 0x54DA35C: ACE_Service_Config::static_svcs() (Service_Config.cpp:394)
==19096== by 0x5542060: ACE_Object_Manager_Preallocations::ACE_Object_Manager_Preallocations() (Object_Manager.cpp:134)
==19096== by 0x55428B2: ACE_Object_Manager::init() (Object_Manager.cpp:257)
==19096== by 0x5542C33: ACE_Object_Manager::ACE_Object_Manager() (Object_Manager.cpp:313)
==19096== by 0x5542E90: ACE_Object_Manager::instance() (Object_Manager.cpp:333)
==19096== by 0x5543D97: ACE_Object_Manager_Manager::ACE_Object_Manager_Manager() (Object_Manager.cpp:758)
==19096== by 0x5543FC4: __static_initialization_and_destruction_0(int, int) (Object_Manager.cpp:774)
==19096== by 0x554400C: global constructors keyed to _ZN18ACE_Object_Manager9instance_E (Object_Manager.cpp:855)
==19096==
==19096==
==19096== 180 bytes in 5 blocks are indirectly lost in loss record 6 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x57A7AF0: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) (in /usr/lib/libstdc++.so.6.0.12)
==19096== by 0x57A8724: (within /usr/lib/libstdc++.so.6.0.12)
==19096== by 0x57A88C1: std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&) (in /usr/lib/libstdc++.so.6.0.12)
==19096== by 0x407E6F: main (main.cc:44)
==19096==
==19096==
==19096== 508 (216 direct, 292 indirect) bytes in 1 blocks are definitely lost in loss record 7 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x407F1F: main (main.cc:44)
==19096==
==19096==
==19096== 508 (216 direct, 292 indirect) bytes in 1 blocks are definitely lost in loss record 8 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x4080B0: main (main.cc:50)
==19096==
==19096==
==19096== 508 (216 direct, 292 indirect) bytes in 1 blocks are definitely lost in loss record 9 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x408241: main (main.cc:55)
==19096==
==19096==
==19096== 508 (216 direct, 292 indirect) bytes in 1 blocks are definitely lost in loss record 10 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x4083D2: main (main.cc:60)
==19096==
==19096==
==19096== 508 (216 direct, 292 indirect) bytes in 1 blocks are definitely lost in loss record 11 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x408563: main (main.cc:65)
==19096==
==19096==
==19096== 1,280 bytes in 5 blocks are indirectly lost in loss record 12 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x40BA8E: ACE_Task<ACE_MT_SYNCH>::ACE_Task(ACE_Thread_Manager*, ACE_Message_Queue<ACE_MT_SYNCH>*) (Task_T.cpp:59)
==19096== by 0x40A8EC: MyThread::MyThread(std::string const&) (MyThread.cc:8)
==19096== by 0x407F39: main (main.cc:44)
==19096==
==19096==
==19096== 4,097 bytes in 1 blocks are still reachable in loss record 13 of 15
==19096== at 0x4C22B9C: operator new[](unsigned long) (vg_replace_malloc.c:274)
==19096== by 0x552D5F9: ACE_Log_Msg::ACE_Log_Msg() (Log_Msg.cpp:692)
==19096== by 0x552CCE8: ACE_Log_Msg::instance() (Log_Msg.cpp:346)
==19096== by 0x554294E: ACE_Object_Manager::init() (Object_Manager.cpp:263)
==19096== by 0x5542C33: ACE_Object_Manager::ACE_Object_Manager() (Object_Manager.cpp:313)
==19096== by 0x5542E90: ACE_Object_Manager::instance() (Object_Manager.cpp:333)
==19096== by 0x5543D97: ACE_Object_Manager_Manager::ACE_Object_Manager_Manager() (Object_Manager.cpp:758)
==19096== by 0x5543FC4: __static_initialization_and_destruction_0(int, int) (Object_Manager.cpp:774)
==19096== by 0x554400C: global constructors keyed to _ZN18ACE_Object_Manager9instance_E (Object_Manager.cpp:855)
==19096== by 0x5596A45: (within /home/boris/work/ace/ACE-5.5.2/ace/libACE.so.5.5.2)
==19096== by 0x54B4812: (within /home/boris/work/ace/ACE-5.5.2/ace/libACE.so.5.5.2)
==19096== by 0x7FEFFFD4F: ???
==19096==
==19096==
==19096== 4,216 bytes in 1 blocks are still reachable in loss record 14 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x552CCDB: ACE_Log_Msg::instance() (Log_Msg.cpp:346)
==19096== by 0x554294E: ACE_Object_Manager::init() (Object_Manager.cpp:263)
==19096== by 0x5542C33: ACE_Object_Manager::ACE_Object_Manager() (Object_Manager.cpp:313)
==19096== by 0x5542E90: ACE_Object_Manager::instance() (Object_Manager.cpp:333)
==19096== by 0x5543D97: ACE_Object_Manager_Manager::ACE_Object_Manager_Manager() (Object_Manager.cpp:758)
==19096== by 0x5543FC4: __static_initialization_and_destruction_0(int, int) (Object_Manager.cpp:774)
==19096== by 0x554400C: global constructors keyed to _ZN18ACE_Object_Manager9instance_E (Object_Manager.cpp:855)
==19096== by 0x5596A45: (within /home/boris/work/ace/ACE-5.5.2/ace/libACE.so.5.5.2)
==19096== by 0x54B4812: (within /home/boris/work/ace/ACE-5.5.2/ace/libACE.so.5.5.2)
==19096== by 0x7FEFFFD4F: ???
==19096== by 0x400D647: call_init (dl-init.c:70)
==19096==
==19096==
==19096== 8,192 bytes in 1 blocks are still reachable in loss record 15 of 15
==19096== at 0x4C22B9C: operator new[](unsigned long) (vg_replace_malloc.c:274)
==19096== by 0x54E2A70: ACE_Service_Repository::open(unsigned long) (Service_Repository.cpp:115)
==19096== by 0x54E2BF0: ACE_Service_Repository::ACE_Service_Repository(unsigned long) (Service_Repository.cpp:129)
==19096== by 0x54E2814: ACE_Service_Repository::instance(unsigned long) (Service_Repository.cpp:63)
==19096== by 0x54DCC99: ACE_Service_Gestalt::ACE_Service_Gestalt(unsigned long, bool, bool) (Service_Gestalt.cpp:243)
==19096== by 0x54DA4AE: ACE_Service_Config::ACE_Service_Config(int, unsigned long, int) (Service_Config.cpp:447)
==19096== by 0x54DB40F: ACE_Singleton<ACE_Service_Config, ACE_Thread_Mutex>::ACE_Singleton() (Singleton.inl:14)
==19096== by 0x54DAF9D: ACE_Singleton<ACE_Service_Config, ACE_Thread_Mutex>::instance() (Singleton.cpp:77)
==19096== by 0x54DA084: ACE_Service_Config::global() (Service_Config.cpp:284)
==19096== by 0x54DA18E: ACE_Service_Config::current() (Service_Config.cpp:354)
==19096== by 0x54DA35C: ACE_Service_Config::static_svcs() (Service_Config.cpp:394)
==19096== by 0x5542060: ACE_Object_Manager_Preallocations::ACE_Object_Manager_Preallocations() (Object_Manager.cpp:134)
==19096==
==19096== LEAK SUMMARY:
==19096== definitely lost: 1,080 bytes in 5 blocks.
==19096== indirectly lost: 1,460 bytes in 10 blocks.
==19096== possibly lost: 0 bytes in 0 blocks.
==19096== still reachable: 16,841 bytes in 9 blocks.
==19096== suppressed: 0 bytes in 0 blocks.
> memory address leak when running many threads which will parsing xml files
> --------------------------------------------------------------------------
>
> Key: XERCESC-1901
> URL: https://issues.apache.org/jira/browse/XERCESC-1901
> Project: Xerces-C++
> Issue Type: Bug
> Components: SAX/SAX2
> Affects Versions: 3.0.1
> Environment: Red Hat Enterprise Linux Server release 5.1 (Tikanga)
> -g++ --version : x86_64-redhat-linux-g++ (GCC) 4.1.1 20070105 (Red Hat 4.1.1-52)
> Reporter: tao xiaochuan
> Attachments: test.zip
>
>
> when running 5 threads , in each thread a xml file will (for test purpost xerces-c-3.0.1\samples\data\long.xml is used)
> be parsed, it is found, by top command , VIRT increase large , by pmap command , 3 (some time 4) memory address block about 64 m
> are found , when by 8 threads , there will be 5 (some time 6) such blocks :
> Address Kbytes Mode Offset Device Mapping
> 00002aaab00b9000 64796 ----- 00002aaab00b9000 000:00000 [ anon ]
> ...
> 00002aaab40b9000 64796 ----- 00002aaab40b9000 000:00000 [ anon ]
> ...
> 00002aaab80bb000 64788 ----- 00002aaab80bb000 000:00000 [ anon ]
> these blocks will not be released even after XMLPlatformUtils::Terminate();
> if there is only one thead , there is no such a problem
> chief code structure:
> main(){
> XMLPlatformUtils::Initialize();
> //start many threads
> //wait for alll threads ends
> XMLPlatformUtils::Terminate();
> }
> in each thread :
> while(true){
> ...
>
> auto_ptr<SAXParser> parser( new SAXParser() );
> parser->setDocumentHandler(this); //the handler do nothing but just extend from HandlerBase
> parser->setErrorHandler(this); //the handler do nothing but just extend from HandlerBase
> parser->setValidationScheme(SAXParser::Val_Never);
> parser->setDoSchema(false);
> parser->setDoNamespaces(true);
> parser->parse(inputFile.c_str());
>
> }
> refer to attachment for detail about the simple test code
> one more related question :
> in the doc , it is said "client application needs to Terminate() (or multiple Terminate() in the case where multiple Initialize() have been invoked before" , does it also suit for more than one threads ?
> for example :
> thread 1 : XMLPlatformUtils::Initialize(); //A
> thread 1 : do some parsing //B
> thread 2 : XMLPlatformUtils::Initialize(); //C
> thread 2 : do some parsing //D
> thread 1 : do some parsing //E
> thread 1 : XMLPlatformUtils::Terminate(); //F
> thread 2 : do some parsing //G
> thread 1 : XMLPlatformUtils::Terminate(); //H
> }
> since these two api are all static , then does C will affect E and does F affect G
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
[jira] Updated: (XERCESC-1901) memory address leak when running
many threads which will parsing xml files
Posted by "tao xiaochuan (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESC-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
tao xiaochuan updated XERCESC-1901:
-----------------------------------
Attachment: test.zip
> memory address leak when running many threads which will parsing xml files
> --------------------------------------------------------------------------
>
> Key: XERCESC-1901
> URL: https://issues.apache.org/jira/browse/XERCESC-1901
> Project: Xerces-C++
> Issue Type: Bug
> Components: SAX/SAX2
> Affects Versions: 3.0.1
> Environment: Red Hat Enterprise Linux Server release 5.1 (Tikanga)
> -g++ --version : x86_64-redhat-linux-g++ (GCC) 4.1.1 20070105 (Red Hat 4.1.1-52)
> Reporter: tao xiaochuan
> Attachments: test.zip
>
>
> when running 5 threads , in each thread a xml file will (for test purpost xerces-c-3.0.1\samples\data\long.xml is used)
> be parsed, it is found, by top command , VIRT increase large , by pmap command , 3 (some time 4) memory address block about 64 m
> are found , when by 8 threads , there will be 5 (some time 6) such blocks :
> Address Kbytes Mode Offset Device Mapping
> 00002aaab00b9000 64796 ----- 00002aaab00b9000 000:00000 [ anon ]
> ...
> 00002aaab40b9000 64796 ----- 00002aaab40b9000 000:00000 [ anon ]
> ...
> 00002aaab80bb000 64788 ----- 00002aaab80bb000 000:00000 [ anon ]
> these blocks will not be released even after XMLPlatformUtils::Terminate();
> if there is only one thead , there is no such a problem
> chief code structure:
> main(){
> XMLPlatformUtils::Initialize();
> //start many threads
> //wait for alll threads ends
> XMLPlatformUtils::Terminate();
> }
> in each thread :
> while(true){
> ...
>
> auto_ptr<SAXParser> parser( new SAXParser() );
> parser->setDocumentHandler(this); //the handler do nothing but just extend from HandlerBase
> parser->setErrorHandler(this); //the handler do nothing but just extend from HandlerBase
> parser->setValidationScheme(SAXParser::Val_Never);
> parser->setDoSchema(false);
> parser->setDoNamespaces(true);
> parser->parse(inputFile.c_str());
>
> }
> refer to attachment for detail about the simple test code
> one more related question :
> in the doc , it is said "client application needs to Terminate() (or multiple Terminate() in the case where multiple Initialize() have been invoked before" , does it also suit for more than one threads ?
> for example :
> thread 1 : XMLPlatformUtils::Initialize(); //A
> thread 1 : do some parsing //B
> thread 2 : XMLPlatformUtils::Initialize(); //C
> thread 2 : do some parsing //D
> thread 1 : do some parsing //E
> thread 1 : XMLPlatformUtils::Terminate(); //F
> thread 2 : do some parsing //G
> thread 1 : XMLPlatformUtils::Terminate(); //H
> }
> since these two api are all static , then does C will affect E and does F affect G
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org