You are viewing a plain text version of this content. The canonical link for it is here.
Posted to c-dev@xerces.apache.org by "Boris Kolpackov (JIRA)" <xe...@xml.apache.org> on 2009/12/14 20:35:21 UTC
[jira] Commented: (XERCESC-1901) memory address leak when running
many threads which will parsing xml files
[ https://issues.apache.org/jira/browse/XERCESC-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12790307#action_12790307 ]
Boris Kolpackov commented on XERCESC-1901:
------------------------------------------
I ran your test under valgrind. While there are a number of leaks in it (e.g., you don't free ACE task), none of them are Xerces-C++ related (output below). Can you run valgrind and see what you get?
All thread exit
==19096==
==19096== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 9 from 1)
==19096== malloc/free: in use at exit: 19,381 bytes in 24 blocks.
==19096== malloc/free: 30,730 allocs, 30,706 frees, 50,514,225 bytes allocated.
==19096== For counts of detected errors, rerun with: -v
==19096== searching for pointers to 24 not-freed blocks.
==19096== checked 677,216 bytes.
==19096==
==19096== 8 bytes in 1 blocks are still reachable in loss record 1 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x54DB6B6: ACE_TSS<ACE_Service_Config::TSS_Resources>::make_TSS_TYPE() const (TSS_T.cpp:60)
==19096== by 0x54DB4D6: ACE_TSS<ACE_Service_Config::TSS_Resources>::ts_get() const (TSS_T.cpp:219)
==19096== by 0x54DB1DA: ACE_TSS<ACE_Service_Config::TSS_Resources>::operator ACE_Service_Config::TSS_Resources*() const (TSS_T.cpp:53)
==19096== by 0x54DA333: ACE_Service_Config::current_i(ACE_Service_Gestalt*) (Service_Config.cpp:378)
==19096== by 0x54DA196: ACE_Service_Config::current() (Service_Config.cpp:354)
==19096== by 0x54DA35C: ACE_Service_Config::static_svcs() (Service_Config.cpp:394)
==19096== by 0x5542060: ACE_Object_Manager_Preallocations::ACE_Object_Manager_Preallocations() (Object_Manager.cpp:134)
==19096== by 0x55428B2: ACE_Object_Manager::init() (Object_Manager.cpp:257)
==19096== by 0x5542C33: ACE_Object_Manager::ACE_Object_Manager() (Object_Manager.cpp:313)
==19096== by 0x5542E90: ACE_Object_Manager::instance() (Object_Manager.cpp:333)
==19096== by 0x5543D97: ACE_Object_Manager_Manager::ACE_Object_Manager_Manager() (Object_Manager.cpp:758)
==19096==
==19096==
==19096== 64 bytes in 1 blocks are still reachable in loss record 2 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x54DA286: ACE_Service_Config::current_i(ACE_Service_Gestalt*) (Service_Config.cpp:375)
==19096== by 0x54DA196: ACE_Service_Config::current() (Service_Config.cpp:354)
==19096== by 0x54DA35C: ACE_Service_Config::static_svcs() (Service_Config.cpp:394)
==19096== by 0x5542060: ACE_Object_Manager_Preallocations::ACE_Object_Manager_Preallocations() (Object_Manager.cpp:134)
==19096== by 0x55428B2: ACE_Object_Manager::init() (Object_Manager.cpp:257)
==19096== by 0x5542C33: ACE_Object_Manager::ACE_Object_Manager() (Object_Manager.cpp:313)
==19096== by 0x5542E90: ACE_Object_Manager::instance() (Object_Manager.cpp:333)
==19096== by 0x5543D97: ACE_Object_Manager_Manager::ACE_Object_Manager_Manager() (Object_Manager.cpp:758)
==19096== by 0x5543FC4: __static_initialization_and_destruction_0(int, int) (Object_Manager.cpp:774)
==19096== by 0x554400C: global constructors keyed to _ZN18ACE_Object_Manager9instance_E (Object_Manager.cpp:855)
==19096== by 0x5596A45: (within /home/boris/work/ace/ACE-5.5.2/ace/libACE.so.5.5.2)
==19096==
==19096==
==19096== 72 bytes in 1 blocks are still reachable in loss record 3 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x54E2803: ACE_Service_Repository::instance(unsigned long) (Service_Repository.cpp:63)
==19096== by 0x54DCC99: ACE_Service_Gestalt::ACE_Service_Gestalt(unsigned long, bool, bool) (Service_Gestalt.cpp:243)
==19096== by 0x54DA4AE: ACE_Service_Config::ACE_Service_Config(int, unsigned long, int) (Service_Config.cpp:447)
==19096== by 0x54DB40F: ACE_Singleton<ACE_Service_Config, ACE_Thread_Mutex>::ACE_Singleton() (Singleton.inl:14)
==19096== by 0x54DAF9D: ACE_Singleton<ACE_Service_Config, ACE_Thread_Mutex>::instance() (Singleton.cpp:77)
==19096== by 0x54DA084: ACE_Service_Config::global() (Service_Config.cpp:284)
==19096== by 0x54DA18E: ACE_Service_Config::current() (Service_Config.cpp:354)
==19096== by 0x54DA35C: ACE_Service_Config::static_svcs() (Service_Config.cpp:394)
==19096== by 0x5542060: ACE_Object_Manager_Preallocations::ACE_Object_Manager_Preallocations() (Object_Manager.cpp:134)
==19096== by 0x55428B2: ACE_Object_Manager::init() (Object_Manager.cpp:257)
==19096== by 0x5542C33: ACE_Object_Manager::ACE_Object_Manager() (Object_Manager.cpp:313)
==19096==
==19096==
==19096== 80 bytes in 1 blocks are still reachable in loss record 4 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x54DAF90: ACE_Singleton<ACE_Service_Config, ACE_Thread_Mutex>::instance() (Singleton.cpp:77)
==19096== by 0x54DA084: ACE_Service_Config::global() (Service_Config.cpp:284)
==19096== by 0x54DA18E: ACE_Service_Config::current() (Service_Config.cpp:354)
==19096== by 0x54DA35C: ACE_Service_Config::static_svcs() (Service_Config.cpp:394)
==19096== by 0x5542060: ACE_Object_Manager_Preallocations::ACE_Object_Manager_Preallocations() (Object_Manager.cpp:134)
==19096== by 0x55428B2: ACE_Object_Manager::init() (Object_Manager.cpp:257)
==19096== by 0x5542C33: ACE_Object_Manager::ACE_Object_Manager() (Object_Manager.cpp:313)
==19096== by 0x5542E90: ACE_Object_Manager::instance() (Object_Manager.cpp:333)
==19096== by 0x5543D97: ACE_Object_Manager_Manager::ACE_Object_Manager_Manager() (Object_Manager.cpp:758)
==19096== by 0x5543FC4: __static_initialization_and_destruction_0(int, int) (Object_Manager.cpp:774)
==19096== by 0x554400C: global constructors keyed to _ZN18ACE_Object_Manager9instance_E (Object_Manager.cpp:855)
==19096==
==19096==
==19096== 180 bytes in 5 blocks are indirectly lost in loss record 6 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x57A7AF0: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) (in /usr/lib/libstdc++.so.6.0.12)
==19096== by 0x57A8724: (within /usr/lib/libstdc++.so.6.0.12)
==19096== by 0x57A88C1: std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&) (in /usr/lib/libstdc++.so.6.0.12)
==19096== by 0x407E6F: main (main.cc:44)
==19096==
==19096==
==19096== 508 (216 direct, 292 indirect) bytes in 1 blocks are definitely lost in loss record 7 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x407F1F: main (main.cc:44)
==19096==
==19096==
==19096== 508 (216 direct, 292 indirect) bytes in 1 blocks are definitely lost in loss record 8 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x4080B0: main (main.cc:50)
==19096==
==19096==
==19096== 508 (216 direct, 292 indirect) bytes in 1 blocks are definitely lost in loss record 9 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x408241: main (main.cc:55)
==19096==
==19096==
==19096== 508 (216 direct, 292 indirect) bytes in 1 blocks are definitely lost in loss record 10 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x4083D2: main (main.cc:60)
==19096==
==19096==
==19096== 508 (216 direct, 292 indirect) bytes in 1 blocks are definitely lost in loss record 11 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x408563: main (main.cc:65)
==19096==
==19096==
==19096== 1,280 bytes in 5 blocks are indirectly lost in loss record 12 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x40BA8E: ACE_Task<ACE_MT_SYNCH>::ACE_Task(ACE_Thread_Manager*, ACE_Message_Queue<ACE_MT_SYNCH>*) (Task_T.cpp:59)
==19096== by 0x40A8EC: MyThread::MyThread(std::string const&) (MyThread.cc:8)
==19096== by 0x407F39: main (main.cc:44)
==19096==
==19096==
==19096== 4,097 bytes in 1 blocks are still reachable in loss record 13 of 15
==19096== at 0x4C22B9C: operator new[](unsigned long) (vg_replace_malloc.c:274)
==19096== by 0x552D5F9: ACE_Log_Msg::ACE_Log_Msg() (Log_Msg.cpp:692)
==19096== by 0x552CCE8: ACE_Log_Msg::instance() (Log_Msg.cpp:346)
==19096== by 0x554294E: ACE_Object_Manager::init() (Object_Manager.cpp:263)
==19096== by 0x5542C33: ACE_Object_Manager::ACE_Object_Manager() (Object_Manager.cpp:313)
==19096== by 0x5542E90: ACE_Object_Manager::instance() (Object_Manager.cpp:333)
==19096== by 0x5543D97: ACE_Object_Manager_Manager::ACE_Object_Manager_Manager() (Object_Manager.cpp:758)
==19096== by 0x5543FC4: __static_initialization_and_destruction_0(int, int) (Object_Manager.cpp:774)
==19096== by 0x554400C: global constructors keyed to _ZN18ACE_Object_Manager9instance_E (Object_Manager.cpp:855)
==19096== by 0x5596A45: (within /home/boris/work/ace/ACE-5.5.2/ace/libACE.so.5.5.2)
==19096== by 0x54B4812: (within /home/boris/work/ace/ACE-5.5.2/ace/libACE.so.5.5.2)
==19096== by 0x7FEFFFD4F: ???
==19096==
==19096==
==19096== 4,216 bytes in 1 blocks are still reachable in loss record 14 of 15
==19096== at 0x4C232DC: operator new(unsigned long) (vg_replace_malloc.c:230)
==19096== by 0x552CCDB: ACE_Log_Msg::instance() (Log_Msg.cpp:346)
==19096== by 0x554294E: ACE_Object_Manager::init() (Object_Manager.cpp:263)
==19096== by 0x5542C33: ACE_Object_Manager::ACE_Object_Manager() (Object_Manager.cpp:313)
==19096== by 0x5542E90: ACE_Object_Manager::instance() (Object_Manager.cpp:333)
==19096== by 0x5543D97: ACE_Object_Manager_Manager::ACE_Object_Manager_Manager() (Object_Manager.cpp:758)
==19096== by 0x5543FC4: __static_initialization_and_destruction_0(int, int) (Object_Manager.cpp:774)
==19096== by 0x554400C: global constructors keyed to _ZN18ACE_Object_Manager9instance_E (Object_Manager.cpp:855)
==19096== by 0x5596A45: (within /home/boris/work/ace/ACE-5.5.2/ace/libACE.so.5.5.2)
==19096== by 0x54B4812: (within /home/boris/work/ace/ACE-5.5.2/ace/libACE.so.5.5.2)
==19096== by 0x7FEFFFD4F: ???
==19096== by 0x400D647: call_init (dl-init.c:70)
==19096==
==19096==
==19096== 8,192 bytes in 1 blocks are still reachable in loss record 15 of 15
==19096== at 0x4C22B9C: operator new[](unsigned long) (vg_replace_malloc.c:274)
==19096== by 0x54E2A70: ACE_Service_Repository::open(unsigned long) (Service_Repository.cpp:115)
==19096== by 0x54E2BF0: ACE_Service_Repository::ACE_Service_Repository(unsigned long) (Service_Repository.cpp:129)
==19096== by 0x54E2814: ACE_Service_Repository::instance(unsigned long) (Service_Repository.cpp:63)
==19096== by 0x54DCC99: ACE_Service_Gestalt::ACE_Service_Gestalt(unsigned long, bool, bool) (Service_Gestalt.cpp:243)
==19096== by 0x54DA4AE: ACE_Service_Config::ACE_Service_Config(int, unsigned long, int) (Service_Config.cpp:447)
==19096== by 0x54DB40F: ACE_Singleton<ACE_Service_Config, ACE_Thread_Mutex>::ACE_Singleton() (Singleton.inl:14)
==19096== by 0x54DAF9D: ACE_Singleton<ACE_Service_Config, ACE_Thread_Mutex>::instance() (Singleton.cpp:77)
==19096== by 0x54DA084: ACE_Service_Config::global() (Service_Config.cpp:284)
==19096== by 0x54DA18E: ACE_Service_Config::current() (Service_Config.cpp:354)
==19096== by 0x54DA35C: ACE_Service_Config::static_svcs() (Service_Config.cpp:394)
==19096== by 0x5542060: ACE_Object_Manager_Preallocations::ACE_Object_Manager_Preallocations() (Object_Manager.cpp:134)
==19096==
==19096== LEAK SUMMARY:
==19096== definitely lost: 1,080 bytes in 5 blocks.
==19096== indirectly lost: 1,460 bytes in 10 blocks.
==19096== possibly lost: 0 bytes in 0 blocks.
==19096== still reachable: 16,841 bytes in 9 blocks.
==19096== suppressed: 0 bytes in 0 blocks.
> memory address leak when running many threads which will parsing xml files
> --------------------------------------------------------------------------
>
> Key: XERCESC-1901
> URL: https://issues.apache.org/jira/browse/XERCESC-1901
> Project: Xerces-C++
> Issue Type: Bug
> Components: SAX/SAX2
> Affects Versions: 3.0.1
> Environment: Red Hat Enterprise Linux Server release 5.1 (Tikanga)
> -g++ --version : x86_64-redhat-linux-g++ (GCC) 4.1.1 20070105 (Red Hat 4.1.1-52)
> Reporter: tao xiaochuan
> Attachments: test.zip
>
>
> when running 5 threads , in each thread a xml file will (for test purpost xerces-c-3.0.1\samples\data\long.xml is used)
> be parsed, it is found, by top command , VIRT increase large , by pmap command , 3 (some time 4) memory address block about 64 m
> are found , when by 8 threads , there will be 5 (some time 6) such blocks :
> Address Kbytes Mode Offset Device Mapping
> 00002aaab00b9000 64796 ----- 00002aaab00b9000 000:00000 [ anon ]
> ...
> 00002aaab40b9000 64796 ----- 00002aaab40b9000 000:00000 [ anon ]
> ...
> 00002aaab80bb000 64788 ----- 00002aaab80bb000 000:00000 [ anon ]
> these blocks will not be released even after XMLPlatformUtils::Terminate();
> if there is only one thead , there is no such a problem
> chief code structure:
> main(){
> XMLPlatformUtils::Initialize();
> //start many threads
> //wait for alll threads ends
> XMLPlatformUtils::Terminate();
> }
> in each thread :
> while(true){
> ...
>
> auto_ptr<SAXParser> parser( new SAXParser() );
> parser->setDocumentHandler(this); //the handler do nothing but just extend from HandlerBase
> parser->setErrorHandler(this); //the handler do nothing but just extend from HandlerBase
> parser->setValidationScheme(SAXParser::Val_Never);
> parser->setDoSchema(false);
> parser->setDoNamespaces(true);
> parser->parse(inputFile.c_str());
>
> }
> refer to attachment for detail about the simple test code
> one more related question :
> in the doc , it is said "client application needs to Terminate() (or multiple Terminate() in the case where multiple Initialize() have been invoked before" , does it also suit for more than one threads ?
> for example :
> thread 1 : XMLPlatformUtils::Initialize(); //A
> thread 1 : do some parsing //B
> thread 2 : XMLPlatformUtils::Initialize(); //C
> thread 2 : do some parsing //D
> thread 1 : do some parsing //E
> thread 1 : XMLPlatformUtils::Terminate(); //F
> thread 2 : do some parsing //G
> thread 1 : XMLPlatformUtils::Terminate(); //H
> }
> since these two api are all static , then does C will affect E and does F affect G
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org