You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficserver.apache.org by "Bryan Call (JIRA)" <ji...@apache.org> on 2016/08/23 00:40:21 UTC
[jira] [Commented] (TS-4195) out of traffic_manager causes
a double free in traffic_server
[ https://issues.apache.org/jira/browse/TS-4195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431872#comment-15431872 ]
Bryan Call commented on TS-4195:
--------------------------------
I also see this one:
{noformat}
==23255==ERROR: AddressSanitizer: attempting double-free on 0x619000000f80 in thread T0 ([ET_NET 0]):
#0 0x2b24e55bbac0 in free (/lib64/libasan.so.3+0xc6ac0)
#1 0x2b24e9155214 in __run_exit_handlers (/lib64/libc.so.6+0x39214)
#2 0x2b24e9155234 in __GI_exit (/lib64/libc.so.6+0x39234)
#3 0x587239 in proxy_signal_handler /home/bcall/dev/apache/trafficserver/proxy/Main.cc:409
#4 0x2b24e80fac2f (/lib64/libpthread.so.0+0x10c2f)
#5 0x2b24e921f4b2 in __GI_epoll_wait (/lib64/libc.so.6+0x1034b2)
#6 0xc31421 in NetHandler::mainNetEvent(int, Event*) /home/bcall/dev/apache/trafficserver/iocore/net/UnixNet.cc:423
#7 0xd13ce0 in Continuation::handleEvent(int, void*) /home/bcall/dev/apache/trafficserver/iocore/eventsystem/I_Continuation.h:153
#8 0xd13ce0 in EThread::process_event(Event*, int) /home/bcall/dev/apache/trafficserver/iocore/eventsystem/UnixEThread.cc:148
#9 0xd16bc6 in EThread::execute() /home/bcall/dev/apache/trafficserver/iocore/eventsystem/UnixEThread.cc:275
#10 0x49ac50 in main /home/bcall/dev/apache/trafficserver/proxy/Main.cc:1956
#11 0x2b24e913c730 in __libc_start_main (/lib64/libc.so.6+0x20730)
#12 0x4aa598 in _start (/usr/local/bin/traffic_server+0x4aa598)
0x619000000f80 is located 0 bytes inside of 1040-byte region [0x619000000f80,0x619000001390)
freed by thread T1 here:
=================================================================
#0 0x2b24e55bbac0 in free (/lib64/libasan.so.3+0xc6ac0)
#1 0x2b24e9155214 in __run_exit_handlers (/lib64/libc.so.6+0x39214)
==23255==ERROR: LeakSanitizer: detected memory leaks
previously allocated by thread T0 ([ET_NET 0]) here:
Direct leak of 257 byte(s) in 1 object(s) allocated from:
#0 0x2b24e55bbfe0 in calloc (/lib64/libasan.so.3+0xc6fe0)
#1 0x2b24e91553e8 in __new_exitfn (/lib64/libc.so.6+0x393e8)
#0 0x2b24e55bbe20 in malloc (/lib64/libasan.so.3+0xc6e20)
Thread T1 created by T0 ([ET_NET 0]) here:
#1 0x2b24e64f30d5 in ats_malloc /home/bcall/dev/apache/trafficserver/lib/ts/ink_memory.cc:59
#2 0x4988d5 in ats_scoped_str::ats_scoped_str(unsigned long) ../lib/ts/ink_memory.h:442
#3 0x4988d5 in main /home/bcall/dev/apache/trafficserver/proxy/Main.cc:1608
#0 0x2b24e5526458 in pthread_create (/lib64/libasan.so.3+0x31458)
#4 0x2b24e913c730 in __libc_start_main (/lib64/libc.so.6+0x20730)
#1 0x49833c in ink_thread_create ../lib/ts/ink_thread.h:147
#2 0x49833c in ProcessManager::start() ../mgmt/ProcessManager.h:65
#3 0x49833c in initialize_process_manager /home/bcall/dev/apache/trafficserver/proxy/Main.cc:503
#4 0x49833c in main /home/bcall/dev/apache/trafficserver/proxy/Main.cc:1567
Direct leak of 40 byte(s) in 1 object(s) allocated from:
#5 0x2b24e913c730 in __libc_start_main (/lib64/libc.so.6+0x20730)
#0 0x2b24e55bbe20 in malloc (/lib64/libasan.so.3+0xc6e20)
SUMMARY: AddressSanitizer: double-free (/lib64/libasan.so.3+0xc6ac0) in free
#1 0x2b24e64f30d5 in ats_malloc /home/bcall/dev/apache/trafficserver/lib/ts/ink_memory.cc:59
==23255==ABORTING
#2 0xd13640 in Thread::start(char const*, unsigned long, void* (*)(void*), void*) /home/bcall/dev/apache/trafficserver/iocore/eventsystem/Thread.cc:92
[TrafficManager] ==> signal #2
{noformat}
> <ctrl-c>out of traffic_manager causes a double free in traffic_server
> ---------------------------------------------------------------------
>
> Key: TS-4195
> URL: https://issues.apache.org/jira/browse/TS-4195
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
> Reporter: Leif Hedstrom
> Assignee: Bryan Call
> Priority: Blocker
> Fix For: 7.0.0
>
>
> While testing stuff, I was running traffic_manager from command line, and then I get a crash from traffic_server:
> {code}
> root@loki 407/0 # ./bin/traffic_manager
> [E. Mgmt] log ==> [TrafficManager] using root directory '/opt/ats'
> traffic_server: using root directory '/opt/ats'
> ^C[TrafficManager] ==> Cleaning up and reissuing signal #2
> traffic_server: Interrupt (Signal sent by the kernel 0 0)
> 9083 sent by kill()*** Error in `/opt/ats/bin/traffic_server': corrupted double-linked list: 0x00000000028f8940 ***
> ======= Backtrace: =========
> /lib64/libc.so.6(+0x77da5)[0x2ad58f3fcda5]
> /lib64/libc.so.6(+0x80c06)[0x2ad58f405c06]
> /lib64/libc.so.6(cfree+0x4c)[0x2ad58f408cac]
> /lib64/libc.so.6(+0x39685)[0x2ad58f3be685]
> /lib64/libc.so.6(+0x396a5)[0x2ad58f3be6a5]
> /opt/ats/bin/traffic_server[0x4e300atraffic_server: Segmentation fault (Address not mapped to object [0x55b02140])
> traffic_server - STACK TRACE:
> /lib64/libc.so.6(nanosleep+0x2d)[0x2ad58f44d7ad]
> /opt/ats/bin/traffic_server(_Z19crash_logger_invokeiP9siginfo_tPv+0x8e)[0x4abece]
> /lib64/libpthread.so.0(+0x109f0)[0x2ad58e3709f0]
> /lib64/libc.so.6(sleep+0xd4)[0x2ad58f44d644]
> /opt/ats/bin/traffic_server(_Z19startProcessManagerPv+0xb1)[0x69b8a1]
> /lib64/libpthread.so.0(+0x760a)[0x2ad58e36760a]
> /lib64/libc.so.6(clone+0x6d)[0x2ad58f487a4d]
> ======= Memory map: ========
> /lib64/libc.so.6(+0x395ad)[0x2ad58f3be5ad]
> 00400000-008a6000 r-xp 00000000 00:24 1775473 /opt/ats/bin/traffic_server
> 00aa6000-00ab3000 r--p 004a6000 00:24 1775473 /opt/ats/bin/traffic_server
> 00ab3000-00ab9000 rw-p 004b3000 00:24 1775473 /opt/ats/bin/traffic_server
> 00ab9000-01097000 rw-p 00000000 00:00 0
> 028dd000-02cb9000 rw-p 00000000 00:00 0 [heap]
> 2ad58c52c000-2ad58c54d000 r-xp 00000000 00:24 1389899 /usr/lib64/ld-2.22.so
> 2ad58c54d000-2ad58c550000 rw-p 00000000 00:00 0
> 2ad58c550000-2ad58c560000 rwxp 00000000 00:00 0
> 2ad58c56b000-2ad58c6ed000 rw-p 00000000 00:00 0
> 2ad58c6ed000-2ad58c6fd000 rwxp 00000000 00:00 0
> 2ad58c6fd000-2ad58c748000 rw-p 00000000 00:00 0
> 2ad58c74c000-2ad58c74d000 r--p 00020000 00:24 1389899 /usr/lib64/ld-2.22.so
> 2ad58c74d000-2ad58c74e000 rw-p 00021000 00:24 1389899 /usr/lib64/ld-2.22.so
> 2ad58c74e000-2ad58c74f000 rw-p 00000000 00:00 0
> 2ad58c74f000-2ad58c790000 r-xp 00000000 00:24 1775306 /opt/ats/lib/libtsutil.so.6.2.0
> 2ad58c790000-2ad58c990000 ---p 00041000 00:24 1775306 /opt/ats/lib/libtsutil.so.6.2.0
> 2ad58c990000-2ad58c991000 r--p 00041000 00:24 1775306 /opt/ats/lib/libtsutil.so.6.2.0
> 2ad58c991000-2ad58c993000 rw-p 00042000 00:24 1775306 /opt/ats/lib/libtsutil.so.6.2.0
> 2ad58c993000-2ad58c994000 rw-p 00000000 00:00 0
> 2ad58c994000-2ad58c9cb000 r-xp 00000000 00:24 1393339 /usr/lib64/libhwloc.so.5.6.6
> 2ad58c9cb000-2ad58cbcb000 ---p 00037000 00:24 1393339 /usr/lib64/libhwloc.so.5.6.6
> 2ad58cbcb000-2ad58cbcc000 r--p 00037000 00:24 1393339 /usr/lib64/libhwloc.so.5.6.6
> 2ad58cbcc000-2ad58cbcd000 rw-p 00038000 00:24 1393339 /usr/lib64/libhwloc.so.5.6.6
> 2ad58cbcd000-2ad58cd77000 r-xp 00000000 00:24 1441754 /usr/lib64/libtcl8.6.so
> 2ad58cd77000-2ad58cf77000 ---p 001aa000 00:24 1441754 /lib64/libc.so.6( /usr/lib64/libtcl8.6.so
> 2ad58cf77000-2ad58cf86000 r--p 001aa000 00:24 1441754 /usr/lib64/libtcl8.6.so
> 2ad58cf86000-2ad58cf87000 rw-p 001b9000 00:24 1441754 /usr/lib64/libtcl8.6.so
> 2ad58cf87000-2ad58cf88000 rw-p 00000000 00:00 0
> 2ad58cf88000-2ad58cf9f000 r-xp 00000000 00:24 1389936 /usr/lib64/libresolv-2.22.so
> 2ad58cf9f000-2ad58d19f000 ---p 00017000 00:24 1389936 /usr/lib64/libresolv-2.22.so
> 2ad58d19f000-2ad58d1a0000 r--p 00017000 00:24 1389936 /usr/lib64/libresolv-2.22.so
> 2ad58d1a0000-2ad58d1a1000 rw-p 00018000 00:24 1389936 /usr/lib64/libresolv-2.22.so
> 2ad58d1a1000-2ad58d1a3000 rw-p 00000000 00:00 0
> 2ad58d1a3000-2ad58d212000 r-xp 00000000 00:24 1635380 /usr/lib64/libssl.so.1.0.2f
> 2ad58d212000-2ad58d411000 ---p 0006f000 00:24 1635380 /usr/lib64/libssl.so.1.0.2f
> 2ad58d411000-2ad58d416000 r--p 0006e000 00:24 1635380 /usr/lib64/libssl.so.1.0.2f
> 2ad58d416000-2ad58d41d000 rw-p 00073000 00:24 1635380 /usr/lib64/libssl.so.1.0.2f
> 2ad58d41d000-2ad58d64d000 r-xp 00000000 00:24 1635378 /usr/lib64/libcrypto.so.1.0.2f
> 2ad58d64d000-2ad58d84c000 ---p 00230000 00:24 1635378 /usr/lib64/libcrypto.so.1.0.2f
> 2ad58d84c000-2ad58d868000 r--p 0022f000 00:24 1635378 /usr/lib64/libcrypto.so.1.0.2f
> 2ad58d868000-2ad58d875000 rw-p 0024b000 00:24 1635378 /usr/lib64/libcrypto.so.1.0.2f
> 2ad58d875000-2ad58d879000 rw-p 00000000 00:00 0
> 2ad58d879000-2ad58d87d000 r-xp 00000000 00:24 1390959 /usr/lib64/libcap.so.2.24
> 2ad58d87d000-2ad58da7c000 ---p 00004000 00:24 1390959 /usr/lib64/libcap.so.2.24
> 2ad58da7c000-2ad58da7d000 r--p 00003000 00:24 1390959 /usr/lib64/libcap.so.2.24
> 2ad58da7d000-2ad58da7e000 rw-p 00004000 00:24 1390959 /usr/lib64/libcap.so.2.24
> 2ad58da7e000-2ad58daed000 r-xp 00000000 00:24 1390389 +0x /usr/lib64/libpcre.so.1.2.6
> 2ad58daed000-2ad58dcec000 ---p 0006f000 00:24 1390389 /usr/lib64/libpcre.so.1.2.6
> 2ad58dcec000-2ad58dced000 r--p 0006e000 00:24 1390389 /usr/lib64/libpcre.so.1.2.6
> 2ad58dced000-2ad58dcee000 rw-p 0006f000 00:24 1390389 /usr/lib64/libpcre.so.1.2.6
> 2ad58dcee000-2ad58dd13000 r-xp 00000000 00:24 1390456 /usr/lib64/liblzma.so.5.2.1
> 2ad58dd13000-2ad58df12000 ---p 00025000 00:24 1390456 /usr/lib64/liblzma.so.5.2.1
> 2ad58df12000-2ad58df13000 r--p 00024000 00:24 1390456 /usr/lib64/liblzma.so.5.2.1
> 2ad58df13000-2ad58df14000 rw-p 00000000 00:00 0
> 2ad58df14000-2ad58df29000 r-xp 00000000 00:24 1390421 /usr/lib64/libz.so.1.2.8
> 2ad58df29000-2ad58e128000 ---p 00015000 00:24 1390421 /usr/lib64/libz.so.1.2.8
> 2ad58e128000-2ad58e129000 r--p 00014000 00:24 1390421 /usr/lib64/libz.so.1.2.8
> 2ad58e129000-2ad58e12a000 rw-p 00015000 00:24 1390421 /usr/lib64/libz.so.1.2.8
> 2ad58e12a000-2ad58e131000 r-xp 00000000 00:24 1389910 /usr/lib64/libcrypt-2.22.so
> 2ad58e131000-2ad58e330000 ---p 00007000 00:24 1389910 /usr/lib64/libcrypt-2.22.so
> 2ad58e330000-2ad58e331000 r--p 00006000 00:24 1389910 /usr/lib64/libcrypt-2.22.so
> 2ad58e331000-2ad58e332000 rw-p 00007000 00:24 1389910 /usr/lib64/libcrypt-2.22.so
> 2ad58e332000-2ad58e360000 rw-p 00000000 00:00 0
> 2ad58e360000-2ad58e378000 r-xp 00000000 00:24 1389934 /usr/lib64/libpthread-2.22.so
> 2ad58e378000-2ad58e577000 ---p 00018000 00:24 1389934 /usr/lib64/libpthread-2.22.so
> 2ad58e577000-2ad58e578000 r--p 00017000 00:24 1389934 /usr/lib64/libpthread-2.22.so
> 2ad58e578000-2ad58e579000 rw-p 00018000 00:24 1389934 /usr/lib64/libpthread-2.22.so
> 2ad58e579000-2ad58e57d000 rw-p 00000000 00:00 0
> 2ad58e57d000-2ad58e580000 r-xp 00000000 00:396a5)[0x2ad58f3be6a524 1389912 /usr/lib64/libdl-2.22.so
> 2ad58e580000-2ad58e77f000 ---p 00003000 00:24 1389912 /usr/lib64/libdl-2.22.so
> 2ad58e77f000-2ad58e780000 r--p 00002000 00:24 1389912 /usr/lib64/libdl-2.22.so
> 2ad58e780000-2ad58e781000 rw-p 00003000 00:24 1389912 /usr/lib64/libdl-2.22.so
> 2ad58e781000-2ad58e8df000 r-xp 00000000 00:24 1390463 /usr/lib64/libxml2.so.2.9.3
> 2ad58e8df000-2ad58eadf000 ---p 0015e000 00:24 1390463 /usr/lib64/libxml2.so.2.9.3
> 2ad58eadf000-2ad58eae7000 r--p 0015e000 00:24 1390463 /usr/lib64/libxml2.so.2.9.3
> 2ad58eae7000-2ad58eae9000 rw-p 00166000 00:24 1390463 /usr/lib64/libxml2.so.2.9.3
> 2ad58eae9000-2ad58eaea000 rw-p 00000000 00:00 0
> 2ad58eaea000-2ad58ec5c000 r-xp 00000000 00:24 1390225 /usr/lib64/libstdc++.so.6.0.21
> 2ad58ec5c000-2ad58ee5c000 ---p 00172000 00:24 1390225 /usr/lib64/libstdc++.so.6.0.21
> 2ad58ee5c000-2]
> ad58ee66000 r--p 00172000 00:24 1390225 /usr/lib64/libstdc++.so.6.0.21
> 2ad58ee66000-2ad58ee68000 rw-p 0017c000 00:24 1390225 /usr/lib64/libstdc++.so.6.0.21
> 2ad58ee68000-2ad58ee6c000 rw-p 00000000 00:00 0
> 2ad58ee6c000-2ad58ef6d000 r-xp 00000000 00:24 1389914 /usr/lib64/libm-2.22.so
> 2ad58ef6d000-2ad58f16c000 ---p 00101000 00:24 1389914 /usr/lib64/libm-2.22.so
> 2ad58f16c000-2ad58f16d000 r--p 00100000 00:24 1389914 /usr/lib64/libm-2.22.so
> 2ad58f16d000-2ad58f16e000 rw-p 00101000 00:24 1389914 /usr/lib64/libm-2.22.so
> 2ad58f16e000-2ad58f184000 r-xp 00000000 00:24 1381023 /usr/lib64/libgcc_s-5.3.1-20151207.so.1
> 2ad58f184000-2ad58f383000 ---p 00016000 00:24 1381023 /usr/lib64/libgcc_s-5.3.1-20151207.so.1
> 2ad58f383000-2ad58f384000 r--p 00015000 00:24 1381023 /usr/lib64/libgcc_s-5.3.1-20151207.so.1
> 2ad58f384000-2ad58f385000 rw-p 00016000 00:24 1381023 /usr/lib64/libgcc_s-5.3.1-20151207.so.1
> 2ad58f385000-2ad58f53c000 r-xp 00000000 00:24 1389906 /usr/lib64/libc-2.22.so
> 2ad58f53c000-2ad58f73c000 ---p 001b7000 00:24 1389906 /usr/lib64/libc-2.22.so
> 2ad58f73c000-2ad58f740000 r--p 001b7000 00:24 1389906 /usr/lib64/libc-2.22.so
> 2ad58f740000-2ad58f742000 rw-p 001bb000 00:24 1389906 /usr/lib64/libc-2.22.so
> 2ad58f742000-2ad58f746000 rw-p 00000000 00:00 0
> 2ad58f746000-2ad58f750000 r-xp 00000000 00:24 1393293 /usr/lib64/libnuma.so.1.0.0
> 2ad58f750000-2ad58f94f000 ---p 0000a000 00:24 1393293 /usr/lib64/libnuma.so.1.0.0
> 2ad58f94f000-2ad58f950000 r--p 00009000 00:24 1393293 /usr/lib64/libnuma.so.1.0.0
> 2ad58f950000-2ad58f951000 rw-p 0000a000 00:24 1393293 /usr/lib64/libnuma.so.1.0.0
> 2ad58f951000-2ad58f95a000 r-xp 00000000 00:24 1669444 /usr/lib64/libltdl.so.7.3.1
> 2ad58f95a000-2ad58fb590/opt/ats/bin/traffic_server[0x4e300a]
> /lib64/libpthread.so.0
> 9091 sent by tkill()(+0x109f0)traffic_server[0x2ad58e3709f0]
> - STACK TRACE:
> /lib64/libc.so.6(epoll_wait+0x33)[0x2ad58f488043]
> /opt/ats/bin/traffic_server(_Z19crash_logger_invokeiP9siginfo_tPv+0x8e)[0x4abece]
> /lib64/libpthread.so.0(+0x109f0)[0x2ad58e3709f0]
> /lib64/libc.so.6(gsignal+0x38)[0x2ad58f3b9a98]
> /lib64/libc.so.6(abort+0x16a)[0x2ad58f3bb69a]
> /opt/ats/bin/traffic_server(_ZN10NetHandler12mainNetEventEiP5Event+0x6f)[0x77c53f]
> /opt/ats/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8a)[0x7c37ea]
> /opt/ats/bin/traffic_server(_ZN7EThread7executeEv+0x619)[0x7c4449]
> /opt/ats/bin/traffic_server(main+0x16fa)[0x4915ea]
> /lib64/libc.so.6(__libc_start_main+0xf0)[0x2ad58f3a5580]
> /lib64/libc.so.6(+0x77daa)[0x2ad58f3fcdaa]
> /opt/ats/bin/traffic_server(_start+0x29)[0x496b79]
> [TrafficManager] ==> signal #2
> {code}
> [~bcall] got one using ASAN, which seems to point towards a double free.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)