You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficserver.apache.org by "Leif Hedstrom (JIRA)" <ji...@apache.org> on 2016/06/15 20:02:09 UTC
[jira] [Commented] (TS-4424) proxy.config.ssl.max_record_size=-1
(dynamic rec size) can cause out-of-bound memory access
[ https://issues.apache.org/jira/browse/TS-4424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332441#comment-15332441 ]
Leif Hedstrom commented on TS-4424:
-----------------------------------
I realized I never posted the gdb info I tracked down, here it is just in case:
{code}
(gdb) bt
#0 __memmove_ssse3 () at ../sysdeps/x86_64/multiarch/memcpy-ssse3.S:118
#1 0x00007ffff6ef92c2 in __asan_memcpy (to=0x62900007d210, from=0x2aaad79fffff, size=16384)
at ../../../../libsanitizer/asan/asan_interceptors.cc:370
#2 0x00007ffff5eaec74 in ssl3_write_bytes () from /opt/openssl/lib/libssl.so.1.0.0
#3 0x0000000000c02101 in SSLWriteBuffer (ssl=0x618000086080, buf=buf@entry=0x2aaad79e0000, nbytes=nbytes@entry=1300,
nwritten=@0x7fffef4d1670: 0) at SSLUtils.cc:2186
#4 0x0000000000bd2a31 in SSLNetVConnection::load_buffer_and_write (this=<optimized out>, towrite=<optimized out>, buf=...,
total_written=<optimized out>, needs=<optimized out>) at SSLNetVConnection.cc:708
#5 0x0000000000c51a34 in write_to_net_io (nh=nh@entry=0x7fffefae8480, vc=0x2aaaafa139e0, thread=<optimized out>)
at UnixNetVConnection.cc:515
#6 0x0000000000c56bc1 in write_to_net (nh=nh@entry=0x7fffefae8480, vc=vc@entry=0x2aaaafa139e0, thread=<optimized out>)
at UnixNetVConnection.cc:424
#7 0x0000000000c0e58b in NetHandler::mainNetEvent (this=0x7fffefae8480, event=<optimized out>, e=<optimized out>)
at UnixNet.cc:531
#8 0x0000000000cfe2e4 in handleEvent (data=0x2aaaaae05c40, event=5, this=<optimized out>) at I_Continuation.h:153
#9 EThread::process_event (this=this@entry=0x7fffefae4800, e=0x2aaaaae05c40, calling_code=calling_code@entry=5)
at UnixEThread.cc:148
#10 0x0000000000d0128b in EThread::execute (this=0x7fffefae4800) at UnixEThread.cc:275
#11 0x0000000000cfd069 in spawn_thread_internal (a=0x604000016a10) at Thread.cc:84
#12 0x00007ffff4b17dc5 in start_thread (arg=0x7fffef4d2700) at pthread_create.c:308
#13 0x00007ffff3dbeced in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
(gdb) frame 5
#5 0x0000000000c51a34 in write_to_net_io (nh=nh@entry=0x7fffefae8480, vc=0x2aaaafa139e0, thread=<optimized out>)
at UnixNetVConnection.cc:515
515 int64_t r = vc->load_buffer_and_write(towrite, buf, total_written, needs);
(gdb) print towrite
$8 = <optimized out>
(gdb) print buf
$9 = (MIOBufferAccessor &) @0x2aaaafa13b80: {mbuf = 0x2aaaaf8012c0, entry = 0x2aaaaf801328}
(gdb) print total_written
$10 = <optimized out>
(gdb) print needs
$11 = <optimized out>
(gdb) frame 4
#4 0x0000000000bd2a31 in SSLNetVConnection::load_buffer_and_write (this=<optimized out>, towrite=<optimized out>, buf=...,
total_written=<optimized out>, needs=<optimized out>) at SSLNetVConnection.cc:708
708 err = SSLWriteBuffer(ssl, current_block, l, num_really_written);
(gdb) print ssl
value has been optimized out
(gdb) print current_block
$12 = 0x2aaad79e0000 "984bef62a18d239\",\"branch\":[{\"SHA1\":\"28c5d9e05ee55cefb8c6aeaad984bef62a18d239\",\"name\":\"origin/pr/4/merge\"}]},\"revision\":{\"SHA1\":\"28c5d9e05ee55cefb8c6aeaad984bef62a18d239\",\"branch\":[{\"SHA1\":\"28c5d9e05ee"...
(gdb) print l
$13 = 1300
(gdb) print num_really_written
$14 = <optimized out>
(gdb) frame 3
#3 0x0000000000c02101 in SSLWriteBuffer (ssl=0x618000086080, buf=buf@entry=0x2aaad79e0000, nbytes=nbytes@entry=1300,
nwritten=@0x7fffef4d1670: 0) at SSLUtils.cc:2186
2186 int ret = SSL_write(ssl, buf, (int)nbytes);
(gdb) print ssl
$15 = (SSL *) 0x618000086080
(gdb) print *ssl
$16 = {version = 771, type = 8192, method = 0x7ffff60ee0a0 <TLSv1_2_server_method_data.16131>, rbio = 0x60b00024e310,
wbio = 0x60b00024e260, bbio = 0x0, rwstate = 1, in_handshake = 0, handshake_func = 0x7ffff5ea2f20 <ssl3_accept>, server = 1,
new_session = 0, quiet_shutdown = 0, shutdown = 0, state = 3, rstate = 240, init_buf = 0x0, init_msg = 0x62a000756204,
init_num = 0, init_off = 0, packet = 0x62d0000f0403 "", packet_length = 0, s2 = 0x0, s3 = 0x61a0000a0e80, d1 = 0x0,
read_ahead = 0, msg_callback = 0x0, msg_callback_arg = 0x0, hit = 0, param = 0x6060007433c0, cipher_list = 0x0,
cipher_list_by_id = 0x0, mac_flags = 0, enc_read_ctx = 0x60f0000603a0, read_hash = 0x604000097190, expand = 0x0,
enc_write_ctx = 0x60f0000602b0, write_hash = 0x604000097110, compress = 0x0, cert = 0x617000481800, sid_ctx_length = 32,
sid_ctx = "mm\032\377\313i\374l\022\263\267M\377TfQ\252\217\256\204\242\rÚ\211=*(\357\375\347G", session = 0x613000073bc0,
generate_session_id = 0x0, verify_mode = 0, verify_callback = 0x0, info_callback = 0x0, error = 0, error_code = 0,
psk_client_callback = 0x0, psk_server_callback = 0x0, ctx = 0x61800000f080, debug = 0, verify_result = 0, ex_data = {
sk = 0x6030001c6960, dummy = 0}, client_CA = 0x0, references = 1, options = 2203782143, mode = 16, max_cert_list = 102400,
first_packet = 0, client_version = 771, max_send_fragment = 16384, tlsext_debug_cb = 0x0, tlsext_debug_arg = 0x0,
tlsext_hostname = 0x0, servername_done = 0, tlsext_status_type = -1, tlsext_status_expected = 0, tlsext_ocsp_ids = 0x0,
tlsext_ocsp_exts = 0x0, tlsext_ocsp_resp = 0x0, tlsext_ocsp_resplen = -1, tlsext_ticket_expected = 1,
tlsext_ecpointformatlist_length = 0, tlsext_ecpointformatlist = 0x0, tlsext_ellipticcurvelist_length = 0,
tlsext_ellipticcurvelist = 0x0, tlsext_opaque_prf_input = 0x0, tlsext_opaque_prf_input_len = 0, tlsext_session_ticket = 0x0,
tls_session_ticket_ext_cb = 0x0, tls_session_ticket_ext_cb_arg = 0x0, tls_session_secret_cb = 0x0,
tls_session_secret_cb_arg = 0x0, initial_ctx = 0x61800000f080, next_proto_negotiated = 0x0,
next_proto_negotiated_len = 0 '\000', srtp_profiles = 0x0, srtp_profile = 0x0, tlsext_heartbeat = 1, tlsext_hb_pending = 0,
tlsext_hb_seq = 0, renegotiate = 0, srp_ctx = {SRP_cb_arg = 0x0, TLS_ext_srp_username_callback = 0x0,
SRP_verify_param_callback = 0x0, SRP_give_srp_client_pwd_callback = 0x0, login = 0x0, N = 0x0, g = 0x0, s = 0x0, B = 0x0,
A = 0x0, a = 0x0, b = 0x0, v = 0x0, info = 0x0, strength = 1024, srp_Mask = 0}, alpn_client_proto_list = 0x0,
alpn_client_proto_list_len = 0}
(gdb) print nbytes
$17 = 1300
(gdb) frame 2
#2 0x00007ffff5eaec74 in ssl3_write_bytes () from /opt/openssl/lib/libssl.so.1.0.0
(gdb) frame 1
#1 0x00007ffff6ef92c2 in __asan_memcpy (to=0x62900007d210, from=0x2aaad79fffff, size=16384)
at ../../../../libsanitizer/asan/asan_interceptors.cc:370
370 ../../../../libsanitizer/asan/asan_interceptors.cc: No such file or directory.
(gdb) frame 0
#0 __memmove_ssse3 () at ../sysdeps/x86_64/multiarch/memcpy-ssse3.S:118
118 movdqu -16(%rsi, %rdx), %xmm0
{code}
> proxy.config.ssl.max_record_size=-1 (dynamic rec size) can cause out-of-bound memory access
> -------------------------------------------------------------------------------------------
>
> Key: TS-4424
> URL: https://issues.apache.org/jira/browse/TS-4424
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
> Affects Versions: 6.2.0
> Reporter: Leif Hedstrom
> Assignee: Susan Hinrichs
> Labels: crash
> Fix For: 7.0.0
>
>
> From a few days ago:
> {code}
> [May 2 16:09:34.060] Manager {0x7f44c4e94800} WARNING: Be aware that access control checks for HTTP/2 connections are not active!
> [May 2 16:09:34.060] Manager {0x7f44c4e94800} WARNING: Be aware that access control checks for HTTP/2 connections are not active!
> =================================================================
> ==17268==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x62a0064b5000 at pc 0x2b8fd0a73615 bp 0x7ffd34d416e0 sp 0x7ffd34d40e90
> READ of size 16384 at 0x62a0064b5000 thread T0 ([ET_NET 0])
> #0 0x2b8fd0a73614 in __asan_memcpy ../../../../libsanitizer/asan/asan_interceptors.cc:367
> #1 0x2b8fd26f7b63 in ssl3_write_bytes (/opt/openssl/lib/libssl.so.1.0.0+0x29b63)
> #2 0xbfe2e0 in SSLWriteBuffer(ssl_st*, void const*, long, long&) /usr/local/src/trafficserver/iocore/net/SSLUtils.cc:2041
> #3 0xbd2a6a in SSLNetVConnection::load_buffer_and_write(long, long&, long&, MIOBufferAccessor&, int&) /usr/local/src/trafficserver/iocore/net/SSLNetVConnection.cc:735
> #4 0xc48dad in write_to_net_io(NetHandler*, UnixNetVConnection*, EThread*) /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:511
> #5 0xc0a8ba in NetHandler::mainNetEvent(int, Event*) /usr/local/src/trafficserver/iocore/net/UnixNet.cc:529
> #6 0xcf6da3 in Continuation::handleEvent(int, void*) /usr/local/src/trafficserver/iocore/eventsystem/I_Continuation.h:153
> #7 0xcf6da3 in EThread::process_event(Event*, int) /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:129
> #8 0xcf9d4a in EThread::execute() /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:256
> #9 0x498ad5 in main /usr/local/src/trafficserver/proxy/Main.cc:1909
> #10 0x2b8fd475bb14 in __libc_start_main (/lib64/libc.so.6+0x21b14)
> #11 0x4a8244 (/opt/ats/bin/traffic_server+0x4a8244)
> 0x62a0064b5000 is located 0 bytes to the right of 16384-byte region [0x62a0064b1000,0x62a0064b5000)
> allocated by thread T0 ([ET_NET 0]) here:
> #0 0x2b8fd0a7f9ae in __interceptor_posix_memalign ../../../../libsanitizer/asan/asan_malloc_linux.cc:105
> #1 0x2b8fd19b2ca9 in ats_memalign /usr/local/src/trafficserver/lib/ts/ink_memory.cc:105
> #2 0x2b8fd19b3f3e in ink_freelist_new /usr/local/src/trafficserver/lib/ts/ink_queue.cc:183
> #3 0x7d5670 in Allocator::alloc_void() ../../lib/ts/Allocator.h:63
> #4 0x7d5670 in IOBufferData::alloc(long, AllocType) ../../iocore/eventsystem/P_IOBuffer.h:282
> #5 0x7d5670 in new_IOBufferData_internal(char const*, long, AllocType) ../../iocore/eventsystem/P_IOBuffer.h:253
> #6 0x7d5670 in IOBufferBlock::alloc(long) ../../iocore/eventsystem/P_IOBuffer.h:396
> #7 0x7d5670 in Http2Frame::alloc(int) /usr/local/src/trafficserver/proxy/http2/Http2ClientSession.h:96
> #8 0x7d5670 in Http2ConnectionState::send_data_frame(Http2Stream*) /usr/local/src/trafficserver/proxy/http2/Http2ConnectionState.cc:959
> #9 0x7ea209 in Http2Stream::update_write_request(IOBufferReader*, long, bool) /usr/local/src/trafficserver/proxy/http2/Http2Stream.cc:462
> #10 0x7efe96 in Http2Stream::reenable(VIO*) /usr/local/src/trafficserver/proxy/http2/Http2Stream.cc:482
> #11 0x777c49 in VIO::reenable() ../../iocore/eventsystem/P_VIO.h:111
> #12 0x777c49 in HttpTunnel::producer_handler(int, HttpTunnelProducer*) /usr/local/src/trafficserver/proxy/http/HttpTunnel.cc:1199
> #13 0x77a0ee in HttpTunnel::main_handler(int, void*) /usr/local/src/trafficserver/proxy/http/HttpTunnel.cc:1568
> #14 0x9a2467 in Continuation::handleEvent(int, void*) ../../iocore/eventsystem/I_Continuation.h:153
> #15 0x9a2467 in CacheVC::calluser(int) ../../iocore/cache/P_CacheInternal.h:623
> #16 0xb21d2c in CacheVC::openReadMain(int, Event*) /usr/local/src/trafficserver/iocore/cache/CacheRead.cc:717
> #17 0x9a284c in Continuation::handleEvent(int, void*) ../../iocore/eventsystem/I_Continuation.h:153
> #18 0x9a284c in CacheVC::callcont(int) ../../iocore/cache/P_CacheInternal.h:642
> #19 0xb30df9 in CacheVC::openReadStartHead(int, Event*) /usr/local/src/trafficserver/iocore/cache/CacheRead.cc:1162
> #20 0xb29c07 in Continuation::handleEvent(int, void*) ../../iocore/eventsystem/I_Continuation.h:153
> #21 0xb29c07 in Cache::open_read(Continuation*, ats::CryptoHash const*, HTTPHdr*, CacheLookupHttpConfig*, CacheFragType, char const*, int) /usr/local/src/trafficserver/iocore/cache/CacheRead.cc:159
> #22 0x648a63 in HttpCacheSM::do_cache_open_read(HttpCacheKey const&) /usr/local/src/trafficserver/proxy/http/HttpCacheSM.cc:237
> #23 0x648a63 in HttpCacheSM::open_read(HttpCacheKey const*, URL*, HTTPHdr*, CacheLookupHttpConfig*, long) /usr/local/src/trafficserver/proxy/http/HttpCacheSM.cc:270
> #24 0x67fb65 in HttpSM::do_cache_lookup_and_read() /usr/local/src/trafficserver/proxy/http/HttpSM.cc:4457
> #25 0x6be022 in HttpSM::set_next_state() /usr/local/src/trafficserver/proxy/http/HttpSM.cc:7168
> #26 0x6b03c9 in HttpSM::state_api_callout(int, void*) /usr/local/src/trafficserver/proxy/http/HttpSM.cc:1490
> #27 0x6be667 in HttpSM::set_next_state() /usr/local/src/trafficserver/proxy/http/HttpSM.cc:7058
> #28 0x6bd219 in HttpSM::set_next_state() /usr/local/src/trafficserver/proxy/http/HttpSM.cc:7072
> #29 0x6b03c9 in HttpSM::state_api_callout(int, void*) /usr/local/src/trafficserver/proxy/http/HttpSM.cc:1490
> #30 0x6be667 in HttpSM::set_next_state() /usr/local/src/trafficserver/proxy/http/HttpSM.cc:7058
> #31 0x6b03c9 in HttpSM::state_api_callout(int, void*) /usr/local/src/trafficserver/proxy/http/HttpSM.cc:1490
> #32 0x6c2bef in HttpSM::state_api_callback(int, void*) /usr/local/src/trafficserver/proxy/http/HttpSM.cc:1287
> #33 0x56c006 in TSHttpTxnReenable /usr/local/src/trafficserver/proxy/InkAPI.cc:5635
> #34 0x2b8fd9cbff45 in stats_origin /usr/local/src/trafficserver/plugins/stats_over_http/stats_over_http.c:280
> #35 0x5391e3 in INKContInternal::handle_event(int, void*) /usr/local/src/trafficserver/proxy/InkAPI.cc:993
> #36 0x6affe4 in HttpSM::state_api_callout(int, void*) /usr/local/src/trafficserver/proxy/http/HttpSM.cc:1408
> #37 0x6be667 in HttpSM::set_next_state() /usr/local/src/trafficserver/proxy/http/HttpSM.cc:7058
> #38 0x69574d in HttpSM::state_read_client_request_header(int, void*) /usr/local/src/trafficserver/proxy/http/HttpSM.cc:769
> #39 0x6c30f7 in HttpSM::main_handler(int, void*) /usr/local/src/trafficserver/proxy/http/HttpSM.cc:2600
> SUMMARY: AddressSanitizer: heap-buffer-overflow ../../../../libsanitizer/asan/asan_interceptors.cc:367 __asan_memcpy
> Shadow bytes around the buggy address:
> 0x0c5480c8e9b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x0c5480c8e9c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x0c5480c8e9d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x0c5480c8e9e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x0c5480c8e9f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> =>0x0c5480c8ea00:[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
> 0x0c5480c8ea10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
> 0x0c5480c8ea20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
> 0x0c5480c8ea30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
> 0x0c5480c8ea40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
> 0x0c5480c8ea50: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
> Shadow byte legend (one shadow byte represents 8 application bytes):
> Addressable: 00
> Partially addressable: 01 02 03 04 05 06 07
> Heap left redzone: fa
> Heap right redzone: fb
> Freed heap region: fd
> Stack left redzone: f1
> Stack mid redzone: f2
> Stack right redzone: f3
> Stack partial redzone: f4
> Stack after return: f5
> Stack use after scope: f8
> Global redzone: f9
> Global init order: f6
> Poisoned by user: f7
> Container overflow: fc
> Array cookie: ac
> Intra object redzone: bb
> ASan internal: fe
> ==17268==ABORTING
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)