You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficserver.apache.org by "Leif Hedstrom (JIRA)" <ji...@apache.org> on 2016/06/15 20:02:09 UTC

[jira] [Commented] (TS-4424) proxy.config.ssl.max_record_size=-1 (dynamic rec size) can cause out-of-bound memory access

    [ https://issues.apache.org/jira/browse/TS-4424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332441#comment-15332441 ] 

Leif Hedstrom commented on TS-4424:
-----------------------------------

I realized I never posted the gdb info I tracked down, here it is just in case:

{code}
(gdb) bt
#0  __memmove_ssse3 () at ../sysdeps/x86_64/multiarch/memcpy-ssse3.S:118
#1  0x00007ffff6ef92c2 in __asan_memcpy (to=0x62900007d210, from=0x2aaad79fffff, size=16384)
    at ../../../../libsanitizer/asan/asan_interceptors.cc:370
#2  0x00007ffff5eaec74 in ssl3_write_bytes () from /opt/openssl/lib/libssl.so.1.0.0
#3  0x0000000000c02101 in SSLWriteBuffer (ssl=0x618000086080, buf=buf@entry=0x2aaad79e0000, nbytes=nbytes@entry=1300,
    nwritten=@0x7fffef4d1670: 0) at SSLUtils.cc:2186
#4  0x0000000000bd2a31 in SSLNetVConnection::load_buffer_and_write (this=<optimized out>, towrite=<optimized out>, buf=...,
    total_written=<optimized out>, needs=<optimized out>) at SSLNetVConnection.cc:708
#5  0x0000000000c51a34 in write_to_net_io (nh=nh@entry=0x7fffefae8480, vc=0x2aaaafa139e0, thread=<optimized out>)
    at UnixNetVConnection.cc:515
#6  0x0000000000c56bc1 in write_to_net (nh=nh@entry=0x7fffefae8480, vc=vc@entry=0x2aaaafa139e0, thread=<optimized out>)
    at UnixNetVConnection.cc:424
#7  0x0000000000c0e58b in NetHandler::mainNetEvent (this=0x7fffefae8480, event=<optimized out>, e=<optimized out>)
    at UnixNet.cc:531
#8  0x0000000000cfe2e4 in handleEvent (data=0x2aaaaae05c40, event=5, this=<optimized out>) at I_Continuation.h:153
#9  EThread::process_event (this=this@entry=0x7fffefae4800, e=0x2aaaaae05c40, calling_code=calling_code@entry=5)
    at UnixEThread.cc:148
#10 0x0000000000d0128b in EThread::execute (this=0x7fffefae4800) at UnixEThread.cc:275
#11 0x0000000000cfd069 in spawn_thread_internal (a=0x604000016a10) at Thread.cc:84
#12 0x00007ffff4b17dc5 in start_thread (arg=0x7fffef4d2700) at pthread_create.c:308
#13 0x00007ffff3dbeced in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
(gdb) frame 5
#5  0x0000000000c51a34 in write_to_net_io (nh=nh@entry=0x7fffefae8480, vc=0x2aaaafa139e0, thread=<optimized out>)
    at UnixNetVConnection.cc:515
515       int64_t r = vc->load_buffer_and_write(towrite, buf, total_written, needs);
(gdb) print towrite
$8 = <optimized out>
(gdb) print buf
$9 = (MIOBufferAccessor &) @0x2aaaafa13b80: {mbuf = 0x2aaaaf8012c0, entry = 0x2aaaaf801328}
(gdb) print total_written
$10 = <optimized out>
(gdb) print needs
$11 = <optimized out>
(gdb) frame 4
#4  0x0000000000bd2a31 in SSLNetVConnection::load_buffer_and_write (this=<optimized out>, towrite=<optimized out>, buf=...,
    total_written=<optimized out>, needs=<optimized out>) at SSLNetVConnection.cc:708
708         err = SSLWriteBuffer(ssl, current_block, l, num_really_written);
(gdb) print ssl
value has been optimized out
(gdb) print current_block
$12 = 0x2aaad79e0000 "984bef62a18d239\",\"branch\":[{\"SHA1\":\"28c5d9e05ee55cefb8c6aeaad984bef62a18d239\",\"name\":\"origin/pr/4/merge\"}]},\"revision\":{\"SHA1\":\"28c5d9e05ee55cefb8c6aeaad984bef62a18d239\",\"branch\":[{\"SHA1\":\"28c5d9e05ee"...
(gdb) print l
$13 = 1300
(gdb) print num_really_written
$14 = <optimized out>
(gdb) frame 3
#3  0x0000000000c02101 in SSLWriteBuffer (ssl=0x618000086080, buf=buf@entry=0x2aaad79e0000, nbytes=nbytes@entry=1300,
    nwritten=@0x7fffef4d1670: 0) at SSLUtils.cc:2186
2186      int ret = SSL_write(ssl, buf, (int)nbytes);
(gdb) print ssl
$15 = (SSL *) 0x618000086080
(gdb) print *ssl
$16 = {version = 771, type = 8192, method = 0x7ffff60ee0a0 <TLSv1_2_server_method_data.16131>, rbio = 0x60b00024e310,
  wbio = 0x60b00024e260, bbio = 0x0, rwstate = 1, in_handshake = 0, handshake_func = 0x7ffff5ea2f20 <ssl3_accept>, server = 1,
  new_session = 0, quiet_shutdown = 0, shutdown = 0, state = 3, rstate = 240, init_buf = 0x0, init_msg = 0x62a000756204,
  init_num = 0, init_off = 0, packet = 0x62d0000f0403 "", packet_length = 0, s2 = 0x0, s3 = 0x61a0000a0e80, d1 = 0x0,
  read_ahead = 0, msg_callback = 0x0, msg_callback_arg = 0x0, hit = 0, param = 0x6060007433c0, cipher_list = 0x0,
  cipher_list_by_id = 0x0, mac_flags = 0, enc_read_ctx = 0x60f0000603a0, read_hash = 0x604000097190, expand = 0x0,
  enc_write_ctx = 0x60f0000602b0, write_hash = 0x604000097110, compress = 0x0, cert = 0x617000481800, sid_ctx_length = 32,
  sid_ctx = "mm\032\377\313i\374l\022\263\267M\377TfQ\252\217\256\204\242\rÚ\211=*(\357\375\347G", session = 0x613000073bc0,
  generate_session_id = 0x0, verify_mode = 0, verify_callback = 0x0, info_callback = 0x0, error = 0, error_code = 0,
  psk_client_callback = 0x0, psk_server_callback = 0x0, ctx = 0x61800000f080, debug = 0, verify_result = 0, ex_data = {
    sk = 0x6030001c6960, dummy = 0}, client_CA = 0x0, references = 1, options = 2203782143, mode = 16, max_cert_list = 102400,
  first_packet = 0, client_version = 771, max_send_fragment = 16384, tlsext_debug_cb = 0x0, tlsext_debug_arg = 0x0,
  tlsext_hostname = 0x0, servername_done = 0, tlsext_status_type = -1, tlsext_status_expected = 0, tlsext_ocsp_ids = 0x0,
  tlsext_ocsp_exts = 0x0, tlsext_ocsp_resp = 0x0, tlsext_ocsp_resplen = -1, tlsext_ticket_expected = 1,
  tlsext_ecpointformatlist_length = 0, tlsext_ecpointformatlist = 0x0, tlsext_ellipticcurvelist_length = 0,
  tlsext_ellipticcurvelist = 0x0, tlsext_opaque_prf_input = 0x0, tlsext_opaque_prf_input_len = 0, tlsext_session_ticket = 0x0,
  tls_session_ticket_ext_cb = 0x0, tls_session_ticket_ext_cb_arg = 0x0, tls_session_secret_cb = 0x0,
  tls_session_secret_cb_arg = 0x0, initial_ctx = 0x61800000f080, next_proto_negotiated = 0x0,
  next_proto_negotiated_len = 0 '\000', srtp_profiles = 0x0, srtp_profile = 0x0, tlsext_heartbeat = 1, tlsext_hb_pending = 0,
  tlsext_hb_seq = 0, renegotiate = 0, srp_ctx = {SRP_cb_arg = 0x0, TLS_ext_srp_username_callback = 0x0,
    SRP_verify_param_callback = 0x0, SRP_give_srp_client_pwd_callback = 0x0, login = 0x0, N = 0x0, g = 0x0, s = 0x0, B = 0x0,
    A = 0x0, a = 0x0, b = 0x0, v = 0x0, info = 0x0, strength = 1024, srp_Mask = 0}, alpn_client_proto_list = 0x0,
  alpn_client_proto_list_len = 0}
(gdb) print nbytes
$17 = 1300
(gdb) frame 2
#2  0x00007ffff5eaec74 in ssl3_write_bytes () from /opt/openssl/lib/libssl.so.1.0.0
(gdb) frame 1
#1  0x00007ffff6ef92c2 in __asan_memcpy (to=0x62900007d210, from=0x2aaad79fffff, size=16384)
    at ../../../../libsanitizer/asan/asan_interceptors.cc:370
370     ../../../../libsanitizer/asan/asan_interceptors.cc: No such file or directory.
(gdb) frame 0
#0  __memmove_ssse3 () at ../sysdeps/x86_64/multiarch/memcpy-ssse3.S:118
118             movdqu  -16(%rsi, %rdx), %xmm0
{code}

> proxy.config.ssl.max_record_size=-1 (dynamic rec size) can cause out-of-bound memory access
> -------------------------------------------------------------------------------------------
>
>                 Key: TS-4424
>                 URL: https://issues.apache.org/jira/browse/TS-4424
>             Project: Traffic Server
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 6.2.0
>            Reporter: Leif Hedstrom
>            Assignee: Susan Hinrichs
>              Labels: crash
>             Fix For: 7.0.0
>
>
> From a few days ago:
> {code}
> [May  2 16:09:34.060] Manager {0x7f44c4e94800} WARNING: Be aware that access control checks for HTTP/2 connections are not active!
> [May  2 16:09:34.060] Manager {0x7f44c4e94800} WARNING: Be aware that access control checks for HTTP/2 connections are not active!
> =================================================================
> ==17268==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x62a0064b5000 at pc 0x2b8fd0a73615 bp 0x7ffd34d416e0 sp 0x7ffd34d40e90
> READ of size 16384 at 0x62a0064b5000 thread T0 ([ET_NET 0])
>     #0 0x2b8fd0a73614 in __asan_memcpy ../../../../libsanitizer/asan/asan_interceptors.cc:367
>     #1 0x2b8fd26f7b63 in ssl3_write_bytes (/opt/openssl/lib/libssl.so.1.0.0+0x29b63)
>     #2 0xbfe2e0 in SSLWriteBuffer(ssl_st*, void const*, long, long&) /usr/local/src/trafficserver/iocore/net/SSLUtils.cc:2041
>     #3 0xbd2a6a in SSLNetVConnection::load_buffer_and_write(long, long&, long&, MIOBufferAccessor&, int&) /usr/local/src/trafficserver/iocore/net/SSLNetVConnection.cc:735
>     #4 0xc48dad in write_to_net_io(NetHandler*, UnixNetVConnection*, EThread*) /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:511
>     #5 0xc0a8ba in NetHandler::mainNetEvent(int, Event*) /usr/local/src/trafficserver/iocore/net/UnixNet.cc:529
>     #6 0xcf6da3 in Continuation::handleEvent(int, void*) /usr/local/src/trafficserver/iocore/eventsystem/I_Continuation.h:153
>     #7 0xcf6da3 in EThread::process_event(Event*, int) /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:129
>     #8 0xcf9d4a in EThread::execute() /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:256
>     #9 0x498ad5 in main /usr/local/src/trafficserver/proxy/Main.cc:1909
>     #10 0x2b8fd475bb14 in __libc_start_main (/lib64/libc.so.6+0x21b14)
>     #11 0x4a8244  (/opt/ats/bin/traffic_server+0x4a8244)
> 0x62a0064b5000 is located 0 bytes to the right of 16384-byte region [0x62a0064b1000,0x62a0064b5000)
> allocated by thread T0 ([ET_NET 0]) here:
>     #0 0x2b8fd0a7f9ae in __interceptor_posix_memalign ../../../../libsanitizer/asan/asan_malloc_linux.cc:105
>     #1 0x2b8fd19b2ca9 in ats_memalign /usr/local/src/trafficserver/lib/ts/ink_memory.cc:105
>     #2 0x2b8fd19b3f3e in ink_freelist_new /usr/local/src/trafficserver/lib/ts/ink_queue.cc:183
>     #3 0x7d5670 in Allocator::alloc_void() ../../lib/ts/Allocator.h:63
>     #4 0x7d5670 in IOBufferData::alloc(long, AllocType) ../../iocore/eventsystem/P_IOBuffer.h:282
>     #5 0x7d5670 in new_IOBufferData_internal(char const*, long, AllocType) ../../iocore/eventsystem/P_IOBuffer.h:253
>     #6 0x7d5670 in IOBufferBlock::alloc(long) ../../iocore/eventsystem/P_IOBuffer.h:396
>     #7 0x7d5670 in Http2Frame::alloc(int) /usr/local/src/trafficserver/proxy/http2/Http2ClientSession.h:96
>     #8 0x7d5670 in Http2ConnectionState::send_data_frame(Http2Stream*) /usr/local/src/trafficserver/proxy/http2/Http2ConnectionState.cc:959
>     #9 0x7ea209 in Http2Stream::update_write_request(IOBufferReader*, long, bool) /usr/local/src/trafficserver/proxy/http2/Http2Stream.cc:462
>     #10 0x7efe96 in Http2Stream::reenable(VIO*) /usr/local/src/trafficserver/proxy/http2/Http2Stream.cc:482
>     #11 0x777c49 in VIO::reenable() ../../iocore/eventsystem/P_VIO.h:111
>     #12 0x777c49 in HttpTunnel::producer_handler(int, HttpTunnelProducer*) /usr/local/src/trafficserver/proxy/http/HttpTunnel.cc:1199
>     #13 0x77a0ee in HttpTunnel::main_handler(int, void*) /usr/local/src/trafficserver/proxy/http/HttpTunnel.cc:1568
>     #14 0x9a2467 in Continuation::handleEvent(int, void*) ../../iocore/eventsystem/I_Continuation.h:153
>     #15 0x9a2467 in CacheVC::calluser(int) ../../iocore/cache/P_CacheInternal.h:623
>     #16 0xb21d2c in CacheVC::openReadMain(int, Event*) /usr/local/src/trafficserver/iocore/cache/CacheRead.cc:717
>     #17 0x9a284c in Continuation::handleEvent(int, void*) ../../iocore/eventsystem/I_Continuation.h:153
>     #18 0x9a284c in CacheVC::callcont(int) ../../iocore/cache/P_CacheInternal.h:642
>     #19 0xb30df9 in CacheVC::openReadStartHead(int, Event*) /usr/local/src/trafficserver/iocore/cache/CacheRead.cc:1162
>     #20 0xb29c07 in Continuation::handleEvent(int, void*) ../../iocore/eventsystem/I_Continuation.h:153
>     #21 0xb29c07 in Cache::open_read(Continuation*, ats::CryptoHash const*, HTTPHdr*, CacheLookupHttpConfig*, CacheFragType, char const*, int) /usr/local/src/trafficserver/iocore/cache/CacheRead.cc:159
>     #22 0x648a63 in HttpCacheSM::do_cache_open_read(HttpCacheKey const&) /usr/local/src/trafficserver/proxy/http/HttpCacheSM.cc:237
>     #23 0x648a63 in HttpCacheSM::open_read(HttpCacheKey const*, URL*, HTTPHdr*, CacheLookupHttpConfig*, long) /usr/local/src/trafficserver/proxy/http/HttpCacheSM.cc:270
>     #24 0x67fb65 in HttpSM::do_cache_lookup_and_read() /usr/local/src/trafficserver/proxy/http/HttpSM.cc:4457
>     #25 0x6be022 in HttpSM::set_next_state() /usr/local/src/trafficserver/proxy/http/HttpSM.cc:7168
>     #26 0x6b03c9 in HttpSM::state_api_callout(int, void*) /usr/local/src/trafficserver/proxy/http/HttpSM.cc:1490
>     #27 0x6be667 in HttpSM::set_next_state() /usr/local/src/trafficserver/proxy/http/HttpSM.cc:7058
>     #28 0x6bd219 in HttpSM::set_next_state() /usr/local/src/trafficserver/proxy/http/HttpSM.cc:7072
>     #29 0x6b03c9 in HttpSM::state_api_callout(int, void*) /usr/local/src/trafficserver/proxy/http/HttpSM.cc:1490
>     #30 0x6be667 in HttpSM::set_next_state() /usr/local/src/trafficserver/proxy/http/HttpSM.cc:7058
>     #31 0x6b03c9 in HttpSM::state_api_callout(int, void*) /usr/local/src/trafficserver/proxy/http/HttpSM.cc:1490
>     #32 0x6c2bef in HttpSM::state_api_callback(int, void*) /usr/local/src/trafficserver/proxy/http/HttpSM.cc:1287
>     #33 0x56c006 in TSHttpTxnReenable /usr/local/src/trafficserver/proxy/InkAPI.cc:5635
>     #34 0x2b8fd9cbff45 in stats_origin /usr/local/src/trafficserver/plugins/stats_over_http/stats_over_http.c:280
>     #35 0x5391e3 in INKContInternal::handle_event(int, void*) /usr/local/src/trafficserver/proxy/InkAPI.cc:993
>     #36 0x6affe4 in HttpSM::state_api_callout(int, void*) /usr/local/src/trafficserver/proxy/http/HttpSM.cc:1408
>     #37 0x6be667 in HttpSM::set_next_state() /usr/local/src/trafficserver/proxy/http/HttpSM.cc:7058
>     #38 0x69574d in HttpSM::state_read_client_request_header(int, void*) /usr/local/src/trafficserver/proxy/http/HttpSM.cc:769
>     #39 0x6c30f7 in HttpSM::main_handler(int, void*) /usr/local/src/trafficserver/proxy/http/HttpSM.cc:2600
> SUMMARY: AddressSanitizer: heap-buffer-overflow ../../../../libsanitizer/asan/asan_interceptors.cc:367 __asan_memcpy
> Shadow bytes around the buggy address:
>   0x0c5480c8e9b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x0c5480c8e9c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x0c5480c8e9d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x0c5480c8e9e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x0c5480c8e9f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> =>0x0c5480c8ea00:[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>   0x0c5480c8ea10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>   0x0c5480c8ea20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>   0x0c5480c8ea30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>   0x0c5480c8ea40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>   0x0c5480c8ea50: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
> Shadow byte legend (one shadow byte represents 8 application bytes):
>   Addressable:           00
>   Partially addressable: 01 02 03 04 05 06 07
>   Heap left redzone:       fa
>   Heap right redzone:      fb
>   Freed heap region:       fd
>   Stack left redzone:      f1
>   Stack mid redzone:       f2
>   Stack right redzone:     f3
>   Stack partial redzone:   f4
>   Stack after return:      f5
>   Stack use after scope:   f8
>   Global redzone:          f9
>   Global init order:       f6
>   Poisoned by user:        f7
>   Container overflow:      fc
>   Array cookie:            ac
>   Intra object redzone:    bb
>   ASan internal:           fe
> ==17268==ABORTING
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)