[jira] [Created] (THRIFT-5482) Memory leak during SSL handshake

["Anshul Mohan Gupta (Jira)" <ji...@apache.org>]

Anshul Mohan Gupta created THRIFT-5482:
------------------------------------------

             Summary: Memory leak during SSL handshake
                 Key: THRIFT-5482
                 URL: https://issues.apache.org/jira/browse/THRIFT-5482
             Project: Thrift
          Issue Type: Bug
          Components: C++ - Library
    Affects Versions: 0.12.0
         Environment: Build env: Windows 2019

Issue observed: Windows 2012R2

OpenSSL: 1.1.1k
            Reporter: Anshul Mohan Gupta


We observed continuous memory increase in the thrift server during SSL handshake and this memory increase was crashing the windows hosts as there was no memory left for other applications to work. During further investigation, we found that ERR_remove_state has been made stub in the later versions of OpenSSL, and recommended function to clear the thread state is OPENSSL_thread_stop. Once I made this change we did not observe any memory leak further. We are using OpenSSL 1.1.1k with thrift 0.12.0. Hence I am putting the patch for the same. Thanks!

I am adding call stack sample leak trace

{code}

Call stack sample 2
Address   0x0000000e`00bdd770
Allocation Time  00:00:45 since tracking started
Allocation Size 64 Bytes


Function
LeakTrack+1f109
!_malloc_base+44
!CRYPTO_zalloc+59
!BN_CTX_new+20
!rsa_ossl_public_decrypt+d8
!pkey_rsa_verify+245
!EVP_DigestVerify+15f
!tls_process_cert_verify+42a
!read_state_machine+24f
!state_machine+2d2
!SSL_do_handshake+bc
!apache::thrift::transport::TSSLSocket::initializeHandshake+dc
!apache::thrift::transport::TSSLSocket::read+3c
!apache::thrift::transport::TBufferedTransport::readSlow+56
!apache::thrift::transport::TBufferBase::readAll+88
!apache::thrift::protocol::TVirtualProtocol<apache::thrift::protocol::TBinaryProtocolT<apache::thrift::transport::TTransport,apache::thrift::protocol::TNetworkBigEndian>,apache::thrift::protocol::TProtocolDefaults>::readMessageBegin_virt+4c
!apache::thrift::TMultiplexedProcessor::process+87
!apache::thrift::server::TConnectedClient::run+16a
!apache::thrift::server::TThreadedServer::TConnectedClientRunner::run+17
!apache::thrift::concurrency::StdThread::threadMain+6c
!std::thread::_Invoke<std::tuple<int (__cdecl*)(std::shared_ptr<Reducted>),std::shared_ptr<Reducted> >,0,1>+3e
!thread_start<unsigned int (__cdecl*)(void * __ptr64)>+50
kernel32!BaseThreadInitThunk+22
ntdll!RtlUserThreadStart+34

{code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)