[jira] [Commented] (KUDU-2395) Thread spike with all threads blocked in libnss

["Todd Lipcon (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KUDU-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16831928#comment-16831928 ] 

Todd Lipcon commented on KUDU-2395:
-----------------------------------

Worth noting that this can also occur with stacks in GetLoggedInUser like this:
{code}
      0x7fd25227642b __lll_lock_wait
      0x7fd252271dcb _L_lock_812
      0x7fd252271c98 __GI___pthread_mutex_lock
      0x7fd247cfcfc3 _nss_files_getpwuid_r
      0x7fd2502fc52e __getpwuid_r
           0x1b97356 kudu::GetLoggedInUser()
           0x1a0f453 kudu::rpc::Proxy::Proxy()
{code}

That particular case was fixed by 52b50b7a91c61925a7bc42992fe1001e74425d4d in Kudu 1.8, though.

> Thread spike with all threads blocked in libnss
> -----------------------------------------------
>
>                 Key: KUDU-2395
>                 URL: https://issues.apache.org/jira/browse/KUDU-2395
>             Project: Kudu
>          Issue Type: Bug
>          Components: consensus, tserver, util
>            Reporter: Todd Lipcon
>            Priority: Minor
>
> I saw the thread count on a server under a load test spike from 280 threads (fairly constant) to 3400 threads (briefly). I checked the diagnostics log and found that there are several thousand threads in a stack like:
> {code}
> 0x7facce018606 _nss_files_gethostbyname2_r
>   0x345a703645 <unknown>
>   0x345a6d0b3b <unknown>
>   0x345a6d2d80 <unknown>
>      0x1c9366c kudu::(anonymous namespace)::GetAddrInfo()
>      0x1c95fbe kudu::HostPort::ResolveAddresses()
>       0xac4b78 kudu::consensus::(anonymous namespace)::CreateConsensusServiceProxyForHost()
>       0xac5058 kudu::consensus::RpcPeerProxyFactory::NewProxy()
>       0xb0b212 kudu::consensus::LeaderElection::LeaderElection()
>       0xafab80 kudu::consensus::RaftConsensus::StartElection()
>       0xafd20c kudu::consensus::RaftConsensus::ReportFailureDetectedTask()
>      0x1ccf4ed kudu::FunctionRunnable::Run()
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)