You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kudu.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2018/03/31 00:00:30 UTC
[jira] [Commented] (KUDU-2395) Thread spike with all threads
blocked in libnss
[ https://issues.apache.org/jira/browse/KUDU-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16421089#comment-16421089 ]
Todd Lipcon commented on KUDU-2395:
-----------------------------------
It's worth noting this server does not have nscd running. We should probably recommend usage of 'nscd' and consider implementing our own DNS cache of some sort. KUDU-75 is also relevant, which would make this asynchronous and avoid creating thousands of threads even if DNS is slow.
> Thread spike with all threads blocked in libnss
> -----------------------------------------------
>
> Key: KUDU-2395
> URL: https://issues.apache.org/jira/browse/KUDU-2395
> Project: Kudu
> Issue Type: Bug
> Components: consensus, tserver, util
> Reporter: Todd Lipcon
> Priority: Major
>
> I saw the thread count on a server under a load test spike from 280 threads (fairly constant) to 3400 threads (briefly). I checked the diagnostics log and found that there are several thousand threads in a stack like:
> {code}
> 0x7facce018606 _nss_files_gethostbyname2_r
> 0x345a703645 <unknown>
> 0x345a6d0b3b <unknown>
> 0x345a6d2d80 <unknown>
> 0x1c9366c kudu::(anonymous namespace)::GetAddrInfo()
> 0x1c95fbe kudu::HostPort::ResolveAddresses()
> 0xac4b78 kudu::consensus::(anonymous namespace)::CreateConsensusServiceProxyForHost()
> 0xac5058 kudu::consensus::RpcPeerProxyFactory::NewProxy()
> 0xb0b212 kudu::consensus::LeaderElection::LeaderElection()
> 0xafab80 kudu::consensus::RaftConsensus::StartElection()
> 0xafd20c kudu::consensus::RaftConsensus::ReportFailureDetectedTask()
> 0x1ccf4ed kudu::FunctionRunnable::Run()
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)