[jira] [Resolved] (YARN-8428) YARN service has ZooKeeper connection leaks

["Eric Yang (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-8428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eric Yang resolved YARN-8428.
-----------------------------
    Resolution: Won't Fix

YARN ServiceClient is using a single connection to connect to ZooKeeper for deleting DNS entries.  The number of ZooKeeper connection required by the framework depends on number of YARN service applications running on the cluster.  This is a design trade off for using ZooKeeper as storage for coordinating distributed applications.  The only solution is to increase number of ZooKeeper concurrent connections, i.e. maxClientCnxns=2000 to increase number of YARN services can run on the cluster.  Close this as working as designed.

> YARN service has ZooKeeper connection leaks
> -------------------------------------------
>
>                 Key: YARN-8428
>                 URL: https://issues.apache.org/jira/browse/YARN-8428
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: yarn-native-services
>    Affects Versions: 3.1.0
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>            Priority: Critical
>
> There are two leaks discovered during stress test.  First, YARN service application master uses RegistryDNS to create DNS record for YARN service.  Data is first persisted on ZooKeeper, and RegistryDNS will lookup data in ZooKeeper to create DNS entries.  YARN service AM does not close ZooKeeper connection after the record has been created.  Second, when deleting DNS record from ZooKeeper, ZooKeeper connection is left open.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org