You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-issues@hadoop.apache.org by "Akira AJISAKA (JIRA)" <ji...@apache.org> on 2015/02/07 04:09:36 UTC

[jira] [Updated] (HDFS-7597) Clients seeking over webhdfs may crash the NN

     [ https://issues.apache.org/jira/browse/HDFS-7597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Akira AJISAKA updated HDFS-7597:
--------------------------------
    Status: Open  (was: Patch Available)

> Clients seeking over webhdfs may crash the NN
> ---------------------------------------------
>
>                 Key: HDFS-7597
>                 URL: https://issues.apache.org/jira/browse/HDFS-7597
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: webhdfs
>    Affects Versions: 2.0.0-alpha
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Critical
>         Attachments: HDFS-7597.patch, HDFS-7597.patch
>
>
> Webhdfs seeks involve closing the current connection, and reissuing a new open request with the new offset.  The RPC layer caches connections so the DN keeps a lingering connection open to the NN.  Connection caching is in part based on UGI.  Although the client used the same token for the new offset request, the UGI is different which forces the DN to open another unnecessary connection to the NN.
> A job that performs many seeks will easily crash the NN due to fd exhaustion.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)