[jira] [Updated] (OOZIE-1682) remove fs.glob.max for FS action

["Ryota Egashira (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/OOZIE-1682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ryota Egashira updated OOZIE-1682:
----------------------------------

    Attachment: OOZIE-1682.patch

> remove fs.glob.max for FS action
> --------------------------------
>
>                 Key: OOZIE-1682
>                 URL: https://issues.apache.org/jira/browse/OOZIE-1682
>             Project: Oozie
>          Issue Type: Bug
>            Reporter: Ryota Egashira
>         Attachments: OOZIE-1682.patch
>
>
> OOZIE-1471 introduces glob support for FS action. glob max value (default to 1000) limits # of files to do operation at one time (if exceed, fail workflow) in order to protect namenode.  but we see cases where user handle more files in FS action, which requires admin to bump up max value and restart oozie-server every time. also it's difficult to determine what is a right value.
> NN becomes slower in such a case where thousands of mapper do -lsr at the same time, but it is not the case for oozie FS action.
> since NN becomes matured and improved in stability (for example, HDFS-985),  how about removing safeguard on oozie-side to make operation easier? 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)