[jira] [Commented] (OOZIE-1682) remove fs.glob.max for FS action

["Andras Salamon (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OOZIE-1682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16901336#comment-16901336 ] 

Andras Salamon commented on OOZIE-1682:
---------------------------------------

In the meantime OOZIE-2215 added support for blob in FS EL function so if we want to revert OOZIE-1471 we also have to revert OOZIE-2215.

The default is already increased from 1000 to 50000, and if someone doesn't like this feature it's enough to just increase the value to an even larger value after oozie install.

> remove fs.glob.max for FS action
> --------------------------------
>
>                 Key: OOZIE-1682
>                 URL: https://issues.apache.org/jira/browse/OOZIE-1682
>             Project: Oozie
>          Issue Type: Bug
>            Reporter: Ryota Egashira
>            Priority: Major
>         Attachments: OOZIE-1682.patch
>
>
> OOZIE-1471 introduces glob support for FS action. glob max value (default to 1000) limits # of files to do operation at one time (if exceed, fail workflow) in order to protect namenode.  but we see cases where user handle more files in FS action, which requires admin to bump up max value and restart oozie-server every time. also it's difficult to determine what is a right value.
> NN becomes slower in such a case where thousands of mapper do -lsr at the same time, but it is not the case for oozie FS action.
> since NN becomes matured and improved in stability (for example, HDFS-985),  how about removing safeguard on oozie-side to make operation easier? 



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)