[jira] [Updated] (MAPREDUCE-5603) Ability to disable FileInputFormat listLocatedStatus optimization to save client memory

["Jason Lowe (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-5603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Lowe updated MAPREDUCE-5603:
----------------------------------

          Resolution: Won't Fix
    Target Version/s: 2.3.0, 3.0.0, 0.23.11  (was: 3.0.0, 0.23.11, 2.3.0)
              Status: Resolved  (was: Patch Available)

Closing this as won't fix for now.  This was originally filed as a stop-gap in case there were any situations where the client needed the extra memory and simply couldn't use a larger heap.  However after running with the listLocatedStatus fix for quite some time we've only seen a couple of instances where the client needed more memory to handle the situation and it was an easy fix to simply increase the memory used by the client.

There's plenty of confs in Hadoop as it is, and I'd rather not add yet another conf if it's not necessary.  We can reopen this if the need arises.

> Ability to disable FileInputFormat listLocatedStatus optimization to save client memory
> ---------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5603
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5603
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: client, mrv2
>    Affects Versions: 0.23.10, 2.2.0
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>            Priority: Minor
>         Attachments: MAPREDUCE-5603.patch, MAPREDUCE-5603.patch
>
>
> It would be nice if users had the option to disable the listLocatedStatus optimization in FileInputFormat to save client memory.



--
This message was sent by Atlassian JIRA
(v6.2#6252)