[jira] Updated: (MAPREDUCE-1904) Reducing locking contention in TaskTracker.MapOutputServlet's LocalDirAllocator

["Rajesh Balamohan (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rajesh Balamohan updated MAPREDUCE-1904:
----------------------------------------

    Attachment: LocalDirAllocator.JPG
                LocalDirAllocator_Monitor.JPG

CPU profiler output of a TaskTracker "without" the patch. Profiler output shows that ~4% of time being spent on internal methods of LocalDirAllocator.*.getLocalPathToRead(). Rest of them time is spent on the method itself. This along with the monitor profiling shows synchonization to be the bottleneck in getLocalPathToRead().

> Reducing locking contention in TaskTracker.MapOutputServlet's LocalDirAllocator
> -------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1904
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1904
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: tasktracker
>    Affects Versions: 0.20.1
>            Reporter: Rajesh Balamohan
>         Attachments: LocalDirAllocator.JPG, LocalDirAllocator_Monitor.JPG, MAPREDUCE-1904-RC10.patch, MAPREDUCE-1904-trunk.patch, profiler output after applying the patch.jpg, TaskTracker- yourkit profiler output .jpg, Thread profiler output showing contention.jpg
>
>
> While profiling tasktracker with Sort benchmark, it was observed that threads block on LocalDirAllocator.getLocalPathToRead() in order to get the index file and temporary map output file.
> As LocalDirAllocator is tied up with ServetContext,  only one instance would be available per tasktracker httpserver.  Given the jobid & mapid, LocalDirAllocator retrieves index file path and temporary map output file path. getLocalPathToRead() is internally synchronized.
> Introducing a LRUCache for this lookup reduces the contention heavily (LRUCache with key =jobid +mapid and value=PATH to the file). Size of the LRUCache can be varied based on the environment and I observed a throughput improvement in the order of 4-7% with the introduction of LRUCache.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.