You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Luke Lu (JIRA)" <ji...@apache.org> on 2012/07/05 12:25:35 UTC

[jira] [Created] (MAPREDUCE-4400) Fix performance regression for small jobs/workflows

Luke Lu created MAPREDUCE-4400:
----------------------------------

             Summary: Fix performance regression for small jobs/workflows
                 Key: MAPREDUCE-4400
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: task
    Affects Versions: 1.0.3, 0.20.203.0
            Reporter: Luke Lu
            Assignee: Luke Lu


There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-4400) Fix performance regression for small jobs/workflows

Posted by "Tom White (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409627#comment-13409627 ] 

Tom White commented on MAPREDUCE-4400:
--------------------------------------

The same issue was fixed in trunk and branch-2 in MAPREDUCE-3809 in much the same way. How about backporting that code to branch-1?
                
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-4400
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>    Affects Versions: 0.20.203.0, 1.0.3
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>         Attachments: mapreduce-4400-branch-1.patch
>
>
> There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-4400) Fix performance regression for small jobs/workflows

Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417774#comment-13417774 ] 

Luke Lu commented on MAPREDUCE-4400:
------------------------------------

@Shinivas: have you tried this with mapreduce.tasktracker.outofband.heartbeat=true? (needs a cluster restart of course).
                
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-4400
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>    Affects Versions: 0.20.203.0, 1.0.3
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>         Attachments: mapreduce-4400-branch-1.patch
>
>
> There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-4400) Fix performance regression for small jobs/workflows

Posted by "Tom White (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421423#comment-13421423 ] 

Tom White commented on MAPREDUCE-4400:
--------------------------------------

+1
                
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-4400
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: performance, task
>    Affects Versions: 0.20.203.0, 1.0.3
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>         Attachments: mapreduce-4400-branch-1.patch
>
>
> There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-4400) Fix performance regression for small jobs/workflows

Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406969#comment-13406969 ] 

Luke Lu commented on MAPREDUCE-4400:
------------------------------------

Thanks to John Poelman and Shreyas Subramanya of IBM BigInsights performance QA for noticing the issue and verifying my fix.
                
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-4400
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>    Affects Versions: 0.20.203.0, 1.0.3
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>
> There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (MAPREDUCE-4400) Fix performance regression for small jobs/workflows

Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Luke Lu updated MAPREDUCE-4400:
-------------------------------

    Attachment: mapreduce-4400-branch-1.patch

MAPREDUCE-2450 fixed the communication race condition that can cause occasional 1 minute timeout but make the PROGRESS_INTERVAL sleep pretty much mandatory. Any tasks including setup and cleanup tasks would need to sleep at least 3 seconds to finish.

The patch make the wait interruptable when tasks finish.

With this patch and MAPREDUCE-4399 and outofband heartbeats, PigMix is about 20% faster than 0.20.2. These patches makes Hadoop 1.x series on par with Hadoop 2.x in terms of general performance.
                
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-4400
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>    Affects Versions: 0.20.203.0, 1.0.3
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>         Attachments: mapreduce-4400-branch-1.patch
>
>
> There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (MAPREDUCE-4400) Fix performance regression for small jobs/workflows

Posted by "Matt Foley (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matt Foley updated MAPREDUCE-4400:
----------------------------------

    Fix Version/s:     (was: 1.1.1)
                   1.1.0

Due to delay in 1.1.0, brought this into 1.1.0 from 1.1.1.
                
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-4400
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: performance, task
>    Affects Versions: 0.20.203.0, 1.0.3
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>             Fix For: 1.1.0
>
>         Attachments: mapreduce-4400-branch-1.patch
>
>
> There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Resolved] (MAPREDUCE-4400) Fix performance regression for small jobs/workflows

Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Luke Lu resolved MAPREDUCE-4400.
--------------------------------

             Tags: task, performance
       Resolution: Fixed
    Fix Version/s: 1.1.1
     Hadoop Flags: Reviewed

Thanks for the review and +1, Shrinivas and Tom! Created MAPREDUCE-4477 to track trunk improvement. Committed the change to branch-1 and branch-1.1. It's not in 1.1.0-rc1 though.
                
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-4400
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: performance, task
>    Affects Versions: 0.20.203.0, 1.0.3
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>             Fix For: 1.1.1
>
>         Attachments: mapreduce-4400-branch-1.patch
>
>
> There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-4400) Fix performance regression for small jobs/workflows

Posted by "Shrinivas Joshi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415838#comment-13415838 ] 

Shrinivas Joshi commented on MAPREDUCE-4400:
--------------------------------------------

Hi Luke - In our experiments your patch did achieve the same effect as what MAPREDUCE-4381 was trying to in terms of performance. We noticed good performance gains on Mahout KMeans clustering workload (~ 4%). It would be nice if we can get the branch-1 version of your change reviewed and checked-in in the mean time. Thanks.
                
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-4400
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>    Affects Versions: 0.20.203.0, 1.0.3
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>         Attachments: mapreduce-4400-branch-1.patch
>
>
> There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-4400) Fix performance regression for small jobs/workflows

Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13418539#comment-13418539 ] 

Luke Lu commented on MAPREDUCE-4400:
------------------------------------

Yes. The speed up is more pronounced with outofband heartbeat, which has similar effect of MAPREDUCE-1906 (which is not in branch-1). MRv2 doesn't need this patch as it was addressed by MAPREDUCE-3809. Tom, can we file a separate jira to improve the change in trunk? 

Shrinivas, you're encouraged to review and +1 on the patch :)
                
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-4400
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>    Affects Versions: 0.20.203.0, 1.0.3
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>         Attachments: mapreduce-4400-branch-1.patch
>
>
> There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-4400) Fix performance regression for small jobs/workflows

Posted by "Shrinivas Joshi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13418476#comment-13418476 ] 

Shrinivas Joshi commented on MAPREDUCE-4400:
--------------------------------------------

Can I request a code review and commit of this patch so that it gets integrated in to MRv1 branch in the mean time it is ported to MRv2? Thanks.
                
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-4400
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>    Affects Versions: 0.20.203.0, 1.0.3
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>         Attachments: mapreduce-4400-branch-1.patch
>
>
> There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (MAPREDUCE-4400) Fix performance regression for small jobs/workflows

Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Suresh Srinivas updated MAPREDUCE-4400:
---------------------------------------

    Component/s: performance
    
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-4400
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: performance, task
>    Affects Versions: 0.20.203.0, 1.0.3
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>         Attachments: mapreduce-4400-branch-1.patch
>
>
> There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-4400) Fix performance regression for small jobs/workflows

Posted by "Tom White (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13418699#comment-13418699 ] 

Tom White commented on MAPREDUCE-4400:
--------------------------------------

Luke - yes, please do.
                
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-4400
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>    Affects Versions: 0.20.203.0, 1.0.3
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>         Attachments: mapreduce-4400-branch-1.patch
>
>
> There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-4400) Fix performance regression for small jobs/workflows

Posted by "Shrinivas Joshi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13418913#comment-13418913 ] 

Shrinivas Joshi commented on MAPREDUCE-4400:
--------------------------------------------

As I said earlier, I did verify that this patch was working as expected. It does minimize synchronization over MR3809. This patch looks good to be committed to me.
                
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-4400
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>    Affects Versions: 0.20.203.0, 1.0.3
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>         Attachments: mapreduce-4400-branch-1.patch
>
>
> There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-4400) Fix performance regression for small jobs/workflows

Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409740#comment-13409740 ] 

Luke Lu commented on MAPREDUCE-4400:
------------------------------------

Thanks for the pointer to MAPREDUCE-3809, Tom. IMO, this patch is slightly better as it minimizes synchronization. 
                
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-4400
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>    Affects Versions: 0.20.203.0, 1.0.3
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>         Attachments: mapreduce-4400-branch-1.patch
>
>
> There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-4400) Fix performance regression for small jobs/workflows

Posted by "Tom White (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409752#comment-13409752 ] 

Tom White commented on MAPREDUCE-4400:
--------------------------------------

Maybe have a patch for trunk/branch-2 to bring the two into line then? I think it's good to minimize the number of differences where possible.
                
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-4400
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>    Affects Versions: 0.20.203.0, 1.0.3
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>         Attachments: mapreduce-4400-branch-1.patch
>
>
> There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-4400) Fix performance regression for small jobs/workflows

Posted by "Shrinivas Joshi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13418473#comment-13418473 ] 

Shrinivas Joshi commented on MAPREDUCE-4400:
--------------------------------------------

@Luke: I have not tried with outofband heartbeat property. Do you expect this to show more perf gains along with your patch?
                
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-4400
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>    Affects Versions: 0.20.203.0, 1.0.3
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>         Attachments: mapreduce-4400-branch-1.patch
>
>
> There is a significant performance regression for small jobs/workflows (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs. PigMix has an average 40% regression against 0.20.2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira