You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@pig.apache.org by "Akihiro Matsukawa (JIRA)" <ji...@apache.org> on 2013/10/04 00:06:42 UTC

[jira] [Updated] (PIG-3497) JobControlCompiler should only do reducer estimation when the job has a reduce phase

     [ https://issues.apache.org/jira/browse/PIG-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Akihiro Matsukawa updated PIG-3497:
-----------------------------------

    Attachment: reducer_estimation.patch

> JobControlCompiler should only do reducer estimation when the job has a reduce phase
> ------------------------------------------------------------------------------------
>
>                 Key: PIG-3497
>                 URL: https://issues.apache.org/jira/browse/PIG-3497
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Akihiro Matsukawa
>            Priority: Minor
>         Attachments: reducer_estimation.patch
>
>
> Currently, JobControlCompiler makes an estimation for the number of reducers required (by default based on input size into mappers) regardless of whether there is a reduce phase in the job. This is unnecessary, especially when running more complex custom reducer estimators. 
> Change to only estimate reducers when necessary.



--
This message was sent by Atlassian JIRA
(v6.1#6144)