You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@pig.apache.org by "Aniket Mokashi (JIRA)" <ji...@apache.org> on 2014/01/15 23:34:19 UTC

[jira] [Commented] (PIG-3463) Pig should use hadoop local mode for small jobs

    [ https://issues.apache.org/jira/browse/PIG-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13872710#comment-13872710 ] 

Aniket Mokashi commented on PIG-3463:
-------------------------------------

If pig.auto.local.enabled is set, JCC will modify Configuration of all the jobs with one reducer and input size less than pig.auto.local.input.maxbytes, so that they are forced to run in local mode. Output of local run is also written to hdfs.

> Pig should use hadoop local mode for small jobs
> -----------------------------------------------
>
>                 Key: PIG-3463
>                 URL: https://issues.apache.org/jira/browse/PIG-3463
>             Project: Pig
>          Issue Type: New Feature
>          Components: impl
>    Affects Versions: 0.11.1
>            Reporter: Aniket Mokashi
>            Assignee: Aniket Mokashi
>             Fix For: 0.13.0
>
>         Attachments: PIG-3463-1.patch
>
>
> Pig should use hadoop local mode for small jobs - few mappers, few reducers and few mb of data.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)