You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@pig.apache.org by "Daniel Dai (JIRA)" <ji...@apache.org> on 2015/05/12 09:05:00 UTC

[jira] [Updated] (PIG-3346) New property that controls the number of combined splits

     [ https://issues.apache.org/jira/browse/PIG-3346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai updated PIG-3346:
----------------------------
    Fix Version/s:     (was: 0.15.0)
                   0.16.0

> New property that controls the number of combined splits
> --------------------------------------------------------
>
>                 Key: PIG-3346
>                 URL: https://issues.apache.org/jira/browse/PIG-3346
>             Project: Pig
>          Issue Type: Improvement
>          Components: impl
>            Reporter: Cheolsoo Park
>            Assignee: Cheolsoo Park
>             Fix For: 0.16.0
>
>         Attachments: PIG-3346-2.patch, PIG-3346-3.patch, PIG-3346.patch
>
>
> Currently, the size of combined splits can be configured by the {{pig.maxCombinedSplitSize}} property.
> Although this works fine most of time, it can lead to a undesired situation where a single mapper ends up loading a lot of combined splits. Particularly, this is bad if Pig uploads them from S3.
> So it will be useful if the max number of combined splits can be configured via a property something like {{pig.maxCombinedSplitNum}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)