You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@pig.apache.org by "Rohini Palaniswamy (JIRA)" <ji...@apache.org> on 2016/04/26 01:38:14 UTC

[jira] [Updated] (PIG-3346) New property that controls the number of combined splits

     [ https://issues.apache.org/jira/browse/PIG-3346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rohini Palaniswamy updated PIG-3346:
------------------------------------
    Fix Version/s:     (was: 0.16.0)
                   0.17.0

> New property that controls the number of combined splits
> --------------------------------------------------------
>
>                 Key: PIG-3346
>                 URL: https://issues.apache.org/jira/browse/PIG-3346
>             Project: Pig
>          Issue Type: Improvement
>          Components: impl
>            Reporter: Cheolsoo Park
>            Assignee: Cheolsoo Park
>             Fix For: 0.17.0
>
>         Attachments: PIG-3346-2.patch, PIG-3346-3.patch, PIG-3346.patch
>
>
> Currently, the size of combined splits can be configured by the {{pig.maxCombinedSplitSize}} property.
> Although this works fine most of time, it can lead to a undesired situation where a single mapper ends up loading a lot of combined splits. Particularly, this is bad if Pig uploads them from S3.
> So it will be useful if the max number of combined splits can be configured via a property something like {{pig.maxCombinedSplitNum}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)