[jira] [Commented] (TEZ-393) Handle custom partitioners

["Bikas Saha (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749003#comment-13749003 ] 

Bikas Saha commented on TEZ-393:
--------------------------------

Can you please elaborate on how custom partitioners can cause issues. Assuming that a custom partitioner overrides the default method of hashing keys to partitions, once the partitions are written, it should be transparent whether 1 partition goes to 1 reduce task or multiple partitions go to 1 reduce task. Right?

The decision to enable or disable dynamic reduce concurrency will be set by the user. So the user can choose to not use it.
                
> Handle custom partitioners
> --------------------------
>
>                 Key: TEZ-393
>                 URL: https://issues.apache.org/jira/browse/TEZ-393
>             Project: Apache Tez
>          Issue Type: Sub-task
>            Reporter: Rohini Palaniswamy
>
>   With dynamic allocation of reducers, cases like range partitioning (for ORDER BY) need to be handled properly. Also many users have other custom partitioners. So might be good to have an option to turn of dynamic reducer allocation in case they don't work well with dynamic allocation. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira