You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Alex Bain (JIRA)" <ji...@apache.org> on 2014/01/08 02:48:50 UTC
[jira] [Commented] (PIG-3562) Implement combiner optimizations for
DISTINCT
[ https://issues.apache.org/jira/browse/PIG-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13864973#comment-13864973 ]
Alex Bain commented on PIG-3562:
--------------------------------
ReviewBoard posted at https://reviews.apache.org/r/16717/
> Implement combiner optimizations for DISTINCT
> ---------------------------------------------
>
> Key: PIG-3562
> URL: https://issues.apache.org/jira/browse/PIG-3562
> Project: Pig
> Issue Type: Sub-task
> Components: tez
> Affects Versions: tez-branch
> Reporter: Alex Bain
> Assignee: Alex Bain
> Fix For: tez-branch
>
> Attachments: PIG-3562-0.patch
>
>
> Currently, DISTINCT is implemented in a straightforward manner per https://issues.apache.org/jira/browse/PIG-3538.
> However, we can implement two types of combiner optimizations for DISTINCT, just as the MRCompiler does for map-reduce:
> 1. A simple DistinctCombiner that throws away the duplicate tuples
> 2. An optimizer that transforms certain uses of DISTINCT into an algebraic udf form
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)