You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@datafu.apache.org by "Eyal Allweil (JIRA)" <ji...@apache.org> on 2017/12/14 13:16:00 UTC

[jira] [Closed] (DATAFU-116) Make SetIntersect and SetDifference implement Accumulator

     [ https://issues.apache.org/jira/browse/DATAFU-116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eyal Allweil closed DATAFU-116.
-------------------------------
    Resolution: Won't Fix

Since it seems like Pig doesn't use the Accumulator interface when there are multiple bags in the input, this improvement isn't relevant for these UDF's.

> Make SetIntersect and SetDifference implement Accumulator
> ---------------------------------------------------------
>
>                 Key: DATAFU-116
>                 URL: https://issues.apache.org/jira/browse/DATAFU-116
>             Project: DataFu
>          Issue Type: Improvement
>    Affects Versions: 1.3.0
>            Reporter: Eyal Allweil
>
> SetIntersect and SetDifference accept only sorted bags, and the output is always smaller than the inputs. Therefore an accumulator implementation should be possible and it will improve memory usage (somewhat) and allow Pig to optimize loops with these operations better.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)