You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Amir Youssefi (JIRA)" <ji...@apache.org> on 2009/04/15 04:07:15 UTC
[jira] Commented: (PIG-750) Use combiner when a mix of algebraic
and non-algebraic functions are used
[ https://issues.apache.org/jira/browse/PIG-750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699023#action_12699023 ]
Amir Youssefi commented on PIG-750:
-----------------------------------
Other use-cases we need have in unit tests:
1) foreach X generate SUM(a) * AVG(b), ...
2) foreach X generate 1 / SUM(a)
Currently, there is a work-around suggested to have all algebraic functions calculated in a foreach and then more expressions/mixes are calculated in a second foreach. This way combiner is used in the first foreach and we get combiner speed-up.
> Use combiner when a mix of algebraic and non-algebraic functions are used
> -------------------------------------------------------------------------
>
> Key: PIG-750
> URL: https://issues.apache.org/jira/browse/PIG-750
> Project: Pig
> Issue Type: Improvement
> Reporter: Amir Youssefi
> Priority: Minor
>
> Currently Pig uses combiner when all a,b, c,... are algebraic (e.g. SUM, AVG etc.) in foreach:
> foreach X generate a,b,c,...
> It's a performance improvement if it uses combiner when a mix of algebraic and non-algebraic functions are used as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.